From patchwork Sun Apr 11 06:27:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2B10C433B4 for ; Sun, 11 Apr 2021 06:28:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B449611C2 for ; Sun, 11 Apr 2021 06:28:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235012AbhDKG2T (ORCPT ); Sun, 11 Apr 2021 02:28:19 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:47893 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229792AbhDKG2T (ORCPT ); Sun, 11 Apr 2021 02:28:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122498; x=1649658498; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=N8NWw71q4aSE7f87UU28NQUoPYbfY8JCnxG3yVRDbPA=; b=eSgfZRVjuyDi7bREdEqEfz8IiKv30Hy04ncVC8DvR4J2qetPOTuJ8yFi YYHHl/ksRdGzps+weepq1TAQPGjZL2xK0uDHs3h9IjIGKuohZx90Gz8u8 paiTQ/IpWysdcMx3I1EAqjrt4tR/OqmWMSdK9JHLOEiQRmgTYafSEjUyJ obqxJKkNr0SKJ+t7W1RC/2fv64lPIT2TdTN+qcR0to6VVpVCRywqePw10 HNat0Ph3p4s83KKG5KB3ddGmkt/tskc3XnkavdBbVTkxkvFwdve5xdtRq R2CypPD49ziqwrQIiav8qXNJtdROOjRjddfzESHNx7VEmpxR/o63S1gnP w==; IronPort-SDR: CSi6FTE5CgSWU5QwWTKk53p8MvG0cyWXKbb8N1QM8GCdsE5IJu4bVh4fy3KKlPsVLxXtOLVnh4 EcR0nMbiih0QovRnfPpcfaLKZfRGlC2QRIJCD5CZmywT3T3BZig8kRYla9WQptuG9NMIEOU79t nLmv7i0ITDn8Oho0MOq+hYmbsgXbYHmVibcgJiALEUt0bA35fxyfVCeOo650q+zXVjPQC2acZ8 vHpnyn2Nrlod2hxASUIylbHedMXoCufNnwmJw+toBCrDLcIQHCiOifJwi2X1/ej4UC+im0nvhV ByE= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="268668860" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:01 +0800 IronPort-SDR: KtQUvaW92ZHOzz50GJqIqI2YZFKUiP6inkyJ6ZawZh7LEdDnzof4ci+9cAvFPjtLlqSBbxK1vE ntQrPEQ6c0a6MdmOLVFpeZCfPzR+9Wmkg9Vs2kWX/+yDdWGmNGl53fHr1LfMRaZ9QCJfcXHKvQ x3xZ9A9K4Rpc8aWtA5ok1ilHvKIQqtirAh92dtLyg3D6Q5vmtx3U5EOlEProGAGymW9WCc/oIT S5mimEyJLW+S9SvdAqbymW6QXPN5YmpRNnJI1smTQOA47kMqkJ4O78wiKj8aaj8QAI1iekzrqO cNzD7C8pIurvUb+0XWOliwuk Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:08:54 -0700 IronPort-SDR: 4g2SHL03z4wlJcO8d4k29dRsNDoDzcvYS89DYTdlA4teTExZCOC9ompE1U/Pdzce2+WL3yqXHy NGTepYpOzUXMY/P2EgRKH/rnlT6leXuFU4ccNPBtKZs6D3xyW8KJK9CWcg8s++tDGNNLv5aNmp sU2mR7iB4+qRacCqDJTJJAjJ4HMX2aIgHIhankyoHTYUrTSpHdvfbVp0TvAlDqWGbvBqIPV5iR wRZ2quVlGbmoDcuou1l6g/K0U6NQZSTYu453/KmscY195641qn4boRp0B/kFr2HiqXPvpwRgDE 5Yk= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:27:42 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 01/11] scsi: ufshpb: Cache HPB Control mode on init Date: Sun, 11 Apr 2021 09:27:11 +0300 Message-Id: <20210411062721.10099-2-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org We will use it later, when we'll need to differentiate between device and host control modes. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshcd.h | 2 ++ drivers/scsi/ufs/ufshpb.c | 8 +++++--- drivers/scsi/ufs/ufshpb.h | 2 ++ 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ufs/ufshcd.h b/drivers/scsi/ufs/ufshcd.h index 4dbe9bc60e85..c01f75963750 100644 --- a/drivers/scsi/ufs/ufshcd.h +++ b/drivers/scsi/ufs/ufshcd.h @@ -656,6 +656,7 @@ struct ufs_hba_variant_params { * @hpb_disabled: flag to check if HPB is disabled * @max_hpb_single_cmd: maximum size of single HPB command * @is_legacy: flag to check HPB 1.0 + * @control_mode: either host or device */ struct ufshpb_dev_info { int num_lu; @@ -665,6 +666,7 @@ struct ufshpb_dev_info { bool hpb_disabled; int max_hpb_single_cmd; bool is_legacy; + u8 control_mode; }; #endif diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 86805af9abe7..5285a50b05dd 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -1615,6 +1615,9 @@ static void ufshpb_lu_parameter_init(struct ufs_hba *hba, % (hpb->srgn_mem_size / HPB_ENTRY_SIZE); hpb->pages_per_srgn = DIV_ROUND_UP(hpb->srgn_mem_size, PAGE_SIZE); + + if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) + hpb->is_hcm = true; } static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) @@ -2308,11 +2311,10 @@ void ufshpb_get_dev_info(struct ufs_hba *hba, u8 *desc_buf) { struct ufshpb_dev_info *hpb_dev_info = &hba->ufshpb_dev; int version, ret; - u8 hpb_mode; u32 max_hpb_single_cmd = HPB_MULTI_CHUNK_LOW; - hpb_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; - if (hpb_mode == HPB_HOST_CONTROL) { + hpb_dev_info->control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; + if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) { dev_err(hba->dev, "%s: host control mode is not supported.\n", __func__); hpb_dev_info->hpb_disabled = true; diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index b1128b0ce486..7df30340386a 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -228,6 +228,8 @@ struct ufshpb_lu { u32 entries_per_srgn_shift; u32 pages_per_srgn; + bool is_hcm; + struct ufshpb_stats stats; struct ufshpb_params params; From patchwork Sun Apr 11 06:27:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97DC1C433ED for ; Sun, 11 Apr 2021 06:28:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 71D4B6101B for ; Sun, 11 Apr 2021 06:28:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235185AbhDKG2l (ORCPT ); Sun, 11 Apr 2021 02:28:41 -0400 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:37436 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235017AbhDKG2k (ORCPT ); Sun, 11 Apr 2021 02:28:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122506; x=1649658506; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WPqT2rdAevYmQxqfinYFjjtoO19TlVzPnQtPtcAtiZM=; b=U2/0grjaOfPUYmCFuy7/eEL3mG/LOhruYMxLPjTlTx+QRKtxy3KdVWF9 jXKd3JfUILFFvmDmNppZ8tXyU3ABxyiWclh+sZasqm4HlPl+e72FwnuHp Y87H2GRWHthYVrUmt9Oux12wgUL1SBN9RpQpvqL48TFyHz/iDHcSlRWPG 8vucDgGdMMTyhFXWpi3YhA8KR/1sTWLrKMDdpH1swBPoIpG+wS545b9Ms CAiekSGC0hAm//3VIQeV6RHMCHCQkCNerrYoYzOiJMiKFDhm3DwxGKIDV I+EBTFn8bghhSGp1pW3U294ehY9cxfMWPxlS6kW5nl4RjdDMskOAtDqN2 w==; IronPort-SDR: C/Wxzi2GAAEmowyRxc9zX2nEGRXijKqGCMqQCicXpEoPzDGhhmTaShUHnTOFC86r0Ogs4ajs2k cFCyUUsWJtyon87rX3z2IlF3Jj6G+C+E3AE8wPKjQ1rzM5+ZFh+fecpcv83pSE0+0vBydNHTgO qlFFaEHAbG8dwqNtizYbNP2kKOVNUNbZonJjAKxQsyfz9slnnPkm+8Pc6lbPIcPH4MwwvSs7hN LZpiTeQYGvhWqZ+H5hWDBBmv5StOOaq8yBvKZOv+fVr4GFi8QLdddoFKXsn7leK0BjRY9UpXQH Sy0= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="165243139" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:04 +0800 IronPort-SDR: 1AygAaqfeIfkX47F3bbv2Uo1KkiOZs+R5caRG8Is+zQqthfuyIhK6zXmQ5EFH4U1mlZPqw2GOh WlOpl8viqpuYZA3oRpWQuEpt9eOel+hXeVXeIUu5tvbN/uymy+hq0uIB+LhfUAAThbDPqFFdwg FgmZBSW6z1OB8Qvneytx0dMuuzFuGdNGFp+TGV6gCurLdFWV+o6oWLwHmnNy790ciX26kJVd8w g0hoL59g2ApNOtePPAEEGaaYGml4ZSMlkFmYKHxjCl6EwqBXccIIagxstsK82MouwNtURzfLe+ 9bb0jFSB2zoUqqT93Ow29axc Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:09:01 -0700 IronPort-SDR: k4LiYqI3aXs7VMzlYynluVJULslQLtOqJChTMF0h9Sc3MYVHTfg+08iKCzJB0hLOn9VGNGEswW Rau5xy4S5LvzOGlRkC0gvhh7/Fp/PqtNgnpBiNFzoCdC1I0SHL1cg9irDnM86GT5etMJiTFM0h qw+IJx3ucAMgbfgA+rNzjgtthhPRmSQRAUAFGFPRqK/p31OyhAzR4AucAVDqSYcuQntU1z/CT9 /eogH7SzKJLs1/W1T1zyBSoQCUp4OnAvh9s8Lezku8y45ig6tmOWb3Jxk9Suv/U/U7J6fZ8Fxq m9w= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:27:50 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 02/11] scsi: ufshpb: Add host control mode support to rsp_upiu Date: Sun, 11 Apr 2021 09:27:12 +0300 Message-Id: <20210411062721.10099-3-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In device control mode, the device may recommend the host to either activate or inactivate a region, and the host should follow. Meaning those are not actually recommendations, but more of instructions. On the contrary, in host control mode, the recommendation protocol is slightly changed: a) The device may only recommend the host to update a subregion of an already-active region. And, b) The device may *not* recommend to inactivate a region. Furthermore, in host control mode, the host may choose not to follow any of the device's recommendations. However, in case of a recommendation to update an active and clean subregion, it is better to follow those recommendation because otherwise the host has no other way to know that some internal relocation took place. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 34 +++++++++++++++++++++++++++++++++- drivers/scsi/ufs/ufshpb.h | 2 ++ 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 5285a50b05dd..6111019ca31a 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -166,6 +166,8 @@ static void ufshpb_set_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, else set_bit_len = cnt; + set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + if (rgn->rgn_state != HPB_RGN_INACTIVE && srgn->srgn_state == HPB_SRGN_VALID) bitmap_set(srgn->mctx->ppn_dirty, srgn_offset, set_bit_len); @@ -235,6 +237,11 @@ static bool ufshpb_test_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, return false; } +static inline bool is_rgn_dirty(struct ufshpb_region *rgn) +{ + return test_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); +} + static int ufshpb_fill_ppn_from_page(struct ufshpb_lu *hpb, struct ufshpb_map_ctx *mctx, int pos, int len, u64 *ppn_buf) @@ -712,6 +719,7 @@ static void ufshpb_put_map_req(struct ufshpb_lu *hpb, static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, struct ufshpb_subregion *srgn) { + struct ufshpb_region *rgn; u32 num_entries = hpb->entries_per_srgn; if (!srgn->mctx) { @@ -725,6 +733,10 @@ static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, num_entries = hpb->last_srgn_entries; bitmap_zero(srgn->mctx->ppn_dirty, num_entries); + + rgn = hpb->rgn_tbl + srgn->rgn_idx; + clear_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + return 0; } @@ -1244,6 +1256,18 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, srgn_i = be16_to_cpu(rsp_field->hpb_active_field[i].active_srgn); + rgn = hpb->rgn_tbl + rgn_i; + if (hpb->is_hcm && + (rgn->rgn_state != HPB_RGN_ACTIVE || is_rgn_dirty(rgn))) { + /* + * in host control mode, subregion activation + * recommendations are only allowed to active regions. + * Also, ignore recommendations for dirty regions - the + * host will make decisions concerning those by himself + */ + continue; + } + dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "activate(%d) region %d - %d\n", i, rgn_i, srgn_i); @@ -1251,7 +1275,6 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, ufshpb_update_active_info(hpb, rgn_i, srgn_i); spin_unlock(&hpb->rsp_list_lock); - rgn = hpb->rgn_tbl + rgn_i; srgn = rgn->srgn_tbl + srgn_i; /* blocking HPB_READ */ @@ -1262,6 +1285,14 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, hpb->stats.rb_active_cnt++; } + if (hpb->is_hcm) { + /* + * in host control mode the device is not allowed to inactivate + * regions + */ + goto out; + } + for (i = 0; i < rsp_field->inactive_rgn_cnt; i++) { rgn_i = be16_to_cpu(rsp_field->hpb_inactive_field[i]); dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, @@ -1286,6 +1317,7 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, hpb->stats.rb_inactive_cnt++; } +out: dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, "Noti: #ACT %u #INACT %u\n", rsp_field->active_rgn_cnt, rsp_field->inactive_rgn_cnt); diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 7df30340386a..032672114881 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -121,6 +121,8 @@ struct ufshpb_region { /* below information is used by lru */ struct list_head list_lru_rgn; + unsigned long rgn_flags; +#define RGN_FLAG_DIRTY 0 }; #define for_each_sub_region(rgn, i, srgn) \ From patchwork Sun Apr 11 06:27:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D24FC433B4 for ; Sun, 11 Apr 2021 06:28:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1EE5E611C2 for ; Sun, 11 Apr 2021 06:28:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235152AbhDKG2f (ORCPT ); Sun, 11 Apr 2021 02:28:35 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:46956 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235113AbhDKG2e (ORCPT ); Sun, 11 Apr 2021 02:28:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122521; x=1649658521; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c8dspGgbwbxAFjlh/X7gkBtjw1xkLXGeVV/u/PXbzIQ=; b=auGfpeyem+BPyJ2UOGjEk/2VwNGdSe1IrlyIspQp4SOewnNehswuMDIh s4JYTne0KDzqAJAn0AC1qyVpLtVZ/RlNleltb4l82wdlDVCs3C11hsBBr pY/ddKz/LMgto5xdVjbZ3AAAsldfwfBikKA91qvniWcfPa0C+T1zarSDF NrxJPlJ3TISon1hIySsejx7bpCXgSP4vfNCrQrOmXX8cFtJD6Phz7sP+M PAOYkVbw0Ghf+YmYMnxui7yEKi/W7i/7FW1Uy/QaSVRNO1WIqLiA8Ccn9 tISTYwnNovz6UG0n0VQSHOx4aaP+uXubLusqOsYlHxBI+aNZurDpt73gz w==; IronPort-SDR: WeiyXJjvWpDl9LrFDBLsZRZj4V18dJPJRncmzCy0uHuYgDkhHzyz/yjSaG0P4W+KKZfJdHbL9g FGnvoeVHJD4vfXjUjWasWNeaTOclHdD8xvUFayFsMMx824dnEqLHFel8knE/8hIVraSGBWYI/n 55e9uHf4ZLPvx4Q5diQnGookSsQAtFpTiGL7yM0G/uA3golx4pexg90i0fXZN1wjezAHmQ+Hp2 yQ3hMkE/zIl8ScYAF9HxeOOThL33qku7TK1OIv8cDAK+VMktriIhP7m5aVGcckwjvcbLrWFRxk bho= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="268668873" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:24 +0800 IronPort-SDR: nFOzTUX4TANwFnqfpAoYgB10myP8Oq10hjz8VVG+pcJX14eIVx5xam/v/cY9mf8ZkbOqx9ZdRp GbRuWU6cF2h0vYCQYDU1j3PfFhnfC2+wPs6ELEiFdCrQ6KxQKd4g13QQ+l8dn56WVV1/v5YZg2 VyokpPC8Qgar0YOt0f3IQPK9T4EfXOpcx1S3wb1HQildxxoHDjVUVOnzYM7x27StcLaH69f9sz WMJrwUsy4hfWWgFh2AEWwLdn1pnjQcn9Q0Uu1O7Pr4/olqOl6VHwUmXNcVzDTXa797YMhGo7/7 JAszCaCqvET3cSZkjGL+6CWV Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:09:07 -0700 IronPort-SDR: SCNfS1RvX0BBmYfGMnTHehR55oVb8Jg5Wd6vH5objZpUIfNx+KizVP1HtrACX/T0xlvAafOUMx YmvozW9bp/hp/C6fakMV/oSAJLlMEzFfg1olHHT9d/teAfO2k68NviH7ftrHiUf5DDSJSyn7aG BykSXpfYxUUUdNV6vFmFd5eyxZA4lhUaTsbZYnNCRcmGYFUVLDN1uoKRZnd8tNSdwXd1Az5Q/Y 2kA348VAxn2QpZuLXd/Ay4t42yUov2/TqkcgWpTPv0Odj9MnF6JqndI9mUYmmGb40TRUYdX7VR 0jA= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:27:56 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 03/11] scsi: ufshpb: Transform set_dirty to iterate_rgn Date: Sun, 11 Apr 2021 09:27:13 +0300 Message-Id: <20210411062721.10099-4-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Given a transfer length, set_dirty meticulously runs over all the entries, across subregions and regions if needed. Currently its only use is to mark dirty blocks, but soon HCM may profit from it as well, when managing its read counters. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 6111019ca31a..252fcfb48862 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -144,13 +144,14 @@ static bool ufshpb_is_hpb_rsp_valid(struct ufs_hba *hba, return true; } -static void ufshpb_set_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, - int srgn_idx, int srgn_offset, int cnt) +static void ufshpb_iterate_rgn(struct ufshpb_lu *hpb, int rgn_idx, int srgn_idx, + int srgn_offset, int cnt, bool set_dirty) { struct ufshpb_region *rgn; struct ufshpb_subregion *srgn; int set_bit_len; int bitmap_len; + unsigned long flags; next_srgn: rgn = hpb->rgn_tbl + rgn_idx; @@ -166,11 +167,14 @@ static void ufshpb_set_ppn_dirty(struct ufshpb_lu *hpb, int rgn_idx, else set_bit_len = cnt; - set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + if (set_dirty) + set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); - if (rgn->rgn_state != HPB_RGN_INACTIVE && + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + if (set_dirty && rgn->rgn_state != HPB_RGN_INACTIVE && srgn->srgn_state == HPB_SRGN_VALID) bitmap_set(srgn->mctx->ppn_dirty, srgn_offset, set_bit_len); + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); srgn_offset = 0; if (++srgn_idx == hpb->srgns_per_rgn) { @@ -592,10 +596,8 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) /* If command type is WRITE or DISCARD, set bitmap as drity */ if (ufshpb_is_write_or_discard_cmd(cmd)) { - spin_lock_irqsave(&hpb->rgn_state_lock, flags); - ufshpb_set_ppn_dirty(hpb, rgn_idx, srgn_idx, srgn_offset, - transfer_len); - spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + ufshpb_iterate_rgn(hpb, rgn_idx, srgn_idx, srgn_offset, + transfer_len, true); return 0; } From patchwork Sun Apr 11 06:27:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6529C433B4 for ; Sun, 11 Apr 2021 06:28:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A6CEB6101D for ; Sun, 11 Apr 2021 06:28:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234996AbhDKG26 (ORCPT ); Sun, 11 Apr 2021 02:28:58 -0400 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:37451 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235204AbhDKG2w (ORCPT ); Sun, 11 Apr 2021 02:28:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122517; x=1649658517; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MP/iaopsFhmHH+0iw/GzZpRoFHNh2uNshgBq4m5TKTY=; b=qfEH5EKtLDYOvuw4UBTZi9/vsfBTgGmLmF6490XF1LCgbWFxZTD4WYY+ brMnam4kG1E2veTkf/ubPbWKXO9LNY9ukwE3BZ3wm2cy1Pz6BZDQFpLq5 0XONYael0hP8+dgixtU7xQrmxXY4Tl/7x5Jqkc6easCP+eSE6bM4xMlNe VP+u1EXOnwS2uSpHtKb50tNBoZkK9wL7Z+F6WmOECJwC1nyxydg57seo2 zhBqbFDcW+ZoLQr0JX85ccUmTCOIpNUL62Eltja1AhpsUASMzQfWm9pQ/ lu994QHn3100Lz7TD9PRbM/ykvlkCI9krkzfZgLcevFuSxIOyqQBoFRfI w==; IronPort-SDR: dwfa3blFGWRkB7Yq1ryQ71WbMzpiut7bLtyn3HH9uyH9A0GJcAcuTqzQM9Lhw9b+cAJEDLzf7I befR9noht3Co+hTA2g6INmpDXg86nTUuBiGurMt9W4Z2fbfIn4wmuIX/Wtmjd/AatvQlUVMJp4 wxJilBG49zZKAjb1lpJxhHA1p2Y8I7S8s7AUUB0r7tycFcouyfdUr5fvq43rmIbvZ/dlmnyafj zG01vKS+XZPk9WeGJEEwd0Q2OuzA6abZUx371aPDHJjwLuXv7ZmOWzoki85JFObtDObSWFhKRT dKo= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="165243146" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:18 +0800 IronPort-SDR: PEnxur/yZ0R0Cim2kSZvE4MBSLRruGLNRCBXh9Htf4U3jwNrBtajo31dJD07sIbouS2cnkYQNT Wsk2i+9gRK9D4LH/3xkIEwJMmtqJvyol5rHzpAJTokzbx69ztobLoOTuS0ifrACXHWzHd2ClJY x7CjjMol5dtmBO93rBmVDQstl2GheR13l347yiaDaAA64da2mr2HL8mL3uGYLFXt/drwj4C/7R snBkP0TEWdfK22shmrP0eJDuH4b1tILsamwfm9jOtkYVe6txXKFmoa3kkwpxPYpCodZuTOQ/zv BlcSWzYpxjy1iwKN/47h3DGp Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:07:40 -0700 IronPort-SDR: zzBVCb5VC6j4LqNS9i7OYo5GCX+jMwoXoX59bjQPR0seuKbOxH+x09e4YkTrwbR2kWhxZi9bjt C10DZR5Eqjgl9lEiabZvVbq9OWRPlpWbaNfzMVDrUOh5jnmmBRb8h234F1c9ruSEi17/ipy2rm G5jLPGb8b6LaPlahhd+vKNMdPTWjKMkFSv3UyNz9Xwp54kxBj1FySvB6Sp7fUDgsNsNgjTgey/ 40XHlsmmwqE3wsZamrHoQLnicS/d8wYB4SO8zahzAX/F7WpLY29QjvHhKyrObPB4FxCK3eEY1+ c1U= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:03 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 04/11] scsi: ufshpb: Add reads counter Date: Sun, 11 Apr 2021 09:27:14 +0300 Message-Id: <20210411062721.10099-5-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host control mode, reads are the major source of activation trials. Keep track of those reads counters, for both active as well inactive regions. We reset the read counter upon write - we are only interested in "clean" reads. Keep those counters normalized, as we are using those reads as a comparative score, to make various decisions. If during consecutive normalizations an active region has exhaust its reads - inactivate it. while at it, protect the {active,inactive}_count stats by adding them into the applicable handler. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 94 ++++++++++++++++++++++++++++++++++++--- drivers/scsi/ufs/ufshpb.h | 9 ++++ 2 files changed, 97 insertions(+), 6 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 252fcfb48862..3ab66421dc00 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -16,6 +16,8 @@ #include "ufshpb.h" #include "../sd.h" +#define ACTIVATION_THRESHOLD 8 /* 8 IOs */ + /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; static mempool_t *ufshpb_mctx_pool; @@ -26,6 +28,9 @@ static int tot_active_srgn_pages; static struct workqueue_struct *ufshpb_wq; +static void ufshpb_update_active_info(struct ufshpb_lu *hpb, int rgn_idx, + int srgn_idx); + bool ufshpb_is_allowed(struct ufs_hba *hba) { return !(hba->ufshpb_dev.hpb_disabled); @@ -148,7 +153,7 @@ static void ufshpb_iterate_rgn(struct ufshpb_lu *hpb, int rgn_idx, int srgn_idx, int srgn_offset, int cnt, bool set_dirty) { struct ufshpb_region *rgn; - struct ufshpb_subregion *srgn; + struct ufshpb_subregion *srgn, *prev_srgn = NULL; int set_bit_len; int bitmap_len; unsigned long flags; @@ -167,15 +172,39 @@ static void ufshpb_iterate_rgn(struct ufshpb_lu *hpb, int rgn_idx, int srgn_idx, else set_bit_len = cnt; - if (set_dirty) - set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); - spin_lock_irqsave(&hpb->rgn_state_lock, flags); if (set_dirty && rgn->rgn_state != HPB_RGN_INACTIVE && srgn->srgn_state == HPB_SRGN_VALID) bitmap_set(srgn->mctx->ppn_dirty, srgn_offset, set_bit_len); spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + if (hpb->is_hcm && prev_srgn != srgn) { + bool activate = false; + + spin_lock(&rgn->rgn_lock); + if (set_dirty) { + rgn->reads -= srgn->reads; + srgn->reads = 0; + set_bit(RGN_FLAG_DIRTY, &rgn->rgn_flags); + } else { + srgn->reads++; + rgn->reads++; + if (srgn->reads == ACTIVATION_THRESHOLD) + activate = true; + } + spin_unlock(&rgn->rgn_lock); + + if (activate) { + spin_lock_irqsave(&hpb->rsp_list_lock, flags); + ufshpb_update_active_info(hpb, rgn_idx, srgn_idx); + spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); + dev_dbg(&hpb->sdev_ufs_lu->sdev_dev, + "activate region %d-%d\n", rgn_idx, srgn_idx); + } + + prev_srgn = srgn; + } + srgn_offset = 0; if (++srgn_idx == hpb->srgns_per_rgn) { srgn_idx = 0; @@ -604,6 +633,19 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) if (!ufshpb_is_support_chunk(hpb, transfer_len)) return 0; + if (hpb->is_hcm) { + /* + * in host control mode, reads are the main source for + * activation trials. + */ + ufshpb_iterate_rgn(hpb, rgn_idx, srgn_idx, srgn_offset, + transfer_len, false); + + /* keep those counters normalized */ + if (rgn->reads > hpb->entries_per_srgn) + schedule_work(&hpb->ufshpb_normalization_work); + } + spin_lock_irqsave(&hpb->rgn_state_lock, flags); if (ufshpb_test_ppn_dirty(hpb, rgn_idx, srgn_idx, srgn_offset, transfer_len)) { @@ -755,6 +797,8 @@ static void ufshpb_update_active_info(struct ufshpb_lu *hpb, int rgn_idx, if (list_empty(&srgn->list_act_srgn)) list_add_tail(&srgn->list_act_srgn, &hpb->lh_act_srgn); + + hpb->stats.rb_active_cnt++; } static void ufshpb_update_inactive_info(struct ufshpb_lu *hpb, int rgn_idx) @@ -770,6 +814,8 @@ static void ufshpb_update_inactive_info(struct ufshpb_lu *hpb, int rgn_idx) if (list_empty(&rgn->list_inact_rgn)) list_add_tail(&rgn->list_inact_rgn, &hpb->lh_inact_rgn); + + hpb->stats.rb_inactive_cnt++; } static void ufshpb_activate_subregion(struct ufshpb_lu *hpb, @@ -1090,6 +1136,7 @@ static int ufshpb_evict_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) rgn->rgn_idx); goto out; } + if (!list_empty(&rgn->list_lru_rgn)) { if (ufshpb_check_srgns_issue_state(hpb, rgn)) { ret = -EBUSY; @@ -1284,7 +1331,6 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, if (srgn->srgn_state == HPB_SRGN_VALID) srgn->srgn_state = HPB_SRGN_INVALID; spin_unlock(&hpb->rgn_state_lock); - hpb->stats.rb_active_cnt++; } if (hpb->is_hcm) { @@ -1316,7 +1362,6 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, } spin_unlock(&hpb->rgn_state_lock); - hpb->stats.rb_inactive_cnt++; } out: @@ -1515,6 +1560,36 @@ static void ufshpb_run_inactive_region_list(struct ufshpb_lu *hpb) spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); } +static void ufshpb_normalization_work_handler(struct work_struct *work) +{ + struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, + ufshpb_normalization_work); + int rgn_idx; + + for (rgn_idx = 0; rgn_idx < hpb->rgns_per_lu; rgn_idx++) { + struct ufshpb_region *rgn = hpb->rgn_tbl + rgn_idx; + int srgn_idx; + + spin_lock(&rgn->rgn_lock); + rgn->reads = 0; + for (srgn_idx = 0; srgn_idx < hpb->srgns_per_rgn; srgn_idx++) { + struct ufshpb_subregion *srgn = rgn->srgn_tbl + srgn_idx; + + srgn->reads >>= 1; + rgn->reads += srgn->reads; + } + spin_unlock(&rgn->rgn_lock); + + if (rgn->rgn_state != HPB_RGN_ACTIVE || rgn->reads) + continue; + + /* if region is active but has no reads - inactivate it */ + spin_lock(&hpb->rsp_list_lock); + ufshpb_update_inactive_info(hpb, rgn->rgn_idx); + spin_unlock(&hpb->rsp_list_lock); + } +} + static void ufshpb_map_work_handler(struct work_struct *work) { struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, map_work); @@ -1674,6 +1749,8 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) rgn = rgn_table + rgn_idx; rgn->rgn_idx = rgn_idx; + spin_lock_init(&rgn->rgn_lock); + INIT_LIST_HEAD(&rgn->list_inact_rgn); INIT_LIST_HEAD(&rgn->list_lru_rgn); @@ -1915,6 +1992,9 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&hpb->list_hpb_lu); INIT_WORK(&hpb->map_work, ufshpb_map_work_handler); + if (hpb->is_hcm) + INIT_WORK(&hpb->ufshpb_normalization_work, + ufshpb_normalization_work_handler); hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", sizeof(struct ufshpb_req), 0, 0, NULL); @@ -2014,6 +2094,8 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { + if (hpb->is_hcm) + cancel_work_sync(&hpb->ufshpb_normalization_work); cancel_work_sync(&hpb->map_work); } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 032672114881..87495e59fcf1 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -106,6 +106,10 @@ struct ufshpb_subregion { int rgn_idx; int srgn_idx; bool is_last; + + /* subregion reads - for host mode */ + unsigned int reads; + /* below information is used by rsp_list */ struct list_head list_act_srgn; }; @@ -123,6 +127,10 @@ struct ufshpb_region { struct list_head list_lru_rgn; unsigned long rgn_flags; #define RGN_FLAG_DIRTY 0 + + /* region reads - for host mode */ + spinlock_t rgn_lock; + unsigned int reads; }; #define for_each_sub_region(rgn, i, srgn) \ @@ -212,6 +220,7 @@ struct ufshpb_lu { /* for selecting victim */ struct victim_select_info lru_info; + struct work_struct ufshpb_normalization_work; /* pinned region information */ u32 lu_pinned_start; From patchwork Sun Apr 11 06:27:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C709C433ED for ; Sun, 11 Apr 2021 06:28:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 52458611C2 for ; Sun, 11 Apr 2021 06:28:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235119AbhDKG23 (ORCPT ); Sun, 11 Apr 2021 02:28:29 -0400 Received: from esa1.hgst.iphmx.com ([68.232.141.245]:61184 "EHLO esa1.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235113AbhDKG23 (ORCPT ); Sun, 11 Apr 2021 02:28:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122493; x=1649658493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rL8WCMpfxX9Dd32VH5TCQxuY4q7r9qOQGHdD3tJcJ7E=; b=GfzmTOhKuwchbzvaT77tunnvCDwjWIxd+na4uEIkfH76dIzCupA+DY6H +5g3FSBfaprra57ta2VyiS/9BNqzhud+TTUMIScPpbUQzatfaviQ9PtoV zp8mUSpgrtKlg2EtXLdLF5Z2v/xEv2bA4TDsM7v61B6dZ4qFIu1c6oT7K 5QWSG34dsW+tVJu5XLgw3s66ZIbAnpanoFStCOaCEeHU+oLcHQmVFIW3Z tcQygsz0bF+Raau50tx1r9a9ri5qxa1cbTp2DzAOqCJdinf694CCHKcJd q+rce8lFbc3P40R0/5ahjgf6w1uZossTUcI9+gdghyyOQgfPZcZL13sJw w==; IronPort-SDR: wdJnJb7G7+ZmMdxS3kxAIYgNAO2c4kW0kw6JQM4HFxLXj7/HqRzmJSCf2nik5NytRzMn+Mx40M 5/wfl4/QvocfG+cNxzK1wFJsK5JLEbIzHwpZeD8seqcoh9J9rQxyEFfhDFT4PMCwSc3y5Yu/hT khELmw1USZA1gygRBoxZHKlsqYCTsiDpScZB4O1NzKv+tYsxy8FGUVnnHkcQVsIbgs7LhtxhFF 7qrGCqCokZVq/aHBHkdKdXgpAjluHD+eSSywSZannqlnLzCoMOoZwX5i9P5qXus8SUqWBrxBNe 3EY= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="275295976" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:13 +0800 IronPort-SDR: JPZxlPkiE8bzIIzie/PnX0tZrWHaP6iM3Jgx+twr4KSCYJak8Pz5Pm7h+XgWomz2FKiLAkf+ih FlUiAR6IrSeQqJ4iJYyO5/T8ButJUMiQQ1lundTtwVTlcmeJvjZHJusju7SYvQMZb9Qz2x/ewd w0PESxIcjWK9ov+K8Jy0ZJPWfrBqsfr3ieepaf/pXoBmNCmEjjqJ+5uJ3cvqRFRaHkvcW1raWT pEDUUNTU0aqCakZ6DXEWOkQ/MUi2Epl/pseywa+FmqlBNFRJgi73j8k2M2zQ7UWg7ImhqdsE/S F8QVpBHeHZDfucON1aFKfS3m Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:07:46 -0700 IronPort-SDR: xed44BdbYBi1XXCHm4TyctZ3QqjFs/gmP4OovjgWyMo9sGvQcMECbEwGVWoin4dD69iIe+LKHG MoXgWFRa7e9UWRQiW3fLxKYMPyTsIi/IPFE8w+a9nBDsb7ryyf1s48UvZMuf5DrgXoIYpj+TpM Wy3nVGOGQMbgU3MNuNqvjx38NNC+FXvFI+7FWBtwqimbSkAQMVLiglge2SWikOHwf3YEhWCI8C QgjycqnApqLpLZTVXn0w84miFlkQmCjXUhE+ov5HKRCAHaPi97/dfLwF3DLw3I+h/ibW5iIijb wnY= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:09 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 05/11] scsi: ufshpb: Make eviction depends on region's reads Date: Sun, 11 Apr 2021 09:27:15 +0300 Message-Id: <20210411062721.10099-6-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host mode, eviction is considered an extreme measure. verify that the entering region has enough reads, and the exiting region has much less reads. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 3ab66421dc00..aefb6dc160ee 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -17,6 +17,7 @@ #include "../sd.h" #define ACTIVATION_THRESHOLD 8 /* 8 IOs */ +#define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 5) /* 256 IOs */ /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -1057,6 +1058,13 @@ static struct ufshpb_region *ufshpb_victim_lru_info(struct ufshpb_lu *hpb) if (ufshpb_check_srgns_issue_state(hpb, rgn)) continue; + /* + * in host control mode, verify that the exiting region + * has less reads + */ + if (hpb->is_hcm && rgn->reads > (EVICTION_THRESHOLD >> 1)) + continue; + victim_rgn = rgn; break; } @@ -1229,7 +1237,7 @@ static int ufshpb_issue_map_req(struct ufshpb_lu *hpb, static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) { - struct ufshpb_region *victim_rgn; + struct ufshpb_region *victim_rgn = NULL; struct victim_select_info *lru_info = &hpb->lru_info; unsigned long flags; int ret = 0; @@ -1256,7 +1264,15 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) * It is okay to evict the least recently used region, * because the device could detect this region * by not issuing HPB_READ + * + * in host control mode, verify that the entering + * region has enough reads */ + if (hpb->is_hcm && rgn->reads < EVICTION_THRESHOLD) { + ret = -EACCES; + goto out; + } + victim_rgn = ufshpb_victim_lru_info(hpb); if (!victim_rgn) { dev_warn(&hpb->sdev_ufs_lu->sdev_dev, From patchwork Sun Apr 11 06:27:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26E1AC433ED for ; Sun, 11 Apr 2021 06:28:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 04C8E611C2 for ; Sun, 11 Apr 2021 06:28:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235176AbhDKG2i (ORCPT ); Sun, 11 Apr 2021 02:28:38 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:62885 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235173AbhDKG2g (ORCPT ); Sun, 11 Apr 2021 02:28:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122500; x=1649658500; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EHOf5wvKwq4BI5bS85fMXpHxwsYDRytYEOM+K6W4Tnw=; b=Hg+C7ZwIP0WbIPT9se/yfhVGvJLbtk6Qax8VBvmG9/GTYCtogglpxPpd gOlc2WRhHOWDqTEvTYRyBemlFRg9hG2Mm2VguO6PzsjzRVVsv4xwR/78l SkNmGdJfZ6K42yu3eSoObZImFKpnR0Qt/F4DQUOqHSuQkhBgaAG7vDAhB 4cBZdsCLC4lcUbGmztGk9Y2Qk8sqbbrYnTOj0q47MurOLK6hepoLAaLaB vcZWTZgQtLJP+wx07r3EyjwQouI/1V1hG1tj5Q7gZ5cc98j1KDOO4A3nd 7FaENQBpu0uTMqeAxHIJBiPQx4KNNEPXUK30T2qIVzWZ0mKSuY121dkVW w==; IronPort-SDR: EIBfb6nTLjZuinQKU3AIS9rtABjV3xlxalpR13U17ZW/f/40YCr320yU/WJDeBPtyQyZXEpOXI Yruj5qoTMdQfSRFSq2b1BgAZ9bngAnDcAdKfEhKpz2ZQgTJ2GiFmV0IfbfbYbi+Oa1+qMHpTck aW6Ijr7rux0oJ1p0TrjFIV0vX829YzKtMG6JT0uO9NIisEmsceNFG2xGW94DhgoBNEBcr+BqIw SmTT6JKiAeoKepUYo56eDQr8ppnGe4Lt6VsMnZ+X358LLDy3kt8cGOTzv9Y54/4YXPkKezqsF7 9A0= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="168904970" Received: from h199-255-45-15.hgst.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:19 +0800 IronPort-SDR: /Cp8zX2lowIz93gF2LmQ5qgbhpwnOv99ZYqZfeVm2PpWjukNkOwoGPQ+bmRkwp52RIruwyz0pG 9sgpdDoo7mThDyfwsqLbT/hw3Sm5MQHn7yjokogLQQ6S0kf5hRyt1AEmkhf3BA9HHOZU4rAoao hkfLmI9x15qwKMGnpOZoBlSxLkiil4zrtEsKDySm/dpKKB5bSmphlMw1YmaJXJfW4HTa3i+Obb al7AVII5YZHez+tXYSIyHzT1hLtc9qJ0fZGbCh/Q9ogC++XG1rhePyXl3Zqp+rnafLErM5yJVb kijL9UCyl8BY3gD8277+nAnB Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:07:52 -0700 IronPort-SDR: tAw6ICkfj9Mn96aaC5defGPIQrCIRShmCPWw063U66eZlzqdJ82RhlK6IqJQcbiDr1A/cLeKVp VTdDMXNQ6oMzB3imjEhJzsU3pgMdLSeh+Sg79PlICGJfTUpKRkbstEtF7M8Vaxc4KLbn3ISffh 3kIvCYcmJ8gm+33c9aNbLqDZC/kBFHmC3DChEA5EKlhbX0emGmHMrCwt4nw2knM48HvEs+jAjB nESRBAD4l/DhR21VHc7gmoq9uNBagiCYOA4JCPyeyadBul22l7flIn60ImSaUpSrZDBfnPnPRe vyw= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:15 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 06/11] scsi: ufshpb: Region inactivation in host mode Date: Sun, 11 Apr 2021 09:27:16 +0300 Message-Id: <20210411062721.10099-7-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host mode, the host is expected to send HPB-WRITE-BUFFER with buffer-id = 0x1 when it inactivates a region. Use the map-requests pool as there is no point in assigning a designated cache for umap-requests. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 46 +++++++++++++++++++++++++++++++++------ drivers/scsi/ufs/ufshpb.h | 1 + 2 files changed, 40 insertions(+), 7 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index aefb6dc160ee..28aef5acffa2 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -691,7 +691,8 @@ int ufshpb_prep(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) } static struct ufshpb_req *ufshpb_get_req(struct ufshpb_lu *hpb, - int rgn_idx, enum req_opf dir) + int rgn_idx, enum req_opf dir, + bool atomic) { struct ufshpb_req *rq; struct request *req; @@ -705,7 +706,7 @@ static struct ufshpb_req *ufshpb_get_req(struct ufshpb_lu *hpb, req = blk_get_request(hpb->sdev_ufs_lu->request_queue, dir, BLK_MQ_REQ_NOWAIT); - if ((PTR_ERR(req) == -EWOULDBLOCK) && (--retries > 0)) { + if (!atomic && (PTR_ERR(req) == -EWOULDBLOCK) && (--retries > 0)) { usleep_range(3000, 3100); goto retry; } @@ -736,7 +737,7 @@ static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, struct ufshpb_req *map_req; struct bio *bio; - map_req = ufshpb_get_req(hpb, srgn->rgn_idx, REQ_OP_SCSI_IN); + map_req = ufshpb_get_req(hpb, srgn->rgn_idx, REQ_OP_SCSI_IN, false); if (!map_req) return NULL; @@ -914,6 +915,7 @@ static int ufshpb_execute_umap_req(struct ufshpb_lu *hpb, blk_execute_rq_nowait(NULL, req, 1, ufshpb_umap_req_compl_fn); + hpb->stats.umap_req_cnt++; return 0; } @@ -1091,12 +1093,13 @@ static void ufshpb_purge_active_subregion(struct ufshpb_lu *hpb, } static int ufshpb_issue_umap_req(struct ufshpb_lu *hpb, - struct ufshpb_region *rgn) + struct ufshpb_region *rgn, + bool atomic) { struct ufshpb_req *umap_req; int rgn_idx = rgn ? rgn->rgn_idx : 0; - umap_req = ufshpb_get_req(hpb, rgn_idx, REQ_OP_SCSI_OUT); + umap_req = ufshpb_get_req(hpb, rgn_idx, REQ_OP_SCSI_OUT, atomic); if (!umap_req) return -ENOMEM; @@ -1110,13 +1113,19 @@ static int ufshpb_issue_umap_req(struct ufshpb_lu *hpb, return -EAGAIN; } +static int ufshpb_issue_umap_single_req(struct ufshpb_lu *hpb, + struct ufshpb_region *rgn) +{ + return ufshpb_issue_umap_req(hpb, rgn, true); +} + static int ufshpb_issue_umap_all_req(struct ufshpb_lu *hpb) { - return ufshpb_issue_umap_req(hpb, NULL); + return ufshpb_issue_umap_req(hpb, NULL, false); } static void __ufshpb_evict_region(struct ufshpb_lu *hpb, - struct ufshpb_region *rgn) + struct ufshpb_region *rgn) { struct victim_select_info *lru_info; struct ufshpb_subregion *srgn; @@ -1151,6 +1160,14 @@ static int ufshpb_evict_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) goto out; } + if (hpb->is_hcm) { + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + ret = ufshpb_issue_umap_single_req(hpb, rgn); + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + if (ret) + goto out; + } + __ufshpb_evict_region(hpb, rgn); } out: @@ -1285,6 +1302,18 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) "LRU full (%d), choose victim %d\n", atomic_read(&lru_info->active_cnt), victim_rgn->rgn_idx); + + if (hpb->is_hcm) { + spin_unlock_irqrestore(&hpb->rgn_state_lock, + flags); + ret = ufshpb_issue_umap_single_req(hpb, + victim_rgn); + spin_lock_irqsave(&hpb->rgn_state_lock, + flags); + if (ret) + goto out; + } + __ufshpb_evict_region(hpb, victim_rgn); } @@ -1856,6 +1885,7 @@ ufshpb_sysfs_attr_show_func(rb_noti_cnt); ufshpb_sysfs_attr_show_func(rb_active_cnt); ufshpb_sysfs_attr_show_func(rb_inactive_cnt); ufshpb_sysfs_attr_show_func(map_req_cnt); +ufshpb_sysfs_attr_show_func(umap_req_cnt); static struct attribute *hpb_dev_stat_attrs[] = { &dev_attr_hit_cnt.attr, @@ -1864,6 +1894,7 @@ static struct attribute *hpb_dev_stat_attrs[] = { &dev_attr_rb_active_cnt.attr, &dev_attr_rb_inactive_cnt.attr, &dev_attr_map_req_cnt.attr, + &dev_attr_umap_req_cnt.attr, NULL, }; @@ -1988,6 +2019,7 @@ static void ufshpb_stat_init(struct ufshpb_lu *hpb) hpb->stats.rb_active_cnt = 0; hpb->stats.rb_inactive_cnt = 0; hpb->stats.map_req_cnt = 0; + hpb->stats.umap_req_cnt = 0; } static void ufshpb_param_init(struct ufshpb_lu *hpb) diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 87495e59fcf1..1ea58c17a4de 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -191,6 +191,7 @@ struct ufshpb_stats { u64 rb_inactive_cnt; u64 map_req_cnt; u64 pre_req_cnt; + u64 umap_req_cnt; }; struct ufshpb_lu { From patchwork Sun Apr 11 06:27:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5659C433ED for ; Sun, 11 Apr 2021 06:28:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B71D66101D for ; Sun, 11 Apr 2021 06:28:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235017AbhDKG2m (ORCPT ); Sun, 11 Apr 2021 02:28:42 -0400 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:11067 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235173AbhDKG2l (ORCPT ); Sun, 11 Apr 2021 02:28:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122506; x=1649658506; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cn80eFxaOEOocvOvD/vJ/p5jRgT1O6ba7LzTf9VBSqg=; b=HdHzIRuVEjdVMPbMv3HcBinKuUOpVkHRsDYBwSAsXAgBLD4CmexqQ6y5 7zrDVcL9RjaZ6UjYCfSGAaT/cx9+Xgz9FMmMoXyUMD3kFKhiMdC5z1V7g l1bmj4M5LzIiQDWCwuV3GKsuh1iYy84nuOsfA8G9gn1Psyk79oGY8uWD8 PuiItOVPYhvLDjoZXQtSQRKW2YHdrBNT9aNhYNOODKPUYyyiscU9Ixhl3 AeR0rkUUoNQo2d+3IG9LzzSDOBT/rNOrSNS94rLrRR4cDg+EgX4/q2GIK KlURgbdHfm63DxrIaKvMzJsxGj0srNzt8/8LqLqBTplcWHwPq+FGcMO8u w==; IronPort-SDR: Fh01ICdvHn3+ff2M6OCJyXzugHxRZEXiJznNu7OXYqoaXxJok6zmmTLMhPJ/s3S73g6htGk1jm IhgwkUPw7B07bK1hkWtzg5GFCzxKwOJwX4CcYh9xSU9GlByyCjMRvMuUv++9s18Tt8EaUYx4Rj 5ap/yMSAf1uHpdRPXu0AyYeqxJfdnY6fTGAKAItudUwkWB/kayYmPg14VE1qxcTi5zHaGJgASR lIjY3MXcq0frNt9CHJh/f19l+uzFvRP6phIeadFdMEP35xzPJe68FYSMXFj8wFshA7jkWJSAKb OEo= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="164405563" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:25 +0800 IronPort-SDR: oxC3BI1RzVJyFsMWgbl/UjDOHcRsgZk2j/1hS5FqhPxImKzFNrDBYpiFlQU4unqpv9Z+D7xIXv ITGALGeA03dZT4A+tf2Hj6+4l6p9gXCmOWXci2RM3idJtMuiQ9/dVHSELj5DyV4Nyv3bJx4aef UvlpV504Z8NB0o60cJyFVV5GD3XBUYLFTpirKDdB9tuq/QWSaFzldtmkez5hVxJvMEMpAzh8Fh ZtjbJiHRefjn6bbRrSIADGnbIYJzW2Rx7HVlQhFzlIX2vlHOPNyN9p/HtoNMLJsXUU8zTtDHCG iKy0pCy1lMpMEwCSWMol2w/O Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:07:58 -0700 IronPort-SDR: AGxaAL8z/2J1Dv57aOg9cF9eU9iMYCBddHIx6pKRRewFFC7QpnBfXruC5TIEDg+Hf0XeHzLrjV Kv1+duySmppkKF2OLdJ8eyv//3oVHwaMpZsmmOKphnAh8PC0IqSvTMxe7SonaRw1Ku4mt6jtcE JxSV346gFhYfURSYniPkeGRx9oQJpYzUiPOdw+afFlyLJ/fWXOmo4uNXuwlfkTgRcwAKtcaAaa S4xiviisADLe2gUVKNPbpr/2nCzFfpyVoy4t4ZxVRD0WzVjAPMOPEQRqFve04OdD8fxQM77L45 tIU= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:21 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 07/11] scsi: ufshpb: Add hpb dev reset response Date: Sun, 11 Apr 2021 09:27:17 +0300 Message-Id: <20210411062721.10099-8-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The spec does not define what is the host's recommended response when the device send hpb dev reset response (oper 0x2). We will update all active hpb regions: mark them and do that on the next read. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 32 +++++++++++++++++++++++++++++++- drivers/scsi/ufs/ufshpb.h | 1 + 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 28aef5acffa2..0352d269c1e9 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -195,7 +195,8 @@ static void ufshpb_iterate_rgn(struct ufshpb_lu *hpb, int rgn_idx, int srgn_idx, } spin_unlock(&rgn->rgn_lock); - if (activate) { + if (activate || + test_and_clear_bit(RGN_FLAG_UPDATE, &rgn->rgn_flags)) { spin_lock_irqsave(&hpb->rsp_list_lock, flags); ufshpb_update_active_info(hpb, rgn_idx, srgn_idx); spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); @@ -1417,6 +1418,20 @@ static void ufshpb_rsp_req_region_update(struct ufshpb_lu *hpb, queue_work(ufshpb_wq, &hpb->map_work); } +static void ufshpb_dev_reset_handler(struct ufshpb_lu *hpb) +{ + struct victim_select_info *lru_info = &hpb->lru_info; + struct ufshpb_region *rgn; + unsigned long flags; + + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + + list_for_each_entry(rgn, &lru_info->lh_lru_rgn, list_lru_rgn) + set_bit(RGN_FLAG_UPDATE, &rgn->rgn_flags); + + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); +} + /* * This function will parse recommended active subregion information in sense * data field of response UPIU with SAM_STAT_GOOD state. @@ -1491,6 +1506,18 @@ void ufshpb_rsp_upiu(struct ufs_hba *hba, struct ufshcd_lrb *lrbp) case HPB_RSP_DEV_RESET: dev_warn(&hpb->sdev_ufs_lu->sdev_dev, "UFS device lost HPB information during PM.\n"); + + if (hpb->is_hcm) { + struct scsi_device *sdev; + + __shost_for_each_device(sdev, hba->host) { + struct ufshpb_lu *h = sdev->hostdata; + + if (h) + ufshpb_dev_reset_handler(h); + } + } + break; default: dev_notice(&hpb->sdev_ufs_lu->sdev_dev, @@ -1817,6 +1844,8 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) } else { rgn->rgn_state = HPB_RGN_INACTIVE; } + + rgn->rgn_flags = 0; } return 0; @@ -2144,6 +2173,7 @@ static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { if (hpb->is_hcm) cancel_work_sync(&hpb->ufshpb_normalization_work); + cancel_work_sync(&hpb->map_work); } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 1ea58c17a4de..b863540e28d6 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -127,6 +127,7 @@ struct ufshpb_region { struct list_head list_lru_rgn; unsigned long rgn_flags; #define RGN_FLAG_DIRTY 0 +#define RGN_FLAG_UPDATE 1 /* region reads - for host mode */ spinlock_t rgn_lock; From patchwork Sun Apr 11 06:27:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19D78C433ED for ; Sun, 11 Apr 2021 06:28:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED0CA61207 for ; Sun, 11 Apr 2021 06:28:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235240AbhDKG3E (ORCPT ); Sun, 11 Apr 2021 02:29:04 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:46987 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235204AbhDKG3C (ORCPT ); Sun, 11 Apr 2021 02:29:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122539; x=1649658539; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yb0CxD121uLW/VisKVjoVf7arA1I6CNc7zJIZomsHOs=; b=d0EZmKBPeMPXb96p/McurkiBhHKZ3Kx4RnaNptcDIpVnSQ6uNzkuAAAO 8ceejh73V47f3m3aR67oMN4JIMguU1/HqjOP2e5+0etJWTSbr3sIUpbfg K81msKMVtzUt2zkpM3/iUu3XEg8uaKTBWMeOrQuBKO38AeR9+0drkFVdM la2UK+tpdZyPLRkvguKBLOW8Q0Vsnp2JIXKcq7w5MdG/m2xefJWgL0MxO zCJcDSjC0KmTSIu9ioI9zqPx+VN38TT5yGKhyQ8Z0LyoK8eAlsX1ETAfM 0UG7j99Z7RbQwYAzDf2BEpowi81ejEdk2wFAis/B6Uzbk4X7VDYYn7pXb Q==; IronPort-SDR: 2VtQ2BJ7r7Xvqdk53JihhXBVGSBVbygs9HWAKX1cf3cvu1eOnTh8Fnjg1lIg4peX931Q/sD8jw pLokuGx6TOYHSrViIxTCOpJu+etdnuW/ogKCTZOd5fHHKpnKqFs0HkLAFZYcJOFQjE3gsJsbmn 7uKPnilTEwTIUPcen5INSr+g31yyn+VzUNk1j3abGGElh9CIIcpN56CWykuSylWSvAx8eII/kl kj4fpjSS7myeE5Z7M3oVbPbUh//eqyk3foJOTCGhnKzcUafOzM+vKnwQ1v+fDEx3egUYPL2pBP qgE= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="268668894" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:43 +0800 IronPort-SDR: C6ND4TJw+QYKHOdsRkshfo0u2xyQPodxF4R1M3k+F/xa53btrByLfWXmorGNdsGzoXf6Au0pQb iaY+ljROkbchiXLpjiaKgTrKxKTAacHJkWTryOmMxvYKaQLKbIlUWILhYljmFlZx3fMxKhOfXK X7V6m6GoL1UbFazHnHVT1hCZfPEsnUXae3daQXtmXtTQevHxLwa0UVgRjiuUNLZHJ2+zNHW9m1 xsXtzATG6Pt+7nbxs9ukVcekJMJxizKw0QZUcigcHOLQI1nvr/hJmZIeYoopnrBbsPAqlmckai JmQPjCpZe0lCtmQ4ZYdmgwR8 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:09:38 -0700 IronPort-SDR: Iz1IElV0YZbLenZBnt5wu3HvYKHWF3M/quXChWHkuMJnT+i92VgrfCcz/EXTcB456fHU1ytBRe B9/r5Meggjc4L30YOAuTa/Qd9Hff0G2jpWeXd6YV9sRvhSHLmj8tMIuTKq0ujnAQwYVvjzgxdH poV3STe38xmCq8PoKCs+eoh1++0qK3adLbJJljMU1uLJlbw30hXHTf0cdd05fpA0fEo6zM/piV HpommYF8p2iB+DC0nB4ag18w8bXQcMM8Io8iqJ2UN2T0Dc0GkehZZtGUF49x2yFrc817+5opx1 68M= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:27 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 08/11] scsi: ufshpb: Add "Cold" regions timer Date: Sun, 11 Apr 2021 09:27:18 +0300 Message-Id: <20210411062721.10099-9-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In order not to hang on to “cold” regions, we shall inactivate a region that has no READ access for a predefined amount of time - READ_TO_MS. For that purpose we shall monitor the active regions list, polling it on every POLLING_INTERVAL_MS. On timeout expiry we shall add the region to the "to-be-inactivated" list, unless it is clean and did not exhaust its READ_TO_EXPIRIES - another parameter. All this does not apply to pinned regions. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 74 +++++++++++++++++++++++++++++++++++++-- drivers/scsi/ufs/ufshpb.h | 8 +++++ 2 files changed, 79 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 0352d269c1e9..8d067cf72ae2 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -18,6 +18,9 @@ #define ACTIVATION_THRESHOLD 8 /* 8 IOs */ #define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 5) /* 256 IOs */ +#define READ_TO_MS 1000 +#define READ_TO_EXPIRIES 100 +#define POLLING_INTERVAL_MS 200 /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -1032,12 +1035,63 @@ static int ufshpb_check_srgns_issue_state(struct ufshpb_lu *hpb, return 0; } +static void ufshpb_read_to_handler(struct work_struct *work) +{ + struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, + ufshpb_read_to_work.work); + struct victim_select_info *lru_info = &hpb->lru_info; + struct ufshpb_region *rgn, *next_rgn; + unsigned long flags; + LIST_HEAD(expired_list); + + if (test_and_set_bit(TIMEOUT_WORK_RUNNING, &hpb->work_data_bits)) + return; + + spin_lock_irqsave(&hpb->rgn_state_lock, flags); + + list_for_each_entry_safe(rgn, next_rgn, &lru_info->lh_lru_rgn, + list_lru_rgn) { + bool timedout = ktime_after(ktime_get(), rgn->read_timeout); + + if (timedout) { + rgn->read_timeout_expiries--; + if (is_rgn_dirty(rgn) || + rgn->read_timeout_expiries == 0) + list_add(&rgn->list_expired_rgn, &expired_list); + else + rgn->read_timeout = ktime_add_ms(ktime_get(), + READ_TO_MS); + } + } + + spin_unlock_irqrestore(&hpb->rgn_state_lock, flags); + + list_for_each_entry_safe(rgn, next_rgn, &expired_list, + list_expired_rgn) { + list_del_init(&rgn->list_expired_rgn); + spin_lock_irqsave(&hpb->rsp_list_lock, flags); + ufshpb_update_inactive_info(hpb, rgn->rgn_idx); + spin_unlock_irqrestore(&hpb->rsp_list_lock, flags); + } + + ufshpb_kick_map_work(hpb); + + clear_bit(TIMEOUT_WORK_RUNNING, &hpb->work_data_bits); + + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); +} + static void ufshpb_add_lru_info(struct victim_select_info *lru_info, struct ufshpb_region *rgn) { rgn->rgn_state = HPB_RGN_ACTIVE; list_add_tail(&rgn->list_lru_rgn, &lru_info->lh_lru_rgn); atomic_inc(&lru_info->active_cnt); + if (rgn->hpb->is_hcm) { + rgn->read_timeout = ktime_add_ms(ktime_get(), READ_TO_MS); + rgn->read_timeout_expiries = READ_TO_EXPIRIES; + } } static void ufshpb_hit_lru_info(struct victim_select_info *lru_info, @@ -1825,6 +1879,7 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&rgn->list_inact_rgn); INIT_LIST_HEAD(&rgn->list_lru_rgn); + INIT_LIST_HEAD(&rgn->list_expired_rgn); if (rgn_idx == hpb->rgns_per_lu - 1) { srgn_cnt = ((hpb->srgns_per_lu - 1) % @@ -1846,6 +1901,7 @@ static int ufshpb_alloc_region_tbl(struct ufs_hba *hba, struct ufshpb_lu *hpb) } rgn->rgn_flags = 0; + rgn->hpb = hpb; } return 0; @@ -2069,9 +2125,12 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) INIT_LIST_HEAD(&hpb->list_hpb_lu); INIT_WORK(&hpb->map_work, ufshpb_map_work_handler); - if (hpb->is_hcm) + if (hpb->is_hcm) { INIT_WORK(&hpb->ufshpb_normalization_work, ufshpb_normalization_work_handler); + INIT_DELAYED_WORK(&hpb->ufshpb_read_to_work, + ufshpb_read_to_handler); + } hpb->map_req_cache = kmem_cache_create("ufshpb_req_cache", sizeof(struct ufshpb_req), 0, 0, NULL); @@ -2105,6 +2164,10 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_stat_init(hpb); ufshpb_param_init(hpb); + if (hpb->is_hcm) + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); + return 0; release_pre_req_mempool: @@ -2171,9 +2234,10 @@ static void ufshpb_discard_rsp_lists(struct ufshpb_lu *hpb) static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb) { - if (hpb->is_hcm) + if (hpb->is_hcm) { + cancel_delayed_work_sync(&hpb->ufshpb_read_to_work); cancel_work_sync(&hpb->ufshpb_normalization_work); - + } cancel_work_sync(&hpb->map_work); } @@ -2281,6 +2345,10 @@ void ufshpb_resume(struct ufs_hba *hba) continue; ufshpb_set_state(hpb, HPB_PRESENT); ufshpb_kick_map_work(hpb); + if (hpb->is_hcm) + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(POLLING_INTERVAL_MS)); + } } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index b863540e28d6..448062a94760 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -115,6 +115,7 @@ struct ufshpb_subregion { }; struct ufshpb_region { + struct ufshpb_lu *hpb; struct ufshpb_subregion *srgn_tbl; enum HPB_RGN_STATE rgn_state; int rgn_idx; @@ -132,6 +133,10 @@ struct ufshpb_region { /* region reads - for host mode */ spinlock_t rgn_lock; unsigned int reads; + /* region "cold" timer - for host mode */ + ktime_t read_timeout; + unsigned int read_timeout_expiries; + struct list_head list_expired_rgn; }; #define for_each_sub_region(rgn, i, srgn) \ @@ -223,6 +228,9 @@ struct ufshpb_lu { /* for selecting victim */ struct victim_select_info lru_info; struct work_struct ufshpb_normalization_work; + struct delayed_work ufshpb_read_to_work; + unsigned long work_data_bits; +#define TIMEOUT_WORK_RUNNING 0 /* pinned region information */ u32 lu_pinned_start; From patchwork Sun Apr 11 06:27:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC2C0C433ED for ; Sun, 11 Apr 2021 06:29:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 973A7611C9 for ; Sun, 11 Apr 2021 06:29:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235247AbhDKG3W (ORCPT ); Sun, 11 Apr 2021 02:29:22 -0400 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:37489 "EHLO esa6.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235229AbhDKG3V (ORCPT ); Sun, 11 Apr 2021 02:29:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122547; x=1649658547; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0UZeo9TQDrA2E3f2IoxL7Zq6OFWwZ5enXMf0r9iUqKU=; b=XY4zmnfdB32t5IN7C6DGGj7IDmheNrtbW5LTyOvb6pmXpxg4we6WYJew EfNCT7VQZDsoRlIRAiuxnC+324kL93RYbAVoeE2uNUkP7BtjfWU8U6O7U K4ruWSK9jIQmeRaoRun/k+au7Ts2YM99mv7BbwCFNGOp2ghLo7J7KMRV0 x11nqD/s5VamwmbLWRZxXZMVLQm3dAzSh3a2W/qBDcLxlQhuiwHcv/nKS t1whqxSUNjTkrG8wWGH29Q8Q3qjDr/XBJP0DZEk7iSndez6kbKW9F66bl gqLjaMoenyXAwwaf4pUVMnA7IT7yC53uZLHKFsobVGyITSOueQf91Mh9L A==; IronPort-SDR: yo6bRKRoh4Emw2794dO0LwmoPU7vJeabWFCZ0NeyBR44j3rxkFm4+Ul+2+mNOH3Vko45B2W6Nh 2G4oeU51SamZxGqhGnwPQJIqzMIgiA43MeYjBCUrdlPKLaPCfX2IxhcuEd5ylUzGzpJywa0dZr I8hnkA3AJlxbhOS4q1qtMid6MjubQ6NYJjB77P6pMqdmSFs7d2kRdvY66ZHV7rDTOiP2WPs0wO olKIx2kL7GbWUeTtDY+rrGsX/VnlsJWIb2Ww4SvRHMXdxDC6RQ+lvx4nyoWCvFpzH1Eo3fiRLH 3mA= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="165243171" Received: from uls-op-cesaip02.wdc.com (HELO uls-op-cesaep02.wdc.com) ([199.255.45.15]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:47 +0800 IronPort-SDR: zFv/BTLGgbekTnYs7OTD6NfpjO1d63yv2FwUSEiCFphZoKlU/TRmkZZSLf4p4YYXXluOhO2lj2 aoJzjJYRufvHO4ZdiRdeaEosxcHsQ0HYlMBAWfMImOsAjMMO8aERQidTQpmW6GNI6zduiOBKD6 kSUkwtVNcNq+kclIq4Uu1dKW0a1KFrfPkU8x8j24WzzMHLx1CAWYNNQwOWIv9ihKIq5dBDbt4Y +yMqLgxHDVNVmsCBqA/l1zXciBVifpg7l2WhYXMqIVJBhjXi+8CutK1sMjkkGh989g6R11iqq0 tU6CW8v0cBKPH6+0fVkgHjq3 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep02.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:08:10 -0700 IronPort-SDR: C8yhFzzFapXYqTAGcifNdWCaiEZeCJ7fNJdmU+t43i3zNUYT8iPGTkj4uhxjKpHJNYgVr0Cd2/ ul85c6p1TcL4Io+DNSHXEXRmD8UrSJYXPS0WRVoxdLEpm0RHcAcQH8Nd9dEIqk6SOHsdwVQ2yv xxaJjoHTNaGkhC6BHBSgGBL2oKW3q9VYOwR7pTMjgmlLhoq7VNLmNF3X1RZWa2ZqtewduMKlE6 CXm++bH12Mn/z3yIJCZSisx79Lzt4CvLw1Qjdbo19LJSOJJup24OeEC2PIN5eLrbjWBajBXHqd 2ZE= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:33 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 09/11] scsi: ufshpb: Limit the number of inflight map requests Date: Sun, 11 Apr 2021 09:27:19 +0300 Message-Id: <20210411062721.10099-10-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In host control mode the host is the originator of map requests. To not flood the device with map requests, use a simple throttling mechanism that limits the number of inflight map requests. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 11 +++++++++++ drivers/scsi/ufs/ufshpb.h | 1 + 2 files changed, 12 insertions(+) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 8d067cf72ae2..0f72ea3f5d71 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -21,6 +21,7 @@ #define READ_TO_MS 1000 #define READ_TO_EXPIRIES 100 #define POLLING_INTERVAL_MS 200 +#define THROTTLE_MAP_REQ_DEFAULT 1 /* memory management */ static struct kmem_cache *ufshpb_mctx_cache; @@ -741,6 +742,14 @@ static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, struct ufshpb_req *map_req; struct bio *bio; + if (hpb->is_hcm && + hpb->num_inflight_map_req >= THROTTLE_MAP_REQ_DEFAULT) { + dev_info(&hpb->sdev_ufs_lu->sdev_dev, + "map_req throttle. inflight %d throttle %d", + hpb->num_inflight_map_req, THROTTLE_MAP_REQ_DEFAULT); + return NULL; + } + map_req = ufshpb_get_req(hpb, srgn->rgn_idx, REQ_OP_SCSI_IN, false); if (!map_req) return NULL; @@ -755,6 +764,7 @@ static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, map_req->rb.srgn_idx = srgn->srgn_idx; map_req->rb.mctx = srgn->mctx; + hpb->num_inflight_map_req++; return map_req; } @@ -764,6 +774,7 @@ static void ufshpb_put_map_req(struct ufshpb_lu *hpb, { bio_put(map_req->bio); ufshpb_put_req(hpb, map_req); + hpb->num_inflight_map_req--; } static int ufshpb_clear_dirty_bitmap(struct ufshpb_lu *hpb, diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index 448062a94760..cfa0abac21db 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -217,6 +217,7 @@ struct ufshpb_lu { struct ufshpb_req *pre_req; int num_inflight_pre_req; int throttle_pre_req; + int num_inflight_map_req; struct list_head lh_pre_req_free; int cur_read_id; int pre_req_min_tr_len; From patchwork Sun Apr 11 06:27:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2FD5C433ED for ; Sun, 11 Apr 2021 06:28:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE5D761026 for ; Sun, 11 Apr 2021 06:28:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235230AbhDKG27 (ORCPT ); Sun, 11 Apr 2021 02:28:59 -0400 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:11090 "EHLO esa5.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235223AbhDKG26 (ORCPT ); Sun, 11 Apr 2021 02:28:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122523; x=1649658523; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ao9KGLtiOtOXMs3JVdpAkduZXjxNqr5ZEmzsn/ujCvI=; b=k3rwPB+JprapyB1aIMYbGjD+/jdcj47QrCJi87NZq7sEeUx31zkURg4D aKimkzcZQc9mP3iNIXeVxUR9MFGcHb1KTsRmlbVI/BBtCj4WNHgONp0l4 9YiR8oDSk86+74hW5fb2zhP5CPs/y2be+bzrduReC3svxA/q78hm+Y+uw ekjMLIuk28P/fug/G5JdPpH9k0CjtxvhcpBn9cCs/rGv4h54p+DwqLJtQ 0RZvbxE/wTT0ysvQQbRpRLbwps6JRU5yI+7CRQmdor2jeQIpiFFU+VGVx sVRvXLfIQ4TPGASZ9FMffGVMSOwvbFH2jGqMVbzHINXkiHYyW0mr27ol7 w==; IronPort-SDR: t/xg9quJFQ3C1mVEotuXyfSrMh9alcw1jWSheluQ/oQ17INjhiMtK/S9thtsmLbb4KfzAGlv+s 1qYJhS+wxX85yL5i2xJU7y+lZ5Qqpd765E4doOQYh4xeYiI48lkgQzPOcQMGfRn8Ug/Gfy1Sgp 3Iu+BpnoCYrsDa1ZC6rb+Y5J1KMCoqZAE4/ySEjEZbWN7CZh0kN5oEDptrtrDXD1MgwdImlsPU u8spNAicqcygmbP6OumL6HgQNe59n/O7RJZLpMcxrKQP7Zway8eibH4OComIygtFfhj89Wesis 4cw= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="164405572" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:42 +0800 IronPort-SDR: yMJI9sM3IbxphBiym3u9uDW55Lny7xsrQiUdhebx7BIuKwLiHEWwjcTuJ14LQPCQ4isAgflzk6 vmVGouq/am8hZYt1B4VdmO41+D3e2qQeDxDofg+B4TcWLo+3GuO5VsMPt/FR0JRt4tw/ijSYir sc3qDDzoIXsUCRSRp2lQtmdL40LWZa1lAqj1K4ose+WeivY60y1okCxlZbn9hmTq/uyZhrU32p ZWl7Ay6WO5pNv8xh2m9pot/7nXxMIUdXfBXZ7+nrvIZ/bT7CLq5473sam+R3lIIXAxBQKZmkis 5fRaCNv1aDtlOCbdBXvY9pgx Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:09:50 -0700 IronPort-SDR: 87XWyqQF05GCLMz+d9rSV+5/ATvtE3UjFpwFtsNreeqaZSV6wAPRu9JIstVCd7fQYCznk9496I 4BK71sO///uleBL1kKLsvNMjAZHOCwgsASxB2FGoKKM8j/XOQ060N0pPSRvseVsdkU4NXZ/nMA Z8u9ZChuZec0+DJ6qE92CIaB0rYw/f30RSFronryxxSaHTZ2obuph+oX2/6IXq853tRM6oGwBz 9VvcyFRrbP8sCiiAZXLGFK3FUGsQLm4I95qbYOLd0eleo5lSz72HNaVgCKIFIpmQxw/aPBADDv DhI= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:38 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 10/11] scsi: ufshpb: Add support for host control mode Date: Sun, 11 Apr 2021 09:27:20 +0300 Message-Id: <20210411062721.10099-11-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Support devices that report they are using host control mode. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- drivers/scsi/ufs/ufshpb.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 0f72ea3f5d71..3c4001486cf1 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -2587,12 +2587,6 @@ void ufshpb_get_dev_info(struct ufs_hba *hba, u8 *desc_buf) u32 max_hpb_single_cmd = HPB_MULTI_CHUNK_LOW; hpb_dev_info->control_mode = desc_buf[DEVICE_DESC_PARAM_HPB_CONTROL]; - if (hpb_dev_info->control_mode == HPB_HOST_CONTROL) { - dev_err(hba->dev, "%s: host control mode is not supported.\n", - __func__); - hpb_dev_info->hpb_disabled = true; - return; - } version = get_unaligned_be16(desc_buf + DEVICE_DESC_PARAM_HPB_VER); if ((version != HPB_SUPPORT_VERSION) && From patchwork Sun Apr 11 06:27:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Avri Altman X-Patchwork-Id: 12196039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FD0CC433ED for ; Sun, 11 Apr 2021 06:28:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B4CF611C2 for ; Sun, 11 Apr 2021 06:28:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235255AbhDKG3G (ORCPT ); Sun, 11 Apr 2021 02:29:06 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:62921 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235229AbhDKG3E (ORCPT ); Sun, 11 Apr 2021 02:29:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1618122529; x=1649658529; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=N2Vouw2HbP8wkXO1V4kWeoPnrRgg/r94xtKRdSa9PaA=; b=hXOliZzWyZyRoQS7d7j1VstJ9qjw9AwUm2EmhBdhEZ0IWyFt+4cQAMMq c4c0E5b/AVFfzEItsG+OsrjuSZsMEFd3JaX2IFyK7k76IpEbFXuGnJ/iv sEXizVKY3/E8+feEjs76M8Ck1Bg7wkdWNfh8TIedQLi2C/sVIfe7T65Jn e/82t6oIcb7vTotMkY6950fO/h0lkiU6Qg9ZUD2VturffgTsRLQwmWlmC cR0uBJ93R/4cbxJdoNXYRtKUNPzQ5jLb9kWLK6v0F9d5gSwezj+JhTqOX Y+YHdgKbs8w/RCHOZFf3nv++/RwIu7bkHCPbqjmyRFqb65pPcu/vMvhHi w==; IronPort-SDR: 3vc2zIieGgFsYrTzfXV+KSPCBxT6j7GeyAKdFp1ai3aSGaSuU1rDFPU1Mkp013ZwjDAgMNZYFi PgRLMTULmZh0fRKLzivohQkaYNBmvGSZHM4MZvn0Dn81PEJ+DIweXXC0Y/Xv48EfN4KebC/Ef9 zqcW9WRnDJPa8Rlw8lfNrRg2+DeYuozve2qzMF6mpOC5HAwrgwdl56Gh3kAzg662Ri2Epi+Yid 7SBKgolJ3hzv61jALsaAmCnTeWqzHpAJbYnlep/gxnje/PmVAqsCcSaMQPeH1MvQ/fvRKszu77 M2o= X-IronPort-AV: E=Sophos;i="5.82,213,1613404800"; d="scan'208";a="168904986" Received: from h199-255-45-14.hgst.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 11 Apr 2021 14:28:48 +0800 IronPort-SDR: XGhEStwW2h5oq+xC4csWCTJEEJrlONGTpzkksLMdwg8/bdI2COnciIGD1bus/jSVz0swKAii6O b/1gF7KgFeKYlCqKVf0hIv5W/HvWWEAqs5M/tzXmzXYvEjZXh2QLT9S6nH5E4uIuxP1mcYsmVA W7KUixM2vwg7Vx2FAFCD4pf+z69CfAxpmXF4J4acoWCweS9ptUgo9AeVFQXmAKc5agwUwH3bF2 TX1ujpYY1DcWurlYtc78cYOr1Y82D8ekZ/yVAjJXmY9lxXpzbu32VSwPsbd7XeC3d2w/I7ViP8 GfxhNyD+f9JpiNFUJklI6mUq Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Apr 2021 23:09:56 -0700 IronPort-SDR: myCEweg95SD362k6i323zgAq0c6vLYYW7U3bfSYLP7JibjaX1OTjp26I/LTTA1/Foh424PB+zx 41Lq5AvlRu8bbGMJv+bYfa4/VyZAezLQvnl9fFIlFnGvWybnFOHuNCAot5h+qiT6OoUXMClNZA K1DbeHumBWCpnzNWdAg98uMzfpP0cvSrZlf6QxGtmPntL11UBO7cgZdSVy7sig94gXTeutEkNm hvpgEthhCK/Nr01M5y4H1gWYKChCgmGEk8H1qgYCiSxI1TpxAUauToAcC8GzD1Jifw284A9PmY dEw= WDCIronportException: Internal Received: from bxygm33.sdcorp.global.sandisk.com ([10.0.231.247]) by uls-op-cesaip02.wdc.com with ESMTP; 10 Apr 2021 23:28:44 -0700 From: Avri Altman To: "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org Cc: gregkh@linuxfoundation.org, Bart Van Assche , yongmyung lee , Daejun Park , alim.akhtar@samsung.com, asutoshd@codeaurora.org, Zang Leigang , Avi Shchislowski , Bean Huo , cang@codeaurora.org, stanley.chu@mediatek.com, Avri Altman Subject: [PATCH v8 11/11] scsi: ufshpb: Make host mode parameters configurable Date: Sun, 11 Apr 2021 09:27:21 +0300 Message-Id: <20210411062721.10099-12-avri.altman@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210411062721.10099-1-avri.altman@wdc.com> References: <20210411062721.10099-1-avri.altman@wdc.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org We can make use of this commit, to elaborate some more of the host control mode logic, explaining what role play each and every variable. While at it, allow those parameters to be configurable. Signed-off-by: Avri Altman Reviewed-by: Daejun Park --- Documentation/ABI/testing/sysfs-driver-ufs | 84 +++++- drivers/scsi/ufs/ufshpb.c | 288 +++++++++++++++++++-- drivers/scsi/ufs/ufshpb.h | 20 ++ 3 files changed, 365 insertions(+), 27 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-driver-ufs b/Documentation/ABI/testing/sysfs-driver-ufs index 419adf450b89..133af2114165 100644 --- a/Documentation/ABI/testing/sysfs-driver-ufs +++ b/Documentation/ABI/testing/sysfs-driver-ufs @@ -1323,14 +1323,76 @@ Description: This entry shows the maximum HPB data size for using single HPB The file is read only. -What: /sys/bus/platform/drivers/ufshcd/*/flags/wb_enable -Date: March 2021 -Contact: Daejun Park -Description: This entry shows the status of HPB. - - == ============================ - 0 HPB is not enabled. - 1 HPB is enabled - == ============================ - - The file is read only. +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/activation_thld +Date: February 2021 +Contact: Avri Altman +Description: In host control mode, reads are the major source of activation + trials. once this threshold hs met, the region is added to the + "to-be-activated" list. Since we reset the read counter upon + write, this include sending a rb command updating the region + ppn as well. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/normalization_factor +Date: February 2021 +Contact: Avri Altman +Description: In host control mode, We think of the regions as "buckets". + Those buckets are being filled with reads, and emptied on write. + We use entries_per_srgn - the amount of blocks in a subregion as + our bucket size. This applies because HPB1.0 only concern a + single-block reads. Once the bucket size is crossed, we trigger + a normalization work - not only to avoid overflow, but mainly + because we want to keep those counters normalized, as we are + using those reads as a comparative score, to make various decisions. + The normalization is dividing (shift right) the read counter by + the normalization_factor. If during consecutive normalizations + an active region has exhaust its reads - inactivate it. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_enter +Date: February 2021 +Contact: Avri Altman +Description: Region deactivation is often due to the fact that eviction took + place: a region become active on the expense of another. This is + happening when the max-active-regions limit has crossed. + In host mode, eviction is considered an extreme measure. We + want to verify that the entering region has enough reads, and + the exiting region has much less reads. eviction_thld_enter is + the min reads that a region must have in order to be considered + as a candidate to evict other region. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/eviction_thld_exit +Date: February 2021 +Contact: Avri Altman +Description: same as above for the exiting region. A region is consider to + be a candidate to be evicted, only if it has less reads than + eviction_thld_exit. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_ms +Date: February 2021 +Contact: Avri Altman +Description: In order not to hang on to “cold” regions, we shall inactivate + a region that has no READ access for a predefined amount of + time - read_timeout_ms. If read_timeout_ms has expired, and the + region is dirty - it is less likely that we can make any use of + HPB-READing it. So we inactivate it. Still, deactivation has + its overhead, and we may still benefit from HPB-READing this + region if it is clean - see read_timeout_expiries. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/read_timeout_expiries +Date: February 2021 +Contact: Avri Altman +Description: if the region read timeout has expired, but the region is clean, + just re-wind its timer for another spin. Do that as long as it + is clean and did not exhaust its read_timeout_expiries threshold. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/timeout_polling_interval_ms +Date: February 2021 +Contact: Avri Altman +Description: the frequency in which the delayed worker that checks the + read_timeouts is awaken. + +What: /sys/class/scsi_device/*/device/hpb_param_sysfs/inflight_map_req +Date: February 2021 +Contact: Avri Altman +Description: in host control mode the host is the originator of map requests. + To not flood the device with map requests, use a simple throttling + mechanism that limits the number of inflight map requests. diff --git a/drivers/scsi/ufs/ufshpb.c b/drivers/scsi/ufs/ufshpb.c index 3c4001486cf1..d22f3ad8fc32 100644 --- a/drivers/scsi/ufs/ufshpb.c +++ b/drivers/scsi/ufs/ufshpb.c @@ -17,7 +17,6 @@ #include "../sd.h" #define ACTIVATION_THRESHOLD 8 /* 8 IOs */ -#define EVICTION_THRESHOLD (ACTIVATION_THRESHOLD << 5) /* 256 IOs */ #define READ_TO_MS 1000 #define READ_TO_EXPIRIES 100 #define POLLING_INTERVAL_MS 200 @@ -194,7 +193,7 @@ static void ufshpb_iterate_rgn(struct ufshpb_lu *hpb, int rgn_idx, int srgn_idx, } else { srgn->reads++; rgn->reads++; - if (srgn->reads == ACTIVATION_THRESHOLD) + if (srgn->reads == hpb->params.activation_thld) activate = true; } spin_unlock(&rgn->rgn_lock); @@ -743,10 +742,11 @@ static struct ufshpb_req *ufshpb_get_map_req(struct ufshpb_lu *hpb, struct bio *bio; if (hpb->is_hcm && - hpb->num_inflight_map_req >= THROTTLE_MAP_REQ_DEFAULT) { + hpb->num_inflight_map_req >= hpb->params.inflight_map_req) { dev_info(&hpb->sdev_ufs_lu->sdev_dev, "map_req throttle. inflight %d throttle %d", - hpb->num_inflight_map_req, THROTTLE_MAP_REQ_DEFAULT); + hpb->num_inflight_map_req, + hpb->params.inflight_map_req); return NULL; } @@ -1053,6 +1053,7 @@ static void ufshpb_read_to_handler(struct work_struct *work) struct victim_select_info *lru_info = &hpb->lru_info; struct ufshpb_region *rgn, *next_rgn; unsigned long flags; + unsigned int poll; LIST_HEAD(expired_list); if (test_and_set_bit(TIMEOUT_WORK_RUNNING, &hpb->work_data_bits)) @@ -1071,7 +1072,7 @@ static void ufshpb_read_to_handler(struct work_struct *work) list_add(&rgn->list_expired_rgn, &expired_list); else rgn->read_timeout = ktime_add_ms(ktime_get(), - READ_TO_MS); + hpb->params.read_timeout_ms); } } @@ -1089,8 +1090,9 @@ static void ufshpb_read_to_handler(struct work_struct *work) clear_bit(TIMEOUT_WORK_RUNNING, &hpb->work_data_bits); + poll = hpb->params.timeout_polling_interval_ms; schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + msecs_to_jiffies(poll)); } static void ufshpb_add_lru_info(struct victim_select_info *lru_info, @@ -1100,8 +1102,11 @@ static void ufshpb_add_lru_info(struct victim_select_info *lru_info, list_add_tail(&rgn->list_lru_rgn, &lru_info->lh_lru_rgn); atomic_inc(&lru_info->active_cnt); if (rgn->hpb->is_hcm) { - rgn->read_timeout = ktime_add_ms(ktime_get(), READ_TO_MS); - rgn->read_timeout_expiries = READ_TO_EXPIRIES; + rgn->read_timeout = + ktime_add_ms(ktime_get(), + rgn->hpb->params.read_timeout_ms); + rgn->read_timeout_expiries = + rgn->hpb->params.read_timeout_expiries; } } @@ -1130,7 +1135,8 @@ static struct ufshpb_region *ufshpb_victim_lru_info(struct ufshpb_lu *hpb) * in host control mode, verify that the exiting region * has less reads */ - if (hpb->is_hcm && rgn->reads > (EVICTION_THRESHOLD >> 1)) + if (hpb->is_hcm && + rgn->reads > hpb->params.eviction_thld_exit) continue; victim_rgn = rgn; @@ -1351,7 +1357,8 @@ static int ufshpb_add_region(struct ufshpb_lu *hpb, struct ufshpb_region *rgn) * in host control mode, verify that the entering * region has enough reads */ - if (hpb->is_hcm && rgn->reads < EVICTION_THRESHOLD) { + if (hpb->is_hcm && + rgn->reads < hpb->params.eviction_thld_enter) { ret = -EACCES; goto out; } @@ -1702,6 +1709,7 @@ static void ufshpb_normalization_work_handler(struct work_struct *work) struct ufshpb_lu *hpb = container_of(work, struct ufshpb_lu, ufshpb_normalization_work); int rgn_idx; + u8 factor = hpb->params.normalization_factor; for (rgn_idx = 0; rgn_idx < hpb->rgns_per_lu; rgn_idx++) { struct ufshpb_region *rgn = hpb->rgn_tbl + rgn_idx; @@ -1712,7 +1720,7 @@ static void ufshpb_normalization_work_handler(struct work_struct *work) for (srgn_idx = 0; srgn_idx < hpb->srgns_per_rgn; srgn_idx++) { struct ufshpb_subregion *srgn = rgn->srgn_tbl + srgn_idx; - srgn->reads >>= 1; + srgn->reads >>= factor; rgn->reads += srgn->reads; } spin_unlock(&rgn->rgn_lock); @@ -2036,8 +2044,247 @@ requeue_timeout_ms_store(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RW(requeue_timeout_ms); +ufshpb_sysfs_param_show_func(activation_thld); +static ssize_t +activation_thld_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.activation_thld = val; + + return count; +} +static DEVICE_ATTR_RW(activation_thld); + +ufshpb_sysfs_param_show_func(normalization_factor); +static ssize_t +normalization_factor_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0 || val > ilog2(hpb->entries_per_srgn)) + return -EINVAL; + + hpb->params.normalization_factor = val; + + return count; +} +static DEVICE_ATTR_RW(normalization_factor); + +ufshpb_sysfs_param_show_func(eviction_thld_enter); +static ssize_t +eviction_thld_enter_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= hpb->params.eviction_thld_exit) + return -EINVAL; + + hpb->params.eviction_thld_enter = val; + + return count; +} +static DEVICE_ATTR_RW(eviction_thld_enter); + +ufshpb_sysfs_param_show_func(eviction_thld_exit); +static ssize_t +eviction_thld_exit_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= hpb->params.activation_thld) + return -EINVAL; + + hpb->params.eviction_thld_exit = val; + + return count; +} +static DEVICE_ATTR_RW(eviction_thld_exit); + +ufshpb_sysfs_param_show_func(read_timeout_ms); +static ssize_t +read_timeout_ms_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + /* read_timeout >> timeout_polling_interval */ + if (val < hpb->params.timeout_polling_interval_ms * 2) + return -EINVAL; + + hpb->params.read_timeout_ms = val; + + return count; +} +static DEVICE_ATTR_RW(read_timeout_ms); + +ufshpb_sysfs_param_show_func(read_timeout_expiries); +static ssize_t +read_timeout_expiries_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0) + return -EINVAL; + + hpb->params.read_timeout_expiries = val; + + return count; +} +static DEVICE_ATTR_RW(read_timeout_expiries); + +ufshpb_sysfs_param_show_func(timeout_polling_interval_ms); +static ssize_t +timeout_polling_interval_ms_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + /* timeout_polling_interval << read_timeout */ + if (val <= 0 || val > hpb->params.read_timeout_ms / 2) + return -EINVAL; + + hpb->params.timeout_polling_interval_ms = val; + + return count; +} +static DEVICE_ATTR_RW(timeout_polling_interval_ms); + +ufshpb_sysfs_param_show_func(inflight_map_req); +static ssize_t inflight_map_req_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufshpb_lu *hpb = ufshpb_get_hpb_data(sdev); + int val; + + if (!hpb) + return -ENODEV; + + if (!hpb->is_hcm) + return -EOPNOTSUPP; + + if (kstrtouint(buf, 0, &val)) + return -EINVAL; + + if (val <= 0 || val > hpb->sdev_ufs_lu->queue_depth - 1) + return -EINVAL; + + hpb->params.inflight_map_req = val; + + return count; +} +static DEVICE_ATTR_RW(inflight_map_req); + +static void ufshpb_hcm_param_init(struct ufshpb_lu *hpb) +{ + hpb->params.activation_thld = ACTIVATION_THRESHOLD; + hpb->params.normalization_factor = 1; + hpb->params.eviction_thld_enter = (ACTIVATION_THRESHOLD << 5); + hpb->params.eviction_thld_exit = (ACTIVATION_THRESHOLD << 4); + hpb->params.read_timeout_ms = READ_TO_MS; + hpb->params.read_timeout_expiries = READ_TO_EXPIRIES; + hpb->params.timeout_polling_interval_ms = POLLING_INTERVAL_MS; + hpb->params.inflight_map_req = THROTTLE_MAP_REQ_DEFAULT; +} + static struct attribute *hpb_dev_param_attrs[] = { &dev_attr_requeue_timeout_ms.attr, + &dev_attr_activation_thld.attr, + &dev_attr_normalization_factor.attr, + &dev_attr_eviction_thld_enter.attr, + &dev_attr_eviction_thld_exit.attr, + &dev_attr_read_timeout_ms.attr, + &dev_attr_read_timeout_expiries.attr, + &dev_attr_timeout_polling_interval_ms.attr, + &dev_attr_inflight_map_req.attr, NULL, }; @@ -2121,6 +2368,8 @@ static void ufshpb_stat_init(struct ufshpb_lu *hpb) static void ufshpb_param_init(struct ufshpb_lu *hpb) { hpb->params.requeue_timeout_ms = HPB_REQUEUE_TIME_MS; + if (hpb->is_hcm) + ufshpb_hcm_param_init(hpb); } static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) @@ -2175,9 +2424,13 @@ static int ufshpb_lu_hpb_init(struct ufs_hba *hba, struct ufshpb_lu *hpb) ufshpb_stat_init(hpb); ufshpb_param_init(hpb); - if (hpb->is_hcm) + if (hpb->is_hcm) { + unsigned int poll; + + poll = hpb->params.timeout_polling_interval_ms; schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + msecs_to_jiffies(poll)); + } return 0; @@ -2356,10 +2609,13 @@ void ufshpb_resume(struct ufs_hba *hba) continue; ufshpb_set_state(hpb, HPB_PRESENT); ufshpb_kick_map_work(hpb); - if (hpb->is_hcm) - schedule_delayed_work(&hpb->ufshpb_read_to_work, - msecs_to_jiffies(POLLING_INTERVAL_MS)); + if (hpb->is_hcm) { + unsigned int poll = + hpb->params.timeout_polling_interval_ms; + schedule_delayed_work(&hpb->ufshpb_read_to_work, + msecs_to_jiffies(poll)); + } } } diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufshpb.h index cfa0abac21db..68a5af0ff682 100644 --- a/drivers/scsi/ufs/ufshpb.h +++ b/drivers/scsi/ufs/ufshpb.h @@ -185,8 +185,28 @@ struct victim_select_info { atomic_t active_cnt; }; +/** + * ufshpb_params - ufs hpb parameters + * @requeue_timeout_ms - requeue threshold of wb command (0x2) + * @activation_thld - min reads [IOs] to activate/update a region + * @normalization_factor - shift right the region's reads + * @eviction_thld_enter - min reads [IOs] for the entering region in eviction + * @eviction_thld_exit - max reads [IOs] for the exiting region in eviction + * @read_timeout_ms - timeout [ms] from the last read IO to the region + * @read_timeout_expiries - amount of allowable timeout expireis + * @timeout_polling_interval_ms - frequency in which timeouts are checked + * @inflight_map_req - number of inflight map requests + */ struct ufshpb_params { unsigned int requeue_timeout_ms; + unsigned int activation_thld; + unsigned int normalization_factor; + unsigned int eviction_thld_enter; + unsigned int eviction_thld_exit; + unsigned int read_timeout_ms; + unsigned int read_timeout_expiries; + unsigned int timeout_polling_interval_ms; + unsigned int inflight_map_req; }; struct ufshpb_stats {