From patchwork Mon Nov 18 10:31:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11249297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16BAB14C0 for ; Mon, 18 Nov 2019 10:31:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EB9BF20855 for ; Mon, 18 Nov 2019 10:31:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NHh2PD3C" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726706AbfKRKbp (ORCPT ); Mon, 18 Nov 2019 05:31:45 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:23797 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726518AbfKRKbo (ORCPT ); Mon, 18 Nov 2019 05:31:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574073102; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QBUQpEvhUSGPyW0TNCDWONDd8thoA0ntji5C32j1HOM=; b=NHh2PD3CxryF93hocFfzs/VRO6KJRMJTq5aGqD5tbK7iRfuiheI7fJLiirZqjZgHLyfoja OBsP97Hfp/MhCZdnB5vNSKOTrbQlwQJ77ITxF41NWgJBtxLe6hu7nILCQ3EtVFaMgsihGa oO4cprw0jKegIdB/12K/5hjLIDT24Hs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-207-R0u2oME6PaG-zSQCmkh7eQ-1; Mon, 18 Nov 2019 05:31:39 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9C7AD1802CE0; Mon, 18 Nov 2019 10:31:36 +0000 (UTC) Received: from localhost (ovpn-8-23.pek2.redhat.com [10.72.8.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 013D4600CD; Mon, 18 Nov 2019 10:31:32 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, Ming Lei , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , Kashyap Desai , Sumit Saxena , Shivasharan S , "Ewan D . Milne" , Christoph Hellwig , Hannes Reinecke , Bart Van Assche Subject: [PATCH 1/4] scsi: megaraid_sas: use private counter for tracking inflight per-LUN commands Date: Mon, 18 Nov 2019 18:31:14 +0800 Message-Id: <20191118103117.978-2-ming.lei@redhat.com> In-Reply-To: <20191118103117.978-1-ming.lei@redhat.com> References: <20191118103117.978-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: R0u2oME6PaG-zSQCmkh7eQ-1 X-Mimecast-Spam-Score: 0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Prepare for bypassing sdev->device_busy for improving performance on fast SCSI storage device, so sdev->device_busy can't be relied any more. megaraid_sas may need one such counter for balancing load among LUNs in some specific setting, so add one private counter for this purpose. Cc: Sathya Prakash Cc: Chaitra P B Cc: Suganath Prabu Subramani Cc: Kashyap Desai Cc: Sumit Saxena Cc: Shivasharan S Cc: Ewan D. Milne Cc: Christoph Hellwig , Cc: Hannes Reinecke Cc: Bart Van Assche Signed-off-by: Ming Lei Reviewed-by: Hannes Reinecke --- drivers/scsi/megaraid/megaraid_sas.h | 1 + drivers/scsi/megaraid/megaraid_sas_base.c | 15 +++++++++++++-- drivers/scsi/megaraid/megaraid_sas_fusion.c | 13 +++++++++---- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/drivers/scsi/megaraid/megaraid_sas.h b/drivers/scsi/megaraid/megaraid_sas.h index a6e788c02ff4..f9562c02705b 100644 --- a/drivers/scsi/megaraid/megaraid_sas.h +++ b/drivers/scsi/megaraid/megaraid_sas.h @@ -2025,6 +2025,7 @@ struct MR_PRIV_DEVICE { u8 interface_type; u8 task_abort_tmo; u8 target_reset_tmo; + atomic_t active_cmds; }; struct megasas_cmd; diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c index 42cf38c1ea99..3afead3bc96e 100644 --- a/drivers/scsi/megaraid/megaraid_sas_base.c +++ b/drivers/scsi/megaraid/megaraid_sas_base.c @@ -1759,6 +1759,7 @@ megasas_queue_command(struct Scsi_Host *shost, struct scsi_cmnd *scmd) { struct megasas_instance *instance; struct MR_PRIV_DEVICE *mr_device_priv_data; + int ret; instance = (struct megasas_instance *) scmd->device->host->hostdata; @@ -1821,7 +1822,11 @@ megasas_queue_command(struct Scsi_Host *shost, struct scsi_cmnd *scmd) goto out_done; } - return instance->instancet->build_and_issue_cmd(instance, scmd); + atomic_inc(&mr_device_priv_data->active_cmds); + ret = instance->instancet->build_and_issue_cmd(instance, scmd); + if (ret) + atomic_dec(&mr_device_priv_data->active_cmds); + return ret; out_done: scmd->scsi_done(scmd); @@ -2100,6 +2105,7 @@ static int megasas_slave_alloc(struct scsi_device *sdev) atomic_set(&mr_device_priv_data->r1_ldio_hint, instance->r1_ldio_hint_default); + atomic_set(&mr_device_priv_data->active_cmds, 0); return 0; } @@ -3475,12 +3481,15 @@ megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd, unsigned long flags; struct fusion_context *fusion = instance->ctrl_context; u32 opcode, status; + struct MR_PRIV_DEVICE *mr_device_priv_data = NULL; /* flag for the retry reset */ cmd->retry_for_fw_reset = 0; - if (cmd->scmd) + if (cmd->scmd) { cmd->scmd->SCp.ptr = NULL; + mr_device_priv_data = cmd->scmd->device->hostdata; + } switch (hdr->cmd) { case MFI_CMD_INVALID: @@ -3519,6 +3528,7 @@ megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd, if (exception) { atomic_dec(&instance->fw_outstanding); + atomic_dec(&mr_device_priv_data->active_cmds); scsi_dma_unmap(cmd->scmd); cmd->scmd->scsi_done(cmd->scmd); @@ -3567,6 +3577,7 @@ megasas_complete_cmd(struct megasas_instance *instance, struct megasas_cmd *cmd, } atomic_dec(&instance->fw_outstanding); + atomic_dec(&mr_device_priv_data->active_cmds); scsi_dma_unmap(cmd->scmd); cmd->scmd->scsi_done(cmd->scmd); diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.c b/drivers/scsi/megaraid/megaraid_sas_fusion.c index e301458bcbae..10ed3bc3b643 100644 --- a/drivers/scsi/megaraid/megaraid_sas_fusion.c +++ b/drivers/scsi/megaraid/megaraid_sas_fusion.c @@ -2728,7 +2728,7 @@ megasas_build_ldio_fusion(struct megasas_instance *instance, u8 *raidLUN; unsigned long spinlock_flags; struct MR_LD_RAID *raid = NULL; - struct MR_PRIV_DEVICE *mrdev_priv; + struct MR_PRIV_DEVICE *mrdev_priv = scp->device->hostdata; struct RAID_CONTEXT *rctx; struct RAID_CONTEXT_G35 *rctx_g35; @@ -2826,7 +2826,7 @@ megasas_build_ldio_fusion(struct megasas_instance *instance, } if ((instance->perf_mode == MR_BALANCED_PERF_MODE) && - atomic_read(&scp->device->device_busy) > + atomic_read(&mrdev_priv->active_cmds) > (io_info.data_arms * MR_DEVICE_HIGH_IOPS_DEPTH)) cmd->request_desc->SCSIIO.MSIxIndex = mega_mod64((atomic64_add_return(1, &instance->high_iops_outstanding) / @@ -2849,7 +2849,6 @@ megasas_build_ldio_fusion(struct megasas_instance *instance, * with the SLD bit asserted. */ if (io_info.r1_alt_dev_handle != MR_DEVHANDLE_INVALID) { - mrdev_priv = scp->device->hostdata; if (atomic_inc_return(&instance->fw_outstanding) > (instance->host->can_queue)) { @@ -3159,7 +3158,7 @@ megasas_build_syspd_fusion(struct megasas_instance *instance, cmd->request_desc->SCSIIO.DevHandle = io_request->DevHandle; if ((instance->perf_mode == MR_BALANCED_PERF_MODE) && - atomic_read(&scmd->device->device_busy) > MR_DEVICE_HIGH_IOPS_DEPTH) + atomic_read(&mr_device_priv_data->active_cmds) > MR_DEVICE_HIGH_IOPS_DEPTH) cmd->request_desc->SCSIIO.MSIxIndex = mega_mod64((atomic64_add_return(1, &instance->high_iops_outstanding) / MR_HIGH_IOPS_BATCH_COUNT), instance->low_latency_index_start); @@ -3550,6 +3549,7 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex, while (d_val.u.low != cpu_to_le32(UINT_MAX) && d_val.u.high != cpu_to_le32(UINT_MAX)) { + struct MR_PRIV_DEVICE *mr_device_priv_data = NULL; smid = le16_to_cpu(reply_desc->SMID); cmd_fusion = fusion->cmd_list[smid - 1]; @@ -3585,6 +3585,8 @@ complete_cmd_fusion(struct megasas_instance *instance, u32 MSIxIndex, } /* Fall through - and complete IO */ case MEGASAS_MPI2_FUNCTION_LD_IO_REQUEST: /* LD-IO Path */ + mr_device_priv_data = scmd_local->device->hostdata; + atomic_dec(&mr_device_priv_data->active_cmds); atomic_dec(&instance->fw_outstanding); if (cmd_fusion->r1_alt_dev_handle == MR_DEVHANDLE_INVALID) { map_cmd_status(fusion, scmd_local, status, @@ -4865,6 +4867,7 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason) /* Now return commands back to the OS */ for (i = 0 ; i < instance->max_scsi_cmds; i++) { + struct MR_PRIV_DEVICE *mr_device_priv_data = NULL; cmd_fusion = fusion->cmd_list[i]; /*check for extra commands issued by driver*/ if (instance->adapter_type >= VENTURA_SERIES) { @@ -4893,6 +4896,8 @@ int megasas_reset_fusion(struct Scsi_Host *shost, int reason) megasas_return_cmd_fusion(instance, cmd_fusion); scsi_dma_unmap(scmd_local); scmd_local->scsi_done(scmd_local); + mr_device_priv_data = scmd_local->device->hostdata; + atomic_dec(&mr_device_priv_data->active_cmds); } } From patchwork Mon Nov 18 10:31:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11249299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD1E6930 for ; Mon, 18 Nov 2019 10:31:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B1BBA20730 for ; Mon, 18 Nov 2019 10:31:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XhdwF3Mv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726708AbfKRKbw (ORCPT ); Mon, 18 Nov 2019 05:31:52 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:50122 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726518AbfKRKbw (ORCPT ); Mon, 18 Nov 2019 05:31:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574073110; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kQGaSPSMYZf0wTy4mSjYgqZcEjcQZsd+fMlyCkP8oTA=; b=XhdwF3MvMaOJZcLcwwnWUCulR72OjbamI5440PD0DBqrCk09d1CB3gIFwlgBxpwYg6t6Y6 lBUeo3q/IVJQ8SYBAWI8huCWhQhpv9ltsApY4yD2wKRlWp2BifVMhKCyg3lW6hBdd1SNL9 ylsWmUV3Bueh90BtALcxsyFatCdLFCE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-322-YV0MO35nMzaGbPBoC5Cesw-1; Mon, 18 Nov 2019 05:31:46 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8CF5F801FD2; Mon, 18 Nov 2019 10:31:42 +0000 (UTC) Received: from localhost (ovpn-8-23.pek2.redhat.com [10.72.8.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id F37A519C6A; Mon, 18 Nov 2019 10:31:38 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, Ming Lei , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , Kashyap Desai , Sumit Saxena , Shivasharan S , "Ewan D . Milne" , Christoph Hellwig , Hannes Reinecke , Bart Van Assche Subject: [PATCH 2/4] scsi: mpt3sas: use private counter for tracking inflight per-LUN commands Date: Mon, 18 Nov 2019 18:31:15 +0800 Message-Id: <20191118103117.978-3-ming.lei@redhat.com> In-Reply-To: <20191118103117.978-1-ming.lei@redhat.com> References: <20191118103117.978-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: YV0MO35nMzaGbPBoC5Cesw-1 X-Mimecast-Spam-Score: 0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Prepare for bypassing sdev->device_busy for improving performance on fast SCSI storage device, so sdev->device_busy can't be relied any more. mpt3sas may need one such counter for balancing load among LUNs in some specific setting, so add one private counter for this purpose. Cc: Sathya Prakash Cc: Chaitra P B Cc: Suganath Prabu Subramani Cc: Kashyap Desai Cc: Sumit Saxena Cc: Shivasharan S Cc: Ewan D. Milne Cc: Christoph Hellwig , Cc: Hannes Reinecke Cc: Bart Van Assche Signed-off-by: Ming Lei Reviewed-by: Hannes Reinecke --- drivers/scsi/mpt3sas/mpt3sas_base.c | 3 ++- drivers/scsi/mpt3sas/mpt3sas_base.h | 1 + drivers/scsi/mpt3sas/mpt3sas_scsih.c | 15 +++++++++++++-- 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index fea3cb6a090b..9c2d374b3157 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -3480,12 +3480,13 @@ static inline u8 _base_get_high_iops_msix_index(struct MPT3SAS_ADAPTER *ioc, struct scsi_cmnd *scmd) { + struct MPT3SAS_DEVICE *sas_device_priv_data = scmd->device->hostdata; /** * Round robin the IO interrupts among the high iops * reply queues in terms of batch count 16 when outstanding * IOs on the target device is >=8. */ - if (atomic_read(&scmd->device->device_busy) > + if (atomic_read(&sas_device_priv_data->active_cmds) > MPT3SAS_DEVICE_HIGH_IOPS_DEPTH) return base_mod64(( atomic64_add_return(1, &ioc->high_iops_outstanding) / diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h b/drivers/scsi/mpt3sas/mpt3sas_base.h index faca0a5e71f8..9e9f319bb636 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.h +++ b/drivers/scsi/mpt3sas/mpt3sas_base.h @@ -467,6 +467,7 @@ struct MPT3SAS_DEVICE { * thing while a SATL command is pending. */ unsigned long ata_command_pending; + atomic_t active_cmds; }; diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c index c8e512ba6d39..194960dae1d6 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c +++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c @@ -1770,6 +1770,7 @@ scsih_slave_alloc(struct scsi_device *sdev) sas_device_priv_data->lun = sdev->lun; sas_device_priv_data->flags = MPT_DEVICE_FLAGS_INIT; + atomic_set(&sas_device_priv_data->active_cmds, 0); starget = scsi_target(sdev); sas_target_priv_data = starget->hostdata; @@ -2884,6 +2885,7 @@ scsih_abort(struct scsi_cmnd *scmd) ioc->remove_host) { sdev_printk(KERN_INFO, scmd->device, "device been deleted! scmd(%p)\n", scmd); + atomic_dec(&sas_device_priv_data->active_cmds); scmd->result = DID_NO_CONNECT << 16; scmd->scsi_done(scmd); r = SUCCESS; @@ -2993,7 +2995,7 @@ scsih_dev_reset(struct scsi_cmnd *scmd) MPI2_SCSITASKMGMT_TASKTYPE_LOGICAL_UNIT_RESET, 0, 0, tr_timeout, tr_method); /* Check for busy commands after reset */ - if (r == SUCCESS && atomic_read(&scmd->device->device_busy)) + if (r == SUCCESS && atomic_read(&sas_device_priv_data->active_cmds)) r = FAILED; out: sdev_printk(KERN_INFO, scmd->device, "device reset: %s scmd(%p)\n", @@ -4517,9 +4519,12 @@ _scsih_flush_running_cmds(struct MPT3SAS_ADAPTER *ioc) int count = 0; for (smid = 1; smid <= ioc->scsiio_depth; smid++) { + struct MPT3SAS_DEVICE *sas_device_priv_data; + scmd = mpt3sas_scsih_scsi_lookup_get(ioc, smid); if (!scmd) continue; + sas_device_priv_data = scmd->device->hostdata; count++; _scsih_set_satl_pending(scmd, false); st = scsi_cmd_priv(scmd); @@ -4529,6 +4534,7 @@ _scsih_flush_running_cmds(struct MPT3SAS_ADAPTER *ioc) scmd->result = DID_NO_CONNECT << 16; else scmd->result = DID_RESET << 16; + atomic_dec(&sas_device_priv_data->active_cmds); scmd->scsi_done(scmd); } dtmprintk(ioc, ioc_info(ioc, "completing %d cmds\n", count)); @@ -4756,11 +4762,14 @@ scsih_qcmd(struct Scsi_Host *shost, struct scsi_cmnd *scmd) mpi_request->LUN); memcpy(mpi_request->CDB.CDB32, scmd->cmnd, scmd->cmd_len); + atomic_inc(&sas_device_priv_data->active_cmds); + if (mpi_request->DataLength) { pcie_device = sas_target_priv_data->pcie_dev; if (ioc->build_sg_scmd(ioc, scmd, smid, pcie_device)) { mpt3sas_base_free_smid(ioc, smid); _scsih_set_satl_pending(scmd, false); + atomic_dec(&sas_device_priv_data->active_cmds); goto out; } } else @@ -5207,6 +5216,7 @@ _scsih_smart_predicted_fault(struct MPT3SAS_ADAPTER *ioc, u16 handle) static u8 _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) { + struct MPT3SAS_DEVICE *sas_device_priv_data; Mpi25SCSIIORequest_t *mpi_request; Mpi2SCSIIOReply_t *mpi_reply; struct scsi_cmnd *scmd; @@ -5216,7 +5226,6 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) u8 scsi_state; u8 scsi_status; u32 log_info; - struct MPT3SAS_DEVICE *sas_device_priv_data; u32 response_code = 0; mpi_reply = mpt3sas_base_get_reply_virt_addr(ioc, reply); @@ -5225,6 +5234,7 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) if (scmd == NULL) return 1; + sas_device_priv_data = scmd->device->hostdata; _scsih_set_satl_pending(scmd, false); mpi_request = mpt3sas_base_get_msg_frame(ioc, smid); @@ -5431,6 +5441,7 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) scsi_dma_unmap(scmd); mpt3sas_base_free_smid(ioc, smid); + atomic_dec(&sas_device_priv_data->active_cmds); scmd->scsi_done(scmd); return 0; } From patchwork Mon Nov 18 10:31:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11249303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19B88930 for ; Mon, 18 Nov 2019 10:31:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id ED5242075E for ; Mon, 18 Nov 2019 10:31:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JtiOg2iQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726716AbfKRKbz (ORCPT ); Mon, 18 Nov 2019 05:31:55 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:37308 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726518AbfKRKbz (ORCPT ); Mon, 18 Nov 2019 05:31:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574073114; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L14bP1v4AxvDNzpocYgioYdse5rLV822J0Dg6UrPweU=; b=JtiOg2iQvr3Owznbsqf962FZCaT7xPUQF3Mpu3/GuQuKwOvnHvg74y0cwDd52OhyH6SnZQ mzUSdCgkC7FfAaFG2IMwkHtaqdpxGVswsyiYmWmpC9lWIvzJMA9z/O3rdK2kM7wJtzNCNH FIBCVenQtfTri2tEcQQGdrvoWggQhKY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-48-z39Ibvi6N0Getux7GzBQLw-1; Mon, 18 Nov 2019 05:31:51 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0402E107B7C1; Mon, 18 Nov 2019 10:31:49 +0000 (UTC) Received: from localhost (ovpn-8-23.pek2.redhat.com [10.72.8.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2C8218783; Mon, 18 Nov 2019 10:31:44 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, Ming Lei , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , Kashyap Desai , Sumit Saxena , Shivasharan S , "Ewan D . Milne" , Christoph Hellwig , Hannes Reinecke , Bart Van Assche Subject: [PATCH 3/4] scsi: sd: register request queue after sd_revalidate_disk is done Date: Mon, 18 Nov 2019 18:31:16 +0800 Message-Id: <20191118103117.978-4-ming.lei@redhat.com> In-Reply-To: <20191118103117.978-1-ming.lei@redhat.com> References: <20191118103117.978-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-MC-Unique: z39Ibvi6N0Getux7GzBQLw-1 X-Mimecast-Spam-Score: 0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Prepare for improving SSD performance in the following patch, which needs to read queue flag of QUEUE_FLAG_NONROT in IO path. So we have to freeze queue before changing this flag in sd_revalidate_disk(). However, queue freezing becomes quite slow after the queue is registered because RCU period is involved. So delay registering queue after sd_revalidate_disk() is done for avoiding slow queue freezing which will be added to sd_revalidate_disk() in the following patch. Cc: Sathya Prakash Cc: Chaitra P B Cc: Suganath Prabu Subramani Cc: Kashyap Desai Cc: Sumit Saxena Cc: Shivasharan S Cc: Ewan D. Milne Cc: Christoph Hellwig , Cc: Hannes Reinecke Cc: Bart Van Assche Signed-off-by: Ming Lei Reviewed-by: Hannes Reinecke --- drivers/scsi/sd.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 03163ac5fe95..0744c34468e1 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -3368,12 +3368,14 @@ static int sd_probe(struct device *dev) } blk_pm_runtime_init(sdp->request_queue, dev); - device_add_disk(dev, gd, NULL); + device_add_disk_no_queue_reg(dev, gd); if (sdkp->capacity) sd_dif_config_host(sdkp); sd_revalidate_disk(gd); + blk_register_queue(gd); + if (sdkp->security) { sdkp->opal_dev = init_opal_dev(sdp, &sd_sec_submit); if (sdkp->opal_dev) From patchwork Mon Nov 18 10:31:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 11249309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E34A3184E for ; Mon, 18 Nov 2019 10:32:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C312320878 for ; Mon, 18 Nov 2019 10:32:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KUoHkbUv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726728AbfKRKcC (ORCPT ); Mon, 18 Nov 2019 05:32:02 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:38461 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726717AbfKRKcB (ORCPT ); Mon, 18 Nov 2019 05:32:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574073119; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=umM+99ErFcMwK7HA93tRo1cw8l8CdV7G6pVpKtihSRg=; b=KUoHkbUvjjHRJ0tuvNCnK7USxSwa2kwNk3x/sVkM8oMJtIIySSjSUDa3E1Z8rjGNTluHs4 tqTl4mlU7HiKayWplGrC516CY0tOr+iR6bGiT/ua56+z+hFuZRJSTfWAulYG7GpWxaVp80 1ZIWgUdsKam9G6H76DYNVMfkPz6z9x0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-107-D7VhB1bQN2u8zg82HQRrdg-1; Mon, 18 Nov 2019 05:31:56 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7DA511800D7D; Mon, 18 Nov 2019 10:31:54 +0000 (UTC) Received: from localhost (ovpn-8-23.pek2.redhat.com [10.72.8.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3913175E30; Mon, 18 Nov 2019 10:31:50 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, "James E . J . Bottomley" , "Martin K . Petersen" , linux-scsi@vger.kernel.org, Ming Lei , Sathya Prakash , Chaitra P B , Suganath Prabu Subramani , Kashyap Desai , Sumit Saxena , Shivasharan S , "Ewan D . Milne" , Christoph Hellwig , Hannes Reinecke , Bart Van Assche Subject: [PATCH 4/4] scsi: core: don't limit per-LUN queue depth for SSD Date: Mon, 18 Nov 2019 18:31:17 +0800 Message-Id: <20191118103117.978-5-ming.lei@redhat.com> In-Reply-To: <20191118103117.978-1-ming.lei@redhat.com> References: <20191118103117.978-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: D7VhB1bQN2u8zg82HQRrdg-1 X-Mimecast-Spam-Score: 0 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org SCSI core uses the atomic variable of sdev->device_busy to track in-flight IO requests dispatched to this scsi device. IO request may be submitted from any CPU, so the cost for maintaining the shared atomic counter can be very big on big NUMA machine with lots of CPU cores. sdev->queue_depth is usually used for two purposes: 1) improve IO merge; 2) fair IO request scattered among all LUNs. blk-mq already provides fair request allocation among all active shared request queues(LUNs), see hctx_may_queue(). NVMe doesn't have such per-request-queue(namespace) queue depth, so it is reasonable to ignore the limit for SCSI SSD too. Also IO merge won't play big role for reaching top SSD performance. With this patch, big cost for tracking in-flight per-LUN requests via atomic variable can be saved. Given QUEUE_FLAG_NONROT is read in IO path, we have to freeze queue before changing this flag. Cc: Sathya Prakash Cc: Chaitra P B Cc: Suganath Prabu Subramani Cc: Kashyap Desai Cc: Sumit Saxena Cc: Shivasharan S Cc: Ewan D. Milne Cc: Christoph Hellwig , Cc: Hannes Reinecke Cc: Bart Van Assche Signed-off-by: Ming Lei --- block/blk-sysfs.c | 14 +++++++++++++- drivers/scsi/scsi_lib.c | 24 ++++++++++++++++++------ drivers/scsi/sd.c | 4 ++++ 3 files changed, 35 insertions(+), 7 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index fca9b158f4a0..9cc0dd5f88a1 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -281,6 +281,18 @@ QUEUE_SYSFS_BIT_FNS(random, ADD_RANDOM, 0); QUEUE_SYSFS_BIT_FNS(iostats, IO_STAT, 0); #undef QUEUE_SYSFS_BIT_FNS +static ssize_t queue_store_nonrot_freeze(struct request_queue *q, + const char *page, size_t count) +{ + size_t ret; + + blk_mq_freeze_queue(q); + ret = queue_store_nonrot(q, page, count); + blk_mq_unfreeze_queue(q); + + return ret; +} + static ssize_t queue_zoned_show(struct request_queue *q, char *page) { switch (blk_queue_zoned_model(q)) { @@ -642,7 +654,7 @@ static struct queue_sysfs_entry queue_write_zeroes_max_entry = { static struct queue_sysfs_entry queue_nonrot_entry = { .attr = {.name = "rotational", .mode = 0644 }, .show = queue_show_nonrot, - .store = queue_store_nonrot, + .store = queue_store_nonrot_freeze, }; static struct queue_sysfs_entry queue_zoned_entry = { diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 62a86a82c38d..72655a049efd 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -354,7 +354,8 @@ void scsi_device_unbusy(struct scsi_device *sdev, struct scsi_cmnd *cmd) if (starget->can_queue > 0) atomic_dec(&starget->target_busy); - atomic_dec(&sdev->device_busy); + if (!blk_queue_nonrot(sdev->request_queue)) + atomic_dec(&sdev->device_busy); } static void scsi_kick_queue(struct request_queue *q) @@ -410,7 +411,8 @@ static void scsi_single_lun_run(struct scsi_device *current_sdev) static inline bool scsi_device_is_busy(struct scsi_device *sdev) { - if (atomic_read(&sdev->device_busy) >= sdev->queue_depth) + if (!blk_queue_nonrot(sdev->request_queue) && + atomic_read(&sdev->device_busy) >= sdev->queue_depth) return true; if (atomic_read(&sdev->device_blocked) > 0) return true; @@ -1283,8 +1285,12 @@ static inline int scsi_dev_queue_ready(struct request_queue *q, struct scsi_device *sdev) { unsigned int busy; + bool bypass = blk_queue_nonrot(sdev->request_queue); - busy = atomic_inc_return(&sdev->device_busy) - 1; + if (!bypass) + busy = atomic_inc_return(&sdev->device_busy) - 1; + else + busy = 0; if (atomic_read(&sdev->device_blocked)) { if (busy) goto out_dec; @@ -1298,12 +1304,16 @@ static inline int scsi_dev_queue_ready(struct request_queue *q, "unblocking device at zero depth\n")); } + if (bypass) + return 1; + if (busy >= sdev->queue_depth) goto out_dec; return 1; out_dec: - atomic_dec(&sdev->device_busy); + if (!bypass) + atomic_dec(&sdev->device_busy); return 0; } @@ -1624,7 +1634,8 @@ static void scsi_mq_put_budget(struct blk_mq_hw_ctx *hctx) struct request_queue *q = hctx->queue; struct scsi_device *sdev = q->queuedata; - atomic_dec(&sdev->device_busy); + if (!blk_queue_nonrot(sdev->request_queue)) + atomic_dec(&sdev->device_busy); } static bool scsi_mq_get_budget(struct blk_mq_hw_ctx *hctx) @@ -1731,7 +1742,8 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, case BLK_STS_OK: break; case BLK_STS_RESOURCE: - if (atomic_read(&sdev->device_busy) || + if ((!blk_queue_nonrot(sdev->request_queue) && + atomic_read(&sdev->device_busy)) || scsi_device_blocked(sdev)) ret = BLK_STS_DEV_RESOURCE; break; diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 0744c34468e1..c3d47117700d 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -2927,7 +2927,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp) rot = get_unaligned_be16(&buffer[4]); if (rot == 1) { + blk_mq_freeze_queue(q); blk_queue_flag_set(QUEUE_FLAG_NONROT, q); + blk_mq_unfreeze_queue(q); blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q); } @@ -3123,7 +3125,9 @@ static int sd_revalidate_disk(struct gendisk *disk) * cause this to be updated correctly and any device which * doesn't support it should be treated as rotational. */ + blk_mq_freeze_queue(q); blk_queue_flag_clear(QUEUE_FLAG_NONROT, q); + blk_mq_unfreeze_queue(q); blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q); if (scsi_device_supports_vpd(sdp)) {