From patchwork Fri Aug 4 15:48:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13342058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEF65C41513 for ; Fri, 4 Aug 2023 15:49:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232184AbjHDPtC (ORCPT ); Fri, 4 Aug 2023 11:49:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232176AbjHDPs6 (ORCPT ); Fri, 4 Aug 2023 11:48:58 -0400 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8351C3C3D; Fri, 4 Aug 2023 08:48:51 -0700 (PDT) Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-686b91c2744so1658841b3a.0; Fri, 04 Aug 2023 08:48:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691164131; x=1691768931; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KQtAnIArA9iV45Y4VwwqcwMGiJtLwJXQ9EsDGHUi6lM=; b=d2zmke1GtAqOm2rjGdpCs7xLj2MTKpDp7TECW3hWN2rbRX6BtV7iKHqpu5JkoW2uTE PFKCL9i01sFU7rp0lYMwTg2EtF/awXXmw1XZO8VHbMHej87trt2QVUNPwFj86q0sF0GC LLJJ2phe/CLGmY1qTfw2aDrtl3K0q991FlQCUAVY7htxvvnlcuDrFKJa4gVnPbWx0XjB y+aUMh3UCUN2JlHTXOnfDXJVmJot1uRGnMA0TRLHWy/0M6ro1WB1s7KyZl6MfjRr2FGz vrGBj1mivv9CJXxAgrZCaU200eJ3GJNJBLitTG7v6ZAX2fb510rsIgttgDEeyYOdhWSf Zc1A== X-Gm-Message-State: AOJu0Ywe+VOw5NA2rRYGwtAKblpEQ5rKnGwtW/KsRGWhgpyfJqZz6lmb nK5QUNSUroJ8RBzPuStPT4M= X-Google-Smtp-Source: AGHT+IE48FqADNLKlkekkHruc3Q1Lm15ReNNdc+DPvYTsEztXHmAxkjlOsp1OBR7lysU26fELmMznQ== X-Received: by 2002:a05:6a20:970e:b0:122:10f9:f635 with SMTP id hr14-20020a056a20970e00b0012210f9f635mr1925562pzc.19.1691164130831; Fri, 04 Aug 2023 08:48:50 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:4af5:d063:a66d:403b]) by smtp.gmail.com with ESMTPSA id h8-20020a62b408000000b00640ddad2e0dsm1760839pfn.47.2023.08.04.08.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 08:48:50 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Avri Altman , Damien Le Moal , Ming Lei , "James E.J. Bottomley" , Stanley Chu , Can Guo , Bean Huo , Asutosh Das , "Bao D. Nguyen" , Arthur Simchaev Subject: [PATCH v6 7/7] scsi: ufs: Disable zone write locking Date: Fri, 4 Aug 2023 08:48:05 -0700 Message-ID: <20230804154821.3232094-8-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.640.ga95def55d0-goog In-Reply-To: <20230804154821.3232094-1-bvanassche@acm.org> References: <20230804154821.3232094-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From the UFSHCI 4.0 specification, about the legacy (single queue) mode: "The host controller always process transfer requests in-order according to the order submitted to the list. In case of multiple commands with single doorbell register ringing (batch mode), The dispatch order for these transfer requests by host controller will base on their index in the List. A transfer request with lower index value will be executed before a transfer request with higher index value." From the UFSHCI 4.0 specification, about the MCQ mode: "Command Submission 1. Host SW writes an Entry to SQ 2. Host SW updates SQ doorbell tail pointer Command Processing 3. After fetching the Entry, Host Controller updates SQ doorbell head pointer 4. Host controller sends COMMAND UPIU to UFS device" In other words, for both legacy and MCQ mode, UFS controllers are required to forward commands to the UFS device in the order these commands have been received from the host. Notes: - For legacy mode this is only correct if the host submits one command at a time. The UFS driver does this. - Also in legacy mode, the command order is not preserved if auto-hibernation is enabled in the UFS controller. Hence, enable zone write locking if auto-hibernation is enabled. This patch improves performance as follows on my test setup: - With the mq-deadline scheduler: 2.5x more IOPS for small writes. - When not using an I/O scheduler compared to using mq-deadline with zone locking: 4x more IOPS for small writes. Cc: Martin K. Petersen Cc: Avri Altman Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Can Guo --- drivers/ufs/core/ufshcd.c | 40 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index ae7b868f9c26..3c99516c38fa 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -4337,23 +4337,53 @@ int ufshcd_uic_hibern8_exit(struct ufs_hba *hba) } EXPORT_SYMBOL_GPL(ufshcd_uic_hibern8_exit); +static void ufshcd_update_no_zone_write_lock(struct ufs_hba *hba, + bool set_no_zone_write_lock) +{ + struct scsi_device *sdev; + + shost_for_each_device(sdev, hba->host) + blk_freeze_queue_start(sdev->request_queue); + shost_for_each_device(sdev, hba->host) { + struct request_queue *q = sdev->request_queue; + + blk_mq_freeze_queue_wait(q); + if (set_no_zone_write_lock) + blk_queue_flag_set(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); + else + blk_queue_flag_clear(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); + blk_mq_unfreeze_queue(q); + } +} + void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit) { unsigned long flags; - bool update = false; + bool prev_state, new_state, update = false; if (!ufshcd_is_auto_hibern8_supported(hba)) return; spin_lock_irqsave(hba->host->host_lock, flags); + prev_state = ufshcd_is_auto_hibern8_enabled(hba); if (hba->ahit != ahit) { hba->ahit = ahit; update = true; } + new_state = ufshcd_is_auto_hibern8_enabled(hba); spin_unlock_irqrestore(hba->host->host_lock, flags); if (!update) return; + if (!is_mcq_enabled(hba) && !prev_state && new_state) { + /* + * Auto-hibernation will be enabled. Enable write locking for + * zoned writes since auto-hibernation may cause reordering of + * zoned writes when using the legacy mode of the UFS host + * controller. + */ + ufshcd_update_no_zone_write_lock(hba, false); + } if (!pm_runtime_suspended(&hba->ufs_device_wlun->sdev_gendev)) { ufshcd_rpm_get_sync(hba); ufshcd_hold(hba); @@ -4361,6 +4391,13 @@ void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit) ufshcd_release(hba); ufshcd_rpm_put_sync(hba); } + if (!is_mcq_enabled(hba) && prev_state && !new_state) { + /* + * Auto-hibernation has been disabled. Disable write locking + * for zoned writes. + */ + ufshcd_update_no_zone_write_lock(hba, true); + } } EXPORT_SYMBOL_GPL(ufshcd_auto_hibern8_update); @@ -5140,6 +5177,7 @@ static int ufshcd_slave_configure(struct scsi_device *sdev) ufshcd_hpb_configure(hba, sdev); + blk_queue_flag_set(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); blk_queue_update_dma_pad(q, PRDT_DATA_BYTE_COUNT_PAD - 1); if (hba->quirks & UFSHCD_QUIRK_4KB_DMA_ALIGNMENT) blk_queue_update_dma_alignment(q, SZ_4K - 1);