From patchwork Mon Oct 23 20:36:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13433498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F05DAC25B45 for ; Mon, 23 Oct 2023 20:36:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229853AbjJWUg5 (ORCPT ); Mon, 23 Oct 2023 16:36:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229512AbjJWUgw (ORCPT ); Mon, 23 Oct 2023 16:36:52 -0400 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83031D78; Mon, 23 Oct 2023 13:36:50 -0700 (PDT) Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-68fb85afef4so3086453b3a.1; Mon, 23 Oct 2023 13:36:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698093410; x=1698698210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s/5xet03ew9Pw9gn1HDVrR9Wequ2bFLVgw3o3jyYqZc=; b=HiKIE6YoYCVEi0uAyHezkJf67zIHaZR3BUEMOqURrAAUN/OcelwBElKStiF6iGz2Dz ot6g+6aIVNaecCcH372B27wKcYSLf03Y4o7iBTo5jFMqa7vaKRd3G+iK49qltrmRpfae 2WnTJguJIKh5CCzuKwcTgSKnuMpav7aoPbUXqCwWk5AnzK4ufUJQ8Q5b07neXmoLOWhZ OsAvnjkKqRB+vpmIAk4tCZmo0GgNB2xJ75DIe+L0jbDjyvGJ4KLuQByD7wzjbd3NDVgu 7Ob3BEZ6T/8Czq/7GxiF5yQHDY7Xjqg2eCYh0tUUJI/mpKPnekVUzxmuEdX0Py121G/L O/uw== X-Gm-Message-State: AOJu0YyPtPHGkajkGucpvJCV4jTLB0+YTNLdXMGkUwIFq9krbZRQbs5X 4g8Qn+JRGw4xx7pvLijwk4A= X-Google-Smtp-Source: AGHT+IGfTywYz4f7hTgPSzcr9zr9xBTXBy9oK25vNgKXe7U+OYCUZ1DoMhbIDX4X8u7Lep3r509uxA== X-Received: by 2002:a05:6a20:158b:b0:154:a1e3:f967 with SMTP id h11-20020a056a20158b00b00154a1e3f967mr674807pzj.47.1698093409835; Mon, 23 Oct 2023 13:36:49 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:14f9:170e:9304:1c4e]) by smtp.gmail.com with ESMTPSA id g29-20020aa79ddd000000b0068be4ce33easm5776025pfq.96.2023.10.23.13.36.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 13:36:49 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Ming Lei , Keith Busch , Damien Le Moal , Yu Kuai , Ed Tsai , Matthias Brugger Subject: [PATCH v4 1/3] block: Introduce flag BLK_MQ_F_DISABLE_FAIR_TAG_SHARING Date: Mon, 23 Oct 2023 13:36:33 -0700 Message-ID: <20231023203643.3209592-2-bvanassche@acm.org> X-Mailer: git-send-email 2.42.0.758.gaed0368e0e-goog In-Reply-To: <20231023203643.3209592-1-bvanassche@acm.org> References: <20231023203643.3209592-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The fair sharing algorithm has a negative performance impact for storage devices for which the full queue depth is required to reach peak performance, e.g. UFS devices. This is because it takes long after a request queue became inactive until tags are reassigned to the active request queue(s). Since making tag sharing fair is not needed if the request processing latency is similar for all request queues, introduce the hctx / tag set flag BLK_MQ_F_DISABLE_FAIR_TAG_SHARING. Cc: Cc: Christoph Hellwig Cc: Martin K. Petersen Cc: Ming Lei Cc: Keith Busch Cc: Damien Le Moal Cc: Yu Kuai Cc: Ed Tsai Signed-off-by: Bart Van Assche --- block/blk-mq-debugfs.c | 1 + block/blk-mq.h | 3 ++- include/linux/blk-mq.h | 1 + 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index 5cbeb9344f2f..f41408103106 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -198,6 +198,7 @@ static const char *const hctx_flag_name[] = { HCTX_FLAG_NAME(NO_SCHED), HCTX_FLAG_NAME(STACKING), HCTX_FLAG_NAME(TAG_HCTX_SHARED), + HCTX_FLAG_NAME(DISABLE_FAIR_TAG_SHARING), }; #undef HCTX_FLAG_NAME diff --git a/block/blk-mq.h b/block/blk-mq.h index f75a9ecfebde..eda6bd0611ea 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -416,7 +416,8 @@ static inline bool hctx_may_queue(struct blk_mq_hw_ctx *hctx, { unsigned int depth, users; - if (!hctx || !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED)) + if (!hctx || !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) || + (hctx->flags & BLK_MQ_F_DISABLE_FAIR_TAG_SHARING)) return true; /* diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 1ab3081c82ed..b12a0be839aa 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -662,6 +662,7 @@ enum { * or shared hwqs instead of 'mq-deadline'. */ BLK_MQ_F_NO_SCHED_BY_DEFAULT = 1 << 7, + BLK_MQ_F_DISABLE_FAIR_TAG_SHARING = 1 << 8, BLK_MQ_F_ALLOC_POLICY_START_BIT = 8, BLK_MQ_F_ALLOC_POLICY_BITS = 1, From patchwork Mon Oct 23 20:36:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13433499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73CD2C25B49 for ; Mon, 23 Oct 2023 20:36:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230080AbjJWUg6 (ORCPT ); Mon, 23 Oct 2023 16:36:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229798AbjJWUgy (ORCPT ); Mon, 23 Oct 2023 16:36:54 -0400 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 559FBD7A; Mon, 23 Oct 2023 13:36:52 -0700 (PDT) Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6b497c8575aso3730468b3a.1; Mon, 23 Oct 2023 13:36:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698093412; x=1698698212; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Rzazhz3pJNtMrLPLf65MSg+veUxWS3Czg4sZVKyrr/E=; b=wxRMDoyN71mw9CbLXb5kcucBvjy1UsZEJFPQQgrv/198/wLnuvzM2rhB0h61RNCBnI O4Jyy1hCgM/fRedaWPtzJR+0Xgz5//p2/9HpUi2fIDJr74EMqMMOJJq4IvwbuteIG4TZ veonT7mr46KWkvRTVEprE8agqRmQgZ6dxFvZ4CQ7tSlklSvq2x9tvf/uyZY+UyJIqWnD EjwJgtTjZDQblarpLAO/IV0ba7VEwTzz5zecARjlK20eq2ZCQ91OgJwPPQ/w3FPtb6ce FTalljQlBM79xABgmr+B/rD4irkV+tDYepnfKjCKK4dZkTlJe1Pu05ZsUcDgPeV44Ywp M0Jw== X-Gm-Message-State: AOJu0Yy1KAnM6NokGKWkBqyAhUHjdS8K/guMb3b0+1B6D+w0HHVGSbkd GU4X0KmflaDa71luqTXuwnE= X-Google-Smtp-Source: AGHT+IGyq9vTRFR0XF2Np0Mp0z5zonOlx+6qHuBgaPVX49kLwfxfO4P3ts8iW0SERptq2G7TknmDcg== X-Received: by 2002:a05:6a00:174c:b0:6ba:2ba7:b9cb with SMTP id j12-20020a056a00174c00b006ba2ba7b9cbmr12112155pfc.12.1698093411408; Mon, 23 Oct 2023 13:36:51 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:14f9:170e:9304:1c4e]) by smtp.gmail.com with ESMTPSA id g29-20020aa79ddd000000b0068be4ce33easm5776025pfq.96.2023.10.23.13.36.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 13:36:50 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Ming Lei , Keith Busch , Damien Le Moal , Yu Kuai , Ed Tsai , "James E.J. Bottomley" , Matthias Brugger Subject: [PATCH v4 2/3] scsi: core: Support disabling fair tag sharing Date: Mon, 23 Oct 2023 13:36:34 -0700 Message-ID: <20231023203643.3209592-3-bvanassche@acm.org> X-Mailer: git-send-email 2.42.0.758.gaed0368e0e-goog In-Reply-To: <20231023203643.3209592-1-bvanassche@acm.org> References: <20231023203643.3209592-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Allow SCSI drivers to disable the block layer fair tag sharing algorithm. Cc: Christoph Hellwig Cc: Martin K. Petersen Cc: Ming Lei Cc: Keith Busch Cc: Damien Le Moal Cc: Yu Kuai Cc: Ed Tsai Signed-off-by: Bart Van Assche --- drivers/scsi/hosts.c | 1 + drivers/scsi/scsi_lib.c | 2 ++ include/scsi/scsi_host.h | 6 ++++++ 3 files changed, 9 insertions(+) diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c index d7f51b84f3c7..872f87001374 100644 --- a/drivers/scsi/hosts.c +++ b/drivers/scsi/hosts.c @@ -442,6 +442,7 @@ struct Scsi_Host *scsi_host_alloc(const struct scsi_host_template *sht, int priv shost->no_write_same = sht->no_write_same; shost->host_tagset = sht->host_tagset; shost->queuecommand_may_block = sht->queuecommand_may_block; + shost->disable_fair_tag_sharing = sht->disable_fair_tag_sharing; if (shost_eh_deadline == -1 || !sht->eh_host_reset_handler) shost->eh_deadline = -1; diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index c2f647a7c1b0..68683988c466 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1985,6 +1985,8 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost) BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy); if (shost->queuecommand_may_block) tag_set->flags |= BLK_MQ_F_BLOCKING; + if (shost->disable_fair_tag_sharing) + tag_set->flags |= BLK_MQ_F_DISABLE_FAIR_TAG_SHARING; tag_set->driver_data = shost; if (shost->host_tagset) tag_set->flags |= BLK_MQ_F_TAG_HCTX_SHARED; diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 49f768d0ff37..2cd8a41b1f1c 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -461,6 +461,9 @@ struct scsi_host_template { /* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */ unsigned queuecommand_may_block:1; + /* See also BLK_MQ_F_DISABLE_FAIR_TAG_SHARING. */ + unsigned disable_fair_tag_sharing:1; + /* * Countdown for host blocking with no commands outstanding. */ @@ -659,6 +662,9 @@ struct Scsi_Host { /* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */ unsigned queuecommand_may_block:1; + /* See also BLK_MQ_F_DISABLE_FAIR_TAG_SHARING. */ + unsigned disable_fair_tag_sharing:1; + /* Host responded with short (<36 bytes) INQUIRY result */ unsigned short_inquiry:1; From patchwork Mon Oct 23 20:36:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13433500 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 549CEC25B45 for ; Mon, 23 Oct 2023 20:37:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230483AbjJWUhF (ORCPT ); Mon, 23 Oct 2023 16:37:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231172AbjJWUhD (ORCPT ); Mon, 23 Oct 2023 16:37:03 -0400 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E70D5D7F; Mon, 23 Oct 2023 13:37:00 -0700 (PDT) Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6bb4abb8100so3041320b3a.2; Mon, 23 Oct 2023 13:37:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698093420; x=1698698220; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AMnsyk1g8r/3tKoyiKaNRWtn6PBqRflWAg/7J6/KnIY=; b=V/bF7KEPCf4ZVdyJNX231IE7z9gyj4gspeSFSKELybd6EbxYwda47dCs66Nyn/Zg6A EJE+BkkVbbXvYIDhDLm++glwRWcPzvPloqDQxDd6wPYc7EKiMEVnC2Vk10nDONInhG3T n8GyFRI7e5hDzLzGYfprvNfHxwR+b5tdBIKOpGpJZUpbUszsAixOhDaXxm+Ns6xg+T1r 5DMDSp9d4sdclMyKRy85MjYoN/YHCTfZf2FsGQE/b2kJYo6VZuYupi+jvnslL4sytbOS 82wFz8vKgK49d4hkKM0VOt8+96kmi3J/tdNxrqTULCsK4dGlMa5OLRQsPEPnPxOKrhM+ 4hCg== X-Gm-Message-State: AOJu0YwqtpcCMYGQEn+ZcvfWWvRA4tkDt4FreiapJaqlNOnBLweHXVzG WQEoPP/vlbSjQvIeHHkTah5aTgFsOf4= X-Google-Smtp-Source: AGHT+IFpLVY+L0n1w4PTLQloDCUpQcBgfWyDbIutAs9c/T9RzALjy2COti+bZOeDSl3+yYazHs5lzA== X-Received: by 2002:a05:6a21:360c:b0:17b:4f43:afd1 with SMTP id yg12-20020a056a21360c00b0017b4f43afd1mr567295pzb.58.1698093420246; Mon, 23 Oct 2023 13:37:00 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:14f9:170e:9304:1c4e]) by smtp.gmail.com with ESMTPSA id g29-20020aa79ddd000000b0068be4ce33easm5776025pfq.96.2023.10.23.13.36.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Oct 2023 13:36:59 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, "Martin K . Petersen" , Christoph Hellwig , Bart Van Assche , Ming Lei , Keith Busch , Damien Le Moal , Yu Kuai , Ed Tsai , "James E.J. Bottomley" , Matthias Brugger , Stanley Chu , Can Guo , Manivannan Sadhasivam , Asutosh Das , Bean Huo , "Bao D. Nguyen" , Arthur Simchaev Subject: [PATCH v4 3/3] scsi: ufs: Disable fair tag sharing Date: Mon, 23 Oct 2023 13:36:35 -0700 Message-ID: <20231023203643.3209592-4-bvanassche@acm.org> X-Mailer: git-send-email 2.42.0.758.gaed0368e0e-goog In-Reply-To: <20231023203643.3209592-1-bvanassche@acm.org> References: <20231023203643.3209592-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Disable the block layer fair tag sharing algorithm because it significantly reduces performance of UFS devices with a maximum queue depth of 32. Cc: Christoph Hellwig Cc: Martin K. Petersen Cc: Ming Lei Cc: Keith Busch Cc: Damien Le Moal Cc: Yu Kuai Cc: Ed Tsai Signed-off-by: Bart Van Assche Reviewed-by: Avri Altman --- drivers/ufs/core/ufshcd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index c2df07545f96..ed04d1263e02 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -8866,6 +8866,7 @@ static const struct scsi_host_template ufshcd_driver_template = { .max_host_blocked = 1, .track_queue_depth = 1, .skip_settle_delay = 1, + .disable_fair_tag_sharing = 1, .sdev_groups = ufshcd_driver_groups, .rpm_autosuspend_delay = RPM_AUTOSUSPEND_DELAY_MS, };