From patchwork Wed Jul 26 19:34:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BF3EC001E0 for ; Wed, 26 Jul 2023 19:35:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229519AbjGZTfG (ORCPT ); Wed, 26 Jul 2023 15:35:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230154AbjGZTfE (ORCPT ); Wed, 26 Jul 2023 15:35:04 -0400 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C0001BE3 for ; Wed, 26 Jul 2023 12:35:03 -0700 (PDT) Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-6687466137bso199314b3a.0 for ; Wed, 26 Jul 2023 12:35:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400103; x=1691004903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SRbmBTjr8uNkUCS+MmlXaDe5TIY3it4ulF691kaB6aA=; b=I23imBklO8Sob93UDKsgy3SUBdVyicuHu8zEF3uWilJl/Vy4J9+1ofci2DOf+pPiDO +oRzJNPPXIDhAuVOMmWq27P8278ePi6TyKxK1KqdxS9NPr+1TO5mss1SG646D+atJYxE +AHIzyBFAyMsLzZc5lpL9U7TA8a+/AWMqt+zZOkgHNhrErVLEi1Wix39yDO5XTgqfxta Wq0SpOw20hcVvIp09XFBie6yMpIQeXxFvk2TEaat9rbElxmAbh+3q56fTQ1dd7oDcB8+ GGha8P/6Yr/gHc1odrFrZW8jwJ1bq8GEp96C8lEJqUiF+kbz3PHQ21r612YpttjJzGZJ YKLA== X-Gm-Message-State: ABy/qLbPL8AWYT7WYdOR8pHGZbK/zuU5QBryYIUew8IKjNnkci95ZxbV 3SaJZ7/XJWxvGBXb4SL21vk= X-Google-Smtp-Source: APBJJlHw5dCGZMqy8PYK9aNa8fLWNecqEk1y46e/Od/tm+rMRhAis2mlPbG83XhJkBDIJqAzwX85LQ== X-Received: by 2002:a05:6a00:994:b0:64f:7a9c:cb15 with SMTP id u20-20020a056a00099400b0064f7a9ccb15mr3573711pfg.11.1690400102601; Wed, 26 Jul 2023 12:35:02 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:02 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Ming Lei Subject: [PATCH v4 1/7] block: Introduce the flag QUEUE_FLAG_NO_ZONE_WRITE_LOCK Date: Wed, 26 Jul 2023 12:34:05 -0700 Message-ID: <20230726193440.1655149-2-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Writes in sequential write required zones must happen at the write pointer. Even if the submitter of the write commands (e.g. a filesystem) submits writes for sequential write required zones in order, the block layer or the storage controller may reorder these write commands. The zone locking mechanism in the mq-deadline I/O scheduler serializes write commands for sequential zones. Some but not all storage controllers require this serialization. Introduce a new flag such that block drivers can request that zone write locking is disabled. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- include/linux/blkdev.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 2f5371b8482c..066ac395f62f 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -534,6 +534,11 @@ struct request_queue { #define QUEUE_FLAG_NONROT 6 /* non-rotational device (SSD) */ #define QUEUE_FLAG_VIRT QUEUE_FLAG_NONROT /* paravirt device */ #define QUEUE_FLAG_IO_STAT 7 /* do disk/partitions IO accounting */ +/* + * Do not use the zone write lock for sequential writes for sequential write + * required zones. + */ +#define QUEUE_FLAG_NO_ZONE_WRITE_LOCK 8 #define QUEUE_FLAG_NOXMERGES 9 /* No extended merges */ #define QUEUE_FLAG_ADD_RANDOM 10 /* Contributes to random pool */ #define QUEUE_FLAG_SYNCHRONOUS 11 /* always completes in submit context */ @@ -597,6 +602,11 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_skip_tagset_quiesce(q) \ test_bit(QUEUE_FLAG_SKIP_TAGSET_QUIESCE, &(q)->queue_flags) +static inline bool blk_queue_no_zone_write_lock(struct request_queue *q) +{ + return test_bit(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, &q->queue_flags); +} + extern void blk_set_pm_only(struct request_queue *q); extern void blk_clear_pm_only(struct request_queue *q); From patchwork Wed Jul 26 19:34:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1BB5C001DC for ; Wed, 26 Jul 2023 19:35:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230112AbjGZTfG (ORCPT ); Wed, 26 Jul 2023 15:35:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229454AbjGZTfG (ORCPT ); Wed, 26 Jul 2023 15:35:06 -0400 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C91E5212D for ; Wed, 26 Jul 2023 12:35:04 -0700 (PDT) Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-686f0d66652so53245b3a.2 for ; Wed, 26 Jul 2023 12:35:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400104; x=1691004904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i3oGvYfH0cU7md9y+FMhEYGBL6XNBYy8/TQQQlpG39Q=; b=ZAfGeQ7FPty9vbmrE2rhkFdf1noUSwD0N9/bHDjm6swKHD6GKxD/IBuyE4DF2ja2lV KLp0T8xP0n9pMx2VUlSBVHzmKoXTPL2E2jGBUP6nOyMiTnjegtAtGgjn2WjHuERdZ81T 2RZLjCBZ7ZeCtixJ5//phC9d4gXaZV0cCaaxALbwb1DYQk9n94tBxQcZ1rH85c0UIggc o5gCpVld7AGMouCMXmCbLTPLmhK380ZNOZ7Qh/xcfIyGi5q0yRCISZlzFkYRHgptlRCM 7Tp+Ps1CfI1L7EpB0AK+pZa1R43vtr02L/hk52bIBH9fP3hMaJjCytJjhhiChfH6cvAz fVCA== X-Gm-Message-State: ABy/qLYPIJHxvg+YjNVq5d1yiq1D6gusMmrfpHgRnqdQyVervvw9MK11 rU3My2Rq8llDXCzMelEXlys= X-Google-Smtp-Source: APBJJlFelBAEVPoBu10uzOA25sUJObjVXGPmSfPjcBVgY5zC51qyNlBAeBM8vmeDeAiFsuXK0twn6Q== X-Received: by 2002:a05:6a00:1ad3:b0:682:5634:3df1 with SMTP id f19-20020a056a001ad300b0068256343df1mr3721388pfv.10.1690400103909; Wed, 26 Jul 2023 12:35:03 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:03 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Ming Lei Subject: [PATCH v4 2/7] block: Introduce the flag REQ_NO_ZONE_WRITE_LOCK Date: Wed, 26 Jul 2023 12:34:06 -0700 Message-ID: <20230726193440.1655149-3-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Not all software that supports zoned storage allocates and submits zoned writes in LBA order per zone. Introduce the REQ_NO_WRITE_LOCK flag such that submitters of zoned writes can indicate that zoned writes are allocated and submitted in LBA order per zone. Introduce the blk_no_zone_write_lock() function to make it easy to test whether both QUEUE_FLAG_NO_ZONE_WRITE_LOCK and REQ_NO_ZONE_WRITE_LOCK are set. Make flush requests inherit the REQ_NO_ZONE_WRITE_LOCK flag from the request they are derived from. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/blk-flush.c | 3 ++- include/linux/blk-mq.h | 11 +++++++++++ include/linux/blk_types.h | 8 ++++++++ 3 files changed, 21 insertions(+), 1 deletion(-) diff --git a/block/blk-flush.c b/block/blk-flush.c index e73dc22d05c1..038350ed7cce 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -336,7 +336,8 @@ static void blk_kick_flush(struct request_queue *q, struct blk_flush_queue *fq, flush_rq->internal_tag = first_rq->internal_tag; flush_rq->cmd_flags = REQ_OP_FLUSH | REQ_PREFLUSH; - flush_rq->cmd_flags |= (flags & REQ_DRV) | (flags & REQ_FAILFAST_MASK); + flush_rq->cmd_flags |= flags & + (REQ_FAILFAST_MASK | REQ_NO_ZONE_WRITE_LOCK | REQ_DRV); flush_rq->rq_flags |= RQF_FLUSH_SEQ; flush_rq->end_io = flush_end_io; /* diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 01e8c31db665..dc8ec26ba2d0 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1170,6 +1170,12 @@ static inline bool blk_rq_is_seq_zoned_write(struct request *rq) blk_rq_zone_is_seq(rq); } +static inline bool blk_no_zone_write_lock(struct request *rq) +{ + return blk_queue_no_zone_write_lock(rq->q) && + rq->cmd_flags & REQ_NO_ZONE_WRITE_LOCK; +} + bool blk_req_needs_zone_write_lock(struct request *rq); bool blk_req_zone_write_trylock(struct request *rq); void __blk_req_zone_write_lock(struct request *rq); @@ -1205,6 +1211,11 @@ static inline bool blk_rq_is_seq_zoned_write(struct request *rq) return false; } +static inline bool blk_no_zone_write_lock(struct request *rq) +{ + return true; +} + static inline bool blk_req_needs_zone_write_lock(struct request *rq) { return false; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 0bad62cca3d0..68f7934d8fa6 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -416,6 +416,12 @@ enum req_flag_bits { __REQ_PREFLUSH, /* request for cache flush */ __REQ_RAHEAD, /* read ahead, can fail anytime */ __REQ_BACKGROUND, /* background IO */ + /* + * Do not use zone write locking. Setting this flag is only safe if + * the request submitter allocates and submit requests in LBA order per + * zone. + */ + __REQ_NO_ZONE_WRITE_LOCK, __REQ_NOWAIT, /* Don't wait if request will block */ __REQ_POLLED, /* caller polls for completion using bio_poll */ __REQ_ALLOC_CACHE, /* allocate IO from cache if available */ @@ -448,6 +454,8 @@ enum req_flag_bits { #define REQ_PREFLUSH (__force blk_opf_t)(1ULL << __REQ_PREFLUSH) #define REQ_RAHEAD (__force blk_opf_t)(1ULL << __REQ_RAHEAD) #define REQ_BACKGROUND (__force blk_opf_t)(1ULL << __REQ_BACKGROUND) +#define REQ_NO_ZONE_WRITE_LOCK \ + (__force blk_opf_t)(1ULL << __REQ_NO_ZONE_WRITE_LOCK) #define REQ_NOWAIT (__force blk_opf_t)(1ULL << __REQ_NOWAIT) #define REQ_POLLED (__force blk_opf_t)(1ULL << __REQ_POLLED) #define REQ_ALLOC_CACHE (__force blk_opf_t)(1ULL << __REQ_ALLOC_CACHE) From patchwork Wed Jul 26 19:34:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52BA2C0015E for ; Wed, 26 Jul 2023 19:35:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230095AbjGZTfJ (ORCPT ); Wed, 26 Jul 2023 15:35:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229454AbjGZTfH (ORCPT ); Wed, 26 Jul 2023 15:35:07 -0400 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA8BD2D43 for ; Wed, 26 Jul 2023 12:35:05 -0700 (PDT) Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-686efdeabaeso137156b3a.3 for ; Wed, 26 Jul 2023 12:35:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400105; x=1691004905; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Spi0QoOx8xiu22EDvlyX8nDXg1t+SawetU4mZevhtFk=; b=VYG9TXj7Awes2O8G7wdYPU56BQhlnh1LtLs6PjL3f2DIkxWIpEQnyPkhavL3K7At9I QO2W4/hNUVZiwR5QeGwsDHPh7yQKDef7BnjOBjQ3/Qn8ISP48tSMgtWxstzAxng6Ko1G A+LoPtRE5CQxQhal5kcRquuuW2pRAZbPr/+3EszIxP+q/XYLq0ADo5LJhf6Gk20GEdl0 Z/XvkPxWNOGR6FB86NaIhc3x6ICC1IbAIRXrqaXhVltU6mgnvjsvNEM9tJwtyB9vBQwN UP4bIvSz28JcRIn31mPA18cX2tGFMN3lLg2V/9hpMgLuzHctg4aEA4SJfH4oa1ZV/yco T2IA== X-Gm-Message-State: ABy/qLaaU5udA0nfQBPs2v6ctRPctbksO3ai8UXxWIjZfH4SM9qtcyg9 0RL4HMDHhJnH4oTgIywEgfQ= X-Google-Smtp-Source: APBJJlEMDh1KZg3grPe1Zdftvn5YnV5i/pG6YClz0QeU5WruwdXSJEDm+ZRXS/k/Pap1UZ/83u5p/A== X-Received: by 2002:a05:6a00:181e:b0:686:9385:4644 with SMTP id y30-20020a056a00181e00b0068693854644mr3454388pfa.2.1690400105218; Wed, 26 Jul 2023 12:35:05 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:04 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Ming Lei Subject: [PATCH v4 3/7] block/mq-deadline: Only use zone locking if necessary Date: Wed, 26 Jul 2023 12:34:07 -0700 Message-ID: <20230726193440.1655149-4-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Measurements have shown that limiting the queue depth to one for zoned writes has a significant negative performance impact on zoned UFS devices. Hence this patch that disables zone locking by the mq-deadline scheduler if zoned writes are submitted in order and if the storage controller preserves the command order. This patch is based on the following assumptions: - It happens infrequently that zoned write requests are reordered by the block layer. - The I/O priority of all write requests is the same per zone. - Either no I/O scheduler is used or an I/O scheduler is used that submits write requests per zone in LBA order. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 02a916ba62ee..9a64577fe942 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -338,6 +338,17 @@ static struct request *deadline_skip_seq_writes(struct deadline_data *dd, return rq; } +/* + * Use write locking if either QUEUE_FLAG_NO_ZONE_WRITE_LOCK or + * REQ_NO_ZONE_WRITE_LOCK has not been set. Not using zone write locking is + * only safe if the submitter allocates and submit requests in LBA order per + * zone and if the block driver preserves the request order. + */ +static bool dd_use_write_locking(struct request *rq) +{ + return blk_queue_is_zoned(rq->q) && !blk_no_zone_write_lock(rq); +} + /* * For the specified data direction, return the next request to * dispatch using arrival ordered lists. @@ -353,7 +364,7 @@ deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, return NULL; rq = rq_entry_fifo(per_prio->fifo_list[data_dir].next); - if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q)) + if (data_dir == DD_READ || !dd_use_write_locking(rq)) return rq; /* @@ -398,7 +409,7 @@ deadline_next_request(struct deadline_data *dd, struct dd_per_prio *per_prio, if (!rq) return NULL; - if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q)) + if (data_dir == DD_READ || !dd_use_write_locking(rq)) return rq; /* @@ -526,8 +537,9 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, } /* - * For a zoned block device, if we only have writes queued and none of - * them can be dispatched, rq will be NULL. + * For a zoned block device that requires write serialization, if we + * only have writes queued and none of them can be dispatched, rq will + * be NULL. */ if (!rq) return NULL; @@ -552,7 +564,8 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, /* * If the request needs its target zone locked, do it. */ - blk_req_zone_write_lock(rq); + if (dd_use_write_locking(rq)) + blk_req_zone_write_lock(rq); rq->rq_flags |= RQF_STARTED; return rq; } @@ -933,7 +946,7 @@ static void dd_finish_request(struct request *rq) atomic_inc(&per_prio->stats.completed); - if (blk_queue_is_zoned(q)) { + if (dd_use_write_locking(rq)) { unsigned long flags; spin_lock_irqsave(&dd->zone_lock, flags); From patchwork Wed Jul 26 19:34:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E19F9C001DC for ; Wed, 26 Jul 2023 19:35:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229454AbjGZTf0 (ORCPT ); Wed, 26 Jul 2023 15:35:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230386AbjGZTfZ (ORCPT ); Wed, 26 Jul 2023 15:35:25 -0400 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5DE11BD9 for ; Wed, 26 Jul 2023 12:35:24 -0700 (PDT) Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6686a05bc66so185302b3a.1 for ; Wed, 26 Jul 2023 12:35:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400124; x=1691004924; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0QJms5YcXrYmxRKHOnm5/t0MTK2LPWvAB4sY3WYCkgg=; b=jw8pOwaInslUgKOgL4ZUj2aRjzPidK6rrrjkAiYwQ/ngrjSB0GK7flr0hvKoZuOhlu i5TpcCwZKri2lAqLYgzP6cm1c7FNH0VBpOLF6y7h11aG1Et3dyv9enNT5C+WtslHj+VB KGuMRrHj2tmFGgqUzHgKAJ1q7KPHX0uVjboXsjkUWClOoFs2FSErCUptmS5v2R1kKgJu Nk/t9GDpkCqV+vGac1vic5JiPJmbmkn4ir3lsPFHRtyqz7eWiG3Nv0SonizWU17JHCo/ Gh5Y1nDDu8kqUF0dFWAo72QHjx8m6onB2Cj5x6hH5b7aVh5ATxSkJETNdralRhRkk+E4 ZF8A== X-Gm-Message-State: ABy/qLYPZZqMsscYYJhah83WyI5P7dorUpLw3dIRYpxqD/Fl0G6jFGrj dHbNG1oE3OPNBycZUiiG9aE= X-Google-Smtp-Source: APBJJlHzzJbtAKD+r8hZoZVS1wZg5/pijBTWmP7BXVswstGrNs2LOYSPWm5UPzK0FQlnHP8NKUM6wA== X-Received: by 2002:a05:6a20:7f9e:b0:132:c7de:f71c with SMTP id d30-20020a056a207f9e00b00132c7def71cmr3353038pzj.59.1690400124346; Wed, 26 Jul 2023 12:35:24 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:23 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Ming Lei , Chaitanya Kulkarni , Johannes Thumshirn , Vincent Fu , Keith Busch , Akinobu Mita , "Shin'ichiro Kawasaki" Subject: [PATCH v4 4/7] block/null_blk: Support disabling zone write locking Date: Wed, 26 Jul 2023 12:34:08 -0700 Message-ID: <20230726193440.1655149-5-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add a new configfs attribute for disabling zone write locking. Tests show a performance of 250 K IOPS with no I/O scheduler, 6 K IOPS with mq-deadline and write locking enabled and 123 K IOPS with mq-deadline and write locking disabled. This shows that disabling write locking results in about 20 times more IOPS for this particular test case. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal Reviewed-by: Chaitanya Kulkarni Reviewed-by: Nitesh Shetty --- drivers/block/null_blk/main.c | 2 ++ drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/zoned.c | 3 +++ 3 files changed, 6 insertions(+) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 864013019d6b..5c0578137f51 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -424,6 +424,7 @@ NULLB_DEVICE_ATTR(zone_capacity, ulong, NULL); NULLB_DEVICE_ATTR(zone_nr_conv, uint, NULL); NULLB_DEVICE_ATTR(zone_max_open, uint, NULL); NULLB_DEVICE_ATTR(zone_max_active, uint, NULL); +NULLB_DEVICE_ATTR(no_zone_write_lock, bool, NULL); NULLB_DEVICE_ATTR(virt_boundary, bool, NULL); NULLB_DEVICE_ATTR(no_sched, bool, NULL); NULLB_DEVICE_ATTR(shared_tag_bitmap, bool, NULL); @@ -569,6 +570,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_zone_max_active, &nullb_device_attr_zone_readonly, &nullb_device_attr_zone_offline, + &nullb_device_attr_no_zone_write_lock, &nullb_device_attr_virt_boundary, &nullb_device_attr_no_sched, &nullb_device_attr_shared_tag_bitmap, diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index 929f659dd255..b521096bcc3f 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -117,6 +117,7 @@ struct nullb_device { bool memory_backed; /* if data is stored in memory */ bool discard; /* if support discard */ bool zoned; /* if device is zoned */ + bool no_zone_write_lock; bool virt_boundary; /* virtual boundary on/off for the device */ bool no_sched; /* no IO scheduler for the device */ bool shared_tag_bitmap; /* use hostwide shared tags */ diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 55c5b48bc276..31c8364a63e9 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -96,6 +96,9 @@ int null_init_zoned_dev(struct nullb_device *dev, struct request_queue *q) spin_lock_init(&dev->zone_res_lock); + if (dev->no_zone_write_lock) + blk_queue_flag_set(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); + if (dev->zone_nr_conv >= dev->nr_zones) { dev->zone_nr_conv = dev->nr_zones - 1; pr_info("changed the number of conventional zones to %u", From patchwork Wed Jul 26 19:34:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98B50C0015E for ; Wed, 26 Jul 2023 19:35:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230386AbjGZTf2 (ORCPT ); Wed, 26 Jul 2023 15:35:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230154AbjGZTf1 (ORCPT ); Wed, 26 Jul 2023 15:35:27 -0400 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5F041BD6 for ; Wed, 26 Jul 2023 12:35:26 -0700 (PDT) Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-666e916b880so159587b3a.2 for ; Wed, 26 Jul 2023 12:35:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400126; x=1691004926; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zprMuNObpOq9L09JcLTQAP64ZkbUp975VRr/Q4ibEpY=; b=HUius+jhTivSXxXWO2s7duyxjGJeKUSk0weIajIC80uBEFlNszGKZmcIkN4J6pmiNK TrtRMOmdVUFoRh6jkDSKTSblfwXkgkvemur0hT9/m56UUcLEDSt/RGO8bSViQ8vS8W24 1hQ6IfQF7aWtHOmX5oAw25AIUdX/0td23UzjCK477DPSRSGiWJ2W7Z2v1wuKuo7zMw0r 4tZu6GAh4CwlJqqRnmIK+FVtV6EhWEzZQ1O0iBcAULHwkBN9pgCdfBmmWm63nzn+Zxkm +MLhzT4klMXauZ9TrvPOmle2KTr3nvIJEbXmUnp741ahtErCoSfsZ0aa06a5Ird1R4We gEow== X-Gm-Message-State: ABy/qLZPuAW7VLGWTTFWLCLfSgu/pFCwCKWawpZX+qSv9VUvepq8/nST FxdIOE/tGp/mqqV4oqTEjOM= X-Google-Smtp-Source: APBJJlGRYk+RYq+zWfYqIVvAuOawHJD6I+nfkq1qq28WSZQ8pHfig6wxygsXEcrkdawgIzWmnf/hjQ== X-Received: by 2002:a05:6a00:2d04:b0:682:3126:961a with SMTP id fa4-20020a056a002d0400b006823126961amr2746652pfb.5.1690400126009; Wed, 26 Jul 2023 12:35:26 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:25 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Ming Lei , "James E.J. Bottomley" Subject: [PATCH v4 5/7] scsi: Retry unaligned zoned writes Date: Wed, 26 Jul 2023 12:34:09 -0700 Message-ID: <20230726193440.1655149-6-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org If zoned writes (REQ_OP_WRITE) for a sequential write required zone have a starting LBA that differs from the write pointer, e.g. because zoned writes have been reordered, then the storage device will respond with an UNALIGNED WRITE COMMAND error. Send commands that failed with an unaligned write error to the SCSI error handler. Let the SCSI error handler sort SCSI commands per LBA before resubmitting these. If zone write locking is disabled, increase the number of retries for write commands sent to a sequential zone to the maximum number of outstanding commands because in the worst case the number of times reordered zoned writes have to be retried is (number of outstanding writes per sequential zone) - 1. Cc: Martin K. Petersen Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- drivers/scsi/scsi_error.c | 38 ++++++++++++++++++++++++++++++++++++++ drivers/scsi/scsi_lib.c | 1 + drivers/scsi/sd.c | 2 ++ include/scsi/scsi.h | 1 + 4 files changed, 42 insertions(+) diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c index c67cdcdc3ba8..9dc354a24d4b 100644 --- a/drivers/scsi/scsi_error.c +++ b/drivers/scsi/scsi_error.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -698,6 +699,17 @@ enum scsi_disposition scsi_check_sense(struct scsi_cmnd *scmd) fallthrough; case ILLEGAL_REQUEST: + /* + * Unaligned write command. This indicates that zoned writes + * have been received by the device in the wrong order. If zone + * write locking is disabled, retry after all pending commands + * have completed. + */ + if (sshdr.asc == 0x21 && sshdr.ascq == 0x04 && + blk_no_zone_write_lock(scsi_cmd_to_rq(scmd)) && + !scsi_noretry_cmd(scmd) && scsi_cmd_retry_allowed(scmd)) + return NEEDS_DELAYED_RETRY; + if (sshdr.asc == 0x20 || /* Invalid command operation code */ sshdr.asc == 0x21 || /* Logical block address out of range */ sshdr.asc == 0x22 || /* Invalid function */ @@ -2223,6 +2235,25 @@ void scsi_eh_flush_done_q(struct list_head *done_q) } EXPORT_SYMBOL(scsi_eh_flush_done_q); +/* + * Returns a negative value if @_a has a lower LBA than @_b, zero if + * both have the same LBA and a positive value otherwise. + */ +static int scsi_cmp_lba(void *priv, const struct list_head *_a, + const struct list_head *_b) +{ + struct scsi_cmnd *a = list_entry(_a, typeof(*a), eh_entry); + struct scsi_cmnd *b = list_entry(_b, typeof(*b), eh_entry); + const sector_t pos_a = blk_rq_pos(scsi_cmd_to_rq(a)); + const sector_t pos_b = blk_rq_pos(scsi_cmd_to_rq(b)); + + if (pos_a < pos_b) + return -1; + if (pos_a > pos_b) + return 1; + return 0; +} + /** * scsi_unjam_host - Attempt to fix a host which has a cmd that failed. * @shost: Host to unjam. @@ -2258,6 +2289,13 @@ static void scsi_unjam_host(struct Scsi_Host *shost) SCSI_LOG_ERROR_RECOVERY(1, scsi_eh_prt_fail_stats(shost, &eh_work_q)); + /* + * Sort pending SCSI commands in LBA order. This is important if one of + * the SCSI devices associated with @shost is a zoned block device for + * which zone write locking is disabled. + */ + list_sort(NULL, &eh_work_q, scsi_cmp_lba); + if (!scsi_eh_get_sense(&eh_work_q, &eh_done_q)) scsi_eh_ready_devs(shost, &eh_work_q, &eh_done_q); diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 59176946ab56..69da8aee13df 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1443,6 +1443,7 @@ static void scsi_complete(struct request *rq) case ADD_TO_MLQUEUE: scsi_queue_insert(cmd, SCSI_MLQUEUE_DEVICE_BUSY); break; + case NEEDS_DELAYED_RETRY: default: scsi_eh_scmd_add(cmd); break; diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 68b12afa0721..7b5877323245 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -1235,6 +1235,8 @@ static blk_status_t sd_setup_read_write_cmnd(struct scsi_cmnd *cmd) cmd->transfersize = sdp->sector_size; cmd->underflow = nr_blocks << 9; cmd->allowed = sdkp->max_retries; + if (blk_no_zone_write_lock(rq) && blk_rq_is_seq_zoned_write(rq)) + cmd->allowed += rq->q->nr_requests; cmd->sdb.length = nr_blocks * sdp->sector_size; SCSI_LOG_HLQUEUE(1, diff --git a/include/scsi/scsi.h b/include/scsi/scsi.h index ec093594ba53..6600db046227 100644 --- a/include/scsi/scsi.h +++ b/include/scsi/scsi.h @@ -93,6 +93,7 @@ static inline int scsi_status_is_check_condition(int status) * Internal return values. */ enum scsi_disposition { + NEEDS_DELAYED_RETRY = 0x2000, NEEDS_RETRY = 0x2001, SUCCESS = 0x2002, FAILED = 0x2003, From patchwork Wed Jul 26 19:34:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328494 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8887BC0015E for ; Wed, 26 Jul 2023 19:35:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230154AbjGZTfk (ORCPT ); Wed, 26 Jul 2023 15:35:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229488AbjGZTfj (ORCPT ); Wed, 26 Jul 2023 15:35:39 -0400 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCF232D49 for ; Wed, 26 Jul 2023 12:35:34 -0700 (PDT) Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-666ecf9a081so211290b3a.2 for ; Wed, 26 Jul 2023 12:35:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400134; x=1691004934; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ad4uaRIsD1CTI297aCaSL2DUM2sDqDwUwe7bpkX95ps=; b=C5vl82d4fZA7FKJKxPFOnYjcZ0uNctW0W8ti7+sweT8SwL30pxPOOViRn7m0+BPBgP MIKoR0XPpbqVBMLei7Vaz2JM7aBu+3vwREnV6o3sj/L/jBXT+zaW5WSxq3gsVK/OYI3c GKLG+fjyuSmiRJLrwvdqEte54G19BcfYCOaiPh5vJyL7574DNlr23MI8mcAcf2amXG9z kgDZpmZyAdRtK55Wllk/rNS81L2YcctFvkzssg3SFY3QkOHPo1ufvQLdBet3r3/rjLhZ abX6k0u0JrGmlHqV9rNOFJHaegye0FQa5QrOEx/uSZLnOM2oDFPiMrUcNTJnkBdOOdYF DmiQ== X-Gm-Message-State: ABy/qLaH+Te3+iJgtk/V4+yuSe5jw+qcPOvc5H9TL8vpnNELrtzdG0nY TuA9H+P8d1ppwLv01f9VS6s= X-Google-Smtp-Source: APBJJlHKkDNFJUz4pidZXHrDgyFD0ZFV8/s5z47RgVnXZNxYemosPcsdWV9a9NRYb+jIJl/Qfy84Cg== X-Received: by 2002:a05:6a00:c8a:b0:675:667c:f261 with SMTP id a10-20020a056a000c8a00b00675667cf261mr3958073pfv.2.1690400134158; Wed, 26 Jul 2023 12:35:34 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:33 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Avri Altman , Ming Lei , "James E.J. Bottomley" , Stanley Chu , Can Guo , Asutosh Das , "Bao D. Nguyen" , Bean Huo , Arthur Simchaev Subject: [PATCH v4 6/7] scsi: ufs: Disable zone write locking Date: Wed, 26 Jul 2023 12:34:10 -0700 Message-ID: <20230726193440.1655149-7-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From the UFSHCI 4.0 specification, about the legacy (single queue) mode: "The host controller always process transfer requests in-order according to the order submitted to the list. In case of multiple commands with single doorbell register ringing (batch mode), The dispatch order for these transfer requests by host controller will base on their index in the List. A transfer request with lower index value will be executed before a transfer request with higher index value." From the UFSHCI 4.0 specification, about the MCQ mode: "Command Submission 1. Host SW writes an Entry to SQ 2. Host SW updates SQ doorbell tail pointer Command Processing 3. After fetching the Entry, Host Controller updates SQ doorbell head pointer 4. Host controller sends COMMAND UPIU to UFS device" In other words, for both legacy and MCQ mode, UFS controllers are required to forward commands to the UFS device in the order these commands have been received from the host. Notes: - For legacy mode this is only correct if the host submits one command at a time. The UFS driver does this. - Also in legacy mode, the command order is not preserved if auto-hibernation is enabled in the UFS controller. Hence, enable zone write locking if auto-hibernation is enabled. This patch improves performance as follows on my test setup: - With the mq-deadline scheduler: 2.5x more IOPS for small writes. - When not using an I/O scheduler compared to using mq-deadline with zone locking: 4x more IOPS for small writes. Cc: Martin K. Petersen Cc: Avri Altman Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- drivers/ufs/core/ufshcd.c | 45 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 129446775796..0f7f91e2cda9 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -4337,29 +4337,67 @@ int ufshcd_uic_hibern8_exit(struct ufs_hba *hba) } EXPORT_SYMBOL_GPL(ufshcd_uic_hibern8_exit); +void ufshcd_update_no_zone_write_lock(struct ufs_hba *hba, + bool set_no_zone_write_lock) +{ + struct scsi_device *sdev; + + shost_for_each_device(sdev, hba->host) + blk_freeze_queue_start(sdev->request_queue); + shost_for_each_device(sdev, hba->host) { + struct request_queue *q = sdev->request_queue; + + blk_mq_freeze_queue_wait(q); + if (set_no_zone_write_lock) + blk_queue_flag_set(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); + else + blk_queue_flag_clear(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); + blk_mq_unfreeze_queue(q); + } +} + void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit) { unsigned long flags; - bool update = false; + bool prev_state, new_state, update = false; if (!ufshcd_is_auto_hibern8_supported(hba)) return; spin_lock_irqsave(hba->host->host_lock, flags); + prev_state = ufshcd_is_auto_hibern8_enabled(hba); if (hba->ahit != ahit) { hba->ahit = ahit; update = true; } + new_state = ufshcd_is_auto_hibern8_enabled(hba); spin_unlock_irqrestore(hba->host->host_lock, flags); - if (update && - !pm_runtime_suspended(&hba->ufs_device_wlun->sdev_gendev)) { + if (!update) + return; + if (!is_mcq_enabled(hba) && !prev_state && new_state) { + /* + * Auto-hibernation will be enabled. Enable write locking for + * zoned writes since auto-hibernation may cause reordering of + * zoned writes when using the legacy mode of the UFS host + * controller. + */ + ufshcd_update_no_zone_write_lock(hba, false); + } + if (!pm_runtime_suspended(&hba->ufs_device_wlun->sdev_gendev)) { ufshcd_rpm_get_sync(hba); ufshcd_hold(hba); ufshcd_auto_hibern8_enable(hba); ufshcd_release(hba); ufshcd_rpm_put_sync(hba); } + if (!is_mcq_enabled(hba) && prev_state && !new_state) { + /* + * Auto-hibernation has been disabled. Disable write locking + * for zoned writes. + */ + ufshcd_update_no_zone_write_lock(hba, true); + } } EXPORT_SYMBOL_GPL(ufshcd_auto_hibern8_update); @@ -5139,6 +5177,7 @@ static int ufshcd_slave_configure(struct scsi_device *sdev) ufshcd_hpb_configure(hba, sdev); + blk_queue_flag_set(QUEUE_FLAG_NO_ZONE_WRITE_LOCK, q); blk_queue_update_dma_pad(q, PRDT_DATA_BYTE_COUNT_PAD - 1); if (hba->quirks & UFSHCD_QUIRK_4KB_DMA_ALIGNMENT) blk_queue_update_dma_alignment(q, SZ_4K - 1); From patchwork Wed Jul 26 19:34:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13328493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4E44C001DC for ; Wed, 26 Jul 2023 19:35:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230049AbjGZTfk (ORCPT ); Wed, 26 Jul 2023 15:35:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230154AbjGZTfj (ORCPT ); Wed, 26 Jul 2023 15:35:39 -0400 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EFB4E73 for ; Wed, 26 Jul 2023 12:35:36 -0700 (PDT) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-68336d06620so224585b3a.1 for ; Wed, 26 Jul 2023 12:35:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690400135; x=1691004935; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5g5dDMJrDnmVBYbURJxSyl07IG82zCyXPOwukTCAfBk=; b=kS0zfG5wc3qrKU+m4nxmvpPQazxysvfGX1GLM2Vr44NswB2vj/kBDVr4J1A3MwdLVr K8vw1CHN+2IiRFL/HTiL+PZ3v4cQpY7wDadlAadiKew3HLIfCYCWFk92jzuCxtZTmGKe +xzH9kuxHSm6OfWvW2tV3fUu7x8lBfL6sRw9aK9DXoGqRvJJjS3cW/qMV3boZkFqcEyr dgvfmD1U3+kafvF3maYLPqwSofc6p5nQC8MBjeoG3KGEfIsykUJ5MACRmB9EfHaAh6Qg vTz2w+eLr1k5SmZ2h/DGbCd4GERFVI4eIcY4A1wMFEZCT3cJq5O5gY3kFmc+keyU+Y4P n45A== X-Gm-Message-State: ABy/qLYUrQwDnyQSq4zJZSMfAOFK3hH+6KpZhWbvVMXhwb1JSShOxjOx G2Q8QJ+iwuUtDMbUpS+9iSSTbW00QC8= X-Google-Smtp-Source: APBJJlEl1g3vhRzz8R5sWC2162DA2g4F9kH2BjVdNcMky9ozXLdMMwIc8F+aKXhZ1uOZw8DyI8A9Pw== X-Received: by 2002:a05:6a00:1503:b0:64d:46b2:9a58 with SMTP id q3-20020a056a00150300b0064d46b29a58mr3889745pfu.26.1690400135435; Wed, 26 Jul 2023 12:35:35 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:32d2:d535:b137:7ba3]) by smtp.gmail.com with ESMTPSA id x52-20020a056a000bf400b00682ba300cd1sm11846685pfu.29.2023.07.26.12.35.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Jul 2023 12:35:35 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , "Martin K . Petersen" , Jaegeuk Kim , Damien Le Moal , Bart Van Assche , Ming Lei , Chao Yu Subject: [PATCH v4 7/7] fs/f2fs: Disable zone write locking Date: Wed, 26 Jul 2023 12:34:11 -0700 Message-ID: <20230726193440.1655149-8-bvanassche@acm.org> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog In-Reply-To: <20230726193440.1655149-1-bvanassche@acm.org> References: <20230726193440.1655149-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Set the REQ_NO_ZONE_WRITE_LOCK flag to inform the block layer that F2FS allocates and submits zoned writes in LBA order per zone. Cc: Jaegeuk Kim Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- fs/f2fs/data.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 5882afe71d82..87ef089f1e88 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -468,8 +468,8 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages) struct bio *bio; bdev = f2fs_target_device(sbi, fio->new_blkaddr, §or); - bio = bio_alloc_bioset(bdev, npages, - fio->op | fio->op_flags | f2fs_io_flags(fio), + bio = bio_alloc_bioset(bdev, npages, fio->op | fio->op_flags | + f2fs_io_flags(fio) | REQ_NO_ZONE_WRITE_LOCK, GFP_NOIO, &f2fs_bioset); bio->bi_iter.bi_sector = sector; if (is_read_io(fio->op)) {