From patchwork Mon Apr 24 20:33:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 806B4C77B61 for ; Mon, 24 Apr 2023 20:33:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232605AbjDXUdi (ORCPT ); Mon, 24 Apr 2023 16:33:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230430AbjDXUdh (ORCPT ); Mon, 24 Apr 2023 16:33:37 -0400 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 808DC55AF for ; Mon, 24 Apr 2023 13:33:35 -0700 (PDT) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-63b87d23729so4060408b3a.0 for ; Mon, 24 Apr 2023 13:33:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368415; x=1684960415; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Meycj9u3JfaVnzaUnu3BPLSoAESUUt4GW23IhKc0tM8=; b=S5TN9v4lhNUu2ozRVhvy1rcn4CVaRTpqX3lrloIuUkMKuqYYGtH7Tx8wKaS/vWiun6 U6igTQPM/3CfcgSz+o+EOKQWLI5TFxEMPzV3XMwt0yMJ9Y11ncS+te20HVgfNAcalCrS 28WHzo5PBe9eahmmAnd+g85VnYyL1nLSfBAXBUo8jFjxy4zFBEqkkjbPfw2+ySAGn6UL BT+7SrZ5NGYCa7iMLdV1Jj+Y1gHkbN1BBzsxAYiIMmJS25YBnYlY7mOjOENwOUWhn9sC 7n97RxUyxD4ECuZXX4l09m8nTBHxz1r+ZBIQTNOT0dv7QX4GeOvRUe/Y/qDalHJqsC9E aw+w== X-Gm-Message-State: AAQBX9dIUqWfrdemVvRMyeT0T+7V3bRWXi5jQ+c3PLxUbrC2LJGXuY+H m1p85K/U0sFxO5CZ/Q1ywIc= X-Google-Smtp-Source: AKy350ZpImJ7yn99lXYXaYrQ3W9Dko1JZyIBRUe4GrzXOgqV/CXWxdkqzZKdbq/aUAGu0rJtNa02Yg== X-Received: by 2002:a05:6a00:b8a:b0:63d:23a7:ca62 with SMTP id g10-20020a056a000b8a00b0063d23a7ca62mr19807107pfj.19.1682368414921; Mon, 24 Apr 2023 13:33:34 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:34 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 1/9] block: Simplify blk_req_needs_zone_write_lock() Date: Mon, 24 Apr 2023 13:33:21 -0700 Message-ID: <20230424203329.2369688-2-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Remove the blk_rq_is_passthrough() check because it is redundant: blk_req_needs_zone_write_lock() also calls bdev_op_is_zoned_write() and the latter function returns false for pass-through requests. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/blk-zoned.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index fce9082384d6..835d9e937d4d 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -57,9 +57,6 @@ EXPORT_SYMBOL_GPL(blk_zone_cond_str); */ bool blk_req_needs_zone_write_lock(struct request *rq) { - if (blk_rq_is_passthrough(rq)) - return false; - if (!rq->q->disk->seq_zones_wlock) return false; From patchwork Mon Apr 24 20:33:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77881C77B61 for ; Mon, 24 Apr 2023 20:33:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232611AbjDXUdn (ORCPT ); Mon, 24 Apr 2023 16:33:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230430AbjDXUdl (ORCPT ); Mon, 24 Apr 2023 16:33:41 -0400 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF0FE55AE for ; Mon, 24 Apr 2023 13:33:36 -0700 (PDT) Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-63b4e5fdb1eso6085491b3a.1 for ; Mon, 24 Apr 2023 13:33:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368416; x=1684960416; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/tdPWENE116TMSBILZv2mWrtRSpu1nCQ7jTo9np+GS8=; b=LCAtN0YKn+EH80A5X9XTHaxMOS1zl/9fqPLfYvnP2ZOqvUsucjZazdNw73R6Xac2av N0jKrQH22crcfcLXafiGlXEB8/0WlFxqLzSh1OHplYwlJnnbq0nzJBlIseKmw+RraJ1v 13mD/eoZAnbHGqL39xsDWNE6H/Vh5e7UU4u6vgR2TLK04c6EYA0L8TB+nKMMjYZCUga2 /rWOoKTaFnEiDnwcjE/FuFmWVGZ0hNepOSVro9Y25LUcJwH8MdiBZjqLwAkSYEThFywb Ta4VpORuk6dponMfcfqR/bcIZI6WrGJfyXjVU0xeNequABrEXWjVR+wnMRRdRVJHDOpk 2rlw== X-Gm-Message-State: AAQBX9cvviyrzA0i1L1Aio8qWeaKPlFvUJpfRPT8/OlwG1UQbVYMZxF0 TQz1/CDdlH5VFKeOneFd2aI= X-Google-Smtp-Source: AKy350YAi3/DSSlzNpGv43wAgZJ++DR3PS9RxHF53PxCSJcTxJnPqnGlgk5FcTnpRPgck2WCg4Tf6Q== X-Received: by 2002:a05:6a21:2c94:b0:f0:916:e6f with SMTP id ua20-20020a056a212c9400b000f009160e6fmr13502831pzb.32.1682368416261; Mon, 24 Apr 2023 13:33:36 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:35 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 2/9] block: Micro-optimize blk_req_needs_zone_write_lock() Date: Mon, 24 Apr 2023 13:33:22 -0700 Message-ID: <20230424203329.2369688-3-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Micro-optimize blk_req_needs_zone_write_lock() by inlining bdev_op_is_zoned_write(). This is a micro-optimization because the number of pointers that is dereferenced while testing whether the request queue is associated with a zoned device is reduced. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/blk-mq.h | 5 +++-- block/blk-zoned.c | 11 ++++------- include/linux/blkdev.h | 9 --------- 3 files changed, 7 insertions(+), 18 deletions(-) diff --git a/block/blk-mq.h b/block/blk-mq.h index e876584d3516..6bb1281a61f2 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -367,8 +367,9 @@ static inline void blk_mq_clear_mq_map(struct blk_mq_queue_map *qmap) static inline struct blk_plug *blk_mq_plug( struct bio *bio) { /* Zoned block device write operation case: do not plug the BIO */ - if (IS_ENABLED(CONFIG_BLK_DEV_ZONED) && - bdev_op_is_zoned_write(bio->bi_bdev, bio_op(bio))) + if ((bio_op(bio) == REQ_OP_WRITE || + bio_op(bio) == REQ_OP_WRITE_ZEROES) && + disk_zone_is_seq(bio->bi_bdev->bd_disk, bio->bi_iter.bi_sector)) return NULL; /* diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 835d9e937d4d..4640b75ae66f 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -57,13 +57,10 @@ EXPORT_SYMBOL_GPL(blk_zone_cond_str); */ bool blk_req_needs_zone_write_lock(struct request *rq) { - if (!rq->q->disk->seq_zones_wlock) - return false; - - if (bdev_op_is_zoned_write(rq->q->disk->part0, req_op(rq))) - return blk_rq_zone_is_seq(rq); - - return false; + return rq->q->disk->seq_zones_wlock && + (req_op(rq) == REQ_OP_WRITE || + req_op(rq) == REQ_OP_WRITE_ZEROES) && + blk_rq_zone_is_seq(rq); } EXPORT_SYMBOL_GPL(blk_req_needs_zone_write_lock); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e3242e67a8e3..b700c935e230 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1284,15 +1284,6 @@ static inline unsigned int bdev_zone_no(struct block_device *bdev, sector_t sec) return disk_zone_no(bdev->bd_disk, sec); } -static inline bool bdev_op_is_zoned_write(struct block_device *bdev, - blk_opf_t op) -{ - if (!bdev_is_zoned(bdev)) - return false; - - return op == REQ_OP_WRITE || op == REQ_OP_WRITE_ZEROES; -} - static inline sector_t bdev_zone_sectors(struct block_device *bdev) { struct request_queue *q = bdev_get_queue(bdev); From patchwork Mon Apr 24 20:33:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03E02C7618E for ; Mon, 24 Apr 2023 20:33:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232615AbjDXUdn (ORCPT ); Mon, 24 Apr 2023 16:33:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232620AbjDXUdm (ORCPT ); Mon, 24 Apr 2023 16:33:42 -0400 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1161955AF for ; Mon, 24 Apr 2023 13:33:38 -0700 (PDT) Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-63b70f0b320so6661241b3a.1 for ; Mon, 24 Apr 2023 13:33:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368417; x=1684960417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hPXsN9StPRxhLtrCrqF+PfLLPQj82OhYioJmH8NV21o=; b=Lo0WUvx8a2u2t2fz9NSxHmfKmBYnsB1SZeTK0nVB/kgJuAdjHmTAsskKKKIrnQeisS Y0KWZ7VTPFsigno7ZDmbU2m9nk4PorGOd/awhKsfWC18OfcMc9BkjTzLZztpJEjW08OF NdIWaFUBK6E0J6QAKIhm4My8XbSw0Y+qM4HM9QunJ5rb9wDthO5o+DsM+cERgOOESfEV rErizJR+MdTL22u0KaKhpryf+jpItDYW2m5wxgKi6gtb27ePi+EN9HuOw2wZLMSUcRNn CXngvxGPrehtpwN0/sQp4a360FQYmt83bB1GyaxKA5OrUnGCSvKLhHnYu/ZFAnYXQmYH L5/w== X-Gm-Message-State: AAQBX9egfGCp2DpHABL/vF4lVGg3oupiwgPlK1BMaxfcbQtUMcFNkaok XCYN0DQbzRvoRQFecsG9Q0tulqzC7qQ= X-Google-Smtp-Source: AKy350blhO/jAd+gSLD64QSww/xT4cunqZwUum8e3KvF5uXtlgQV2tSglP4sbYa1vPncHXDqPPm9iw== X-Received: by 2002:a05:6a00:2e0d:b0:63b:7119:64a9 with SMTP id fc13-20020a056a002e0d00b0063b711964a9mr21302100pfb.16.1682368417517; Mon, 24 Apr 2023 13:33:37 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:37 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 3/9] block: Introduce blk_rq_is_seq_zoned_write() Date: Mon, 24 Apr 2023 13:33:23 -0700 Message-ID: <20230424203329.2369688-4-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce the function blk_rq_is_seq_zoned_write(). This function will be used in later patches to preserve the order of zoned writes for which the order needs to be preserved. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/blk-zoned.c | 22 +++++++++++++++++++--- include/linux/blk-mq.h | 6 ++++++ 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 4640b75ae66f..78c39fc505bc 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -52,15 +52,31 @@ const char *blk_zone_cond_str(enum blk_zone_cond zone_cond) } EXPORT_SYMBOL_GPL(blk_zone_cond_str); +/** + * blk_rq_is_seq_zoned_write() - Check if @rq requires write serialization. + * @rq: Request to examine. + * + * Note: REQ_OP_ZONE_APPEND requests do not require serialization. + */ +bool blk_rq_is_seq_zoned_write(struct request *rq) +{ + switch (req_op(rq)) { + case REQ_OP_WRITE: + case REQ_OP_WRITE_ZEROES: + return blk_rq_zone_is_seq(rq); + default: + return false; + } +} +EXPORT_SYMBOL_GPL(blk_rq_is_seq_zoned_write); + /* * Return true if a request is a write requests that needs zone write locking. */ bool blk_req_needs_zone_write_lock(struct request *rq) { return rq->q->disk->seq_zones_wlock && - (req_op(rq) == REQ_OP_WRITE || - req_op(rq) == REQ_OP_WRITE_ZEROES) && - blk_rq_zone_is_seq(rq); + blk_rq_is_seq_zoned_write(rq); } EXPORT_SYMBOL_GPL(blk_req_needs_zone_write_lock); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 06caacd77ed6..e498b85bc470 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1164,6 +1164,7 @@ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) return disk_zone_is_seq(rq->q->disk, blk_rq_pos(rq)); } +bool blk_rq_is_seq_zoned_write(struct request *rq); bool blk_req_needs_zone_write_lock(struct request *rq); bool blk_req_zone_write_trylock(struct request *rq); void __blk_req_zone_write_lock(struct request *rq); @@ -1194,6 +1195,11 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq) return !blk_req_zone_is_write_locked(rq); } #else /* CONFIG_BLK_DEV_ZONED */ +static inline bool blk_rq_is_seq_zoned_write(struct request *rq) +{ + return false; +} + static inline bool blk_req_needs_zone_write_lock(struct request *rq) { return false; From patchwork Mon Apr 24 20:33:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64A5FC77B7F for ; Mon, 24 Apr 2023 20:33:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232622AbjDXUdp (ORCPT ); Mon, 24 Apr 2023 16:33:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232617AbjDXUdo (ORCPT ); Mon, 24 Apr 2023 16:33:44 -0400 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 505DC5B93 for ; Mon, 24 Apr 2023 13:33:39 -0700 (PDT) Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-63b60366047so4053258b3a.1 for ; Mon, 24 Apr 2023 13:33:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368419; x=1684960419; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WF3r3QrptKfudquaNkr3Z1ryI3uc5dg0gMZZjctB2Lw=; b=fyWtG6G+izvvvh7vlM7ImSaiEFhV/YannypjFeTR3NPnc+9ltqA4mqgfvLRbmU12wB hK25VihxSR43kB6CXgJeP6500Z1dbm7oqaA3fBX7Q/iuihe8AueBQATCis2f4YdO8ibN SbL2T+t60gbjulQ1xIIekPRS3LF1JyHgYNLIs+bcKO+B5+okcy2SYFup5bW85HfKDQqw Jk5G5y2fIDMrGsGm97BTzU3VHEq/Xy8TizJgJzs54ZKX4yo9LjqEn/R6lQobxmo3a8Ph daVhqZjbHSiOAw+ZfnjOIkP/iX9mKejdFDrxkhi8fPl6aXoxTdT7XJ1UgmbOmd61pWq1 MdiQ== X-Gm-Message-State: AAQBX9eI6arRVnflHzw+bcX58q+EvhErPxzui+x6rVCC9pcBKpDktK8E dmc9/IDqHlhgUAfjgDoDh7g= X-Google-Smtp-Source: AKy350b+Sr5xvOzebrB+2yXysTKe2PJbmMwalYWJuBuxadmHJisDl3kPlZ4wOoNmbQfu3yNcA47qyQ== X-Received: by 2002:a05:6a20:734e:b0:ef:e240:b559 with SMTP id v14-20020a056a20734e00b000efe240b559mr21393452pzc.46.1682368418786; Mon, 24 Apr 2023 13:33:38 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:38 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 4/9] block: mq-deadline: Clean up deadline_check_fifo() Date: Mon, 24 Apr 2023 13:33:24 -0700 Message-ID: <20230424203329.2369688-5-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Change the return type of deadline_check_fifo() from 'int' into 'bool'. Use time_is_before_eq_jiffies() instead of time_after_eq(). No functionality has been changed. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig --- block/mq-deadline.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 5839a027e0f0..a016cafa54b3 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -272,21 +272,15 @@ static u32 dd_queued(struct deadline_data *dd, enum dd_prio prio) } /* - * deadline_check_fifo returns 0 if there are no expired requests on the fifo, - * 1 otherwise. Requires !list_empty(&dd->fifo_list[data_dir]) + * deadline_check_fifo returns true if and only if there are expired requests + * in the FIFO list. Requires !list_empty(&dd->fifo_list[data_dir]). */ -static inline int deadline_check_fifo(struct dd_per_prio *per_prio, - enum dd_data_dir data_dir) +static inline bool deadline_check_fifo(struct dd_per_prio *per_prio, + enum dd_data_dir data_dir) { struct request *rq = rq_entry_fifo(per_prio->fifo_list[data_dir].next); - /* - * rq is expired! - */ - if (time_after_eq(jiffies, (unsigned long)rq->fifo_time)) - return 1; - - return 0; + return time_is_before_eq_jiffies((unsigned long)rq->fifo_time); } /* From patchwork Mon Apr 24 20:33:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07EFDC77B7E for ; Mon, 24 Apr 2023 20:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232490AbjDXUdp (ORCPT ); Mon, 24 Apr 2023 16:33:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232620AbjDXUdo (ORCPT ); Mon, 24 Apr 2023 16:33:44 -0400 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB65059C3 for ; Mon, 24 Apr 2023 13:33:40 -0700 (PDT) Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-63b62d2f729so4088227b3a.1 for ; Mon, 24 Apr 2023 13:33:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368420; x=1684960420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/NUV6NV2cIUWphd4beZRDY/+nr6v98+rXMDkVE/hCI8=; b=MR19SMIvLxFeyg6A0a21md8iYGQth0bCQB5HHDsvGrDmf2oWZVBv+fDgazQYPrY87l tX+YU8aRFNPdATXQ/o475tW4FiC4PCv4I1+jGKuapfT48fXpcaRY+kR+MZbNfMM+UaSB OlTLuJZtAlwCpgWJpKjb1cgT7Fd7VfEvUlOrIiFXgQzRX3uvQjEacJ7B/MCSMmc+cv3j 2MLE8/mS7QGE7LEFO9iKHVYXVbqEwq2R6y4guTUnF8q4hcLmJG20wI2lTSSkvgs+H8Xt rPcTDevIgX75L7+N0ClPxPRK+TxsPJhFzYardm2KuuklXT40u0+1DZlVE0wP91XPQLjo w1Vg== X-Gm-Message-State: AAQBX9dap0hiCjTrj4WXSyCKs9zOZuolR462z0eGzMTTZajOXWf2syQt 18xSybtgtuGCEey/R+7GSk0zoe160aw= X-Google-Smtp-Source: AKy350b3s8pHwJFgehdY+tGCM4CQdPkK9dnYen4HbPse/bkh++AxtzDQ5puynyQxrezDkkRRiXwM8g== X-Received: by 2002:a05:6a00:1503:b0:63b:5496:7af5 with SMTP id q3-20020a056a00150300b0063b54967af5mr22009493pfu.1.1682368420060; Mon, 24 Apr 2023 13:33:40 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:39 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 5/9] block: mq-deadline: Simplify deadline_skip_seq_writes() Date: Mon, 24 Apr 2023 13:33:25 -0700 Message-ID: <20230424203329.2369688-6-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Make the deadline_skip_seq_writes() code shorter without changing its functionality. Reviewed-by: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig --- block/mq-deadline.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index a016cafa54b3..6276afede9cd 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -304,14 +304,11 @@ static struct request *deadline_skip_seq_writes(struct deadline_data *dd, struct request *rq) { sector_t pos = blk_rq_pos(rq); - sector_t skipped_sectors = 0; - while (rq) { - if (blk_rq_pos(rq) != pos + skipped_sectors) - break; - skipped_sectors += blk_rq_sectors(rq); + do { + pos += blk_rq_sectors(rq); rq = deadline_latter_request(rq); - } + } while (rq && blk_rq_pos(rq) == pos); return rq; } From patchwork Mon Apr 24 20:33:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F3B7C77B76 for ; Mon, 24 Apr 2023 20:33:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232623AbjDXUdo (ORCPT ); Mon, 24 Apr 2023 16:33:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232622AbjDXUdn (ORCPT ); Mon, 24 Apr 2023 16:33:43 -0400 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E32F159FB for ; Mon, 24 Apr 2023 13:33:41 -0700 (PDT) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-63b5c48ea09so4240972b3a.1 for ; Mon, 24 Apr 2023 13:33:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368421; x=1684960421; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F6DzJfMAQ5ymngf0FJt4uPULJnWE+0z/3E6xDXxnI+c=; b=YFvObPzNMRGBewgRFEPI4OKRpPGX9PWiLK6ziC9uvyGGSOMblpPQ5sBFMrbU9orAbn fxHxlOAVj7b7zU1iub4M2nAHgDGlrYIseo6KlRJhb7hjRMfY6z+yf1bCabDQ50ep35d9 5qVWhHkTDHKrWzyh6Ynr6aduDIb/rAnqpTIXVANqwrsS2fxIePoA2T13+oTL4Zv4oBmG Zq2NJT/z2oxbAgKwu2ucjc8oMoM+fEm0Qmo9bZmfUWmrAMxCTjRMaHt1GSBetyjcb1Qq 2KyXN5CGNbKUJOXOVze7NPm6prLNJlPqMI5DEzPQsRdpLwbotVkBiGfNL1rdtq+K63mn RgFw== X-Gm-Message-State: AAQBX9eLrY1Vj1R+UOVe2iHxz091vd4g/5Bdla7bjQ+c102mgXBrKQWz 0Q9BhDI5fsTCCHA24Dn9Q74= X-Google-Smtp-Source: AKy350Z1CH6IKoipdB8eMGFhca5VMyz8C8gtUOdxMKspdGT0vmfcdabuuu7CpbMJwA0wwa6pQ20tRw== X-Received: by 2002:a05:6a00:a06:b0:63b:5c82:e209 with SMTP id p6-20020a056a000a0600b0063b5c82e209mr24051752pfh.10.1682368421280; Mon, 24 Apr 2023 13:33:41 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:40 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 6/9] block: mq-deadline: Improve deadline_skip_seq_writes() Date: Mon, 24 Apr 2023 13:33:26 -0700 Message-ID: <20230424203329.2369688-7-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Make deadline_skip_seq_writes() do what its name suggests, namely to skip sequential writes. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig --- block/mq-deadline.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 6276afede9cd..dbc0feca963e 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -308,7 +308,7 @@ static struct request *deadline_skip_seq_writes(struct deadline_data *dd, do { pos += blk_rq_sectors(rq); rq = deadline_latter_request(rq); - } while (rq && blk_rq_pos(rq) == pos); + } while (rq && blk_rq_pos(rq) == pos && blk_rq_is_seq_zoned_write(rq)); return rq; } From patchwork Mon Apr 24 20:33:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E01FC77B61 for ; Mon, 24 Apr 2023 20:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232620AbjDXUdq (ORCPT ); Mon, 24 Apr 2023 16:33:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230430AbjDXUdo (ORCPT ); Mon, 24 Apr 2023 16:33:44 -0400 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36CDC55AE for ; Mon, 24 Apr 2023 13:33:43 -0700 (PDT) Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-63b50a02bffso4230996b3a.2 for ; Mon, 24 Apr 2023 13:33:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368422; x=1684960422; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ilT4dijW2lWeUw3tyb04PzSIETXm80SktmveCf7Njoo=; b=e+0L0VYgKANGmo75oqYJoHAYgsSyZ1jpsPHgqLXgwbQKC/BpEhZcR+mvzMCmSF/7s6 6VL6lBTY7lzdtjhQOJW4hf4BZAaxTZ7eGfUsw2/CFweNVrL70rG1twRt5u3DAOvac2u6 ZNhRyDnxIQcds0+p8pMHnMeEhH2h96PPJw793jugrREAFUyUxVuLBcE40Zo4Cva6Ia5k Wf+agJa0OO33ImiZzmhDFt4RBYDWwGqG4HwaakUOFD+W1sXC7TB0FEtiE7lwEruc3MXZ FphXeYNt1Hdkkj8IE/8BeZmbux4OvjSrZ5+dsu15kHJNka8VpvMhYbpo2xu4M6tLVAof R4aA== X-Gm-Message-State: AAQBX9dgd0zO6ZsHhIwpKPfCpIDurSdoiD6QsDwWY2eRTd43EXKfNGbK GdRNUBJmXAgsVgQypawXM+DbGVJEBYE= X-Google-Smtp-Source: AKy350ZgYr0ntNWl+NHjbtrkT0NaX4JH/WLl27yXpgpLAZ7pfbSNJNAuQR/xxuy3K1Czb1xKqZy2Og== X-Received: by 2002:a05:6a00:2d0d:b0:63b:8f08:9af3 with SMTP id fa13-20020a056a002d0d00b0063b8f089af3mr21047480pfb.7.1682368422502; Mon, 24 Apr 2023 13:33:42 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:42 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 7/9] block: mq-deadline: Track the dispatch position Date: Mon, 24 Apr 2023 13:33:27 -0700 Message-ID: <20230424203329.2369688-8-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Track the position (sector_t) of the most recently dispatched request instead of tracking a pointer to the next request to dispatch. This patch simplifies the code for inserting and removing a request from the red-black tree. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 45 +++++++++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index dbc0feca963e..bf1dfe9fe9c9 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -74,8 +74,8 @@ struct dd_per_prio { struct list_head dispatch; struct rb_root sort_list[DD_DIR_COUNT]; struct list_head fifo_list[DD_DIR_COUNT]; - /* Next request in FIFO order. Read, write or both are NULL. */ - struct request *next_rq[DD_DIR_COUNT]; + /* Position of the most recently dispatched request. */ + sector_t latest_pos[DD_DIR_COUNT]; struct io_stats_per_prio stats; }; @@ -156,6 +156,25 @@ deadline_latter_request(struct request *rq) return NULL; } +/* Return the first request for which blk_rq_pos() >= pos. */ +static inline struct request *deadline_from_pos(struct dd_per_prio *per_prio, + enum dd_data_dir data_dir, sector_t pos) +{ + struct rb_node *node = per_prio->sort_list[data_dir].rb_node; + struct request *rq, *res = NULL; + + while (node) { + rq = rb_entry_rq(node); + if (blk_rq_pos(rq) >= pos) { + res = rq; + node = node->rb_left; + } else { + node = node->rb_right; + } + } + return res; +} + static void deadline_add_rq_rb(struct dd_per_prio *per_prio, struct request *rq) { @@ -167,11 +186,6 @@ deadline_add_rq_rb(struct dd_per_prio *per_prio, struct request *rq) static inline void deadline_del_rq_rb(struct dd_per_prio *per_prio, struct request *rq) { - const enum dd_data_dir data_dir = rq_data_dir(rq); - - if (per_prio->next_rq[data_dir] == rq) - per_prio->next_rq[data_dir] = deadline_latter_request(rq); - elv_rb_del(deadline_rb_root(per_prio, rq), rq); } @@ -251,10 +265,6 @@ static void deadline_move_request(struct deadline_data *dd, struct dd_per_prio *per_prio, struct request *rq) { - const enum dd_data_dir data_dir = rq_data_dir(rq); - - per_prio->next_rq[data_dir] = deadline_latter_request(rq); - /* * take it off the sort and fifo list */ @@ -363,7 +373,8 @@ deadline_next_request(struct deadline_data *dd, struct dd_per_prio *per_prio, struct request *rq; unsigned long flags; - rq = per_prio->next_rq[data_dir]; + rq = deadline_from_pos(per_prio, data_dir, + per_prio->latest_pos[data_dir]); if (!rq) return NULL; @@ -426,6 +437,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, if (started_after(dd, rq, latest_start)) return NULL; list_del_init(&rq->queuelist); + data_dir = rq_data_dir(rq); goto done; } @@ -433,9 +445,11 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, * batches are currently reads XOR writes */ rq = deadline_next_request(dd, per_prio, dd->last_dir); - if (rq && dd->batching < dd->fifo_batch) + if (rq && dd->batching < dd->fifo_batch) { /* we have a next request are still entitled to batch */ + data_dir = rq_data_dir(rq); goto dispatch_request; + } /* * at this point we are not running a batch. select the appropriate @@ -513,6 +527,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, done: ioprio_class = dd_rq_ioclass(rq); prio = ioprio_class_to_prio[ioprio_class]; + dd->per_prio[prio].latest_pos[data_dir] = blk_rq_pos(rq); dd->per_prio[prio].stats.dispatched++; /* * If the request needs its target zone locked, do it. @@ -1026,8 +1041,10 @@ static int deadline_##name##_next_rq_show(void *data, \ struct request_queue *q = data; \ struct deadline_data *dd = q->elevator->elevator_data; \ struct dd_per_prio *per_prio = &dd->per_prio[prio]; \ - struct request *rq = per_prio->next_rq[data_dir]; \ + struct request *rq; \ \ + rq = deadline_from_pos(per_prio, data_dir, \ + per_prio->latest_pos[data_dir]); \ if (rq) \ __blk_mq_debugfs_rq_show(m, rq); \ return 0; \ From patchwork Mon Apr 24 20:33:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A984AC7618E for ; Mon, 24 Apr 2023 20:33:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232484AbjDXUdr (ORCPT ); Mon, 24 Apr 2023 16:33:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232614AbjDXUdo (ORCPT ); Mon, 24 Apr 2023 16:33:44 -0400 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 218F759D5 for ; Mon, 24 Apr 2023 13:33:44 -0700 (PDT) Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-63b73203e0aso30790583b3a.1 for ; Mon, 24 Apr 2023 13:33:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368424; x=1684960424; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ycm0II5V2vvvBo4OxocwgHymfj0kMbZbh+LKQrfKdNo=; b=eoeB/Zl3scxgV9D/1U4CRXS7+7qjQFx/PkIgRmikwzDNHMa6KcHiiJwf3l/1Uq074C Scvr1F9SuoKS6IivxuwlH2A95TgKeCopGj0nPAUsXuh+Hu/aH4jSxCUNJG8B4ScCH4f2 TKCmAKGz3gZhdKfIVbu5xCSz6SRTe+D8qNtPbbABJs6lgPe5SvpnATA6hLAByElMk0so JkEhjOPdiO3yY/AO4mM5xYkekPZJJwh/i0JGjq5SyI/lsAXlvmW5M6VoMAUEyE3b/TCt xsrKBHO9+sf7PVBPatLQJe7VOaWYBE645LhM9BXw1DhjO/5HsG3htbiFgnVFhyHExT1I cl/Q== X-Gm-Message-State: AAQBX9d2ucMwYoPHX8Dq4l2X8VyC3AJYWWtDV72UPlyw71cNTIFojaLr TUMoS/KYbyKw7B8K/Ifb9BQ= X-Google-Smtp-Source: AKy350Y1W8kO3Ksiqnn+cV2GCVBc3BvYKZ1O3DACYM/7ud5kSNA1CrWNfOzskOKYr69xLiAYfSclLQ== X-Received: by 2002:a05:6a20:3d17:b0:ee:4210:6ca with SMTP id y23-20020a056a203d1700b000ee421006camr17742735pzi.7.1682368423698; Mon, 24 Apr 2023 13:33:43 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:43 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 8/9] block: mq-deadline: Handle requeued requests correctly Date: Mon, 24 Apr 2023 13:33:28 -0700 Message-ID: <20230424203329.2369688-9-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Start dispatching from the start of a zone instead of from the starting position of the most recently dispatched request. If a zoned write is requeued with an LBA that is lower than already inserted zoned writes, make sure that it is submitted first. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index bf1dfe9fe9c9..2de6c63190d8 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -156,13 +156,23 @@ deadline_latter_request(struct request *rq) return NULL; } -/* Return the first request for which blk_rq_pos() >= pos. */ +/* + * Return the first request for which blk_rq_pos() >= @pos. For zoned devices, + * return the first request after the highest zone start <= @pos. + */ static inline struct request *deadline_from_pos(struct dd_per_prio *per_prio, enum dd_data_dir data_dir, sector_t pos) { struct rb_node *node = per_prio->sort_list[data_dir].rb_node; struct request *rq, *res = NULL; + if (!node) + return NULL; + + rq = rb_entry_rq(node); + if (blk_rq_is_seq_zoned_write(rq)) + pos -= bdev_offset_from_zone_start(rq->q->disk->part0, pos); + while (node) { rq = rb_entry_rq(node); if (blk_rq_pos(rq) >= pos) { @@ -809,6 +819,8 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, list_add(&rq->queuelist, &per_prio->dispatch); rq->fifo_time = jiffies; } else { + struct list_head *insert_before; + deadline_add_rq_rb(per_prio, rq); if (rq_mergeable(rq)) { @@ -821,7 +833,16 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, * set expire time and add to fifo list */ rq->fifo_time = jiffies + dd->fifo_expire[data_dir]; - list_add_tail(&rq->queuelist, &per_prio->fifo_list[data_dir]); + insert_before = &per_prio->fifo_list[data_dir]; +#ifdef CONFIG_BLK_DEV_ZONED + if (blk_rq_is_seq_zoned_write(rq)) { + struct request *rq2 = deadline_latter_request(rq); + + if (rq2 && blk_rq_zone_no(rq2) == blk_rq_zone_no(rq)) + insert_before = &rq2->queuelist; + } +#endif + list_add_tail(&rq->queuelist, insert_before); } } From patchwork Mon Apr 24 20:33:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13222574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F27FC77B61 for ; Mon, 24 Apr 2023 20:33:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232617AbjDXUds (ORCPT ); Mon, 24 Apr 2023 16:33:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230430AbjDXUdr (ORCPT ); Mon, 24 Apr 2023 16:33:47 -0400 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78E4059D5 for ; Mon, 24 Apr 2023 13:33:45 -0700 (PDT) Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-63f273b219eso2031450b3a.1 for ; Mon, 24 Apr 2023 13:33:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682368425; x=1684960425; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x8a5scYpTbdax7m3bT5pD+twXlnJVENiNHaccxcBHPg=; b=CIxukGrL0nI601nKY1sXM7B+VydG4v7gE0ZbNshJRGfpqwom8IzbsdN9qcOvlSZqE9 QVbF2KecmozTNItIQmQ1ZiE0FXIC2B4Cvl1TNpnPlsO5Y8eauWAIOzxW3ONPpIt6TA5s jZsT9XOwTg6Mwms+XQlY4m88LqsJfJEHFhn0C3t++JDQVKT36cWL6SZ6ehunrJyAFEvm tiVkmOTehQFf83ZSIOXTrym0LM+L33zm3HpEuXVePugacnTxbQLlkHmOhpvJQAbmpDn6 D+O1PcLos2xYWR2AT6dYbZIUEzAmqYjB+7bnw8RQIkEgV+dw+15zAnV4KeUGwFoDYL0R eQ/A== X-Gm-Message-State: AAQBX9e+gzhNBC+SW2XB/6cXn4hUKsNeDTuenUemWm3h2uZnvNeMdEEB td15/aGoab6NCA5iuER9+wQ= X-Google-Smtp-Source: AKy350YI4i8hnlHKpIAE3Bq9gEnAE0k5mMIurjE0EI10DS/bnqjvl25j4aqHLckZy4THojRuSwUPfw== X-Received: by 2002:a05:6a21:3286:b0:ef:b897:5b0a with SMTP id yt6-20020a056a21328600b000efb8975b0amr19755596pzb.19.1682368424916; Mon, 24 Apr 2023 13:33:44 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:56cb:b39a:c8b:8c37]) by smtp.gmail.com with ESMTPSA id k16-20020aa788d0000000b00625616f59a1sm7417505pff.73.2023.04.24.13.33.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 13:33:44 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v3 9/9] block: mq-deadline: Fix handling of at-head zoned writes Date: Mon, 24 Apr 2023 13:33:29 -0700 Message-ID: <20230424203329.2369688-10-bvanassche@acm.org> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog In-Reply-To: <20230424203329.2369688-1-bvanassche@acm.org> References: <20230424203329.2369688-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Before dispatching a zoned write from the FIFO list, check whether there are any zoned writes in the RB-tree for a lower LBA for the same zone. This patch ensures that zoned writes happen in order even if at_head is set for some writes for a zone and not for others. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 2de6c63190d8..ed0da04c51ec 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -341,7 +341,7 @@ static struct request * deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, enum dd_data_dir data_dir) { - struct request *rq; + struct request *rq, *rb_rq, *next; unsigned long flags; if (list_empty(&per_prio->fifo_list[data_dir])) @@ -353,13 +353,19 @@ deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, /* * Look for a write request that can be dispatched, that is one with - * an unlocked target zone. For some HDDs, breaking a sequential - * write stream can lead to lower throughput, so make sure to preserve - * sequential write streams, even if that stream crosses into the next - * zones and these zones are unlocked. + * an unlocked target zone. For each write request from the FIFO list, + * check whether an earlier write request exists in the RB tree. For + * some HDDs, breaking a sequential write stream can lead to lower + * throughput, so make sure to preserve sequential write streams, even + * if that stream crosses into the next zones and these zones are + * unlocked. */ spin_lock_irqsave(&dd->zone_lock, flags); - list_for_each_entry(rq, &per_prio->fifo_list[DD_WRITE], queuelist) { + list_for_each_entry_safe(rq, next, &per_prio->fifo_list[DD_WRITE], + queuelist) { + rb_rq = deadline_from_pos(per_prio, data_dir, blk_rq_pos(rq)); + if (rb_rq && blk_rq_pos(rb_rq) < blk_rq_pos(rq)) + rq = rb_rq; if (blk_req_can_dispatch_to_zone(rq) && (blk_queue_nonrot(rq->q) || !deadline_is_seq_write(dd, rq)))