From patchwork Tue May 16 22:33:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243940 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 446CFC7EE23 for ; Tue, 16 May 2023 22:33:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229678AbjEPWdb (ORCPT ); Tue, 16 May 2023 18:33:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229556AbjEPWda (ORCPT ); Tue, 16 May 2023 18:33:30 -0400 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A99245FCF for ; Tue, 16 May 2023 15:33:29 -0700 (PDT) Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1aad5245632so1386215ad.3 for ; Tue, 16 May 2023 15:33:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276409; x=1686868409; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Meycj9u3JfaVnzaUnu3BPLSoAESUUt4GW23IhKc0tM8=; b=P/O1A28Qjnynqzvfg3mQPe/XpbP/QD1KOK2LO98KxRycYyIMsjFVgCmCVAiCW1Wc3z Mqm0j+xa+NHrrCbbhS1rA1i3695r2YWS0ArI6VaQ2HYaeb6OH8WtHrwmojDjsKJpeozE GqppYZGa1rlz2snHFeT6VChn6vmllTayrGsdh0NvbiiM0YarXXaqWzXi7q7XHv2IvPVm a7EsBHhz4Xk4cDCtayBwpTGdwwRaD79zjsv4R9gI3PG/mB7T+G0FTM0QzxoX5EoISfcJ LB0HpkbG+9dCXtIP2vRBo3oGUNpb6CsGVHIBVp3ql+KILCOWlP/jfX8+n/6oK65pjO7h Qe2A== X-Gm-Message-State: AC+VfDyGHv8JjahVfrXNvoz8bsuUYp1Fvc+h9IvUUyq8phrmhlKagLMv 5x1yrl4uYho1ldQAICNYtt1wJrKnbj4= X-Google-Smtp-Source: ACHHUZ6LHCr6ZwFWNTQjkRstZ5Jro1YbUmD9kAc0Se55zORWO3TMiJtQwIHximvuFu0kpomW/i9rgg== X-Received: by 2002:a17:903:22cd:b0:1a2:19c1:a974 with SMTP id y13-20020a17090322cd00b001a219c1a974mr46891606plg.68.1684276409051; Tue, 16 May 2023 15:33:29 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:28 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 01/11] block: Simplify blk_req_needs_zone_write_lock() Date: Tue, 16 May 2023 15:33:10 -0700 Message-ID: <20230516223323.1383342-2-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Remove the blk_rq_is_passthrough() check because it is redundant: blk_req_needs_zone_write_lock() also calls bdev_op_is_zoned_write() and the latter function returns false for pass-through requests. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke Reviewed-by: Johannes Thumshirn --- block/blk-zoned.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index fce9082384d6..835d9e937d4d 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -57,9 +57,6 @@ EXPORT_SYMBOL_GPL(blk_zone_cond_str); */ bool blk_req_needs_zone_write_lock(struct request *rq) { - if (blk_rq_is_passthrough(rq)) - return false; - if (!rq->q->disk->seq_zones_wlock) return false; From patchwork Tue May 16 22:33:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1357DC77B75 for ; Tue, 16 May 2023 22:33:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229879AbjEPWdc (ORCPT ); Tue, 16 May 2023 18:33:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229818AbjEPWdb (ORCPT ); Tue, 16 May 2023 18:33:31 -0400 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09E693A80 for ; Tue, 16 May 2023 15:33:31 -0700 (PDT) Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1aaf21bb42bso1437555ad.2 for ; Tue, 16 May 2023 15:33:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276410; x=1686868410; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9MJbfIY4s3b5GVikbVG1xJxZkHbW16AzSAn+W99G4rY=; b=eytDAL+uw7+1WutU9gR59ujZ2Xu7vFxXO8v5IeWBjK4s272d/cwhjXSCTMQr3QEJv7 TNGIuSMZL9HCgFedyCQKJZFv7BB+s6aXboRJsUB2L83wRAvVzNJfUoa1LUTSlPuSjnht oX7Ngd4WNDa9zAY6PXBoEk9eoNNiWnADGX3lWNnfP0qtEDJR/pCgsB+jHuvB2buNm9Cg wtBDLdId8yB8VTIxg+qNnPpLmYnG4876tJHmORWzkkro1z7ACorPcR+Rv/vf7nuoSjNy DuqkXAIgZBAtQ7UxJWwKwp1S96D0ky6JwfBj11ZsYjC3W49svjl1fTlD9aGNFkUmPetj Yy0A== X-Gm-Message-State: AC+VfDydKZOtp/rr8P10MKvoaHa3Qf/jEjo9QLj6fSkkt48B1jBnF6Z8 1knQdpcAZKY6fpM2DQMCg5o= X-Google-Smtp-Source: ACHHUZ7lVUtRDE2gpo0joCTbdV8ntpwVRq9a6P0l6WiC7I6TqyP3WfkkE/kTEq3aAfOE7qPsW5OJMg== X-Received: by 2002:a17:903:1249:b0:1ab:220a:9068 with SMTP id u9-20020a170903124900b001ab220a9068mr51204015plh.42.1684276410412; Tue, 16 May 2023 15:33:30 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:30 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Johannes Thumshirn , Pankaj Raghav , Damien Le Moal , Ming Lei Subject: [PATCH v5 02/11] block: Fix the type of the second bdev_op_is_zoned_write() argument Date: Tue, 16 May 2023 15:33:11 -0700 Message-ID: <20230516223323.1383342-3-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Change the type of the second argument of bdev_op_is_zoned_write() from blk_opf_t into enum req_op because this function expects an operation without flags as second argument. Reviewed-by: Johannes Thumshirn Reviewed-by: Pankaj Raghav Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Fixes: 8cafdb5ab94c ("block: adapt blk_mq_plug() to not plug for writes that require a zone lock") Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke --- include/linux/blkdev.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index b441e633f4dd..db24cf98ccfb 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1282,7 +1282,7 @@ static inline unsigned int bdev_zone_no(struct block_device *bdev, sector_t sec) } static inline bool bdev_op_is_zoned_write(struct block_device *bdev, - blk_opf_t op) + enum req_op op) { if (!bdev_is_zoned(bdev)) return false; From patchwork Tue May 16 22:33:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243942 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C6AC7EE23 for ; Tue, 16 May 2023 22:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229881AbjEPWde (ORCPT ); Tue, 16 May 2023 18:33:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229514AbjEPWdd (ORCPT ); Tue, 16 May 2023 18:33:33 -0400 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4BC7A6189 for ; Tue, 16 May 2023 15:33:32 -0700 (PDT) Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1aaf2ede38fso2055125ad.2 for ; Tue, 16 May 2023 15:33:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276412; x=1686868412; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LGoTIX5x/s8cU4eYpFurYc77b8Ta9ocu9yvDP7N1aU8=; b=e3+kc8Ozcin7/PW+upfk8E2gmPlVd51NUjk+FiGS/MfDVCCExN3FgTqNMJO0xfsz7H 37vePXS2SUqcyLPVUiXwHD5OrCZ1L2QQKI+GSDIinjdc7kdVfq1v0j7e31DauQhVF5VG kFhZV5AWU23GmHJdlD8Fq6u2Yz1IGztSWSDBy+19+YXRWXYmxG5nh9CJhmhO+v4TwyOu Cm6GbZ3laYL5ABq04xCOvEpH9NTczkFclMndpWi3y1W6MC2oLWv9iYXx0zyPFfyWK8c3 7Wg/WnfWMkjZqvi2s4GcL9GAFf2fzElUcLMgUApLy8qAxV6w8u9g0fVE+Ig8UiJGliTD 9bYw== X-Gm-Message-State: AC+VfDwk8BmGgpXZqogV/3eII+meJlX3hzB5GKVPOnIQX4rcOTloWVNE Dilwo3kX5cmbkdF58Ob7KkI= X-Google-Smtp-Source: ACHHUZ7fw8/VGw0SsK+6QdEfBJx2x7ejPjaIfTE3QFIp7SDJz74ZoSu0nUXpwkiLnFoSwf1SQwI/rQ== X-Received: by 2002:a17:902:da91:b0:1ae:f37:c1ab with SMTP id j17-20020a170902da9100b001ae0f37c1abmr13435592plx.25.1684276411680; Tue, 16 May 2023 15:33:31 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:31 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 03/11] block: Introduce op_is_zoned_write() Date: Tue, 16 May 2023 15:33:12 -0700 Message-ID: <20230516223323.1383342-4-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce a helper function for checking whether write serialization is required if the operation will be sent to a zoned device. A second caller for op_is_zoned_write() will be introduced in the next patch in this series. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke --- include/linux/blkdev.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index db24cf98ccfb..a4f85781060c 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1281,13 +1281,16 @@ static inline unsigned int bdev_zone_no(struct block_device *bdev, sector_t sec) return disk_zone_no(bdev->bd_disk, sec); } +/* Whether write serialization is required for @op on zoned devices. */ +static inline bool op_is_zoned_write(enum req_op op) +{ + return op == REQ_OP_WRITE || op == REQ_OP_WRITE_ZEROES; +} + static inline bool bdev_op_is_zoned_write(struct block_device *bdev, enum req_op op) { - if (!bdev_is_zoned(bdev)) - return false; - - return op == REQ_OP_WRITE || op == REQ_OP_WRITE_ZEROES; + return bdev_is_zoned(bdev) && op_is_zoned_write(op); } static inline sector_t bdev_zone_sectors(struct block_device *bdev) From patchwork Tue May 16 22:33:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78441C77B7A for ; Tue, 16 May 2023 22:33:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230240AbjEPWdh (ORCPT ); Tue, 16 May 2023 18:33:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229514AbjEPWde (ORCPT ); Tue, 16 May 2023 18:33:34 -0400 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 945C86E8D for ; Tue, 16 May 2023 15:33:33 -0700 (PDT) Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1ae4be0b1f3so1582265ad.0 for ; Tue, 16 May 2023 15:33:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276413; x=1686868413; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wf3oFXc53dbL2Y9+RB2yCOp47YOiZf0uoEqAZxnr7V8=; b=ByQB6PrURpAQntaxL8ZEHsQG8pFzHgoV1TPftC5jc04TxWRjWGSzfYyCX/fTH/J6aU Arvc50E50gSeYkTCPHE8DOcx2z8BXYWSlA1Qqc3BuwRk9fvrr+s3CPCsQkpTYWh+2VYz g6H7+NCJS0p0SI39UfmsfZ1SS1aNLhXqUM3086G40Chm9f4jTh1q3xIhj4qPc63HSpDj fxdc8V/LrxkqEyxIy5RLAbBbjjRoSwHgi7TNGd1TD2mdXwwl6ZPKkXQ4Cz43mws2XZoR I3NVPY8hos2hiVXdic+L60p2RRWHJKXddj+sEdwkpIHNEnhB83KNc4n29gkeLVr4yP7X lv7A== X-Gm-Message-State: AC+VfDxNBi23C3QUYlLHfnzwHOGB6b6/SJDwOLUSeHEyKudm56xmUzww SsW9LQWEiWN/t96Wd9znAl7//2sdpBI= X-Google-Smtp-Source: ACHHUZ4qNFr8s5Y62l/pbKEEd33s2TYuDs0oI+SaHZdUc53nNOIotNZsMfeVe9nyDqu+Q+Aa8JjZCQ== X-Received: by 2002:a17:903:2452:b0:1ad:bc86:851 with SMTP id l18-20020a170903245200b001adbc860851mr29891266pls.45.1684276412959; Tue, 16 May 2023 15:33:32 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:32 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 04/11] block: Introduce blk_rq_is_seq_zoned_write() Date: Tue, 16 May 2023 15:33:13 -0700 Message-ID: <20230516223323.1383342-5-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce the function blk_rq_is_seq_zoned_write(). This function will be used in later patches to preserve the order of zoned writes that require write serialization. This patch includes an optimization: instead of using rq->q->disk->part0->bd_queue to check whether or not the queue is associated with a zoned block device, use rq->q->disk->queue. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke Reviewed-by: Johannes Thumshirn --- block/blk-zoned.c | 5 +---- include/linux/blk-mq.h | 16 ++++++++++++++++ 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 835d9e937d4d..096b6b47561f 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -60,10 +60,7 @@ bool blk_req_needs_zone_write_lock(struct request *rq) if (!rq->q->disk->seq_zones_wlock) return false; - if (bdev_op_is_zoned_write(rq->q->disk->part0, req_op(rq))) - return blk_rq_zone_is_seq(rq); - - return false; + return blk_rq_is_seq_zoned_write(rq); } EXPORT_SYMBOL_GPL(blk_req_needs_zone_write_lock); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 06caacd77ed6..301d72f85486 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1164,6 +1164,17 @@ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) return disk_zone_is_seq(rq->q->disk, blk_rq_pos(rq)); } +/** + * blk_rq_is_seq_zoned_write() - Check if @rq requires write serialization. + * @rq: Request to examine. + * + * Note: REQ_OP_ZONE_APPEND requests do not require serialization. + */ +static inline bool blk_rq_is_seq_zoned_write(struct request *rq) +{ + return op_is_zoned_write(req_op(rq)) && blk_rq_zone_is_seq(rq); +} + bool blk_req_needs_zone_write_lock(struct request *rq); bool blk_req_zone_write_trylock(struct request *rq); void __blk_req_zone_write_lock(struct request *rq); @@ -1194,6 +1205,11 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq) return !blk_req_zone_is_write_locked(rq); } #else /* CONFIG_BLK_DEV_ZONED */ +static inline bool blk_rq_is_seq_zoned_write(struct request *rq) +{ + return false; +} + static inline bool blk_req_needs_zone_write_lock(struct request *rq) { return false; From patchwork Tue May 16 22:33:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243944 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FEE1C77B75 for ; Tue, 16 May 2023 22:33:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230056AbjEPWdi (ORCPT ); Tue, 16 May 2023 18:33:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229966AbjEPWdf (ORCPT ); Tue, 16 May 2023 18:33:35 -0400 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA9815FDC for ; Tue, 16 May 2023 15:33:34 -0700 (PDT) Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-1aaebed5bd6so1491295ad.1 for ; Tue, 16 May 2023 15:33:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276414; x=1686868414; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CxsjF6/Njb1CBZYlUphCM+L4dP+Khele8WXLzm2lN6Q=; b=CanYD2rQalR1i8cFCwQde3MMawsfJkGeKjwyn0rbJESQZ01ullwM0JjSvl37qNiTxI 1g//0vTbwOjM5vtb0RXylKxJNUqzV6ZcXY9cCpxN2RXsWRpHObNTyOb4OkswBUiEYjq8 sI7CdPIMfLgRK1DpjSmLxfWbTqZUBIMUY/veB1urztSwVOEr0FQi0ryURIPJx4kxoxcZ Oh4ooiFpOSYDQKJjYRJwq07P1iCxIfTlKDrUFsBWQwLjFB6r8cNiBGlYIE2H8AMYToyo IFvO1cIlqcMlw81gAJCwiSKdzfO1aA+oAJTIjTw+dPbTxwR8vVqHo59hDCN6MCPbGwTi 6vQA== X-Gm-Message-State: AC+VfDzzd29bZiEHJDmO11/oEY8FesJVwmYp5REdXmSFpHGCx4mpEsu9 9HK9jZz0OzGM0Eax0vGVsOo= X-Google-Smtp-Source: ACHHUZ6IlttNXldUScllloZbVUb7cI+R/A5xHacTilsnUQXqkg0HOe26htG0vBqd8i0Uftn2vRNY5w== X-Received: by 2002:a17:903:1c6:b0:1ac:946e:468e with SMTP id e6-20020a17090301c600b001ac946e468emr36070226plh.57.1684276414207; Tue, 16 May 2023 15:33:34 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:33 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 05/11] block: mq-deadline: Clean up deadline_check_fifo() Date: Tue, 16 May 2023 15:33:14 -0700 Message-ID: <20230516223323.1383342-6-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Change the return type of deadline_check_fifo() from 'int' into 'bool'. Use time_is_before_eq_jiffies() instead of time_after_eq(). No functionality has been changed. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke --- block/mq-deadline.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 5839a027e0f0..a016cafa54b3 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -272,21 +272,15 @@ static u32 dd_queued(struct deadline_data *dd, enum dd_prio prio) } /* - * deadline_check_fifo returns 0 if there are no expired requests on the fifo, - * 1 otherwise. Requires !list_empty(&dd->fifo_list[data_dir]) + * deadline_check_fifo returns true if and only if there are expired requests + * in the FIFO list. Requires !list_empty(&dd->fifo_list[data_dir]). */ -static inline int deadline_check_fifo(struct dd_per_prio *per_prio, - enum dd_data_dir data_dir) +static inline bool deadline_check_fifo(struct dd_per_prio *per_prio, + enum dd_data_dir data_dir) { struct request *rq = rq_entry_fifo(per_prio->fifo_list[data_dir].next); - /* - * rq is expired! - */ - if (time_after_eq(jiffies, (unsigned long)rq->fifo_time)) - return 1; - - return 0; + return time_is_before_eq_jiffies((unsigned long)rq->fifo_time); } /* From patchwork Tue May 16 22:33:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20AD2C7EE24 for ; Tue, 16 May 2023 22:33:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229818AbjEPWdi (ORCPT ); Tue, 16 May 2023 18:33:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230179AbjEPWdh (ORCPT ); Tue, 16 May 2023 18:33:37 -0400 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2210D6189 for ; Tue, 16 May 2023 15:33:36 -0700 (PDT) Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1ab032d9266so2292405ad.0 for ; Tue, 16 May 2023 15:33:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276415; x=1686868415; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kfH3fddu0PSG0m6djX24rpBRUBdl5uuwI2SQRIjlZ2k=; b=Xmf5ofrIco0KKsjkG2yefYzOerQASaG47gHeGKYc8ZpAe5JeJ4jIhg3i5GAlCepUaN mu0T1m6ZJ7nfHSTdgHTabR030bhKwLQQZnx9ATWfgnyiCijkAEeBxJ4Gi7wC2k7lKTtA HELIxL14tLjHzpEFKx6ad7LEPhMmJDAuI5eDrv0pDfTAiN6iGnLYAL2xI6OJ1JyrvV5y CXU6gx6zMVBKTapjzTxjBVmYJchJEu++V8Bb/IFHUloYsDTPGSpl8Z2e7sq9XINuYb12 QaDL573iocFhXcls/EYKzcrBMxUEqCMo/VANQ/omQgskO0sGlj/wBo52u1fxdyrFQiw6 w4Hw== X-Gm-Message-State: AC+VfDxJI4+w4LgysDJol5Ms49DpRWF+PusrZ0X9w3n3doQp6F1id685 FYtvxL/rU01OWrS8tG3e9j0= X-Google-Smtp-Source: ACHHUZ5QorBj7rU2QW5rUVVaQ0p2J7tzMTV6Wi/agny+qZG85hBR1Oogf9aKlW1oz4SQ58Z2VpDg1Q== X-Received: by 2002:a17:902:da84:b0:1ac:5b6b:df4c with SMTP id j4-20020a170902da8400b001ac5b6bdf4cmr45287312plx.69.1684276415510; Tue, 16 May 2023 15:33:35 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:35 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 06/11] block: mq-deadline: Simplify deadline_skip_seq_writes() Date: Tue, 16 May 2023 15:33:15 -0700 Message-ID: <20230516223323.1383342-7-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Make the deadline_skip_seq_writes() code shorter without changing its functionality. Reviewed-by: Damien Le Moal Reviewed-by: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke --- block/mq-deadline.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index a016cafa54b3..6276afede9cd 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -304,14 +304,11 @@ static struct request *deadline_skip_seq_writes(struct deadline_data *dd, struct request *rq) { sector_t pos = blk_rq_pos(rq); - sector_t skipped_sectors = 0; - while (rq) { - if (blk_rq_pos(rq) != pos + skipped_sectors) - break; - skipped_sectors += blk_rq_sectors(rq); + do { + pos += blk_rq_sectors(rq); rq = deadline_latter_request(rq); - } + } while (rq && blk_rq_pos(rq) == pos); return rq; } From patchwork Tue May 16 22:33:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5E79C77B75 for ; Tue, 16 May 2023 22:33:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230358AbjEPWdk (ORCPT ); Tue, 16 May 2023 18:33:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230324AbjEPWdj (ORCPT ); Tue, 16 May 2023 18:33:39 -0400 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6329E6EB3 for ; Tue, 16 May 2023 15:33:37 -0700 (PDT) Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1ae3fe67980so2078645ad.3 for ; Tue, 16 May 2023 15:33:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276417; x=1686868417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YwyPTOfw7CLhqYC/Hv0uFzhwBy5QE+LBN1Uiqc5OArM=; b=FA3x9o8E7yp+W37E/30CyOSyAS24Cx+UnknqP50jtLtbrjYJaR0Gw7NuQq0szj7fh3 n1rQP/jegd6GWJFHai1Ovbq1P1f0gBYT1e9HkA2irqTU1VdTx6KX5zkxpBO0hIRJUtLr SBPawyKMtByN16pm6jC17NPTYkTZiM527rn3eRy4iuZDlKINY6ntbl0a52M40XjYKe5T lGa/7k9wcqgQbaeF6kHJ0ttv23hczrXaP6yICT2tAoz4NktKuo2t/uRj5I5F/mfXPcGf EkMl2MLZH9zy9QWjc0UvX1DME0FKEiPNKFRZFAUxeLnOfoNB81L+VpijxggWpV1YnnC8 wVNw== X-Gm-Message-State: AC+VfDyTtD7iptQsridR5MA3jSTy5RpbRHSqpEooadopeiYF4zG0AIfb Cofl5fiDOV6XhCoguS1ZvoE= X-Google-Smtp-Source: ACHHUZ5gZwV9UXtEGen9F3KbVsj/mGSpUON0h12rpHEowTIbm0HpDFWkWry96yyAor4HVwOKBdlpKg== X-Received: by 2002:a17:902:9897:b0:1ab:253e:6906 with SMTP id s23-20020a170902989700b001ab253e6906mr36360906plp.67.1684276416752; Tue, 16 May 2023 15:33:36 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:36 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 07/11] block: mq-deadline: Improve deadline_skip_seq_writes() Date: Tue, 16 May 2023 15:33:16 -0700 Message-ID: <20230516223323.1383342-8-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Make deadline_skip_seq_writes() do what its name suggests, namely to skip sequential writes. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 6276afede9cd..dbc0feca963e 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -308,7 +308,7 @@ static struct request *deadline_skip_seq_writes(struct deadline_data *dd, do { pos += blk_rq_sectors(rq); rq = deadline_latter_request(rq); - } while (rq && blk_rq_pos(rq) == pos); + } while (rq && blk_rq_pos(rq) == pos && blk_rq_is_seq_zoned_write(rq)); return rq; } From patchwork Tue May 16 22:33:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C39CC77B7A for ; Tue, 16 May 2023 22:33:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229514AbjEPWdn (ORCPT ); Tue, 16 May 2023 18:33:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55732 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230060AbjEPWdl (ORCPT ); Tue, 16 May 2023 18:33:41 -0400 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADB056A65 for ; Tue, 16 May 2023 15:33:38 -0700 (PDT) Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1aaea43def7so1537475ad.2 for ; Tue, 16 May 2023 15:33:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276418; x=1686868418; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rWu1E1H6Cpdlf2SWVyhyPtFI9G5HnS2JOK38qbcbgYY=; b=ex+BpXksq0QY1ruGhKGgJTWymOBq8soCZvbTakCUVGV1WScmUrMaZYaqnYfbgxG4Sr vJiyQpuVi3x8Fso+ocR74KnQQEjQorZ9rfixdKGgJipDxxq7nGxL/+8S67uSTBp4Jex2 utDm7GmgglrFYlf2B7VZ81OK4RJCGHOrJNkApO89giDQOzADFHofjC6w1kvLbYcyHCG+ puDq/D3GJ1qzrnB5BX8YwuKZtnAqVFwftVm+Ug/HFyO8rDqlMbPZWSkrZzkmj0bxPlFF bU408ZYGHBvJHPhGpEUvoMleS8W3LsBE1LhvrR3peMVnM4U5qbiAxoQOJKxKEWSPvmlD UtKg== X-Gm-Message-State: AC+VfDzuOt6GV5I6oENyvJ4x92+XFvbMM1vRmsqV7WUx9xPVRiWIidTw UltoGRz3BWimaZAARgqD/1Y= X-Google-Smtp-Source: ACHHUZ7Oh30UsGrpN3NnHKeJJtybQz++TD/j54NVFPNAviNpaYUvGgRMCgHNl7NERedEgR1UOz08jw== X-Received: by 2002:a17:902:bf4b:b0:1ae:29a8:d6d0 with SMTP id u11-20020a170902bf4b00b001ae29a8d6d0mr4984037pls.59.1684276417952; Tue, 16 May 2023 15:33:37 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:37 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 08/11] block: mq-deadline: Reduce lock contention Date: Tue, 16 May 2023 15:33:17 -0700 Message-ID: <20230516223323.1383342-9-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blk_mq_free_requests() calls dd_finish_request() indirectly. Prevent nested locking of dd->lock and dd->zone_lock by moving the code for freeing requests. Cc: Damien Le Moal Cc: Christoph Hellwig Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- block/mq-deadline.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index dbc0feca963e..06af9c28a3bf 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -757,7 +757,7 @@ static bool dd_bio_merge(struct request_queue *q, struct bio *bio, * add rq to rbtree and fifo */ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, - blk_insert_t flags) + blk_insert_t flags, struct list_head *free) { struct request_queue *q = hctx->queue; struct deadline_data *dd = q->elevator->elevator_data; @@ -766,7 +766,6 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, u8 ioprio_class = IOPRIO_PRIO_CLASS(ioprio); struct dd_per_prio *per_prio; enum dd_prio prio; - LIST_HEAD(free); lockdep_assert_held(&dd->lock); @@ -783,10 +782,8 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, rq->elv.priv[0] = (void *)(uintptr_t)1; } - if (blk_mq_sched_try_insert_merge(q, rq, &free)) { - blk_mq_free_requests(&free); + if (blk_mq_sched_try_insert_merge(q, rq, free)) return; - } trace_block_rq_insert(rq); @@ -819,6 +816,7 @@ static void dd_insert_requests(struct blk_mq_hw_ctx *hctx, { struct request_queue *q = hctx->queue; struct deadline_data *dd = q->elevator->elevator_data; + LIST_HEAD(free); spin_lock(&dd->lock); while (!list_empty(list)) { @@ -826,9 +824,11 @@ static void dd_insert_requests(struct blk_mq_hw_ctx *hctx, rq = list_first_entry(list, struct request, queuelist); list_del_init(&rq->queuelist); - dd_insert_request(hctx, rq, flags); + dd_insert_request(hctx, rq, flags, &free); } spin_unlock(&dd->lock); + + blk_mq_free_requests(&free); } /* Callback from inside blk_mq_rq_ctx_init(). */ From patchwork Tue May 16 22:33:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5167C77B75 for ; Tue, 16 May 2023 22:33:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229750AbjEPWdq (ORCPT ); Tue, 16 May 2023 18:33:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230204AbjEPWdp (ORCPT ); Tue, 16 May 2023 18:33:45 -0400 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35A8872A7 for ; Tue, 16 May 2023 15:33:40 -0700 (PDT) Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1ae50da739dso417835ad.1 for ; Tue, 16 May 2023 15:33:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276419; x=1686868419; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EO/K03PaxOhxpK8taM5yLo5Sd/gHkP9xk7+LydZTQV0=; b=JmcsGeL2mbDoEGV4P1CXKib8U2hAEasbrT6L++ZxB1qNzzsJTEJaAN58m1YXJn9Vs7 eaEsZaGMyqgbvCudurHde1I336VPqNvNzmc/dyrAak9Q0eR8UlPeSYAHd2YfDnUe7hd5 8xnVXiB2EJPTHHO33iwKSaMQRqAfQ9Q7LT/oi6p1loFKaMyqRT2rvcq7JPinrxZ5UHlY Uc/cr9oL6FhKQZzjBqMCDn+ZDKM0AhH5O1PXmRI+e/3oHJH2hZ/7eq3M02UbbbYOyjEN eB4phIuzQYs8FdRlqyosmssbkm2JVVUzR2BQxfFDxHGnUeXb/IN48N3WkSRc+Z7bj2yU vaOQ== X-Gm-Message-State: AC+VfDx2Fg/a10/Yq3xqXJfQdNz7s++0rOEODFyWCZYOhna2RXt/auyJ yrht+STUcZ4gAozm63B390YWagYuuFk= X-Google-Smtp-Source: ACHHUZ6SWlK36rQLQOHms/QNCuHezKiaQ2vYXerRS0m1EQ+As6DUFkD0UJ9bLuszrri/AWLZBez37Q== X-Received: by 2002:a17:903:1247:b0:1ac:3605:97ec with SMTP id u7-20020a170903124700b001ac360597ecmr51261510plh.62.1684276419168; Tue, 16 May 2023 15:33:39 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:38 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 09/11] block: mq-deadline: Track the dispatch position Date: Tue, 16 May 2023 15:33:18 -0700 Message-ID: <20230516223323.1383342-10-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Track the position (sector_t) of the most recently dispatched request instead of tracking a pointer to the next request to dispatch. This patch is the basis for patch "Handle requeued requests correctly". Without this patch it would be significantly more complicated to make sure that zoned writes are dispatched in LBA order per zone. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- block/mq-deadline.c | 45 +++++++++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 06af9c28a3bf..6d0b99042c96 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -74,8 +74,8 @@ struct dd_per_prio { struct list_head dispatch; struct rb_root sort_list[DD_DIR_COUNT]; struct list_head fifo_list[DD_DIR_COUNT]; - /* Next request in FIFO order. Read, write or both are NULL. */ - struct request *next_rq[DD_DIR_COUNT]; + /* Position of the most recently dispatched request. */ + sector_t latest_pos[DD_DIR_COUNT]; struct io_stats_per_prio stats; }; @@ -156,6 +156,25 @@ deadline_latter_request(struct request *rq) return NULL; } +/* Return the first request for which blk_rq_pos() >= pos. */ +static inline struct request *deadline_from_pos(struct dd_per_prio *per_prio, + enum dd_data_dir data_dir, sector_t pos) +{ + struct rb_node *node = per_prio->sort_list[data_dir].rb_node; + struct request *rq, *res = NULL; + + while (node) { + rq = rb_entry_rq(node); + if (blk_rq_pos(rq) >= pos) { + res = rq; + node = node->rb_left; + } else { + node = node->rb_right; + } + } + return res; +} + static void deadline_add_rq_rb(struct dd_per_prio *per_prio, struct request *rq) { @@ -167,11 +186,6 @@ deadline_add_rq_rb(struct dd_per_prio *per_prio, struct request *rq) static inline void deadline_del_rq_rb(struct dd_per_prio *per_prio, struct request *rq) { - const enum dd_data_dir data_dir = rq_data_dir(rq); - - if (per_prio->next_rq[data_dir] == rq) - per_prio->next_rq[data_dir] = deadline_latter_request(rq); - elv_rb_del(deadline_rb_root(per_prio, rq), rq); } @@ -251,10 +265,6 @@ static void deadline_move_request(struct deadline_data *dd, struct dd_per_prio *per_prio, struct request *rq) { - const enum dd_data_dir data_dir = rq_data_dir(rq); - - per_prio->next_rq[data_dir] = deadline_latter_request(rq); - /* * take it off the sort and fifo list */ @@ -363,7 +373,8 @@ deadline_next_request(struct deadline_data *dd, struct dd_per_prio *per_prio, struct request *rq; unsigned long flags; - rq = per_prio->next_rq[data_dir]; + rq = deadline_from_pos(per_prio, data_dir, + per_prio->latest_pos[data_dir]); if (!rq) return NULL; @@ -426,6 +437,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, if (started_after(dd, rq, latest_start)) return NULL; list_del_init(&rq->queuelist); + data_dir = rq_data_dir(rq); goto done; } @@ -433,9 +445,11 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, * batches are currently reads XOR writes */ rq = deadline_next_request(dd, per_prio, dd->last_dir); - if (rq && dd->batching < dd->fifo_batch) + if (rq && dd->batching < dd->fifo_batch) { /* we have a next request are still entitled to batch */ + data_dir = rq_data_dir(rq); goto dispatch_request; + } /* * at this point we are not running a batch. select the appropriate @@ -513,6 +527,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, done: ioprio_class = dd_rq_ioclass(rq); prio = ioprio_class_to_prio[ioprio_class]; + dd->per_prio[prio].latest_pos[data_dir] = blk_rq_pos(rq); dd->per_prio[prio].stats.dispatched++; /* * If the request needs its target zone locked, do it. @@ -1026,8 +1041,10 @@ static int deadline_##name##_next_rq_show(void *data, \ struct request_queue *q = data; \ struct deadline_data *dd = q->elevator->elevator_data; \ struct dd_per_prio *per_prio = &dd->per_prio[prio]; \ - struct request *rq = per_prio->next_rq[data_dir]; \ + struct request *rq; \ \ + rq = deadline_from_pos(per_prio, data_dir, \ + per_prio->latest_pos[data_dir]); \ if (rq) \ __blk_mq_debugfs_rq_show(m, rq); \ return 0; \ From patchwork Tue May 16 22:33:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30700C7EE23 for ; Tue, 16 May 2023 22:33:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229496AbjEPWdt (ORCPT ); Tue, 16 May 2023 18:33:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229966AbjEPWds (ORCPT ); Tue, 16 May 2023 18:33:48 -0400 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7305C76A2 for ; Tue, 16 May 2023 15:33:41 -0700 (PDT) Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1ae50da739dso417895ad.1 for ; Tue, 16 May 2023 15:33:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276420; x=1686868420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vAl94Tb/uTywl4h7fTUUfHM03u/h0eEBWXhvRspn1G4=; b=R8DEJXfEhQef132J2GGAqgaYzCCFLeubOLY26I3ZqRIiHkvRSdjlAiyTvyQgg0sZLS Cdi6rOuc9+vtpVEbcZ3Y+PDi3Lb72q7VuUCDj19RFAewfGl9lk1sMZkHRbPZNyTrzHjj jHcIkCOPj+9DM4vR0PHZp8n2H+CNUOQbhIypES0FMcdxRIrLuYZ1wu7/xvWQhTo3LvP5 OtR707Jzidexth56bPWa9tqBVHVMZ318wLGJVEo728Gy98MjCPib1XwdCuYG2QcNQKQs kW44iv37PrM94qTsk9pAi4CPUVedJ+AD7vuWmWBzBQxjkoK9J7aFHWsd/B7zfaeCtV4P cJRA== X-Gm-Message-State: AC+VfDzqpYLiKXf+J1/xp2I0nPlTAOMu45rNwPONgsKAtIJYu5olpFM5 XVgSQkxBmSgkXzvd9+CcHqk= X-Google-Smtp-Source: ACHHUZ4oWgG9SuVNN5R3Gt55wgGg8Ha/K6tI6fhx5ADMsW377RkEaiYdom4EPwX7Bub8H0vL4S9ikQ== X-Received: by 2002:a17:902:d48a:b0:1ac:8db3:d4e5 with SMTP id c10-20020a170902d48a00b001ac8db3d4e5mr35719210plg.69.1684276420475; Tue, 16 May 2023 15:33:40 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:40 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 10/11] block: mq-deadline: Handle requeued requests correctly Date: Tue, 16 May 2023 15:33:19 -0700 Message-ID: <20230516223323.1383342-11-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Start dispatching from the start of a zone instead of from the starting position of the most recently dispatched request. If a zoned write is requeued with an LBA that is lower than already inserted zoned writes, make sure that it is submitted first. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Hannes Reinecke --- block/mq-deadline.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 6d0b99042c96..059727fa4b98 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -156,13 +156,28 @@ deadline_latter_request(struct request *rq) return NULL; } -/* Return the first request for which blk_rq_pos() >= pos. */ +/* + * Return the first request for which blk_rq_pos() >= @pos. For zoned devices, + * return the first request after the highest zone start <= @pos. + */ static inline struct request *deadline_from_pos(struct dd_per_prio *per_prio, enum dd_data_dir data_dir, sector_t pos) { struct rb_node *node = per_prio->sort_list[data_dir].rb_node; struct request *rq, *res = NULL; + if (!node) + return NULL; + + rq = rb_entry_rq(node); + /* + * A zoned write may have been requeued with a starting position that + * is below that of the most recently dispatched request. Hence, for + * zoned writes, start searching from the start of a zone. + */ + if (blk_rq_is_seq_zoned_write(rq)) + pos -= round_down(pos, rq->q->limits.chunk_sectors); + while (node) { rq = rb_entry_rq(node); if (blk_rq_pos(rq) >= pos) { @@ -806,6 +821,8 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, list_add(&rq->queuelist, &per_prio->dispatch); rq->fifo_time = jiffies; } else { + struct list_head *insert_before; + deadline_add_rq_rb(per_prio, rq); if (rq_mergeable(rq)) { @@ -818,7 +835,20 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, * set expire time and add to fifo list */ rq->fifo_time = jiffies + dd->fifo_expire[data_dir]; - list_add_tail(&rq->queuelist, &per_prio->fifo_list[data_dir]); + insert_before = &per_prio->fifo_list[data_dir]; +#ifdef CONFIG_BLK_DEV_ZONED + /* + * Insert zoned writes such that requests are sorted by + * position per zone. + */ + if (blk_rq_is_seq_zoned_write(rq)) { + struct request *rq2 = deadline_latter_request(rq); + + if (rq2 && blk_rq_zone_no(rq2) == blk_rq_zone_no(rq)) + insert_before = &rq2->queuelist; + } +#endif + list_add_tail(&rq->queuelist, insert_before); } } From patchwork Tue May 16 22:33:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13243950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6D5C77B75 for ; Tue, 16 May 2023 22:33:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230204AbjEPWdv (ORCPT ); Tue, 16 May 2023 18:33:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229571AbjEPWdu (ORCPT ); Tue, 16 May 2023 18:33:50 -0400 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6418C7292 for ; Tue, 16 May 2023 15:33:42 -0700 (PDT) Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1aaea3909d1so1850175ad.2 for ; Tue, 16 May 2023 15:33:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684276422; x=1686868422; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6JBGozQfRMCbQ5kzLOjh8kmaOBJhZb4ciuk6Ot10DUo=; b=ZgCZv7AE7ejLhWKzfiCFiaHP8TzoGcPSBdhc8RiIBIMUcRoBmCGlDmW3wnbNvp5g0G TwKDbBNW0XIf5uG6tcgmgJ5152LjLHG+XGq9N9F8GV2XyVh8oT1x7tYvKyTXbXrC9Ij6 fY8+mRRjiWMsNJ9Zji2S7Z3ff7uRD/fwvEDFv16yPzrAJtFIvkiBBiVBNJXpT8Ajdw5Y GL62j1fPSRj+IYkiDbVds92xoUcjpVQQidS4tFx0w5GpVKI7XeXgTix2OluyY932OBaR fGr87PWG0Y3dcdfMNjzHrjpTiACo0Le3TQkLqq+DYPBAIA7XOpiiE2nr132BvYUM9ddN 8t/w== X-Gm-Message-State: AC+VfDxnsSTkd3U9+WjCrDA8scc5HvwqJuaMb/dPklN/HaBdbWnOzZM/ JcGNCq5Fqxr9eKAVWiFLwoQTOyb44jc= X-Google-Smtp-Source: ACHHUZ7HnHYHqLvH0JkIOA3a49YoCkFBS8sxEPeoT9ZkkpPJXPpeYt+kxt0Zion7apkfqpfVDuiTYg== X-Received: by 2002:a17:903:234e:b0:1ac:a9c1:b61d with SMTP id c14-20020a170903234e00b001aca9c1b61dmr32287226plh.11.1684276421769; Tue, 16 May 2023 15:33:41 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:baed:ee38:35e4:f97d]) by smtp.gmail.com with ESMTPSA id d8-20020a170902654800b001ae48d441desm839255pln.148.2023.05.16.15.33.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 15:33:41 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v5 11/11] block: mq-deadline: Fix handling of at-head zoned writes Date: Tue, 16 May 2023 15:33:20 -0700 Message-ID: <20230516223323.1383342-12-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.606.ga4b1b128d6-goog In-Reply-To: <20230516223323.1383342-1-bvanassche@acm.org> References: <20230516223323.1383342-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Before dispatching a zoned write from the FIFO list, check whether there are any zoned writes in the RB-tree with a lower LBA for the same zone. This patch ensures that zoned writes happen in order even if at_head is set for some writes for a zone and not for others. Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- block/mq-deadline.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 059727fa4b98..67989f8d29a5 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -346,7 +346,7 @@ static struct request * deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, enum dd_data_dir data_dir) { - struct request *rq; + struct request *rq, *rb_rq, *next; unsigned long flags; if (list_empty(&per_prio->fifo_list[data_dir])) @@ -364,7 +364,12 @@ deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, * zones and these zones are unlocked. */ spin_lock_irqsave(&dd->zone_lock, flags); - list_for_each_entry(rq, &per_prio->fifo_list[DD_WRITE], queuelist) { + list_for_each_entry_safe(rq, next, &per_prio->fifo_list[DD_WRITE], + queuelist) { + /* Check whether a prior request exists for the same zone. */ + rb_rq = deadline_from_pos(per_prio, data_dir, blk_rq_pos(rq)); + if (rb_rq && blk_rq_pos(rb_rq) < blk_rq_pos(rq)) + rq = rb_rq; if (blk_req_can_dispatch_to_zone(rq) && (blk_queue_nonrot(rq->q) || !deadline_is_seq_write(dd, rq)))