From patchwork Wed May 17 17:42:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B50C6C7EE2E for ; Wed, 17 May 2023 17:45:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229937AbjEQRpi (ORCPT ); Wed, 17 May 2023 13:45:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229489AbjEQRpe (ORCPT ); Wed, 17 May 2023 13:45:34 -0400 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 283CC1FE6 for ; Wed, 17 May 2023 10:45:05 -0700 (PDT) Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6436e004954so1095818b3a.0 for ; Wed, 17 May 2023 10:45:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345471; x=1686937471; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A5JRbS3GKyvmHNAncqIZ0qk+UzhMY/Qxetj+preyd2A=; b=RPSAGyrhPsE8qxZjxXl7tjzsTJndse2SvV9FbR5iQh49iJDo9MsmJlWMSw4uNqsyYy 0PMHwfUYdZjhMN91hmhojoH5oVFNU9bzBnRQ6y48VSBOrixMgs+bVNHe513MKH7e1JSL HfXFtouSvFOnuuXASRAi/dOivw8ODEYAi3UjwzjDAWsR7Dd/wo+4YlNIc1SXdL5BN8QC t/ns16Iq71m7M7dEuvEaxyAb9sVQ2pX23XY2KkO3PlVi6QJWxrP+gJvI+0fxtLxRek7r zR6VjS0P20sm4hvMOZmJSh1UsNrHYqposcqzf//swRlWG72rUIp2O5nna3O8MK3GXqs4 +JdA== X-Gm-Message-State: AC+VfDwzOwf7/G0BmExjOSwhTC6+ZcigGD91lkEIycpL61GYw1B4/HWf 0xIZYqC5+cGOjKtYoZwvTJw= X-Google-Smtp-Source: ACHHUZ4J7NjylBaTKfLrDwxq+PCWQ/k9r9FNhKaNg3NYIXpXsuRYxz3Lsh5QT7FhNfGJ3+TLC+gM/Q== X-Received: by 2002:a05:6a00:14c9:b0:646:2edb:a23 with SMTP id w9-20020a056a0014c900b006462edb0a23mr750469pfu.1.1684345470982; Wed, 17 May 2023 10:44:30 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:30 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Omar Sandoval Subject: [PATCH v6 01/11] block: mq-deadline: Add a word in a source code comment Date: Wed, 17 May 2023 10:42:19 -0700 Message-ID: <20230517174230.897144-2-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add the missing word "and". Cc: Damien Le Moal Suggested-by: Damien Le Moal Fixes: 945ffb60c11d ("mq-deadline: add blk-mq adaptation of the deadline IO scheduler") Signed-off-by: Bart Van Assche Tested-by: Damien Le Moal --- block/mq-deadline.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 5839a027e0f0..cea1b084c69e 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -443,7 +443,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, */ rq = deadline_next_request(dd, per_prio, dd->last_dir); if (rq && dd->batching < dd->fifo_batch) - /* we have a next request are still entitled to batch */ + /* we have a next request and are still entitled to batch */ goto dispatch_request; /* From patchwork Wed May 17 17:42:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69BB1C77B7F for ; Wed, 17 May 2023 17:45:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229601AbjEQRpg (ORCPT ); Wed, 17 May 2023 13:45:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229520AbjEQRpc (ORCPT ); Wed, 17 May 2023 13:45:32 -0400 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43B47AD32 for ; Wed, 17 May 2023 10:45:01 -0700 (PDT) Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-643465067d1so835480b3a.0 for ; Wed, 17 May 2023 10:45:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345472; x=1686937472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O3LBuBNIBav3dorAz0aTtKrVDkeCIbggzsfXX9pDkGI=; b=SN18Xo4dZzS4di4QjhjZgJeuEN78tqdRXI0znMkEgHZgDtBNZlO9/U1Nu2+Gm3dy9Q U7ZdmZ8hh1/ASIVQahntj7gctRBE/5xKPrIPBS0ON32JSYMuxQZl1Dfw1t6nv5b8K6uq RZQF6+YMZjiMgD1DvAR7YYQPW3pIjz+dH8AYmPMcZ7J6bzgOcXl4ieLp6yJqhCiAMDTS zSILaB8c7KSZzO3L3sa52jMx2TrrVvpJr38qlzJwf4aHQehIB8pE/ItukggWgtrPdRFL fuN4flMDZHvvwyM17CtlD1vp+F2kOai9/opzzqXfwgJPNwTVxJgKB3EiA8aRAR+cQ67j 4xAA== X-Gm-Message-State: AC+VfDxpxEvxNLTBMev//AGHc9UXAt+m0ugL8YbaykXAYG1tBcbBtCqY im3x1+XDELgDT9Xgyg88+ec= X-Google-Smtp-Source: ACHHUZ7lIMa/tihI4ODOWbQDaIGekQ1wf8tLslyCeJPJJyKQv7a+SFfdmyb3XkqrWa1EFpECgeyUrw== X-Received: by 2002:a05:6a00:1943:b0:63d:3981:313d with SMTP id s3-20020a056a00194300b0063d3981313dmr624829pfk.10.1684345472380; Wed, 17 May 2023 10:44:32 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:31 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Hannes Reinecke , Johannes Thumshirn , Ming Lei Subject: [PATCH v6 02/11] block: Simplify blk_req_needs_zone_write_lock() Date: Wed, 17 May 2023 10:42:20 -0700 Message-ID: <20230517174230.897144-3-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Remove the blk_rq_is_passthrough() check because it is redundant: blk_req_needs_zone_write_lock() also calls bdev_op_is_zoned_write() and the latter function returns false for pass-through requests. Reviewed-by: Christoph Hellwig Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke Reviewed-by: Johannes Thumshirn Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/blk-zoned.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index fce9082384d6..835d9e937d4d 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -57,9 +57,6 @@ EXPORT_SYMBOL_GPL(blk_zone_cond_str); */ bool blk_req_needs_zone_write_lock(struct request *rq) { - if (blk_rq_is_passthrough(rq)) - return false; - if (!rq->q->disk->seq_zones_wlock) return false; From patchwork Wed May 17 17:42:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F04AC77B7F for ; Wed, 17 May 2023 17:45:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229521AbjEQRpl (ORCPT ); Wed, 17 May 2023 13:45:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229903AbjEQRph (ORCPT ); Wed, 17 May 2023 13:45:37 -0400 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 966AFAD20 for ; Wed, 17 May 2023 10:45:08 -0700 (PDT) Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-6439f186366so779061b3a.2 for ; Wed, 17 May 2023 10:45:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345474; x=1686937474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gy/YzMXYBbHQ6xn4xkeaIqLzTZzU9fXUdvN/JMs2Vas=; b=hKpVAD0SUtL4qLJLrzbWx2Fazrlffoib1pc5hLRIz3yHb/6vS1rmDcBuSXVqM6W5YP IhQf+THU81hqCBQ/ZliAx5+wjOqq+K0uoaY8rn677DZ3+cvhUpO+tJwd4FZxoIBZcuvC X5SrcQzldRkAznFvwNWTUk357uZjt0997mKYK+x6fOrIyHazkpB8r1aucDuOnTcNXEfk UQG+fr9ySgbY1NFvjxwKBnDasdUBQUDOQKgBBlKEocLoEXDQb9+ww2wvfy0wdUiSw21g i8BusqL54M9ZiiZf3x7fVP7bjycvdyjaGBPMO1aeZULUmw9gD5UUBCbXfsRlytkjn2eT tVHw== X-Gm-Message-State: AC+VfDxmdZ0uvm/NfT1VNDW2KRmBnGCOYfV9CilpAS+FpEubBzXJVgnf NLYR8M6umjbiuB4B9fdK8EA= X-Google-Smtp-Source: ACHHUZ7geWLB0MlA+iDGypy54HwYLGSkMVeHzSGC0qhxXaz1j/Z6ZnaUvFtuZ4GozilxvVmJVabNgw== X-Received: by 2002:a05:6a00:138f:b0:64a:a1ba:50fd with SMTP id t15-20020a056a00138f00b0064aa1ba50fdmr610590pfg.22.1684345473991; Wed, 17 May 2023 10:44:33 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:33 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Johannes Thumshirn , Pankaj Raghav , Damien Le Moal , Hannes Reinecke , Ming Lei Subject: [PATCH v6 03/11] block: Fix the type of the second bdev_op_is_zoned_write() argument Date: Wed, 17 May 2023 10:42:21 -0700 Message-ID: <20230517174230.897144-4-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Change the type of the second argument of bdev_op_is_zoned_write() from blk_opf_t into enum req_op because this function expects an operation without flags as second argument. Reviewed-by: Johannes Thumshirn Reviewed-by: Pankaj Raghav Reviewed-by: Christoph Hellwig Reviewed-by: Damien Le Moal Reviewed-by: Hannes Reinecke Cc: Ming Lei Fixes: 8cafdb5ab94c ("block: adapt blk_mq_plug() to not plug for writes that require a zone lock") Signed-off-by: Bart Van Assche --- include/linux/blkdev.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index b441e633f4dd..db24cf98ccfb 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1282,7 +1282,7 @@ static inline unsigned int bdev_zone_no(struct block_device *bdev, sector_t sec) } static inline bool bdev_op_is_zoned_write(struct block_device *bdev, - blk_opf_t op) + enum req_op op) { if (!bdev_is_zoned(bdev)) return false; From patchwork Wed May 17 17:42:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2315FC77B7D for ; Wed, 17 May 2023 17:45:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229814AbjEQRpb (ORCPT ); Wed, 17 May 2023 13:45:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229477AbjEQRpa (ORCPT ); Wed, 17 May 2023 13:45:30 -0400 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2203DC67 for ; Wed, 17 May 2023 10:44:56 -0700 (PDT) Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-643995a47f7so1120608b3a.1 for ; Wed, 17 May 2023 10:44:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345475; x=1686937475; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vPPtJLUNcWWEc/mSEtTotSAvLx2KABsnEcjgBEYWy8I=; b=A3ufuaYibewP3oi6I+JffFeVvxqsZ7tEa0WwVrjfJ7PLcTF2XS/iL0Y9Wwjr+hmebZ CIhHXksfVQSG/sCoA+WHxUlp0kUlhqN6qeBIdtwsOkhmjXaqgw1a2VSH4KVnS4cQAfeC DCae79H/HjRS3gz7VSDComZDL79+2RQY02bNcPHFqroWrz/lWuUJ5PKptjdzvplWF9Fi Q/varWbEgVj7qzMzJ9bqNzSLVPdb9OBZ2nQe/2W1x05dN6DiLhHw8iYCCfgsLLXsCGtt V+J6Vcyr7/UCLBdxpZSauWzzmyC6tA3OlSY/57fdgbpcjWU5J1ddfh1ihXvCm0U8tV5B Vk2g== X-Gm-Message-State: AC+VfDwnpdRAe2xzj4zCbDP4n9NhYEqzY4kfo6KKL/j8lkAB90EDwT1L mPFMBt3LPvCClWSOG6IQ+5RnRlL39dQ= X-Google-Smtp-Source: ACHHUZ7rXVB8dpmsjQzAPgENGHjCqeaNpNez2N/uBPH+ZaLmLMKQK7H/4yaTiAOEvDHQnI2svk8IHw== X-Received: by 2002:a05:6a00:1696:b0:647:7ee8:6248 with SMTP id k22-20020a056a00169600b006477ee86248mr570126pfc.14.1684345475274; Wed, 17 May 2023 10:44:35 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:34 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v6 04/11] block: Introduce op_needs_zoned_write_locking() Date: Wed, 17 May 2023 10:42:22 -0700 Message-ID: <20230517174230.897144-5-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce a helper function for checking whether write serialization is required if the operation will be sent to a zoned device. A second caller for op_needs_zoned_write_locking() will be introduced in the next patch in this series. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- include/linux/blkdev.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index db24cf98ccfb..3952c52d6cd1 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1281,13 +1281,16 @@ static inline unsigned int bdev_zone_no(struct block_device *bdev, sector_t sec) return disk_zone_no(bdev->bd_disk, sec); } +/* Whether write serialization is required for @op on zoned devices. */ +static inline bool op_needs_zoned_write_locking(enum req_op op) +{ + return op == REQ_OP_WRITE || op == REQ_OP_WRITE_ZEROES; +} + static inline bool bdev_op_is_zoned_write(struct block_device *bdev, enum req_op op) { - if (!bdev_is_zoned(bdev)) - return false; - - return op == REQ_OP_WRITE || op == REQ_OP_WRITE_ZEROES; + return bdev_is_zoned(bdev) && op_needs_zoned_write_locking(op); } static inline sector_t bdev_zone_sectors(struct block_device *bdev) From patchwork Wed May 17 17:42:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245426 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F356C77B75 for ; Wed, 17 May 2023 17:45:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229599AbjEQRpd (ORCPT ); Wed, 17 May 2023 13:45:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229789AbjEQRpb (ORCPT ); Wed, 17 May 2023 13:45:31 -0400 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 992965FD0 for ; Wed, 17 May 2023 10:44:58 -0700 (PDT) Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-643995a47f7so1120701b3a.1 for ; Wed, 17 May 2023 10:44:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345477; x=1686937477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=at01WkjHCkf+9Zq8PAzpbm1lmLYzBF4o5syCToTTzgM=; b=Kg9EXEK7vZF/8kqJFzl7Ss1xn4RddAJAavbLYEWs2rSWtrHwk+xzm/WdCK+EpHM+c9 LtB7RnjpL+u15zONZP+ZbVZNMzQaHL23k9UVA5lkPrF1mtlHsJyCnNHYZfi05IrfBLJd OCabpgnccXlyOK8HVW8l6856Nfb3FKc5Ow3jkUr5kIi7Y0Af5ItBsAVe0WXt3gA4UeIB c6lRZ1Zq4BBNjTfP0rjN9lO0vHkYyvFJaWifQGhfad6f81PN15L4+qR6Kv1roWw4BtR/ Iu+n7rsJurlQkkuH3A3NdTeZo+Z2LNgt9qkFvZFKm/vaB2U1GRcz0HnOFL6FnuS90vlR A2Qw== X-Gm-Message-State: AC+VfDzJVZ+lh4ZCfUobdPssk2l1WM29t4J2V/0lMPa5+CcWPRdZ7EOd Fj8yx+TOwq9EecdzeEGEhGE= X-Google-Smtp-Source: ACHHUZ6IO0beZvYlKgRtrx0RC13Cy4nBCOin7GgFoZT8YWX9EmVNvNaF58dHuBVpjcuXABEyb7re3g== X-Received: by 2002:a05:6a20:a11f:b0:106:a78f:be1d with SMTP id q31-20020a056a20a11f00b00106a78fbe1dmr11192501pzk.20.1684345476871; Wed, 17 May 2023 10:44:36 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:36 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v6 05/11] block: Introduce blk_rq_is_seq_zoned_write() Date: Wed, 17 May 2023 10:42:23 -0700 Message-ID: <20230517174230.897144-6-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce the function blk_rq_is_seq_zoned_write(). This function will be used in later patches to preserve the order of zoned writes that require write serialization. This patch includes an optimization: instead of using rq->q->disk->part0->bd_queue to check whether or not the queue is associated with a zoned block device, use rq->q->disk->queue. Cc: Christoph Hellwig Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- block/blk-zoned.c | 5 +---- include/linux/blk-mq.h | 17 +++++++++++++++++ 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 835d9e937d4d..096b6b47561f 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -60,10 +60,7 @@ bool blk_req_needs_zone_write_lock(struct request *rq) if (!rq->q->disk->seq_zones_wlock) return false; - if (bdev_op_is_zoned_write(rq->q->disk->part0, req_op(rq))) - return blk_rq_zone_is_seq(rq); - - return false; + return blk_rq_is_seq_zoned_write(rq); } EXPORT_SYMBOL_GPL(blk_req_needs_zone_write_lock); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 06caacd77ed6..937493249aae 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1164,6 +1164,18 @@ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) return disk_zone_is_seq(rq->q->disk, blk_rq_pos(rq)); } +/** + * blk_rq_is_seq_zoned_write() - Check if @rq requires write serialization. + * @rq: Request to examine. + * + * Note: REQ_OP_ZONE_APPEND requests do not require serialization. + */ +static inline bool blk_rq_is_seq_zoned_write(struct request *rq) +{ + return op_needs_zoned_write_locking(req_op(rq)) && + blk_rq_zone_is_seq(rq); +} + bool blk_req_needs_zone_write_lock(struct request *rq); bool blk_req_zone_write_trylock(struct request *rq); void __blk_req_zone_write_lock(struct request *rq); @@ -1194,6 +1206,11 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq) return !blk_req_zone_is_write_locked(rq); } #else /* CONFIG_BLK_DEV_ZONED */ +static inline bool blk_rq_is_seq_zoned_write(struct request *rq) +{ + return false; +} + static inline bool blk_req_needs_zone_write_lock(struct request *rq) { return false; From patchwork Wed May 17 17:42:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68FFEC7EE2A for ; Wed, 17 May 2023 17:45:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229724AbjEQRpl (ORCPT ); Wed, 17 May 2023 13:45:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229521AbjEQRpg (ORCPT ); Wed, 17 May 2023 13:45:36 -0400 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78F97A275 for ; Wed, 17 May 2023 10:45:06 -0700 (PDT) Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-64cb307d91aso1080267b3a.3 for ; Wed, 17 May 2023 10:45:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345478; x=1686937478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c1Mh8aRLXpq1JQWihv+khZjFLFcjuKpLPc/SIHkC/Ro=; b=Uc6QB004FLWsLM2ucoeJOmk6UKsLLjgUdP52Jrbe1OFhT9i6hvbnZCtdJZ6+CH1mYN SWyS+32qv49bpQhEhWcM4yWHctog9jmjwWS84TPxQw/WXw01ohefUyKUv1K9ECjFFWJp I0r+NTrSaRr+D2oTXCmToB2oO8tlQ1W2jNRJipC3HrVclyizogTgi+LZvnWLDR+Qy+u7 bOpyHxSQVge3W6gjPnIxORQ7THdj0ktUVYW7feN4FnJobhfBEiiIDv14Qu4lZKjcYlf3 RFnVIHNez9nqWEjQTNj5VKje2/NdjYDuHVI2mHfzQnDZ0MdrjfMR7JwcvbChyH6YhljI fIug== X-Gm-Message-State: AC+VfDyoXAWl1DLLEA/TxEWfdiynTVmsd0fXmjESXeeyTbmIbxEx1MGd btbKvbCoJ9Sm4oj0D+nRAig= X-Google-Smtp-Source: ACHHUZ5NBVUMEAdnIfUfe/U5K/xh+AOAl6pB0IE3/rqmMVUBanjOMA+dCLmRrJ7bqpuIlFCfoa28gA== X-Received: by 2002:a05:6a00:138b:b0:64c:ecf7:f4a4 with SMTP id t11-20020a056a00138b00b0064cecf7f4a4mr617684pfg.24.1684345478060; Wed, 17 May 2023 10:44:38 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:37 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Hannes Reinecke , Damien Le Moal , Ming Lei Subject: [PATCH v6 06/11] block: mq-deadline: Clean up deadline_check_fifo() Date: Wed, 17 May 2023 10:42:24 -0700 Message-ID: <20230517174230.897144-7-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Change the return type of deadline_check_fifo() from 'int' into 'bool'. Use time_is_before_eq_jiffies() instead of time_after_eq(). No functionality has been changed. Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- block/mq-deadline.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index cea1b084c69e..cea91ba4a6ea 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -272,21 +272,15 @@ static u32 dd_queued(struct deadline_data *dd, enum dd_prio prio) } /* - * deadline_check_fifo returns 0 if there are no expired requests on the fifo, - * 1 otherwise. Requires !list_empty(&dd->fifo_list[data_dir]) + * deadline_check_fifo returns true if and only if there are expired requests + * in the FIFO list. Requires !list_empty(&dd->fifo_list[data_dir]). */ -static inline int deadline_check_fifo(struct dd_per_prio *per_prio, - enum dd_data_dir data_dir) +static inline bool deadline_check_fifo(struct dd_per_prio *per_prio, + enum dd_data_dir data_dir) { struct request *rq = rq_entry_fifo(per_prio->fifo_list[data_dir].next); - /* - * rq is expired! - */ - if (time_after_eq(jiffies, (unsigned long)rq->fifo_time)) - return 1; - - return 0; + return time_is_before_eq_jiffies((unsigned long)rq->fifo_time); } /* From patchwork Wed May 17 17:42:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B50DC77B75 for ; Wed, 17 May 2023 17:45:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229812AbjEQRpg (ORCPT ); Wed, 17 May 2023 13:45:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229551AbjEQRpd (ORCPT ); Wed, 17 May 2023 13:45:33 -0400 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68251D069 for ; Wed, 17 May 2023 10:45:01 -0700 (PDT) Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-643557840e4so1118723b3a.2 for ; Wed, 17 May 2023 10:45:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345480; x=1686937480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aDnGikLOYG1t6apzhUnxcr96a3tR4AhATdhtBeyLLBM=; b=cRVPa2Uv1ICZsHSL1nw3pU08r4aSrtsZVxOC6N4OjDpsUf0LqwrdxF436l8az5b/mV Rglqj43yWtoUSVi+vs6uZMJvq86ecRlXN69p6kaV6pMKGBK4pSngjO8hQds+6RiKnJu2 fPhpcPnZyH3lTkvNIGLDesnZYzeE72n5b4SIm9Fqdfn8T1hwYkVWSypwQ6PpPoY7DMLt ve6M2Qqbm8vz5txyfz+Jo1obUFGUPrSlQmY+7Q/yRTOGl+78x2miLeeR9DIfbWfkixK6 +MOsfd/X1yqtRl7S0QbIzS9KuUDDXCrCfR2WV0aRf6MOCONF8QVnUElbLphzrPJsaAxe PqdQ== X-Gm-Message-State: AC+VfDz1evNwpz97g6K0gnn+X4WaM7auloOcaE2e9Q2OclitfS6WJre0 9+tUkPsmBf0cyfa34xHWg3c= X-Google-Smtp-Source: ACHHUZ71Yh2gf5KlY7qdUaaDvSPaWfp7f6W1Nl4kXB8zp5P1K+fqUs37Ow0Mybx6UcgVZxu6YEiogw== X-Received: by 2002:a05:6a00:1a89:b0:62d:d045:392 with SMTP id e9-20020a056a001a8900b0062dd0450392mr498191pfv.32.1684345480348; Wed, 17 May 2023 10:44:40 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:39 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Hannes Reinecke , Ming Lei Subject: [PATCH v6 07/11] block: mq-deadline: Simplify deadline_skip_seq_writes() Date: Wed, 17 May 2023 10:42:25 -0700 Message-ID: <20230517174230.897144-8-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Make the deadline_skip_seq_writes() code shorter without changing its functionality. Reviewed-by: Damien Le Moal Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index cea91ba4a6ea..56782ee93522 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -304,14 +304,11 @@ static struct request *deadline_skip_seq_writes(struct deadline_data *dd, struct request *rq) { sector_t pos = blk_rq_pos(rq); - sector_t skipped_sectors = 0; - while (rq) { - if (blk_rq_pos(rq) != pos + skipped_sectors) - break; - skipped_sectors += blk_rq_sectors(rq); + do { + pos += blk_rq_sectors(rq); rq = deadline_latter_request(rq); - } + } while (rq && blk_rq_pos(rq) == pos); return rq; } From patchwork Wed May 17 17:42:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A644AC7EE2D for ; Wed, 17 May 2023 17:45:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229883AbjEQRpm (ORCPT ); Wed, 17 May 2023 13:45:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229551AbjEQRph (ORCPT ); Wed, 17 May 2023 13:45:37 -0400 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27133D2D9 for ; Wed, 17 May 2023 10:45:08 -0700 (PDT) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-643a1fed360so763052b3a.3 for ; Wed, 17 May 2023 10:45:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345482; x=1686937482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EtawUDGVkAQQgu3j8B80kzwNBVCXAkTme75c4XUrVi8=; b=CzEEUi5Bs+g1cDFaCgUPnOCxOPRwLu8BFuNj2GyduMAd9z2+Sjx3m9t83XPSydG1+3 1C+FcTtj6Weu891cKnJaEoEm4pEnFWzU5xJfKFbe67YUyZ3FiGi/rtV2VeOemRH+dE7Q IEl0TpBjyYqEoq3ec/EXCi6JAdnl3tZv8V5KkU5QsroJYUOu2+fJAWI/lB4yt1gKlWVT yxDJQdw0OCzJvnwD8YX2lExyTqnKOoh6nTdVbaXyDBy59xBqeeN1okRLyYKUOPm1HznD AD1QyXrXe4WaF4xsCaVhPK6kT09dqzW1lgH/qC+QEA9kMA1dS65ePGBa7yBc2ri4GYjO Zt8Q== X-Gm-Message-State: AC+VfDyJ7VIeu7y+71IHZ7QsdjnumQMplu+/fiav0DTtcfwqzLR+3G2P Kt0aSe+yfw3EyQc7l+ZaLrE= X-Google-Smtp-Source: ACHHUZ6qPwa2ddHDgtreNJt0ieufZE3y3ID1L+uVL8azLa28s6s4BUWUGB9IwtRaXHZn3wTRwjpF+g== X-Received: by 2002:a05:6a00:1a15:b0:64c:c842:5864 with SMTP id g21-20020a056a001a1500b0064cc8425864mr661984pfv.14.1684345481712; Wed, 17 May 2023 10:44:41 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:41 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Hannes Reinecke , Ming Lei Subject: [PATCH v6 08/11] block: mq-deadline: Reduce lock contention Date: Wed, 17 May 2023 10:42:26 -0700 Message-ID: <20230517174230.897144-9-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org blk_mq_free_requests() calls dd_finish_request() indirectly. Prevent nested locking of dd->lock and dd->zone_lock by moving the code for freeing requests. Reviewed-by: Damien Le Moal Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 56782ee93522..44222d18f6d4 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -757,7 +757,7 @@ static bool dd_bio_merge(struct request_queue *q, struct bio *bio, * add rq to rbtree and fifo */ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, - blk_insert_t flags) + blk_insert_t flags, struct list_head *free) { struct request_queue *q = hctx->queue; struct deadline_data *dd = q->elevator->elevator_data; @@ -766,7 +766,6 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, u8 ioprio_class = IOPRIO_PRIO_CLASS(ioprio); struct dd_per_prio *per_prio; enum dd_prio prio; - LIST_HEAD(free); lockdep_assert_held(&dd->lock); @@ -783,10 +782,8 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, rq->elv.priv[0] = (void *)(uintptr_t)1; } - if (blk_mq_sched_try_insert_merge(q, rq, &free)) { - blk_mq_free_requests(&free); + if (blk_mq_sched_try_insert_merge(q, rq, free)) return; - } trace_block_rq_insert(rq); @@ -819,6 +816,7 @@ static void dd_insert_requests(struct blk_mq_hw_ctx *hctx, { struct request_queue *q = hctx->queue; struct deadline_data *dd = q->elevator->elevator_data; + LIST_HEAD(free); spin_lock(&dd->lock); while (!list_empty(list)) { @@ -826,9 +824,11 @@ static void dd_insert_requests(struct blk_mq_hw_ctx *hctx, rq = list_first_entry(list, struct request, queuelist); list_del_init(&rq->queuelist); - dd_insert_request(hctx, rq, flags); + dd_insert_request(hctx, rq, flags, &free); } spin_unlock(&dd->lock); + + blk_mq_free_requests(&free); } /* Callback from inside blk_mq_rq_ctx_init(). */ From patchwork Wed May 17 17:42:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245430 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18E6CC7EE2A for ; Wed, 17 May 2023 17:45:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229904AbjEQRph (ORCPT ); Wed, 17 May 2023 13:45:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229724AbjEQRpd (ORCPT ); Wed, 17 May 2023 13:45:33 -0400 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C7395FDF for ; Wed, 17 May 2023 10:45:02 -0700 (PDT) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-643990c5319so787146b3a.2 for ; Wed, 17 May 2023 10:45:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345483; x=1686937483; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=POypCGiWuApqDM19tE3Eviu4RKO/A7PZnM2IaNLHCrQ=; b=l6EY0pLXF60w7sdUGiHjzTmaaS+ejjoDoks+3pB9Xw5P2g+CMxpDFu6dd1Yytyd90N 2W0qmdBBBHq34cDKESL2F2fSgg9ZV2SNHolpSJfw7Dwol9cDU8l6IqfMfVRiSVKCC5Yk ucSdF+1cU6A2Xx77rUI4htDaxpbtI5HZ/vO30slS9VGooR3QhkRNoMEmMooJhxZ/Yynu BYdP1a6ZuGxyzIWtvNMTglew1vd4ue3qO31YTTobDokOifJG9eHlt7tcTRuJRg1Xn/08 epF9kBVp0rdx6vBCPHUL+z/cvm19E6FkdaH/g2WTViQWgtwiPjePnwDqUrKOMkfCBZ35 EMMA== X-Gm-Message-State: AC+VfDzwjTIF8TfrTcYIfSPqA5JSPAPypFX4K1cilY9yLj90MGWu0U87 XRfiC2t6L0brtwB9PNsg3Jc= X-Google-Smtp-Source: ACHHUZ7i42svQSYzBtQw/Hfg7xTrA17FOZfO7jlC4nMoeDmgUksCjrg+YrsXESbbp9L5ldQx3/2UNg== X-Received: by 2002:a05:6a00:1252:b0:638:7e00:3737 with SMTP id u18-20020a056a00125200b006387e003737mr526104pfi.23.1684345482951; Wed, 17 May 2023 10:44:42 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:42 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v6 09/11] block: mq-deadline: Track the dispatch position Date: Wed, 17 May 2023 10:42:27 -0700 Message-ID: <20230517174230.897144-10-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Track the position (sector_t) of the most recently dispatched request instead of tracking a pointer to the next request to dispatch. This patch is the basis for patch "Handle requeued requests correctly". Without this patch it would be significantly more complicated to make sure that zoned writes are dispatched in LBA order per zone. Reviewed-by: Christoph Hellwig Reviewed-by: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 45 +++++++++++++++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 14 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 44222d18f6d4..91b689261d30 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -74,8 +74,8 @@ struct dd_per_prio { struct list_head dispatch; struct rb_root sort_list[DD_DIR_COUNT]; struct list_head fifo_list[DD_DIR_COUNT]; - /* Next request in FIFO order. Read, write or both are NULL. */ - struct request *next_rq[DD_DIR_COUNT]; + /* Position of the most recently dispatched request. */ + sector_t latest_pos[DD_DIR_COUNT]; struct io_stats_per_prio stats; }; @@ -156,6 +156,25 @@ deadline_latter_request(struct request *rq) return NULL; } +/* Return the first request for which blk_rq_pos() >= pos. */ +static inline struct request *deadline_from_pos(struct dd_per_prio *per_prio, + enum dd_data_dir data_dir, sector_t pos) +{ + struct rb_node *node = per_prio->sort_list[data_dir].rb_node; + struct request *rq, *res = NULL; + + while (node) { + rq = rb_entry_rq(node); + if (blk_rq_pos(rq) >= pos) { + res = rq; + node = node->rb_left; + } else { + node = node->rb_right; + } + } + return res; +} + static void deadline_add_rq_rb(struct dd_per_prio *per_prio, struct request *rq) { @@ -167,11 +186,6 @@ deadline_add_rq_rb(struct dd_per_prio *per_prio, struct request *rq) static inline void deadline_del_rq_rb(struct dd_per_prio *per_prio, struct request *rq) { - const enum dd_data_dir data_dir = rq_data_dir(rq); - - if (per_prio->next_rq[data_dir] == rq) - per_prio->next_rq[data_dir] = deadline_latter_request(rq); - elv_rb_del(deadline_rb_root(per_prio, rq), rq); } @@ -251,10 +265,6 @@ static void deadline_move_request(struct deadline_data *dd, struct dd_per_prio *per_prio, struct request *rq) { - const enum dd_data_dir data_dir = rq_data_dir(rq); - - per_prio->next_rq[data_dir] = deadline_latter_request(rq); - /* * take it off the sort and fifo list */ @@ -363,7 +373,8 @@ deadline_next_request(struct deadline_data *dd, struct dd_per_prio *per_prio, struct request *rq; unsigned long flags; - rq = per_prio->next_rq[data_dir]; + rq = deadline_from_pos(per_prio, data_dir, + per_prio->latest_pos[data_dir]); if (!rq) return NULL; @@ -426,6 +437,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, if (started_after(dd, rq, latest_start)) return NULL; list_del_init(&rq->queuelist); + data_dir = rq_data_dir(rq); goto done; } @@ -433,9 +445,11 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, * batches are currently reads XOR writes */ rq = deadline_next_request(dd, per_prio, dd->last_dir); - if (rq && dd->batching < dd->fifo_batch) + if (rq && dd->batching < dd->fifo_batch) { /* we have a next request and are still entitled to batch */ + data_dir = rq_data_dir(rq); goto dispatch_request; + } /* * at this point we are not running a batch. select the appropriate @@ -513,6 +527,7 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, done: ioprio_class = dd_rq_ioclass(rq); prio = ioprio_class_to_prio[ioprio_class]; + dd->per_prio[prio].latest_pos[data_dir] = blk_rq_pos(rq); dd->per_prio[prio].stats.dispatched++; /* * If the request needs its target zone locked, do it. @@ -1026,8 +1041,10 @@ static int deadline_##name##_next_rq_show(void *data, \ struct request_queue *q = data; \ struct deadline_data *dd = q->elevator->elevator_data; \ struct dd_per_prio *per_prio = &dd->per_prio[prio]; \ - struct request *rq = per_prio->next_rq[data_dir]; \ + struct request *rq; \ \ + rq = deadline_from_pos(per_prio, data_dir, \ + per_prio->latest_pos[data_dir]); \ if (rq) \ __blk_mq_debugfs_rq_show(m, rq); \ return 0; \ From patchwork Wed May 17 17:42:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EAA8C7EE2D for ; Wed, 17 May 2023 17:45:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229489AbjEQRpj (ORCPT ); Wed, 17 May 2023 13:45:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229759AbjEQRpe (ORCPT ); Wed, 17 May 2023 13:45:34 -0400 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F554AD26 for ; Wed, 17 May 2023 10:45:03 -0700 (PDT) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6434e65d808so1009086b3a.3 for ; Wed, 17 May 2023 10:45:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345485; x=1686937485; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WiPSD3ypK27BFIDaFIHIvt91IAeB1EY3KLLD7n21fic=; b=AHBUfuOYu0qaPlY5Vlq/a17mytu+FhoJ0ukAbDQJRacXJaSixJxi7EHC32EE1DLjzq bNOjVqgIkzPxWdYB36a2vf/1MsbERFHDY7aozcRQ9YZsgK0ksoIsf18o3IdqkN3Vu2a+ o1KZpiDZvyurFzVFYC3Iu/DqHd1cYfu9hxhxhdpaIAnyCpRo8l+6tHABdgxr1BlEdFeq 6wyW0KX3qeqSMUKuTdgbBdSEpg1XWYqoE/mtumeBs46eGywId6SMqdxOlQ4E8qhzmJ3E iOMCQpf2TtKU7LQy7Khh/LnYBGCS4XZxo3h0nIE1k3+PE2fw8Hd94XslNe0/qxw2VYtM Yf4Q== X-Gm-Message-State: AC+VfDwWfS8OfPcC5e2ZNEHMlCzDeKxHyo+YeLrGtgxwZJKdhVZ0V9vZ FQ1ucUZe1z9lxcB+St9zWUE= X-Google-Smtp-Source: ACHHUZ40nl6xOPGnb8krchVrLAd5z4aSvBJEKdHyovJ/nbRwbdNRE8NPEEafil82D5BqHjTkbHvZcg== X-Received: by 2002:a05:6a00:1953:b0:64a:f8c9:a42c with SMTP id s19-20020a056a00195300b0064af8c9a42cmr620253pfk.18.1684345484701; Wed, 17 May 2023 10:44:44 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:43 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Hannes Reinecke , Damien Le Moal , Ming Lei Subject: [PATCH v6 10/11] block: mq-deadline: Handle requeued requests correctly Date: Wed, 17 May 2023 10:42:28 -0700 Message-ID: <20230517174230.897144-11-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Start dispatching from the start of a zone instead of from the starting position of the most recently dispatched request. If a zoned write is requeued with an LBA that is lower than already inserted zoned writes, make sure that it is submitted first. Reviewed-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Cc: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- block/mq-deadline.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 91b689261d30..e90879869c90 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -156,13 +156,28 @@ deadline_latter_request(struct request *rq) return NULL; } -/* Return the first request for which blk_rq_pos() >= pos. */ +/* + * Return the first request for which blk_rq_pos() >= @pos. For zoned devices, + * return the first request after the start of the zone containing @pos. + */ static inline struct request *deadline_from_pos(struct dd_per_prio *per_prio, enum dd_data_dir data_dir, sector_t pos) { struct rb_node *node = per_prio->sort_list[data_dir].rb_node; struct request *rq, *res = NULL; + if (!node) + return NULL; + + rq = rb_entry_rq(node); + /* + * A zoned write may have been requeued with a starting position that + * is below that of the most recently dispatched request. Hence, for + * zoned writes, start searching from the start of a zone. + */ + if (blk_rq_is_seq_zoned_write(rq)) + pos -= round_down(pos, rq->q->limits.chunk_sectors); + while (node) { rq = rb_entry_rq(node); if (blk_rq_pos(rq) >= pos) { @@ -806,6 +821,8 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, list_add(&rq->queuelist, &per_prio->dispatch); rq->fifo_time = jiffies; } else { + struct list_head *insert_before; + deadline_add_rq_rb(per_prio, rq); if (rq_mergeable(rq)) { @@ -818,7 +835,20 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, * set expire time and add to fifo list */ rq->fifo_time = jiffies + dd->fifo_expire[data_dir]; - list_add_tail(&rq->queuelist, &per_prio->fifo_list[data_dir]); + insert_before = &per_prio->fifo_list[data_dir]; +#ifdef CONFIG_BLK_DEV_ZONED + /* + * Insert zoned writes such that requests are sorted by + * position per zone. + */ + if (blk_rq_is_seq_zoned_write(rq)) { + struct request *rq2 = deadline_latter_request(rq); + + if (rq2 && blk_rq_zone_no(rq2) == blk_rq_zone_no(rq)) + insert_before = &rq2->queuelist; + } +#endif + list_add_tail(&rq->queuelist, insert_before); } } From patchwork Wed May 17 17:42:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13245432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F18D8C77B75 for ; Wed, 17 May 2023 17:45:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229477AbjEQRpk (ORCPT ); Wed, 17 May 2023 13:45:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229821AbjEQRpf (ORCPT ); Wed, 17 May 2023 13:45:35 -0400 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C7156E8D for ; Wed, 17 May 2023 10:45:06 -0700 (PDT) Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6434e263962so822574b3a.2 for ; Wed, 17 May 2023 10:45:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345486; x=1686937486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cMWr+MCtsGHi9OfvspnRheUM1pYYUBU4HUhQP6OfIrk=; b=KgySUTQ2k0QgNGyA5EKMXILEjGTgsJLa93Y+vSKbTp7350AP/2zbZbeyD7M28AOmZ2 ZE23NZgeVw3QAKOoIIwWaTHxohtWmd/H09QsdKKSRxiDnFOL1z0jk1cLPeNEm8a+jIzD 9c0n5W8aOUMD6IKKtIfc5TDPv/9we0CrrrIyPdrOBWDraEIcHVVuofiNtUkGgrBqry41 Sy7dUmcW997/U4CD0VQneZaqTXH6p410OzODX75FJ/2rZyQ2P7CFxICdbsMLB+MRT8ny aUrFUzi1Tj+pPK4iED1Cy1jbTHRkybVpkPqGZFh8Wo1jhi85FsJzY2WiPNXU+F7UVNHf tziw== X-Gm-Message-State: AC+VfDy/i5TyupogaPs9IR37GQ1pw/bAdS2Q6+Gv7TOKGun6fhYDg3TK vWdGcpLBCwYOfc1EW//3tL8= X-Google-Smtp-Source: ACHHUZ6P1rJLTdB0rFmkiZVXeteMzqjTcPJn4nDwb4/IziyOcrgkBwFBD+2UPQ8pmiuCgVgq9naoDg== X-Received: by 2002:a05:6a00:80e:b0:62a:4503:53ba with SMTP id m14-20020a056a00080e00b0062a450353bamr524190pfk.26.1684345486119; Wed, 17 May 2023 10:44:46 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d22-20020aa78e56000000b00646e7d2b5a7sm15334410pfr.112.2023.05.17.10.44.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 10:44:45 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Damien Le Moal , Ming Lei Subject: [PATCH v6 11/11] block: mq-deadline: Fix handling of at-head zoned writes Date: Wed, 17 May 2023 10:42:29 -0700 Message-ID: <20230517174230.897144-12-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517174230.897144-1-bvanassche@acm.org> References: <20230517174230.897144-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Before dispatching a zoned write from the FIFO list, check whether there are any zoned writes in the RB-tree with a lower LBA for the same zone. This patch ensures that zoned writes happen in order even if at_head is set for some writes for a zone and not for others. Reviewed-by: Christoph Hellwig Reviewed-by: Damien Le Moal Cc: Ming Lei Signed-off-by: Bart Van Assche --- block/mq-deadline.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index e90879869c90..6aa5daf7ae32 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -346,7 +346,7 @@ static struct request * deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, enum dd_data_dir data_dir) { - struct request *rq; + struct request *rq, *rb_rq, *next; unsigned long flags; if (list_empty(&per_prio->fifo_list[data_dir])) @@ -364,7 +364,12 @@ deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, * zones and these zones are unlocked. */ spin_lock_irqsave(&dd->zone_lock, flags); - list_for_each_entry(rq, &per_prio->fifo_list[DD_WRITE], queuelist) { + list_for_each_entry_safe(rq, next, &per_prio->fifo_list[DD_WRITE], + queuelist) { + /* Check whether a prior request exists for the same zone. */ + rb_rq = deadline_from_pos(per_prio, data_dir, blk_rq_pos(rq)); + if (rb_rq && blk_rq_pos(rb_rq) < blk_rq_pos(rq)) + rq = rb_rq; if (blk_req_can_dispatch_to_zone(rq) && (blk_queue_nonrot(rq->q) || !deadline_is_seq_write(dd, rq)))