From patchwork Mon Jun 27 23:43:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2178C433EF for ; Mon, 27 Jun 2022 23:43:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242310AbiF0Xnv (ORCPT ); Mon, 27 Jun 2022 19:43:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242257AbiF0Xnu (ORCPT ); Mon, 27 Jun 2022 19:43:50 -0400 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4020313D5C for ; Mon, 27 Jun 2022 16:43:50 -0700 (PDT) Received: by mail-pg1-f175.google.com with SMTP id d129so10507867pgc.9 for ; Mon, 27 Jun 2022 16:43:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Sidm+YmX4odLJBDsgWhSml1wP+jSGmGPOzF6NSRNmeI=; b=xScNvigY6syhadqW67hjeJDZu8d3itN0lNBoSaqUAkhyLCpqyLlxv/PPQD6HsoNTdn 3uVRZWQoTkPTKXdGtXHCCfVKJvcM9vz+OszXpNxnaXTxtxBsY7IZ3Zh52oUkBKJf+XPu sc+C4XVWLqfDJRbdSomFWCdeLObRwaPdy64EfwAaMHtcojNuTqkVjbXMChcHFlF7YPfu FchpNdsnI7nHZJ3DhCgAagUMnIH1zZptjd6rL98eAW1COHjSrvc1LiPCULEAQN+mKTMh eE1uK7i5vW9OfPEZmB1JUru65/HyPuuuJ1N8EbGuikzuVk4dy4r+9kxTrgNtC7U2r+O+ MAQw== X-Gm-Message-State: AJIora9YhmyNt9oBhFZPF2ooCI4bap+y4TUqRqO/obSVFz5BSf9r2l74 2rb72ZTJQybed7G33LB/9k4= X-Google-Smtp-Source: AGRyM1uAPZkeaU9+QdHtXkBi+IhjgYqqJ7iU8Liq++XGo7rx5Xoa+zpK5NZieh67sXr33y2YNN/PvQ== X-Received: by 2002:a63:1e49:0:b0:3fd:cf48:3694 with SMTP id p9-20020a631e49000000b003fdcf483694mr15020725pgm.275.1656373429587; Mon, 27 Jun 2022 16:43:49 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:48 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Damien Le Moal Subject: [PATCH v3 1/8] block: Document blk_queue_zone_is_seq() and blk_rq_zone_is_seq() Date: Mon, 27 Jun 2022 16:43:28 -0700 Message-Id: <20220627234335.1714393-2-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Since it is nontrivial to figure out how blk_queue_zone_is_seq() and blk_rq_zone_is_seq() handle sequential write preferred zones, document this. Reviewed-by: Damien Le Moal Signed-off-by: Bart Van Assche Reviewed-by: Chaitanya Kulkarni --- include/linux/blk-mq.h | 7 +++++++ include/linux/blkdev.h | 9 +++++++++ 2 files changed, 16 insertions(+) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index e2d9daf7e8dd..909d47e34b7c 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1124,6 +1124,13 @@ static inline unsigned int blk_rq_zone_no(struct request *rq) return blk_queue_zone_no(rq->q, blk_rq_pos(rq)); } +/** + * blk_rq_zone_is_seq() - Whether a request is for a sequential zone. + * @rq: Request pointer. + * + * Return: true if and only if blk_rq_pos(@rq) refers either to a sequential + * write required or a sequential write preferred zone. + */ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) { return blk_queue_zone_is_seq(rq->q, blk_rq_pos(rq)); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index b2d42201bd5d..6491250ba286 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -687,6 +687,15 @@ static inline unsigned int blk_queue_zone_no(struct request_queue *q, return sector >> ilog2(q->limits.chunk_sectors); } +/** + * blk_queue_zone_is_seq() - Whether a logical block is in a sequential zone. + * @q: Request queue pointer. + * @sector: Offset from start of block device in 512 byte units. + * + * Return: true if and only if @q is associated with a zoned block device and + * @sector refers either to a sequential write required or a sequential write + * preferred zone. + */ static inline bool blk_queue_zone_is_seq(struct request_queue *q, sector_t sector) { From patchwork Mon Jun 27 23:43:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B36FDC43334 for ; Mon, 27 Jun 2022 23:43:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242257AbiF0Xnx (ORCPT ); Mon, 27 Jun 2022 19:43:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242320AbiF0Xnw (ORCPT ); Mon, 27 Jun 2022 19:43:52 -0400 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3AC713D5C for ; Mon, 27 Jun 2022 16:43:51 -0700 (PDT) Received: by mail-pg1-f181.google.com with SMTP id 23so10510939pgc.8 for ; Mon, 27 Jun 2022 16:43:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dVBXKDUUQfCpF8OMFSZm33rUgwa4FkPVmBE8uu4g8Is=; b=x6tS9OwMB1EMc6+4soMuI8Hn145YfIBOE4ct5eQTcdCDs75wEhMM7w/ATbOapNe34u WYGc0qCgbyEdLNuXmfQ8CzSGlq1I10fJl0x0Va0v5GlhvkSg1vOz0KE+DwFEZ69hY/O3 dxma8vbZvhzLIG/EjgVdvFpVFTy37DWHWVuEpvHgFGQIoPmdsYc2aguZBGEjosqfsF5+ YcPdDlcVSraAuhQRWd+J0L/bQcNHqYMkZ6wXr1sE1sxBTHr35HE9UfiKTnibEPh8ECqU oXI06Paf8GrGu2iw8l8a8msaAaHtHzxIqeXe4Z6IBkYVXM+MBwr5HaSYfQPsZl8U1CHz lHug== X-Gm-Message-State: AJIora8RDKeM2JWHpeGoOXF16dS/AZGNjog2wL/0BTSKks1bgk0qrumF UYo6L1lGkOjADwYghsX1hLA= X-Google-Smtp-Source: AGRyM1u2BGr0oKNGwVeP3wFeXrpKPXeHZOEwZGXWH3cDEX+kFozVdiuipx7q9PE1CHDygfTptO8EKg== X-Received: by 2002:a05:6a00:1c94:b0:527:c49a:3249 with SMTP id y20-20020a056a001c9400b00527c49a3249mr1339863pfw.18.1656373431259; Mon, 27 Jun 2022 16:43:51 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:50 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Damien Le Moal Subject: [PATCH v3 2/8] block: Introduce the blk_rq_is_seq_zone_write() function Date: Mon, 27 Jun 2022 16:43:29 -0700 Message-Id: <20220627234335.1714393-3-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce a function that makes it easy to verify whether a write request is for a sequential write required or sequential write preferred zone. Simplify blk_req_needs_zone_write_lock() by using the new function. Reviewed-by: Damien Le Moal Signed-off-by: Bart Van Assche Reviewed-by: Chaitanya Kulkarni --- block/blk-zoned.c | 14 +------------- include/linux/blk-mq.h | 25 ++++++++++++++++++++++++- 2 files changed, 25 insertions(+), 14 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 38cd840d8838..9da8cf1bb378 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -57,19 +57,7 @@ EXPORT_SYMBOL_GPL(blk_zone_cond_str); */ bool blk_req_needs_zone_write_lock(struct request *rq) { - if (!rq->q->seq_zones_wlock) - return false; - - if (blk_rq_is_passthrough(rq)) - return false; - - switch (req_op(rq)) { - case REQ_OP_WRITE_ZEROES: - case REQ_OP_WRITE: - return blk_rq_zone_is_seq(rq); - default: - return false; - } + return rq->q->seq_zones_wlock && blk_rq_is_seq_zone_write(rq); } EXPORT_SYMBOL_GPL(blk_req_needs_zone_write_lock); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 909d47e34b7c..ccfcac9db879 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1129,13 +1129,31 @@ static inline unsigned int blk_rq_zone_no(struct request *rq) * @rq: Request pointer. * * Return: true if and only if blk_rq_pos(@rq) refers either to a sequential - * write required or a sequential write preferred zone. + * write required or to a sequential write preferred zone. */ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) { return blk_queue_zone_is_seq(rq->q, blk_rq_pos(rq)); } +/** + * blk_rq_is_seq_zone_write() - Whether @rq is a write request for a sequential zone. + * @rq: Request to examine. + * + * In this context sequential zone means either a sequential write required or + * to a sequential write preferred zone. + */ +static inline bool blk_rq_is_seq_zone_write(struct request *rq) +{ + switch (req_op(rq)) { + case REQ_OP_WRITE: + case REQ_OP_WRITE_ZEROES: + return blk_rq_zone_is_seq(rq); + default: + return false; + } +} + bool blk_req_needs_zone_write_lock(struct request *rq); bool blk_req_zone_write_trylock(struct request *rq); void __blk_req_zone_write_lock(struct request *rq); @@ -1166,6 +1184,11 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq) return !blk_req_zone_is_write_locked(rq); } #else /* CONFIG_BLK_DEV_ZONED */ +static inline bool blk_rq_is_seq_zone_write(struct request *rq) +{ + return false; +} + static inline bool blk_req_needs_zone_write_lock(struct request *rq) { return false; From patchwork Mon Jun 27 23:43:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAEFBC43334 for ; Mon, 27 Jun 2022 23:43:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242324AbiF0Xnz (ORCPT ); Mon, 27 Jun 2022 19:43:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242322AbiF0Xny (ORCPT ); Mon, 27 Jun 2022 19:43:54 -0400 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A6E817584 for ; Mon, 27 Jun 2022 16:43:53 -0700 (PDT) Received: by mail-pl1-f173.google.com with SMTP id d5so9514572plo.12 for ; Mon, 27 Jun 2022 16:43:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sJDQo1edv17A+CGMDNmnBvmmgGLDtDKHNKtqJiAWE30=; b=IWgwBt5Dpg3Ll++hSGlkVvtKN6ihzowRIPHd82h7L6PWrE5HWmHfRDXtHrEdhzCiga +OXSrpB6np63UESnXTkKzHzB64YnMtWuMGd+qCvU+mxdLWYJ9uOxRqwyyDHnIHGNr7a0 CP0busBSQxyy0ErvOVJ+fajmcYdgI89sui6goDjCyU/RI0RLfjnF+VXCwu8sDx+ZPLNq GfNlR6Vv7O1p+xdkJQJj5w9R1o9VNbk5cmbAeiTz8l6TIBvN7gPoASmBOLuq8orH1Y1m K1q37lC4Qn1B1xCJzeJ+JeAwQMHrVLcVWgIGVjkPhQqP6SpqUrS7pwrWFdUaAc1dkr6H KH9A== X-Gm-Message-State: AJIora9FSBbNrQp/3GkDw7+hH/17bfJz7Ls666kCSNMLCaXQn0QZ412I pdrPOyDfONU7W0DTe6MA1mFY4fq9XIo= X-Google-Smtp-Source: AGRyM1u0rLcDrR1B4LTJyVOdpbLzf8IFL59EIrNA6o+JSHy/lalShn2UyoYKepzVX+xJollSGiOi0A== X-Received: by 2002:a17:902:c94b:b0:16a:6427:ae5d with SMTP id i11-20020a170902c94b00b0016a6427ae5dmr669244pla.127.1656373432879; Mon, 27 Jun 2022 16:43:52 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:52 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Damien Le Moal Subject: [PATCH v3 3/8] block: Introduce a request queue flag for pipelining zoned writes Date: Mon, 27 Jun 2022 16:43:30 -0700 Message-Id: <20220627234335.1714393-4-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Writes in sequential write required zones must happen at the write pointer. Even if the submitter of the write commands (e.g. a filesystem) submits writes for sequential write required zones in order, the block layer or the storage controller may reorder these write commands. The zone locking mechanism in the mq-deadline I/O scheduler serializes write commands for sequential zones. Some but not all storage controllers require this serialization. Introduce a new flag such that block drivers can request pipelining of writes for sequential write required zones. An example of a storage controller standard that requires write serialization is AHCI (Advanced Host Controller Interface). Submitting commands to AHCI controllers happens by writing a bit pattern into a register. Each set bit corresponds to an active command. This mechanism does not preserve command ordering information. Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- include/linux/blkdev.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 6491250ba286..7ee59e507fbf 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -580,6 +580,8 @@ struct request_queue { #define QUEUE_FLAG_HCTX_ACTIVE 28 /* at least one blk-mq hctx is active */ #define QUEUE_FLAG_NOWAIT 29 /* device supports NOWAIT */ #define QUEUE_FLAG_SQ_SCHED 30 /* single queue style io dispatch */ +/* Writes for sequential write required zones may be pipelined. */ +#define QUEUE_FLAG_PIPELINE_ZONED_WRITES 31 #define QUEUE_FLAG_MQ_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_SAME_COMP) | \ @@ -623,6 +625,11 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_nowait(q) test_bit(QUEUE_FLAG_NOWAIT, &(q)->queue_flags) #define blk_queue_sq_sched(q) test_bit(QUEUE_FLAG_SQ_SCHED, &(q)->queue_flags) +static inline bool blk_queue_pipeline_zoned_writes(struct request_queue *q) +{ + return test_bit(QUEUE_FLAG_PIPELINE_ZONED_WRITES, &(q)->queue_flags); +} + extern void blk_set_pm_only(struct request_queue *q); extern void blk_clear_pm_only(struct request_queue *q); From patchwork Mon Jun 27 23:43:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3F61C433EF for ; Mon, 27 Jun 2022 23:43:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242320AbiF0Xn4 (ORCPT ); Mon, 27 Jun 2022 19:43:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242330AbiF0Xnz (ORCPT ); Mon, 27 Jun 2022 19:43:55 -0400 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1652818348 for ; Mon, 27 Jun 2022 16:43:55 -0700 (PDT) Received: by mail-pj1-f45.google.com with SMTP id cv13so10863432pjb.4 for ; Mon, 27 Jun 2022 16:43:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GzC4O0PTfqt87MREOZLKKGbzlxl9HlHXBQPDVQspuew=; b=B8hrHsliov8Isa2nMvXoOBpIYr0axtwXe5Q5xjKvLmpiRbaADeYIqaKKBWAjEixXDX hKjRBlm3vc1JI4ZZktbaW91Y6LUZNgebUd7sF26vCkUMc84uiSheBM3pI9ciT3SusjwY jqVsY3V82pou/DL4rRdl9kVjv3DlSkTj5HBu1d+13YHdvYQMdyKjbwLPXTJ0o66rsDgI cn85Y6wFQXVLSZgl4t1j0+iFoG8HeUzxXU3aetDALKXbvzgc8WWQWXffsQDFCnwa4tF3 tNMenktJ0EvaZ/cqEmKElWFIF1XUB0OLigyDdvbmr53S8mo42VUpOdGkAqQeHUxE1/Sh Mjgw== X-Gm-Message-State: AJIora9vzINje3UMebkCgnwgSVEMoVp+QBNX3D37+UlS/EmVymBUqpvk 4apZ99e9u093s5/HNdLRSL0L0pEuuns= X-Google-Smtp-Source: AGRyM1vjkdj467FkmaLcoGHY6i0/3vmeEhlEhP6ythhqfjQs+aqFBP5DL56hw4b3/jt+gwtq7Ihu1g== X-Received: by 2002:a17:903:11c9:b0:16b:8293:c599 with SMTP id q9-20020a17090311c900b0016b8293c599mr1966017plh.136.1656373434487; Mon, 27 Jun 2022 16:43:54 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:53 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Damien Le Moal Subject: [PATCH v3 4/8] block/mq-deadline: Only use zone locking if necessary Date: Mon, 27 Jun 2022 16:43:31 -0700 Message-Id: <20220627234335.1714393-5-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Measurements have shown that limiting the queue depth to one for zoned writes has a significant negative performance impact on zoned UFS devices. Hence this patch that disables zone locking from the mq-deadline scheduler for storage controllers that support pipelining zoned writes. This patch is based on the following assumptions: - Applications submit write requests to sequential write required zones in order. - The I/O priority of all pipelined write requests is the same per zone. - Pipelined zoned write requests are submitted to a single hardware queue per zone. - If such write requests get reordered by the software or hardware queue mechanism, nr_requests - 1 retries are sufficient to reorder the write requests. - It happens infrequently that zoned write requests are reordered by the block layer. - Either no I/O scheduler is used or an I/O scheduler is used that submits write requests per zone in LBA order. Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- block/blk-zoned.c | 3 ++- block/mq-deadline.c | 18 ++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index 9da8cf1bb378..63c730a18ac4 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -513,7 +513,8 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx, break; case BLK_ZONE_TYPE_SEQWRITE_REQ: case BLK_ZONE_TYPE_SEQWRITE_PREF: - if (!args->seq_zones_wlock) { + if (!blk_queue_pipeline_zoned_writes(q) && + !args->seq_zones_wlock) { args->seq_zones_wlock = blk_alloc_zone_bitmap(q->node, args->nr_zones); if (!args->seq_zones_wlock) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 1a9e835e816c..aaef07a55984 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -292,7 +292,8 @@ deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, return NULL; rq = rq_entry_fifo(per_prio->fifo_list[data_dir].next); - if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q)) + if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q) || + blk_queue_pipeline_zoned_writes(rq->q)) return rq; /* @@ -326,7 +327,8 @@ deadline_next_request(struct deadline_data *dd, struct dd_per_prio *per_prio, if (!rq) return NULL; - if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q)) + if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q) || + blk_queue_pipeline_zoned_writes(rq->q)) return rq; /* @@ -445,8 +447,9 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, } /* - * For a zoned block device, if we only have writes queued and none of - * them can be dispatched, rq will be NULL. + * For a zoned block device that requires write serialization, if we + * only have writes queued and none of them can be dispatched, rq will + * be NULL. */ if (!rq) return NULL; @@ -719,6 +722,8 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, u8 ioprio_class = IOPRIO_PRIO_CLASS(ioprio); struct dd_per_prio *per_prio; enum dd_prio prio; + bool pipelined_seq_write = blk_queue_pipeline_zoned_writes(q) && + blk_rq_is_seq_zone_write(rq); LIST_HEAD(free); lockdep_assert_held(&dd->lock); @@ -743,7 +748,7 @@ static void dd_insert_request(struct blk_mq_hw_ctx *hctx, struct request *rq, trace_block_rq_insert(rq); - if (at_head) { + if (at_head && !pipelined_seq_write) { list_add(&rq->queuelist, &per_prio->dispatch); rq->fifo_time = jiffies; } else { @@ -823,7 +828,8 @@ static void dd_finish_request(struct request *rq) atomic_inc(&per_prio->stats.completed); - if (blk_queue_is_zoned(q)) { + if (blk_queue_is_zoned(rq->q) && + !blk_queue_pipeline_zoned_writes(q)) { unsigned long flags; spin_lock_irqsave(&dd->zone_lock, flags); From patchwork Mon Jun 27 23:43:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC350C43334 for ; Mon, 27 Jun 2022 23:43:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242316AbiF0Xn5 (ORCPT ); Mon, 27 Jun 2022 19:43:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242326AbiF0Xn5 (ORCPT ); Mon, 27 Jun 2022 19:43:57 -0400 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74F2613DFF for ; Mon, 27 Jun 2022 16:43:56 -0700 (PDT) Received: by mail-pg1-f181.google.com with SMTP id 23so10510939pgc.8 for ; Mon, 27 Jun 2022 16:43:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wTaG9EFjLZI6DmzwdDvZnWJOpRPKMWzU1i3ABuclBq0=; b=yqvuQystxZkCo/idQNkzG4IxUafHWMDuKE2f4L5ZQau2mqZkkh73oCqSrwNCNPeF2t tOgQf8juPQA35dONWXIJMXehItva4vSiOhe6/sJClCxZmqiGBFrTD2o/TAfCZJpn5Nna iHpPxGrvn2hagSwft5HaBeydYgG9DuXlKLRefKYylRO/ps0KMNt/wtg/9OzqvA3mMMzB IUPsLYy1/10IeGH+FwLcnFkWkdtGrgt6HDu5wE7OwODQrRrz2Kupz/O1iPcZyGNZ16PD xm+tc3+IV8vWRPaiJZKh9q3wOPya6pbTay3OnhNfOW34anczpyzFQ6YIC0uzZDUmnxxf A+Eg== X-Gm-Message-State: AJIora/wc4d+wOY1PtopkWQN+VTtJ+VjHfAPhaDaxfrK463C2xIod7Eq HKWWX54xQoF2GcRh5jj5am0= X-Google-Smtp-Source: AGRyM1umtTN5+ZtAZMWib/wUHudsWYltW+oxASH6gfqKeMqReYV3TRf2umvHtvkV1sCziV0d8oKAhg== X-Received: by 2002:a05:6a00:2185:b0:520:7276:6570 with SMTP id h5-20020a056a00218500b0052072766570mr1658153pfi.84.1656373436003; Mon, 27 Jun 2022 16:43:56 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:55 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Damien Le Moal Subject: [PATCH v3 5/8] block/null_blk: Refactor null_queue_rq() Date: Mon, 27 Jun 2022 16:43:32 -0700 Message-Id: <20220627234335.1714393-6-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce a local variable for the expression bd->rq since that expression occurs multiple times. This patch does not change any functionality. Reviewed-by: Damien Le Moal Signed-off-by: Bart Van Assche Reviewed-by: Chaitanya Kulkarni --- drivers/block/null_blk/main.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 6b67088f4ea7..fd68e6f4637f 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -1609,10 +1609,11 @@ static enum blk_eh_timer_return null_timeout_rq(struct request *rq, bool res) static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, const struct blk_mq_queue_data *bd) { - struct nullb_cmd *cmd = blk_mq_rq_to_pdu(bd->rq); + struct request *rq = bd->rq; + struct nullb_cmd *cmd = blk_mq_rq_to_pdu(rq); struct nullb_queue *nq = hctx->driver_data; - sector_t nr_sectors = blk_rq_sectors(bd->rq); - sector_t sector = blk_rq_pos(bd->rq); + sector_t nr_sectors = blk_rq_sectors(rq); + sector_t sector = blk_rq_pos(rq); const bool is_poll = hctx->type == HCTX_TYPE_POLL; might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); @@ -1621,14 +1622,14 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, hrtimer_init(&cmd->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); cmd->timer.function = null_cmd_timer_expired; } - cmd->rq = bd->rq; + cmd->rq = rq; cmd->error = BLK_STS_OK; cmd->nq = nq; - cmd->fake_timeout = should_timeout_request(bd->rq); + cmd->fake_timeout = should_timeout_request(rq); - blk_mq_start_request(bd->rq); + blk_mq_start_request(rq); - if (should_requeue_request(bd->rq)) { + if (should_requeue_request(rq)) { /* * Alternate between hitting the core BUSY path, and the * driver driven requeue path @@ -1637,21 +1638,21 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, if (nq->requeue_selection & 1) return BLK_STS_RESOURCE; else { - blk_mq_requeue_request(bd->rq, true); + blk_mq_requeue_request(rq, true); return BLK_STS_OK; } } if (is_poll) { spin_lock(&nq->poll_lock); - list_add_tail(&bd->rq->queuelist, &nq->poll_list); + list_add_tail(&rq->queuelist, &nq->poll_list); spin_unlock(&nq->poll_lock); return BLK_STS_OK; } if (cmd->fake_timeout) return BLK_STS_OK; - return null_handle_cmd(cmd, sector, nr_sectors, req_op(bd->rq)); + return null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); } static void cleanup_queue(struct nullb_queue *nq) From patchwork Mon Jun 27 23:43:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897296 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31482C43334 for ; Mon, 27 Jun 2022 23:44:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242322AbiF0XoA (ORCPT ); Mon, 27 Jun 2022 19:44:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242330AbiF0XoA (ORCPT ); Mon, 27 Jun 2022 19:44:00 -0400 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 006EE186D4 for ; Mon, 27 Jun 2022 16:43:58 -0700 (PDT) Received: by mail-pl1-f176.google.com with SMTP id q18so9518802pld.13 for ; Mon, 27 Jun 2022 16:43:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mWEdUNf63h0nD8vBGcfJSzzSQBu9wEwV18+U9TSXL50=; b=p8CRHLxgPH57V1D3h/gs5X1J9qbqr+P4MWYgzRJadLrAtZjEeNnIwPIOaXtNoexDRT 4G82q6xqLhv+p6RaHnG0r8vmx/bfKeoybkEyMmJAqF9BeQGLluCnlIFPlxXh++5WTha7 bTx46Qs0CfxQvxjTpEp717MBWcA8OH5+RdKisskJ7SXceIrLl/5jDA/VejxGVyYCZhju 3M6Kcb4ylybVkrqosEJ3JURe8PZlAo123YHtkrWqSB1wJbm9dr0esHEaVQ9M6baD559v oJ0KLxmBfD+RE/TF3ic/QQ0Fex2sZbK87/DsYq0gvsL5BUSx4cPiWvFrFbjI4gAh2OXW jR+g== X-Gm-Message-State: AJIora+ksYHAYw6raIcX3IWRhrN4jtBj+o6Iqdb5XuV/IcQASKPSajCf jCTqv3tgirhz0mCSYiGW/ho= X-Google-Smtp-Source: AGRyM1u86Pk5SL67Jr25AXE41LTwmfdfGIJjYS5g0ZXMu5wReHSUpQViorSAIMCOeJ4nj/uY7/NJaA== X-Received: by 2002:a17:902:d4c4:b0:16a:2d0a:b609 with SMTP id o4-20020a170902d4c400b0016a2d0ab609mr1895170plg.5.1656373438093; Mon, 27 Jun 2022 16:43:58 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:57 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Damien Le Moal Subject: [PATCH v3 6/8] block/null_blk: Add support for pipelining zoned writes Date: Mon, 27 Jun 2022 16:43:33 -0700 Message-Id: <20220627234335.1714393-7-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add a new configfs attribute for enabling pipelining of zoned writes. If that attribute has been set, retry zoned writes that are not aligned with the write pointer. The test script below reports 244 K IOPS with no I/O scheduler, 5.79 K IOPS with mq-deadline and pipelining disabled and 79.6 K IOPS with mq-deadline and pipelining enabled. This shows that pipelining results in about 14 times more IOPS for this particular test case. #!/bin/bash for mode in "none 0" "mq-deadline 0" "mq-deadline 1"; do set +e for d in /sys/kernel/config/nullb/*; do [ -d "$d" ] && rmdir "$d" done modprobe -r null_blk set -e read -r iosched pipelining <<<"$mode" modprobe null_blk nr_devices=0 ( cd /sys/kernel/config/nullb mkdir nullb0 cd nullb0 params=( completion_nsec=100000 hw_queue_depth=64 irqmode=2 # NULL_IRQ_TIMER max_sectors=$((4096/512)) memory_backed=1 pipeline_zoned_writes="${pipelining}" size=1 submit_queues=1 zone_size=1 zoned=1 power=1 ) for p in "${params[@]}"; do echo "${p//*=}" > "${p//=*}" done ) udevadm settle dev=/dev/nullb0 [ -b "${dev}" ] params=( --direct=1 --filename="${dev}" --iodepth=64 --iodepth_batch=16 --ioengine=io_uring --ioscheduler="${iosched}" --gtod_reduce=1 --hipri=0 --name=nullb0 --runtime=30 --rw=write --time_based=1 --zonemode=zbd ) fio "${params[@]}" done Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- drivers/block/null_blk/main.c | 9 +++++++++ drivers/block/null_blk/null_blk.h | 3 +++ drivers/block/null_blk/zoned.c | 13 ++++++++++++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index fd68e6f4637f..de9ed9a4ca10 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -408,6 +408,7 @@ NULLB_DEVICE_ATTR(zone_capacity, ulong, NULL); NULLB_DEVICE_ATTR(zone_nr_conv, uint, NULL); NULLB_DEVICE_ATTR(zone_max_open, uint, NULL); NULLB_DEVICE_ATTR(zone_max_active, uint, NULL); +NULLB_DEVICE_ATTR(pipeline_zoned_writes, bool, NULL); NULLB_DEVICE_ATTR(virt_boundary, bool, NULL); static ssize_t nullb_device_power_show(struct config_item *item, char *page) @@ -531,6 +532,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_zone_nr_conv, &nullb_device_attr_zone_max_open, &nullb_device_attr_zone_max_active, + &nullb_device_attr_pipeline_zoned_writes, &nullb_device_attr_virt_boundary, NULL, }; @@ -1626,6 +1628,11 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, cmd->error = BLK_STS_OK; cmd->nq = nq; cmd->fake_timeout = should_timeout_request(rq); + if (!(rq->rq_flags & RQF_DONTPREP)) { + rq->rq_flags |= RQF_DONTPREP; + cmd->retries = 0; + cmd->max_attempts = rq->q->nr_requests; + } blk_mq_start_request(rq); @@ -2042,6 +2049,8 @@ static int null_add_dev(struct nullb_device *dev) nullb->q->queuedata = nullb; blk_queue_flag_set(QUEUE_FLAG_NONROT, nullb->q); blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, nullb->q); + if (dev->pipeline_zoned_writes) + blk_queue_flag_set(QUEUE_FLAG_PIPELINE_ZONED_WRITES, nullb->q); mutex_lock(&lock); nullb->index = ida_simple_get(&nullb_indexes, 0, 0, GFP_KERNEL); diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index 8359b43842f2..bbe2cb17bdbd 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -23,6 +23,8 @@ struct nullb_cmd { unsigned int tag; blk_status_t error; bool fake_timeout; + u16 retries; + u16 max_attempts; struct nullb_queue *nq; struct hrtimer timer; }; @@ -112,6 +114,7 @@ struct nullb_device { bool memory_backed; /* if data is stored in memory */ bool discard; /* if support discard */ bool zoned; /* if device is zoned */ + bool pipeline_zoned_writes; bool virt_boundary; /* virtual boundary on/off for the device */ }; diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 2fdd7b20c224..7138954dc01c 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -403,7 +403,18 @@ static blk_status_t null_zone_write(struct nullb_cmd *cmd, sector_t sector, else cmd->bio->bi_iter.bi_sector = sector; } else if (sector != zone->wp) { - ret = BLK_STS_IOERR; + /* + * In case of a misaligned write and if pipelining of zoned + * writes has been enabled, request the block layer to retry + * until the maximum number of attempts has been reached. If + * the maximum number of attempts has been reached, fail the + * misaligned write. + */ + if (dev->pipeline_zoned_writes && + ++cmd->retries < cmd->max_attempts) + ret = BLK_STS_DEV_RESOURCE; + else + ret = BLK_STS_IOERR; goto unlock; } From patchwork Mon Jun 27 23:43:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D013BCCA47B for ; Mon, 27 Jun 2022 23:44:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236532AbiF0XoC (ORCPT ); Mon, 27 Jun 2022 19:44:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242327AbiF0XoB (ORCPT ); Mon, 27 Jun 2022 19:44:01 -0400 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 825B813CEA for ; Mon, 27 Jun 2022 16:44:00 -0700 (PDT) Received: by mail-pf1-f174.google.com with SMTP id k9so1203409pfg.5 for ; Mon, 27 Jun 2022 16:44:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hqpsHFfHkm5Pkj04n/IH7gcU0rzDX2ha8rnKflqkGE8=; b=CfJ4F4DymiQXqq45+JcFhoykAvpjtb/bsKrSCRkjlU+/uXilCkcVxTrHfYRz0/8gT8 ylwu0XmgU//ieV0pbEU8aehf812kpwXNbYv4zdqh87hGjfRe5JpnOKYO7UveHuFgrhN/ jGX5BxYm1z+5QW5nsvvT6EPOL/LDKsl5E4UURAOJ/G64y5Ecm6UptI0hIr/TKIUgja0k g4XXW3aJGJBBteZbyEkifwTpRzMcU0IszNEwMx6Tx1nB8tdAUjpEhu63qTgRtBLIupWg TyvpDITkSoDRiBKDHPRCGcoQrCY0gzfbCR62gvRjUJolmYIMTRGTXZwHwiE3h+w7Ar3Y K46A== X-Gm-Message-State: AJIora9YTPKQI3u1qfRWjPWondf1uGC/ayjTpJ0P3ICamwTGKcAHBRGx NlXpNTkXjBlKdCxxnCAMyHI= X-Google-Smtp-Source: AGRyM1sZBFKZCombPzuzZmE26KUTSWsB+tN4KOLyrVdEaciwgi9hYcRZ8kE42iQG9zAqoSzWW6leHw== X-Received: by 2002:aa7:82c1:0:b0:525:ad2c:7271 with SMTP id f1-20020aa782c1000000b00525ad2c7271mr1648478pfn.30.1656373439891; Mon, 27 Jun 2022 16:43:59 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.43.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:43:59 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Keith Busch , Sagi Grimberg , Chaitanya Kulkarni Subject: [PATCH v3 7/8] nvme: Make the number of retries command specific Date: Mon, 27 Jun 2022 16:43:34 -0700 Message-Id: <20220627234335.1714393-8-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add support for specifying the number of retries per NVMe command. Cc: Christoph Hellwig Cc: Keith Busch Cc: Sagi Grimberg Cc: Chaitanya Kulkarni Signed-off-by: Bart Van Assche --- drivers/nvme/host/core.c | 3 ++- drivers/nvme/host/nvme.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index b3d9c29aba1e..df9ac7fab9b8 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -339,7 +339,7 @@ static inline enum nvme_disposition nvme_decide_disposition(struct request *req) if (blk_noretry_request(req) || (nvme_req(req)->status & NVME_SC_DNR) || - nvme_req(req)->retries >= nvme_max_retries) + nvme_req(req)->retries >= nvme_req(req)->max_retries) return COMPLETE; if (req->cmd_flags & REQ_NVME_MPATH) { @@ -632,6 +632,7 @@ static inline void nvme_clear_nvme_request(struct request *req) { nvme_req(req)->status = 0; nvme_req(req)->retries = 0; + nvme_req(req)->max_retries = nvme_max_retries; nvme_req(req)->flags = 0; req->rq_flags |= RQF_DONTPREP; } diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 0da94b233fed..ca415cd9571e 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -160,6 +160,7 @@ struct nvme_request { union nvme_result result; u8 genctr; u8 retries; + u8 max_retries; u8 flags; u16 status; struct nvme_ctrl *ctrl; From patchwork Mon Jun 27 23:43:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 12897298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30E82C43334 for ; Mon, 27 Jun 2022 23:44:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235245AbiF0XoD (ORCPT ); Mon, 27 Jun 2022 19:44:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242290AbiF0XoC (ORCPT ); Mon, 27 Jun 2022 19:44:02 -0400 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0739113D5C for ; Mon, 27 Jun 2022 16:44:02 -0700 (PDT) Received: by mail-pj1-f49.google.com with SMTP id g10-20020a17090a708a00b001ea8aadd42bso10964059pjk.0 for ; Mon, 27 Jun 2022 16:44:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ceApt9RGrSOfd7l9Wn70npr+5M+6N/tjqVvDqJkluDs=; b=lYv9zpEXJZqsxOpBG3kvC8rjK/iRwQeA711+ssCujjvMMaMAX2rIvndh0FOMgxTOr2 4nda/VKoRxLBofP9TTDA/RIJAIrXtj3H3cFDiVAa+syfOmrLZPdGDJSHLAd4J3x05fu1 uxXkDs3Ji4uP5xNhTvx7Ppmb/k6ecI4m7AVu9Sr9HR2qEXoCY+lNrXPSoaocDPZs28gJ fi3CyhaR60JApwbcQ9VXQHBeygFHnDILEa/EUFR0yN/iOWmykWctrExpslkUDBw1C+kd JkWUlX6nrI9N3De2h4dfBoDFR+mpkBUND1vYbez9ecCfRvsm9Bt07dldBtwJgD/QR/6C meWA== X-Gm-Message-State: AJIora9xCq3bb4uBm7w3SR7bO9b62lkqd90B70p8/cH262/+mdEmpWIY vYGPfCNFeUxLfiHktFJXlCo= X-Google-Smtp-Source: AGRyM1s+j4p08ihRgx9zgRgKfeC7DLJmopqOaEpDTvmQ72R58xJNoDPhTHDm33HfafeX8aoqhYJIPg== X-Received: by 2002:a17:903:248:b0:168:cf03:eefe with SMTP id j8-20020a170903024800b00168cf03eefemr667420plh.124.1656373441478; Mon, 27 Jun 2022 16:44:01 -0700 (PDT) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:3879:b18f:db0d:205]) by smtp.gmail.com with ESMTPSA id md6-20020a17090b23c600b001e305f5cd22sm7907456pjb.47.2022.06.27.16.44.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jun 2022 16:44:00 -0700 (PDT) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Jaegeuk Kim , Bart Van Assche , Keith Busch , Sagi Grimberg , Chaitanya Kulkarni Subject: [PATCH v3 8/8] nvme: Enable pipelining of zoned writes Date: Mon, 27 Jun 2022 16:43:35 -0700 Message-Id: <20220627234335.1714393-9-bvanassche@acm.org> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog In-Reply-To: <20220627234335.1714393-1-bvanassche@acm.org> References: <20220627234335.1714393-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Enabling pipelining for zoned writes. Increase the number of retries for zoned writes to the maximum number of outstanding commands per hardware queue. Cc: Christoph Hellwig Cc: Keith Busch Cc: Sagi Grimberg Cc: Chaitanya Kulkarni Signed-off-by: Bart Van Assche --- drivers/nvme/host/core.c | 6 ++++++ drivers/nvme/host/zns.c | 2 ++ 2 files changed, 8 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index df9ac7fab9b8..3f71d3d82d5e 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -854,6 +854,12 @@ static inline blk_status_t nvme_setup_rw(struct nvme_ns *ns, if (req->cmd_flags & REQ_RAHEAD) dsmgmt |= NVME_RW_DSM_FREQ_PREFETCH; + if (blk_queue_pipeline_zoned_writes(req->q) && + blk_rq_is_seq_zone_write(req)) + nvme_req(req)->max_retries = + min(0UL + type_max(typeof(nvme_req(req)->max_retries)), + nvme_req(req)->max_retries + req->q->nr_requests); + cmnd->rw.opcode = op; cmnd->rw.flags = 0; cmnd->rw.nsid = cpu_to_le32(ns->head->ns_id); diff --git a/drivers/nvme/host/zns.c b/drivers/nvme/host/zns.c index 9f81beb4df4e..0b10db3b8d3a 100644 --- a/drivers/nvme/host/zns.c +++ b/drivers/nvme/host/zns.c @@ -54,6 +54,8 @@ int nvme_update_zone_info(struct nvme_ns *ns, unsigned lbaf) struct nvme_id_ns_zns *id; int status; + blk_queue_flag_set(QUEUE_FLAG_PIPELINE_ZONED_WRITES, ns->queue); + /* Driver requires zone append support */ if ((le32_to_cpu(log->iocs[nvme_cmd_zone_append]) & NVME_CMD_EFFECTS_CSUPP)) {