From patchwork Mon Jan 9 23:27:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9086C54EBD for ; Mon, 9 Jan 2023 23:28:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237890AbjAIX2K (ORCPT ); Mon, 9 Jan 2023 18:28:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237906AbjAIX1s (ORCPT ); Mon, 9 Jan 2023 18:27:48 -0500 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1163A140F9 for ; Mon, 9 Jan 2023 15:27:48 -0800 (PST) Received: by mail-pf1-f173.google.com with SMTP id c9so7424578pfj.5 for ; Mon, 09 Jan 2023 15:27:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cOlCWcSLEYoc+DC54TwVl5DO3M0BRFT5LbwS4adE59w=; b=laNnJU9WToG2LLlAWI+Env9wpSm7Pf/qNzCQSZe+F6Az4uHeNGBt7yhmLFt5yW4Ol0 SjbtyKafehq+379qBkTc2maxPekppU5yWcNiqgvBSHc0q1x82si1i1UR8TkAFLmrneIr 3y5iR8sHnwcQSBWphy69P1VAORHBqFl1lOwGxWoPOW4F0wk3C2K0j99b/q4LIGKm0dSz IdXZeLbab5gR2q5mdgzQezMDGMsbl+uptPl/nOm6wRojiXDs/A1mZARUihjJn1Y0AmPe q5yJ/IwA4V4oWRzMf9Ytf9yU37fQk/iWq66ktdmrzgfQJ6iszZJpCf0J3B8D9+d95RXa PS8A== X-Gm-Message-State: AFqh2kpjfYkfbQXnQ6L81EG8oobtlWU1ma5Sxb7rbvoHbIV+oOOxOZA2 FnzukCsi8ImCzJTpHuJRieI= X-Google-Smtp-Source: AMrXdXv5nV9jFTli4ksUbS0rt+NPMu18SNEYFKsvpQk2saFobpxLncRV7viwuqa6OCmQ9BY8ZlaYNQ== X-Received: by 2002:aa7:86ce:0:b0:582:f129:7b93 with SMTP id h14-20020aa786ce000000b00582f1297b93mr19843476pfo.33.1673306867500; Mon, 09 Jan 2023 15:27:47 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:46 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , Damien Le Moal Subject: [PATCH 1/8] block: Document blk_queue_zone_is_seq() and blk_rq_zone_is_seq() Date: Mon, 9 Jan 2023 15:27:31 -0800 Message-Id: <20230109232738.169886-2-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Since it is nontrivial to figure out how disk_zone_is_seq() and blk_rq_zone_is_seq() handle sequential write preferred zones, document this. Cc: Damien Le Moal Signed-off-by: Bart Van Assche Reviewed-by: Damien Le Moal --- include/linux/blk-mq.h | 7 +++++++ include/linux/blkdev.h | 9 +++++++++ 2 files changed, 16 insertions(+) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 779fba613bd0..6735db1ad24d 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1155,6 +1155,13 @@ static inline unsigned int blk_rq_zone_no(struct request *rq) return disk_zone_no(rq->q->disk, blk_rq_pos(rq)); } +/** + * blk_rq_zone_is_seq() - Whether a request is for a sequential zone. + * @rq: Request pointer. + * + * Return: true if and only if blk_rq_pos(@rq) refers either to a sequential + * write required or to a sequential write preferred zone. + */ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) { return disk_zone_is_seq(rq->q->disk, blk_rq_pos(rq)); diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index b87ed829ab94..ef93e848b1fd 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -672,6 +672,15 @@ static inline unsigned int disk_zone_no(struct gendisk *disk, sector_t sector) return sector >> ilog2(disk->queue->limits.chunk_sectors); } +/** + * disk_zone_is_seq() - Whether a logical block is in a sequential zone. + * @disk: Disk pointer. + * @sector: Offset from start of block device in 512 byte units. + * + * Return: true if and only if @disk refers to a zoned block device and + * @sector refers either to a sequential write required or a sequential + * write preferred zone. + */ static inline bool disk_zone_is_seq(struct gendisk *disk, sector_t sector) { if (!blk_queue_is_zoned(disk->queue)) From patchwork Mon Jan 9 23:27:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B85FBC54EBD for ; Mon, 9 Jan 2023 23:28:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237884AbjAIX2I (ORCPT ); Mon, 9 Jan 2023 18:28:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237913AbjAIX1u (ORCPT ); Mon, 9 Jan 2023 18:27:50 -0500 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B56DFB49B for ; Mon, 9 Jan 2023 15:27:49 -0800 (PST) Received: by mail-pg1-f172.google.com with SMTP id r18so6984562pgr.12 for ; Mon, 09 Jan 2023 15:27:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S0TPuowUZcajvA95Bw/M67Fj3nH+Km848JrYffvVSDE=; b=thYOdn5GiEAlvvis5b1I/5hhIpMl1Q8Hhk1YXdu9xiGwF3bsaIYj0lVMzEOF9yFpPy yC5eApuYdrDR2Lr3jhq1Ip9sBB+poc5ffmckgsd9/LLA8kyBM7fBRg7Ywv8wec6BxC9d 9Ha7tSkNdIOjW2R/OE5iM7N3BftS0LyqGQDBEXfTwxtA6SDmF8ak8CRQ7d6XmsALPOUl UaquO98wDQVVNhviKMBzpEwl8blBF6A2tf9vZf9XnSS3+ImEffp1qACZcjozv+tmFbo7 ljhCIX6Wm/5wJyuMgp7a4iqloiuUfvobZI8B6uPENEnPUxTFLlQaSrmZD9l+6sIagoBi z2dg== X-Gm-Message-State: AFqh2kqksIFyF5O4TpBF6bd6RTbfPVdhZazqxugGT7O/D174i0IWApcM qFSOCmdhKTFBOJeSYsqNz4c= X-Google-Smtp-Source: AMrXdXvSb5pCqIkemxqgh4iJXcnMha6vtPPZTQvQPVIPql+NIa9oifXL73a6g2CQglcJMVnnsXNl0w== X-Received: by 2002:a05:6a00:d2:b0:588:22ad:2928 with SMTP id e18-20020a056a0000d200b0058822ad2928mr5741993pfj.3.1673306869201; Mon, 09 Jan 2023 15:27:49 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:48 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , Damien Le Moal Subject: [PATCH 2/8] block: Introduce the blk_rq_is_seq_zone_write() function Date: Mon, 9 Jan 2023 15:27:32 -0800 Message-Id: <20230109232738.169886-3-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce a function that makes it easy to verify whether a write request is for a sequential write required or sequential write preferred zone. Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- include/linux/blk-mq.h | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 6735db1ad24d..29f834e5c3a9 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1167,6 +1167,24 @@ static inline unsigned int blk_rq_zone_is_seq(struct request *rq) return disk_zone_is_seq(rq->q->disk, blk_rq_pos(rq)); } +/** + * blk_rq_is_seq_zone_write() - Whether @rq is a write request for a sequential zone. + * @rq: Request to examine. + * + * In this context sequential zone means either a sequential write required or + * to a sequential write preferred zone. + */ +static inline bool blk_rq_is_seq_zone_write(struct request *rq) +{ + switch (req_op(rq)) { + case REQ_OP_WRITE: + case REQ_OP_WRITE_ZEROES: + return blk_rq_zone_is_seq(rq); + default: + return false; + } +} + bool blk_req_needs_zone_write_lock(struct request *rq); bool blk_req_zone_write_trylock(struct request *rq); void __blk_req_zone_write_lock(struct request *rq); @@ -1197,6 +1215,11 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq) return !blk_req_zone_is_write_locked(rq); } #else /* CONFIG_BLK_DEV_ZONED */ +static inline bool blk_rq_is_seq_zone_write(struct request *rq) +{ + return false; +} + static inline bool blk_req_needs_zone_write_lock(struct request *rq) { return false; From patchwork Mon Jan 9 23:27:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56048C67871 for ; Mon, 9 Jan 2023 23:28:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237880AbjAIX2I (ORCPT ); Mon, 9 Jan 2023 18:28:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237916AbjAIX1x (ORCPT ); Mon, 9 Jan 2023 18:27:53 -0500 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5898DB49B for ; Mon, 9 Jan 2023 15:27:52 -0800 (PST) Received: by mail-pg1-f172.google.com with SMTP id b12so7000990pgj.6 for ; Mon, 09 Jan 2023 15:27:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oXlrFhwH4HhJPFIyGLm6XVz2plcq5CCa62cjFU51M2c=; b=ey0ubQptrxbLVVU1Ep4D6JUQXhCeuJeFgXG/0MkgsHrEgmPAdyj/LO89Kue9heKWMK L5Hm2ogJ6IQULq9ongJJfbcHfGBshDeAD4j1VJuliq8AQZc1/T5FQGpMdcnHIOhhB1n9 0o2cmxDQpcroPA9F2AV2lKhacSi1KBXcv+1sCLyEgLBeDi8hPKzYZ5uGFFhY/GiFP0JV mECfsTTue9cIYq1d48J4gLKo0E/rFg7Nv2iHsgXdJnCu9wa5nF44AUMwUnAsZoBbLWK7 P/rh+oEF+hgmOPD8aS8pFrJezEth1D6VECO1dTMo6+FT3f0tfwlArAIpPJITLgd/l0m4 DtjA== X-Gm-Message-State: AFqh2kraOFxI86ghgkpJjVfJfoWU68hEBQRSJ9BEvt01Ofgw0vc8WDvB +EDZgJkBEnOsn+2+uYdxFlE= X-Google-Smtp-Source: AMrXdXuLLP7vdW0TU4ha5YBneNSs7AAwwRAgp53VQUxLBV4Mu8Pjxonk3doxkmTW8f/2ZNoO3hdMKQ== X-Received: by 2002:aa7:96f9:0:b0:582:7007:3fec with SMTP id i25-20020aa796f9000000b0058270073fecmr30005701pfq.11.1673306871791; Mon, 09 Jan 2023 15:27:51 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:50 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , Damien Le Moal Subject: [PATCH 3/8] block: Introduce a request queue flag for pipelining zoned writes Date: Mon, 9 Jan 2023 15:27:33 -0800 Message-Id: <20230109232738.169886-4-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Writes in sequential write required zones must happen at the write pointer. Even if the submitter of the write commands (e.g. a filesystem) submits writes for sequential write required zones in order, the block layer or the storage controller may reorder these write commands. The zone locking mechanism in the mq-deadline I/O scheduler serializes write commands for sequential zones. Some but not all storage controllers require this serialization. Introduce a new flag such that block drivers can request pipelining of writes for sequential write required zones. An example of a storage controller standard that requires write serialization is AHCI (Advanced Host Controller Interface). Submitting commands to AHCI controllers happens by writing a bit pattern into a register. Each set bit corresponds to an active command. This mechanism does not preserve command ordering information. Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- include/linux/blkdev.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index ef93e848b1fd..b4b4cd3f2912 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -553,6 +553,8 @@ struct request_queue { #define QUEUE_FLAG_NONROT 6 /* non-rotational device (SSD) */ #define QUEUE_FLAG_VIRT QUEUE_FLAG_NONROT /* paravirt device */ #define QUEUE_FLAG_IO_STAT 7 /* do disk/partitions IO accounting */ +/* Writes for sequential write required zones may be pipelined. */ +#define QUEUE_FLAG_PIPELINE_ZONED_WRITES 8 #define QUEUE_FLAG_NOXMERGES 9 /* No extended merges */ #define QUEUE_FLAG_ADD_RANDOM 10 /* Contributes to random pool */ #define QUEUE_FLAG_SAME_FORCE 12 /* force complete on same CPU */ @@ -614,6 +616,11 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_skip_tagset_quiesce(q) \ test_bit(QUEUE_FLAG_SKIP_TAGSET_QUIESCE, &(q)->queue_flags) +static inline bool blk_queue_pipeline_zoned_writes(struct request_queue *q) +{ + return test_bit(QUEUE_FLAG_PIPELINE_ZONED_WRITES, &q->queue_flags); +} + extern void blk_set_pm_only(struct request_queue *q); extern void blk_clear_pm_only(struct request_queue *q); From patchwork Mon Jan 9 23:27:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45CEFC54EBE for ; Mon, 9 Jan 2023 23:28:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237462AbjAIX2H (ORCPT ); Mon, 9 Jan 2023 18:28:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237919AbjAIX1z (ORCPT ); Mon, 9 Jan 2023 18:27:55 -0500 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F8A1B49B for ; Mon, 9 Jan 2023 15:27:54 -0800 (PST) Received: by mail-pl1-f179.google.com with SMTP id d3so11288447plr.10 for ; Mon, 09 Jan 2023 15:27:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BH5vLribh8aOPgt9vOhItoFB0+zm3eKjdJ+5DkhJ/Mw=; b=soaSVYpsQVlVmhIhukVN076wtV+vle79el03fpoq5d1zpFXAQD3d6bhEL7tKs9hejN nl8ja1R03840j5ludNOeNjE6l3xKC2Vn9UcBAFgG9Qpb8hRTpgIDp/HjZBY+sZXCy7iN vp1DzLhdGmCBRMUyhlQU/rTrLtDQKx5IoO2W8yr0W5Y8b+2mufm1zrWIj8ZA9Iw7tam8 i2At9JmB6LdHuZeq/AGs2jgatEM2Nv7DiNjY2/HOrn5dqzh+E8kOVlihUvmKZkJ7KjDP rtHzywvWtXXIrYdvflSDman620vFBxb/5V6FMyBJoURYUb46yW6cGoeLR52aBb4C3Ea2 vKFg== X-Gm-Message-State: AFqh2kotne2KcyUPWUrckbqfRQXOCqwXZpshGqyUG6gDAGy6OSRLPWkD +LZPmLL4Px2Qwtf9Egkfr3w= X-Google-Smtp-Source: AMrXdXtvD0lj7JpKXm/529NN5IfUWILW6UFx64x2+zLnMYnxYqWXoyEzKgpeDw8MLaIA762dBUpjuA== X-Received: by 2002:a05:6a20:4e10:b0:aa:4010:8886 with SMTP id gk16-20020a056a204e1000b000aa40108886mr73610178pzb.1.1673306873585; Mon, 09 Jan 2023 15:27:53 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:52 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , Damien Le Moal Subject: [PATCH 4/8] block/mq-deadline: Only use zone locking if necessary Date: Mon, 9 Jan 2023 15:27:34 -0800 Message-Id: <20230109232738.169886-5-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Measurements have shown that limiting the queue depth to one for zoned writes has a significant negative performance impact on zoned UFS devices. Hence this patch that disables zone locking from the mq-deadline scheduler for storage controllers that support pipelining zoned writes. This patch is based on the following assumptions: - Applications submit write requests to sequential write required zones in order. - It happens infrequently that zoned write requests are reordered by the block layer. - The storage controller does not reorder write requests that have been submitted to the same hardware queue. This is the case for UFS: the UFSHCI specification requires that UFS controllers process requests in order per hardware queue. - The I/O priority of all pipelined write requests is the same per zone. - Either no I/O scheduler is used or an I/O scheduler is used that submits write requests per zone in LBA order. If applications submit write requests to sequential write required zones in order, at least one of the pending requests will succeed. Hence, the number of retries that is required is at most (number of pending requests) - 1. Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- block/blk-zoned.c | 3 ++- block/mq-deadline.c | 14 +++++++++----- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/block/blk-zoned.c b/block/blk-zoned.c index db829401d8d0..158638091e39 100644 --- a/block/blk-zoned.c +++ b/block/blk-zoned.c @@ -520,7 +520,8 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx, break; case BLK_ZONE_TYPE_SEQWRITE_REQ: case BLK_ZONE_TYPE_SEQWRITE_PREF: - if (!args->seq_zones_wlock) { + if (!blk_queue_pipeline_zoned_writes(q) && + !args->seq_zones_wlock) { args->seq_zones_wlock = blk_alloc_zone_bitmap(q->node, args->nr_zones); if (!args->seq_zones_wlock) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index f10c2a0d18d4..e41808c0b007 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -339,7 +339,8 @@ deadline_fifo_request(struct deadline_data *dd, struct dd_per_prio *per_prio, return NULL; rq = rq_entry_fifo(per_prio->fifo_list[data_dir].next); - if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q)) + if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q) || + blk_queue_pipeline_zoned_writes(rq->q)) return rq; /* @@ -378,7 +379,8 @@ deadline_next_request(struct deadline_data *dd, struct dd_per_prio *per_prio, if (!rq) return NULL; - if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q)) + if (data_dir == DD_READ || !blk_queue_is_zoned(rq->q) || + blk_queue_pipeline_zoned_writes(rq->q)) return rq; /* @@ -503,8 +505,9 @@ static struct request *__dd_dispatch_request(struct deadline_data *dd, } /* - * For a zoned block device, if we only have writes queued and none of - * them can be dispatched, rq will be NULL. + * For a zoned block device that requires write serialization, if we + * only have writes queued and none of them can be dispatched, rq will + * be NULL. */ if (!rq) return NULL; @@ -893,7 +896,8 @@ static void dd_finish_request(struct request *rq) atomic_inc(&per_prio->stats.completed); - if (blk_queue_is_zoned(q)) { + if (blk_queue_is_zoned(rq->q) && + !blk_queue_pipeline_zoned_writes(q)) { unsigned long flags; spin_lock_irqsave(&dd->zone_lock, flags); From patchwork Mon Jan 9 23:27:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E83DC63797 for ; Mon, 9 Jan 2023 23:28:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237865AbjAIX2H (ORCPT ); Mon, 9 Jan 2023 18:28:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237924AbjAIX14 (ORCPT ); Mon, 9 Jan 2023 18:27:56 -0500 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F24BFB49B for ; Mon, 9 Jan 2023 15:27:55 -0800 (PST) Received: by mail-pl1-f174.google.com with SMTP id v23so6398559plo.1 for ; Mon, 09 Jan 2023 15:27:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GUCV5HwEYttl/XmNL0bKMbcwHvDrjNKe2/xgoVJatNg=; b=lMKYYPrF2EzENM9LfFeJuj1w6OLtJTEjZo2GAYdwCP5yd56+UG1CJ7Kouyp2gjVhIl Zb5VblcWnp40uuPHrvKIZcsTJutvh0cAgla2O02kGTWVdZUDjzIVNOBnG5HIMPC0WShl 5kLoSC8RVErxGciixe4s5KpeWCjC6xhb0kYERnzdn5PpTGHXSSYCNkJrSUX0CKyvp2eI kSlsCWF/rjQpc0WVIo2Nud0W/Lh0vFkK8ep5p4w8oAyyY3cU+FctKmjqrjwRP8HDVEn7 rpaW0d2COCqTSaKCumpcAn/vgH/3Am/g4Qc7Vl411IBz4XAApsk+wWf16lrIaU2r1i3e MRFg== X-Gm-Message-State: AFqh2koGHrnZQA0jmgG3Ixbp0tDNEiMlS1ldGb5OqORz6puSy+wQyEmE bsrTOatXFSx4hbv867pYa8xGpzCkQa0= X-Google-Smtp-Source: AMrXdXuv654MholQ2AhY8DH/FyXNCFIHD8AZRWV3zCJTbZTRdHELibvmb/fVpQ2v5EO7qvdv5pkySw== X-Received: by 2002:a05:6a21:1507:b0:9d:efbe:529a with SMTP id nq7-20020a056a21150700b0009defbe529amr77088396pzb.10.1673306875414; Mon, 09 Jan 2023 15:27:55 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:54 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , Damien Le Moal Subject: [PATCH 5/8] block/null_blk: Refactor null_queue_rq() Date: Mon, 9 Jan 2023 15:27:35 -0800 Message-Id: <20230109232738.169886-6-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Introduce a local variable for the expression bd->rq since that expression occurs multiple times. This patch does not change any functionality. Reviewed-by: Damien Le Moal Signed-off-by: Bart Van Assche --- drivers/block/null_blk/main.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 4c601ca9552a..d401230b1e20 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -1660,10 +1660,11 @@ static enum blk_eh_timer_return null_timeout_rq(struct request *rq) static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, const struct blk_mq_queue_data *bd) { - struct nullb_cmd *cmd = blk_mq_rq_to_pdu(bd->rq); + struct request *rq = bd->rq; + struct nullb_cmd *cmd = blk_mq_rq_to_pdu(rq); struct nullb_queue *nq = hctx->driver_data; - sector_t nr_sectors = blk_rq_sectors(bd->rq); - sector_t sector = blk_rq_pos(bd->rq); + sector_t nr_sectors = blk_rq_sectors(rq); + sector_t sector = blk_rq_pos(rq); const bool is_poll = hctx->type == HCTX_TYPE_POLL; might_sleep_if(hctx->flags & BLK_MQ_F_BLOCKING); @@ -1672,14 +1673,14 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, hrtimer_init(&cmd->timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); cmd->timer.function = null_cmd_timer_expired; } - cmd->rq = bd->rq; + cmd->rq = rq; cmd->error = BLK_STS_OK; cmd->nq = nq; - cmd->fake_timeout = should_timeout_request(bd->rq); + cmd->fake_timeout = should_timeout_request(rq); - blk_mq_start_request(bd->rq); + blk_mq_start_request(rq); - if (should_requeue_request(bd->rq)) { + if (should_requeue_request(rq)) { /* * Alternate between hitting the core BUSY path, and the * driver driven requeue path @@ -1688,21 +1689,21 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, if (nq->requeue_selection & 1) return BLK_STS_RESOURCE; else { - blk_mq_requeue_request(bd->rq, true); + blk_mq_requeue_request(rq, true); return BLK_STS_OK; } } if (is_poll) { spin_lock(&nq->poll_lock); - list_add_tail(&bd->rq->queuelist, &nq->poll_list); + list_add_tail(&rq->queuelist, &nq->poll_list); spin_unlock(&nq->poll_lock); return BLK_STS_OK; } if (cmd->fake_timeout) return BLK_STS_OK; - return null_handle_cmd(cmd, sector, nr_sectors, req_op(bd->rq)); + return null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); } static void cleanup_queue(struct nullb_queue *nq) From patchwork Mon Jan 9 23:27:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CF80C5479D for ; Mon, 9 Jan 2023 23:28:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235477AbjAIX2G (ORCPT ); Mon, 9 Jan 2023 18:28:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237928AbjAIX16 (ORCPT ); Mon, 9 Jan 2023 18:27:58 -0500 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDB48B49B for ; Mon, 9 Jan 2023 15:27:57 -0800 (PST) Received: by mail-pf1-f170.google.com with SMTP id h7so3223015pfq.4 for ; Mon, 09 Jan 2023 15:27:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VTWtv50Jtxyv1f5j/Gay7tfPjI4FYKNdpnJ4KOOz+hI=; b=sJAHTHSB2LbIHJKH6D15a0HMOnbM2VyAKuvAOSqLt2gpuedAR7ZmuezKRvJRovg1W2 X5DvFPv/ZI2BRNzc4AUof0VB5xDuUPosMFC0/W32BBM1UnVBXvpiMb+eXTOuczCX6N33 GW2rMh/ODlhNI6WyZNbCIckCgXFssBrq53dJK6FuLigRjE4Xk25P/MDcj173/S1B07i/ M4ZWzB/U9RImBoQlK/sQrhiL79jV5XzRZPhytXKCRPJrfQhZ2sWJEJlo0qQ/PS20U69s LlMPqePpRo59YYkGfEc28JGPTunwuBxwbBoukbfu1kC5y/bPmFsoGjySNrLYHQ8ymCwq zGdA== X-Gm-Message-State: AFqh2kraCDp1wl+qhWhZawTmZ5Pt+JMxRUEEqvUP0ZaZHtRcPz8Dw+p3 MVZDNNw9N0hWwJ+be3vrY/6qGDwe+bE= X-Google-Smtp-Source: AMrXdXsOLkK3kGqXmlfmjW5YlILZhjRPZKCt+avP/5yZQ5m5REfVOC/GRBr99tDG7JfoEQ+naeTMXw== X-Received: by 2002:a05:6a00:400e:b0:577:9807:543b with SMTP id by14-20020a056a00400e00b005779807543bmr74003698pfb.16.1673306877192; Mon, 09 Jan 2023 15:27:57 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:56 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , Damien Le Moal Subject: [PATCH 6/8] block/null_blk: Add support for pipelining zoned writes Date: Mon, 9 Jan 2023 15:27:36 -0800 Message-Id: <20230109232738.169886-7-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Add a new configfs attribute for enabling pipelining of zoned writes. If that attribute has been set, retry zoned writes that are not aligned with the write pointer. The test script below reports 234 K IOPS with no I/O scheduler, 5.32 K IOPS with mq-deadline and pipelining disabled and 92.2 K IOPS with mq-deadline and pipelining enabled. This shows that pipelining results in about 17 times more IOPS for this particular test case. #!/bin/bash for mode in "none 0" "mq-deadline 0" "mq-deadline 1"; do set +e for d in /sys/kernel/config/nullb/*; do [ -d "$d" ] && rmdir "$d" done modprobe -r null_blk set -e read -r iosched pipelining <<<"$mode" modprobe null_blk nr_devices=0 ( cd /sys/kernel/config/nullb mkdir nullb0 cd nullb0 params=( completion_nsec=100000 hw_queue_depth=64 irqmode=2 # NULL_IRQ_TIMER max_sectors=$((4096/512)) memory_backed=1 pipeline_zoned_writes="${pipelining}" size=1 submit_queues=1 zone_size=1 zoned=1 power=1 ) for p in "${params[@]}"; do echo "${p//*=}" > "${p//=*}" done ) udevadm settle dev=/dev/nullb0 [ -b "${dev}" ] params=( --direct=1 --filename="${dev}" --iodepth=64 --iodepth_batch=16 --ioengine=io_uring --ioscheduler="${iosched}" --gtod_reduce=1 --hipri=0 --name=nullb0 --runtime=30 --rw=write --time_based=1 --zonemode=zbd ) fio "${params[@]}" done Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- drivers/block/null_blk/main.c | 9 +++++++++ drivers/block/null_blk/null_blk.h | 3 +++ drivers/block/null_blk/zoned.c | 13 ++++++++++++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index d401230b1e20..851b55b7284f 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -424,6 +424,7 @@ NULLB_DEVICE_ATTR(zone_capacity, ulong, NULL); NULLB_DEVICE_ATTR(zone_nr_conv, uint, NULL); NULLB_DEVICE_ATTR(zone_max_open, uint, NULL); NULLB_DEVICE_ATTR(zone_max_active, uint, NULL); +NULLB_DEVICE_ATTR(pipeline_zoned_writes, bool, NULL); NULLB_DEVICE_ATTR(virt_boundary, bool, NULL); NULLB_DEVICE_ATTR(no_sched, bool, NULL); NULLB_DEVICE_ATTR(shared_tag_bitmap, bool, NULL); @@ -569,6 +570,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_zone_max_active, &nullb_device_attr_zone_readonly, &nullb_device_attr_zone_offline, + &nullb_device_attr_pipeline_zoned_writes, &nullb_device_attr_virt_boundary, &nullb_device_attr_no_sched, &nullb_device_attr_shared_tag_bitmap, @@ -1677,6 +1679,11 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, cmd->error = BLK_STS_OK; cmd->nq = nq; cmd->fake_timeout = should_timeout_request(rq); + if (!(rq->rq_flags & RQF_DONTPREP)) { + rq->rq_flags |= RQF_DONTPREP; + cmd->retries = 0; + cmd->max_attempts = rq->q->nr_requests; + } blk_mq_start_request(rq); @@ -2109,6 +2116,8 @@ static int null_add_dev(struct nullb_device *dev) nullb->q->queuedata = nullb; blk_queue_flag_set(QUEUE_FLAG_NONROT, nullb->q); blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, nullb->q); + if (dev->pipeline_zoned_writes) + blk_queue_flag_set(QUEUE_FLAG_PIPELINE_ZONED_WRITES, nullb->q); mutex_lock(&lock); rv = ida_simple_get(&nullb_indexes, 0, 0, GFP_KERNEL); diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index eb5972c50be8..c44c3fdb1025 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -23,6 +23,8 @@ struct nullb_cmd { unsigned int tag; blk_status_t error; bool fake_timeout; + u16 retries; + u16 max_attempts; struct nullb_queue *nq; struct hrtimer timer; }; @@ -112,6 +114,7 @@ struct nullb_device { bool memory_backed; /* if data is stored in memory */ bool discard; /* if support discard */ bool zoned; /* if device is zoned */ + bool pipeline_zoned_writes; bool virt_boundary; /* virtual boundary on/off for the device */ bool no_sched; /* no IO scheduler for the device */ bool shared_tag_bitmap; /* use hostwide shared tags */ diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 635ce0648133..cffc4985a7df 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -405,7 +405,18 @@ static blk_status_t null_zone_write(struct nullb_cmd *cmd, sector_t sector, else cmd->bio->bi_iter.bi_sector = sector; } else if (sector != zone->wp) { - ret = BLK_STS_IOERR; + /* + * In case of a misaligned write and if pipelining of zoned + * writes has been enabled, request the block layer to retry + * until the maximum number of attempts has been reached. If + * the maximum number of attempts has been reached, fail the + * misaligned write. + */ + if (dev->pipeline_zoned_writes && + ++cmd->retries < cmd->max_attempts) + ret = BLK_STS_DEV_RESOURCE; + else + ret = BLK_STS_IOERR; goto unlock; } From patchwork Mon Jan 9 23:27:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094522 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AB5AC5479D for ; Mon, 9 Jan 2023 23:28:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237888AbjAIX2J (ORCPT ); Mon, 9 Jan 2023 18:28:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237930AbjAIX2A (ORCPT ); Mon, 9 Jan 2023 18:28:00 -0500 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91439B49B for ; Mon, 9 Jan 2023 15:27:59 -0800 (PST) Received: by mail-pl1-f181.google.com with SMTP id c6so11319595pls.4 for ; Mon, 09 Jan 2023 15:27:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lChsis0Vn/H82l7R/PRp89geNpa9iV9NWjl6ZB1IdXU=; b=NgE1PDgjhouIArIm2OxUI94htfFmgvQT2qSzU+nCMuzYrp6RRj4eNdX56BW+TXYrcI N3hb4+lspb8uyJ+KSCMrLz+SpyH4Zo3Tkyhf96VCpZhBIeSXLC39bK9fc+54CrjiZj3L f5VU0guuvroZyxei8DOB/MBianDBVMem98Lnq+6qz8HC6mKGiD+ReOFY19Zk2udXtROB wREQNDzVYBCW5RlUOqnoHWdOu1xObmfGg0meWivsGFmP5O4eqlIVcJqQfKqcmTnzju2Y QjY7invZfH5pqikFYCI8gGsR9OF2XB0RhmtHnd1ASbBSyUG/UmBDv5bKAFv4qtWU9Wqd QpEQ== X-Gm-Message-State: AFqh2krInZ2GneIHHCihHUU/+iRD6xuvlNtSTLlV6X8SJq4342VJ4Xee knLLSjt/DgcePhzt9UJuo2g= X-Google-Smtp-Source: AMrXdXuv63l6RxGaiOeSyBDN9RNxr4Le+OezyhJ1yKbMtUKA8nC4WWYlm90NbAhnkNWtgYN2aY+9wA== X-Received: by 2002:a05:6a20:1e56:b0:af:98cd:9d82 with SMTP id cy22-20020a056a201e5600b000af98cd9d82mr64456446pzb.26.1673306879000; Mon, 09 Jan 2023 15:27:59 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:58 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , "Martin K . Petersen" , Damien Le Moal Subject: [PATCH 7/8] scsi: Retry unaligned zoned writes Date: Mon, 9 Jan 2023 15:27:37 -0800 Message-Id: <20230109232738.169886-8-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From ZBC-2: "The device server terminates with CHECK CONDITION status, with the sense key set to ILLEGAL REQUEST, and the additional sense code set to UNALIGNED WRITE COMMAND a write command, other than an entire medium write same command, that specifies: a) the starting LBA in a sequential write required zone set to a value that is not equal to the write pointer for that sequential write required zone; or b) an ending LBA that is not equal to the last logical block within a physical block (see SBC-5)." I am not aware of any other conditions that may trigger the UNALIGNED WRITE COMMAND response. Retry unaligned writes in preparation of removing zone locking. Increase the number of retries for write commands sent to a sequential zone to the maximum number of outstanding commands. Cc: Martin K. Petersen Cc: Damien Le Moal Signed-off-by: Bart Van Assche --- drivers/scsi/scsi_error.c | 7 +++++++ drivers/scsi/sd.c | 3 +++ 2 files changed, 10 insertions(+) diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c index a7960ad2d386..e17f36b0b18a 100644 --- a/drivers/scsi/scsi_error.c +++ b/drivers/scsi/scsi_error.c @@ -676,6 +676,13 @@ enum scsi_disposition scsi_check_sense(struct scsi_cmnd *scmd) fallthrough; case ILLEGAL_REQUEST: + /* + * Unaligned write command. Retry immediately if pipelining + * zoned writes has been enabled. + */ + if (sshdr.asc == 0x21 && sshdr.ascq == 0x04 && + blk_queue_pipeline_zoned_writes(sdev->request_queue)) + return NEEDS_RETRY; if (sshdr.asc == 0x20 || /* Invalid command operation code */ sshdr.asc == 0x21 || /* Logical block address out of range */ sshdr.asc == 0x22 || /* Invalid function */ diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 47dafe6b8a66..cd90b54a6597 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -1207,6 +1207,9 @@ static blk_status_t sd_setup_read_write_cmnd(struct scsi_cmnd *cmd) cmd->transfersize = sdp->sector_size; cmd->underflow = nr_blocks << 9; cmd->allowed = sdkp->max_retries; + if (blk_queue_pipeline_zoned_writes(rq->q) && + blk_rq_is_seq_zone_write(rq)) + cmd->allowed += rq->q->nr_requests; cmd->sdb.length = nr_blocks * sdp->sector_size; SCSI_LOG_HLQUEUE(1, From patchwork Mon Jan 9 23:27:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 13094524 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA640C63797 for ; Mon, 9 Jan 2023 23:28:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237867AbjAIX2L (ORCPT ); Mon, 9 Jan 2023 18:28:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237945AbjAIX2C (ORCPT ); Mon, 9 Jan 2023 18:28:02 -0500 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3497CB49B for ; Mon, 9 Jan 2023 15:28:01 -0800 (PST) Received: by mail-pf1-f175.google.com with SMTP id g20so7428173pfb.3 for ; Mon, 09 Jan 2023 15:28:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FWjzdVi+aWCeBtDSYSYYVmiFG7iSPPW9f3buG776fN4=; b=QFKcjRYZASp+O+/2yx24VaWTKvxjwMc73XLcU7QQE9szzflEAnkOXEOKs5AxH0gOTk uJFHWzoAar6k+tl8S+YytlGtJS1PNfPre6P7OAVwLmvPyjtjiWFCYKFNrdfg7dokpsqv KjhLHAcm5UKDlS1NL0waSrzKEK9JFWFeeUN+/1NpNunJN5ziQS82dvGcR6qcHkKGnbUD ewHgwU8BlhuDvyuCPDolb2n1/pn7G9PW06vkIUBNTUDdaW2pwunfzbphyzsKTaYXtbun wITTZwqtXCzR2Krqh1Z+cb6+MLQ7ymcgnGSiglFZ+ppq1OZlrz4LAZQ3IKSEgcAFQiaV q14g== X-Gm-Message-State: AFqh2kq3AMvQ5dR8086bTd2w8QFEpnLBO0quEVHRJ975jCClhuS1yHZw mr/+DvXqH1ZbS16zUut501k= X-Google-Smtp-Source: AMrXdXtbrhAvSS4fg34fXFxwLfYAT7R8ti1/P5zSwM65EmGYclDYBq0GedKs7uAMmDr7p7ZaXe6PQQ== X-Received: by 2002:a62:1d0e:0:b0:57f:ef11:acf8 with SMTP id d14-20020a621d0e000000b0057fef11acf8mr64630016pfd.2.1673306880671; Mon, 09 Jan 2023 15:28:00 -0800 (PST) Received: from bvanassche-linux.mtv.corp.google.com ([2620:15c:211:201:9f06:14dd:484f:e55c]) by smtp.gmail.com with ESMTPSA id s2-20020a625e02000000b0057ef155103asm5032244pfb.155.2023.01.09.15.27.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Jan 2023 15:27:59 -0800 (PST) From: Bart Van Assche To: Jens Axboe Cc: linux-block@vger.kernel.org, Jaegeuk Kim , Avri Altman , Bart Van Assche , "Martin K . Petersen" , Damien Le Moal Subject: [PATCH 8/8] scsi: ufs: Enable zoned write pipelining Date: Mon, 9 Jan 2023 15:27:38 -0800 Message-Id: <20230109232738.169886-9-bvanassche@acm.org> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog In-Reply-To: <20230109232738.169886-1-bvanassche@acm.org> References: <20230109232738.169886-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From the UFSHCI 4.0 specification, about the legacy (single queue) mode: "The host controller always process transfer requests in-order according to the order submitted to the list. In case of multiple commands with single doorbell register ringing (batch mode), The dispatch order for these transfer requests by host controller will base on their index in the List. A transfer request with lower index value will be executed before a transfer request with higher index value." From the UFSHCI 4.0 specification, about the MCQ mode: "Command Submission 1. Host SW writes an Entry to SQ 2. Host SW updates SQ doorbell tail pointer Command Processing 3. After fetching the Entry, Host Controller updates SQ doorbell head pointer 4. Host controller sends COMMAND UPIU to UFS device" In other words, for both legacy and MCQ mode, UFS controllers are required to forward commands to the UFS device in the order these commands have been received from the host. Note: for legacy mode this is only correct if the host submits one command at a time. The UFS driver does this. Cc: Martin K. Petersen Cc: Damien Le Moal Cc: Avri Altman Signed-off-by: Bart Van Assche --- drivers/ufs/core/ufshcd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index e18c9f4463ec..9198505e953b 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -5031,6 +5031,7 @@ static int ufshcd_slave_configure(struct scsi_device *sdev) ufshcd_hpb_configure(hba, sdev); + blk_queue_flag_set(QUEUE_FLAG_PIPELINE_ZONED_WRITES, q); blk_queue_update_dma_pad(q, PRDT_DATA_BYTE_COUNT_PAD - 1); if (hba->quirks & UFSHCD_QUIRK_ALIGN_SG_WITH_PAGE_SIZE) blk_queue_update_dma_alignment(q, PAGE_SIZE - 1);