From patchwork Tue Dec 12 11:11:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488950 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DIPs+xaP" Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF8E61FC1; Tue, 12 Dec 2023 03:12:16 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1d2e6e14865so24558625ad.0; Tue, 12 Dec 2023 03:12:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379536; x=1702984336; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kk71sp729csxdUViLno20l6lcdfkMyL/Wdv2yhvpzTA=; b=DIPs+xaPr2h+wojsYMYPLe4VRNQiZ+F0Om1/bK28wnBz7QtOZYQtxT+46OKFRPopF/ Emq2Rrg/INWW6Lv4IOwCnSywRtZbvLd7gznCEgx7Bngb64jmk+QHL9HoYtSrVquwUAxl YNVCT+UiG95+KtgYj9djEgf6Mw3sTzny4E+7AyOgWSXe8Z+S5N24RGlNPlx6ghmoLwCA 4gsX2lVRF+XwbLaUkB1tzjuXovGB2ufXjawBtPEXcqIeuoMOk8ILyCjcHi5H6fNalB4U EEIJy7WuOMGNOMT81KgFSCyyE3ljytfzSwsxZ/h3uqs5x91sfZpupcZ+/3XEf0Zx3qjZ yi8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379536; x=1702984336; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kk71sp729csxdUViLno20l6lcdfkMyL/Wdv2yhvpzTA=; b=nWTKQFLwUVyVNg6hhgB1d5nrApyIKDfy9wpEmNYiB6UutOGWc908nqCF2J8qmYYmzB Ghm4qgBS/+8zW/ePaj6VfxAai5re5yWB9Bw8SQQ2/Tfgv4vXNFx7xJS8TjeneWIvxE6w 9vXZc4QYhrY1pB+0/tJ/XMjGpBKhzaqSu1KONLnpI6SZKGbZNCWbFRQN7rxFf7ABKVeG 5wOJmxf0X0AwdMoAnDD9fIX9JB6ZKSVv52EadgJ0WUI2mIh5pREIkdPundp4vy1d0cYS +8u30w3B7MAYK0P6Xebbbmv40cQ3/HVTyHn328NElchD4Pa6+Cb9ozAuUOJ8hQaah6aH /ApQ== X-Gm-Message-State: AOJu0Yyz69auDjJDiIuRRisp8hsCx6o5OSfLLWD76x+4KMixzsPdgkdQ sc5mvgvZB7wtiYbLbc4IHZE= X-Google-Smtp-Source: AGHT+IH+ypC0oT6XrZH3uyP+6+iEZx9SsOkdE5qJlw+adVZmAWQ+SlcAzutITFQmVX46kV8vnEcdig== X-Received: by 2002:a17:902:6b02:b0:1d0:90bf:cbb7 with SMTP id o2-20020a1709026b0200b001d090bfcbb7mr2387161plk.66.1702379535806; Tue, 12 Dec 2023 03:12:15 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:15 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 1/5] block: Fix bio IO priority setting Date: Tue, 12 Dec 2023 19:11:46 +0800 Message-Id: <20231212111150.18155-2-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Move bio_set_ioprio() into submit_bio(): 1. Only call bio_set_ioprio() once to set the priority of original bio, the bio that cloned and splited from original bio will auto inherit the priority of original bio in clone process. 2. The IO priority can be passed to module that implement struct gendisk::fops::submit_bio, help resolve some of the IO priority loss issues. This patch depends on commit 82b74cac2849 ("blk-ioprio: Convert from rqos policy to direct call") Fixes: a78418e6a04c ("block: Always initialize bio IO priority on submit") Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- block/blk-core.c | 10 ++++++++++ block/blk-mq.c | 11 ----------- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index fdf25b8d6e78..68158c327aea 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -49,6 +49,7 @@ #include "blk-pm.h" #include "blk-cgroup.h" #include "blk-throttle.h" +#include "blk-ioprio.h" struct dentry *blk_debugfs_root; @@ -809,6 +810,14 @@ void submit_bio_noacct(struct bio *bio) } EXPORT_SYMBOL(submit_bio_noacct); +static void bio_set_ioprio(struct bio *bio) +{ + /* Nobody set ioprio so far? Initialize it based on task's nice value */ + if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) + bio->bi_ioprio = get_current_ioprio(); + blkcg_set_ioprio(bio); +} + /** * submit_bio - submit a bio to the block device layer for I/O * @bio: The &struct bio which describes the I/O @@ -831,6 +840,7 @@ void submit_bio(struct bio *bio) count_vm_events(PGPGOUT, bio_sectors(bio)); } + bio_set_ioprio(bio); submit_bio_noacct(bio); } EXPORT_SYMBOL(submit_bio); diff --git a/block/blk-mq.c b/block/blk-mq.c index e2d11183f62e..a6e2609df9c9 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -40,7 +40,6 @@ #include "blk-stat.h" #include "blk-mq-sched.h" #include "blk-rq-qos.h" -#include "blk-ioprio.h" static DEFINE_PER_CPU(struct llist_head, blk_cpu_done); static DEFINE_PER_CPU(call_single_data_t, blk_cpu_csd); @@ -2922,14 +2921,6 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return rq; } -static void bio_set_ioprio(struct bio *bio) -{ - /* Nobody set ioprio so far? Initialize it based on task's nice value */ - if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) - bio->bi_ioprio = get_current_ioprio(); - blkcg_set_ioprio(bio); -} - /** * blk_mq_submit_bio - Create and send a request to block device. * @bio: Bio pointer. @@ -2963,8 +2954,6 @@ void blk_mq_submit_bio(struct bio *bio) if (!bio_integrity_prep(bio)) return; - bio_set_ioprio(bio); - rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); if (!rq) { if (!bio) From patchwork Tue Dec 12 11:11:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488999 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XdY84KQr" Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39512210A; Tue, 12 Dec 2023 03:12:21 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6cef51f7899so986206b3a.3; Tue, 12 Dec 2023 03:12:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379540; x=1702984340; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3C4XGNOSQh2qdBXT10pq8W1umCRdPJt9fc3zqaA6RvE=; b=XdY84KQrG7tMDj4y7quSw+mtepqATxOJqGxGxKKOL30LIATSZqfVsPblM2NNgRSM+g CIlSEcka139nTtkyUvqxBo90R5UBUYaczBHh9aqaikLhuhz09hvCSPsPpLQIPQCf5MNM XxGvxpYfjmJBO8IPFS1FUmtN9As5vwq0IazsmD9GAz6Mergyq1Q0MHcYTEOSH2eGdnLM 80gK8F3qIr4zDC39KbGPYiPYCUmRj2iQs/cDlQi5WKAtzDmopAX7Mdx6mqUQ2xePo40a Fwu49DAb9ETSubsk4sftfWI99DtldYVJK5w2BmhpYiD/xI+062ju2HPpTEvAUrvmodWg aJ3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379540; x=1702984340; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3C4XGNOSQh2qdBXT10pq8W1umCRdPJt9fc3zqaA6RvE=; b=jsuENeoeRRAuUsdbw+7lOL3RcmBK1DXvObA4Zv+ody36+7zQjbsAGKhFlfy+GVagYE q+tW0rAEjScA5vBrI/956ijYbYmKh3iPtpvj/5rek/w5NkpxHufoRXhkZ1vaIwij6Iax p5r99xt1BVD1MMlSjsAy1jG94mi/xNr//I+VnU93wSz3B6oKzo6rKqG7xpxokdFNl2tJ N360jdWHGRgtwiV92wlEy0wmNAPX4cIdUh6JfBoQPXNksO/wyL2cd9jJd/en6YNVvrEt exvo/bujmAa7AIjcAI7J0/JY6pnoKyqb3Vgz3Yymj+QswH9C5m9peRLuogHb6FINgRkw aSdw== X-Gm-Message-State: AOJu0Yzt4ienSBNpXCZpiog6j2TD6Xp90BT8BIpypgHDXyf6x5CIoIvG fLxSLmnwDLc3AZpwLxXmugU= X-Google-Smtp-Source: AGHT+IH2Bn0L9xCh0Tvq4bOuPTmr0lgYXz5vcEeU0cMWIvfZTFc/5sRqby35e2TOr2XjIqb2AQ+L7w== X-Received: by 2002:a05:6a20:3d83:b0:190:7b07:b7c9 with SMTP id s3-20020a056a203d8300b001907b07b7c9mr2389253pzi.66.1702379540250; Tue, 12 Dec 2023 03:12:20 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:19 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 2/5] dm: Support I/O priority for dm_io() Date: Tue, 12 Dec 2023 19:11:47 +0800 Message-Id: <20231212111150.18155-3-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Add ioprio field in struct dm_io_region, by this field specific I/O priority when call dm_io(). Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-bufio.c | 3 +++ drivers/md/dm-integrity.c | 5 +++++ drivers/md/dm-io.c | 1 + drivers/md/dm-log.c | 1 + drivers/md/dm-raid1.c | 2 ++ drivers/md/dm-snap-persistent.c | 1 + drivers/md/dm-writecache.c | 4 ++++ include/linux/dm-io.h | 2 ++ 8 files changed, 19 insertions(+) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 62eb27639c9b..7f82262aed54 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -1269,6 +1269,7 @@ static void use_dmio(struct dm_buffer *b, enum req_op op, sector_t sector, .bdev = b->c->bdev, .sector = sector, .count = n_sectors, + .ioprio = IOPRIO_DEFAULT, }; if (b->data_mode != DATA_MODE_VMALLOC) { @@ -2125,6 +2126,7 @@ int dm_bufio_issue_flush(struct dm_bufio_client *c) .bdev = c->bdev, .sector = 0, .count = 0, + .ioprio = IOPRIO_DEFAULT, }; if (WARN_ON_ONCE(dm_bufio_in_request())) @@ -2149,6 +2151,7 @@ int dm_bufio_issue_discard(struct dm_bufio_client *c, sector_t block, sector_t c .bdev = c->bdev, .sector = block_to_sector(c, block), .count = block_to_sector(c, count), + .ioprio = IOPRIO_DEFAULT, }; if (WARN_ON_ONCE(dm_bufio_in_request())) diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index e85c688fd91e..7cba183abdce 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -543,6 +543,7 @@ static int sync_rw_sb(struct dm_integrity_c *ic, blk_opf_t opf) io_loc.bdev = ic->meta_dev ? ic->meta_dev->bdev : ic->dev->bdev; io_loc.sector = ic->start; io_loc.count = SB_SECTORS; + io_loc.ioprio = IOPRIO_DEFAULT; if (op == REQ_OP_WRITE) { sb_set_version(ic); @@ -1070,6 +1071,7 @@ static void rw_journal_sectors(struct dm_integrity_c *ic, blk_opf_t opf, io_loc.bdev = ic->meta_dev ? ic->meta_dev->bdev : ic->dev->bdev; io_loc.sector = ic->start + SB_SECTORS + sector; io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { @@ -1187,6 +1189,7 @@ static void copy_from_journal(struct dm_integrity_c *ic, unsigned int section, u io_loc.bdev = ic->dev->bdev; io_loc.sector = target; io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { @@ -1515,6 +1518,7 @@ static void dm_integrity_flush_buffers(struct dm_integrity_c *ic, bool flush_dat fr.io_reg.bdev = ic->dev->bdev, fr.io_reg.sector = 0, fr.io_reg.count = 0, + fr.io_reg.ioprio = IOPRIO_DEFAULT, fr.ic = ic; init_completion(&fr.comp); r = dm_io(&fr.io_req, 1, &fr.io_reg, NULL); @@ -2738,6 +2742,7 @@ static void integrity_recalc(struct work_struct *w) io_loc.bdev = ic->dev->bdev; io_loc.sector = get_data_sector(ic, area, offset); io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index f053ce245814..b40f0a432981 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -354,6 +354,7 @@ static void do_region(const blk_opf_t opf, unsigned int region, &io->client->bios); bio->bi_iter.bi_sector = where->sector + (where->count - remaining); bio->bi_end_io = endio; + bio->bi_ioprio = where->ioprio; store_io_and_region_in_bio(bio, io, region); if (op == REQ_OP_DISCARD || op == REQ_OP_WRITE_ZEROES) { diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c index f9f84236dfcd..e0dacdcd94f1 100644 --- a/drivers/md/dm-log.c +++ b/drivers/md/dm-log.c @@ -309,6 +309,7 @@ static int flush_header(struct log_c *lc) .bdev = lc->header_location.bdev, .sector = 0, .count = 0, + .ioprio = IOPRIO_DEFAULT, }; lc->io_req.bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; diff --git a/drivers/md/dm-raid1.c b/drivers/md/dm-raid1.c index ddcb2bc4a617..2de9b1377de3 100644 --- a/drivers/md/dm-raid1.c +++ b/drivers/md/dm-raid1.c @@ -275,6 +275,7 @@ static int mirror_flush(struct dm_target *ti) io[i].bdev = m->dev->bdev; io[i].sector = 0; io[i].count = 0; + io[i].ioprio = IOPRIO_DEFAULT; } error_bits = -1; @@ -475,6 +476,7 @@ static void map_region(struct dm_io_region *io, struct mirror *m, io->bdev = m->dev->bdev; io->sector = map_sector(m, bio); io->count = bio_sectors(bio); + io->ioprio = bio_prio(bio); } static void hold_bio(struct mirror_set *ms, struct bio *bio) diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c index 15649921f2a9..4aa70b71f1da 100644 --- a/drivers/md/dm-snap-persistent.c +++ b/drivers/md/dm-snap-persistent.c @@ -236,6 +236,7 @@ static int chunk_io(struct pstore *ps, void *area, chunk_t chunk, blk_opf_t opf, .bdev = dm_snap_cow(ps->store->snap)->bdev, .sector = ps->store->chunk_size * chunk, .count = ps->store->chunk_size, + .ioprio = IOPRIO_DEFAULT, }; struct dm_io_request io_req = { .bi_opf = opf, diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c index 074cb785eafc..135d1268246f 100644 --- a/drivers/md/dm-writecache.c +++ b/drivers/md/dm-writecache.c @@ -515,6 +515,7 @@ static void ssd_commit_flushed(struct dm_writecache *wc, bool wait_for_ios) region.bdev = wc->ssd_dev->bdev; region.sector = (sector_t)i * (BITMAP_GRANULARITY >> SECTOR_SHIFT); region.count = (sector_t)(j - i) * (BITMAP_GRANULARITY >> SECTOR_SHIFT); + region.ioprio = IOPRIO_DEFAULT; if (unlikely(region.sector >= wc->metadata_sectors)) break; @@ -555,6 +556,7 @@ static void ssd_commit_superblock(struct dm_writecache *wc) region.bdev = wc->ssd_dev->bdev; region.sector = 0; region.count = max(4096U, wc->block_size) >> SECTOR_SHIFT; + region.ioprio = IOPRIO_DEFAULT; if (unlikely(region.sector + region.count > wc->metadata_sectors)) region.count = wc->metadata_sectors - region.sector; @@ -590,6 +592,7 @@ static void writecache_disk_flush(struct dm_writecache *wc, struct dm_dev *dev) region.bdev = dev->bdev; region.sector = 0; region.count = 0; + region.ioprio = IOPRIO_DEFAULT; req.bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; req.mem.type = DM_IO_KMEM; req.mem.ptr.addr = NULL; @@ -984,6 +987,7 @@ static int writecache_read_metadata(struct dm_writecache *wc, sector_t n_sectors region.bdev = wc->ssd_dev->bdev; region.sector = wc->start_sector; region.count = n_sectors; + region.ioprio = IOPRIO_DEFAULT; req.bi_opf = REQ_OP_READ | REQ_SYNC; req.mem.type = DM_IO_VMA; req.mem.ptr.vma = (char *)wc->memory_map; diff --git a/include/linux/dm-io.h b/include/linux/dm-io.h index 7595142f3fc5..227ee6d77c70 100644 --- a/include/linux/dm-io.h +++ b/include/linux/dm-io.h @@ -20,6 +20,8 @@ struct dm_io_region { struct block_device *bdev; sector_t sector; sector_t count; /* If this is zero the region is ignored. */ + /* Set it to IOPRIO_DEFAULT if you don't know what value to set */ + unsigned short ioprio; }; struct page_list { From patchwork Tue Dec 12 11:11:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13489001 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iIN4XUQt" Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96E041980; Tue, 12 Dec 2023 03:12:25 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-28647f4ebd9so4053662a91.3; Tue, 12 Dec 2023 03:12:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379545; x=1702984345; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4yqIhLm4LzKCMhpYyEZh+xPSurdrvcNEreCvT9oRThc=; b=iIN4XUQtjzFQbiCGZwB7BAXRMFOtyBzXFlkb5KveVLh0f5jy/KCyyU+cUP3zaeBihi IacI1+LBl1EyYz57xy53FU2mrXRISbYd5TARzDOtUv3YTs4JPqBPaZRmgqgIaQYliLWt 9z/5gkfGo9hdzLeyfIoXEsl+ko0RvgtvEVveI2VLIQfBSeKxLIMOxtCflTLgC3vdoQLv TbCiMb4fzbaJcva8Czhq2yyFmNNZk5En1yiQy5IOYWLDlmZVxbaodhpZK3PKGqIK5kZt /0kU2shGsBgAZfFR7EMmgIydL3uI44a6g5t6hoxjhtOoINSt05tUkwzR/zMaavDxgNSy VeaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379545; x=1702984345; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4yqIhLm4LzKCMhpYyEZh+xPSurdrvcNEreCvT9oRThc=; b=qiGe2ZB8vAuzcY6+hvfR+va/BpYt+oJibV2YcIC4ZTBVaixJYqvZomNyqA771ECXUj 4b8uOvBHTETNj6ZeLjBw+rJ4MusCg9yVYimVJmg/C+6SZ/o4RR3VPLEr9SD6M0Hwt0xt CuuH2gY9fH96sGu3wUTv+iQWBInd+5sXlNjdN4DLm4zwoKzWYqyD67RC6LE2WT5SKA05 7BNFoFt8ajuyQWRqVSW4uymLVcTE72IEJ0l4UFtYtrO+amhL2gASBFwNZHvkeXYW5SxP no1A6mWARm1YB+D4Y3N0aWnqd5a+duEO5xntC/WJmAuw1HsMjpnhtlymftHoRgB80OKp +O+g== X-Gm-Message-State: AOJu0YwR4A+kx5Z1ktBxke/kEnje0CGsyPDURULpP5nWmPQRjxyuIBrW xVUpsJcfXlO+Ki328V6fZ+/X8lY8jLk= X-Google-Smtp-Source: AGHT+IGJkbSmMBCSDQCY4qtO/eaJOlAMLjKcM057nI0ijY6goUpjdvEP/pMjSUfVMw9aAZp0rt041g== X-Received: by 2002:a17:90a:d205:b0:286:c398:841b with SMTP id o5-20020a17090ad20500b00286c398841bmr2773861pju.58.1702379544881; Tue, 12 Dec 2023 03:12:24 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:24 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 3/5] dm-bufio: Support I/O priority Date: Tue, 12 Dec 2023 19:11:48 +0800 Message-Id: <20231212111150.18155-4-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Add I/O priority parameter for dm_bufio_read() and dm_bufio_prefetch(). Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-bufio.c | 33 ++++++++++--------- drivers/md/dm-ebs-target.c | 8 ++--- drivers/md/dm-integrity.c | 2 +- drivers/md/dm-snap-persistent.c | 4 +-- drivers/md/dm-verity-fec.c | 4 +-- drivers/md/dm-verity-target.c | 4 +-- drivers/md/persistent-data/dm-block-manager.c | 6 ++-- include/linux/dm-bufio.h | 6 ++-- 8 files changed, 34 insertions(+), 33 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 7f82262aed54..739f5dc52432 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -1256,7 +1256,7 @@ static void dmio_complete(unsigned long error, void *context) } static void use_dmio(struct dm_buffer *b, enum req_op op, sector_t sector, - unsigned int n_sectors, unsigned int offset) + unsigned int n_sectors, unsigned int offset, unsigned short ioprio) { int r; struct dm_io_request io_req = { @@ -1296,7 +1296,7 @@ static void bio_complete(struct bio *bio) } static void use_bio(struct dm_buffer *b, enum req_op op, sector_t sector, - unsigned int n_sectors, unsigned int offset) + unsigned int n_sectors, unsigned int offset, unsigned short ioprio) { struct bio *bio; char *ptr; @@ -1304,13 +1304,14 @@ static void use_bio(struct dm_buffer *b, enum req_op op, sector_t sector, bio = bio_kmalloc(1, GFP_NOWAIT | __GFP_NORETRY | __GFP_NOWARN); if (!bio) { - use_dmio(b, op, sector, n_sectors, offset); + use_dmio(b, op, sector, n_sectors, offset, ioprio); return; } bio_init(bio, b->c->bdev, bio->bi_inline_vecs, 1, op); bio->bi_iter.bi_sector = sector; bio->bi_end_io = bio_complete; bio->bi_private = b; + bio->bi_ioprio = ioprio; ptr = (char *)b->data + offset; len = n_sectors << SECTOR_SHIFT; @@ -1333,7 +1334,7 @@ static inline sector_t block_to_sector(struct dm_bufio_client *c, sector_t block return sector; } -static void submit_io(struct dm_buffer *b, enum req_op op, +static void submit_io(struct dm_buffer *b, enum req_op op, unsigned short ioprio, void (*end_io)(struct dm_buffer *, blk_status_t)) { unsigned int n_sectors; @@ -1363,9 +1364,9 @@ static void submit_io(struct dm_buffer *b, enum req_op op, } if (b->data_mode != DATA_MODE_VMALLOC) - use_bio(b, op, sector, n_sectors, offset); + use_bio(b, op, sector, n_sectors, offset, ioprio); else - use_dmio(b, op, sector, n_sectors, offset); + use_dmio(b, op, sector, n_sectors, offset, ioprio); } /* @@ -1421,7 +1422,7 @@ static void __write_dirty_buffer(struct dm_buffer *b, b->write_end = b->dirty_end; if (!write_list) - submit_io(b, REQ_OP_WRITE, write_endio); + submit_io(b, REQ_OP_WRITE, IOPRIO_DEFAULT, write_endio); else list_add_tail(&b->write_list, write_list); } @@ -1435,7 +1436,7 @@ static void __flush_write_list(struct list_head *write_list) struct dm_buffer *b = list_entry(write_list->next, struct dm_buffer, write_list); list_del(&b->write_list); - submit_io(b, REQ_OP_WRITE, write_endio); + submit_io(b, REQ_OP_WRITE, IOPRIO_DEFAULT, write_endio); cond_resched(); } blk_finish_plug(&plug); @@ -1817,7 +1818,7 @@ static void read_endio(struct dm_buffer *b, blk_status_t status) * and uses dm_bufio_mark_buffer_dirty to write new data back). */ static void *new_read(struct dm_bufio_client *c, sector_t block, - enum new_flag nf, struct dm_buffer **bp) + enum new_flag nf, struct dm_buffer **bp, unsigned short ioprio) { int need_submit = 0; struct dm_buffer *b; @@ -1870,7 +1871,7 @@ static void *new_read(struct dm_bufio_client *c, sector_t block, return NULL; if (need_submit) - submit_io(b, REQ_OP_READ, read_endio); + submit_io(b, REQ_OP_READ, ioprio, read_endio); wait_on_bit_io(&b->state, B_READING, TASK_UNINTERRUPTIBLE); @@ -1890,17 +1891,17 @@ static void *new_read(struct dm_bufio_client *c, sector_t block, void *dm_bufio_get(struct dm_bufio_client *c, sector_t block, struct dm_buffer **bp) { - return new_read(c, block, NF_GET, bp); + return new_read(c, block, NF_GET, bp, IOPRIO_DEFAULT); } EXPORT_SYMBOL_GPL(dm_bufio_get); void *dm_bufio_read(struct dm_bufio_client *c, sector_t block, - struct dm_buffer **bp) + struct dm_buffer **bp, unsigned short ioprio) { if (WARN_ON_ONCE(dm_bufio_in_request())) return ERR_PTR(-EINVAL); - return new_read(c, block, NF_READ, bp); + return new_read(c, block, NF_READ, bp, ioprio); } EXPORT_SYMBOL_GPL(dm_bufio_read); @@ -1910,12 +1911,12 @@ void *dm_bufio_new(struct dm_bufio_client *c, sector_t block, if (WARN_ON_ONCE(dm_bufio_in_request())) return ERR_PTR(-EINVAL); - return new_read(c, block, NF_FRESH, bp); + return new_read(c, block, NF_FRESH, bp, IOPRIO_DEFAULT); } EXPORT_SYMBOL_GPL(dm_bufio_new); void dm_bufio_prefetch(struct dm_bufio_client *c, - sector_t block, unsigned int n_blocks) + sector_t block, unsigned int n_blocks, unsigned short ioprio) { struct blk_plug plug; @@ -1951,7 +1952,7 @@ void dm_bufio_prefetch(struct dm_bufio_client *c, dm_bufio_unlock(c); if (need_submit) - submit_io(b, REQ_OP_READ, read_endio); + submit_io(b, REQ_OP_READ, ioprio, read_endio); dm_bufio_release(b); cond_resched(); diff --git a/drivers/md/dm-ebs-target.c b/drivers/md/dm-ebs-target.c index 435b45201f4d..8198c8a7b416 100644 --- a/drivers/md/dm-ebs-target.c +++ b/drivers/md/dm-ebs-target.c @@ -84,7 +84,7 @@ static int __ebs_rw_bvec(struct ebs_c *ec, enum req_op op, struct bio_vec *bv, /* Avoid reading for writes in case bio vector's page overwrites block completely. */ if (op == REQ_OP_READ || buf_off || bv_len < dm_bufio_get_block_size(ec->bufio)) - ba = dm_bufio_read(ec->bufio, block, &b); + ba = dm_bufio_read(ec->bufio, block, &b, IOPRIO_DEFAULT); else ba = dm_bufio_new(ec->bufio, block, &b); @@ -194,13 +194,13 @@ static void __ebs_process_bios(struct work_struct *ws) bio_list_for_each(bio, &bios) { block1 = __sector_to_block(ec, bio->bi_iter.bi_sector); if (bio_op(bio) == REQ_OP_READ) - dm_bufio_prefetch(ec->bufio, block1, __nr_blocks(ec, bio)); + dm_bufio_prefetch(ec->bufio, block1, __nr_blocks(ec, bio), IOPRIO_DEFAULT); else if (bio_op(bio) == REQ_OP_WRITE && !(bio->bi_opf & REQ_PREFLUSH)) { block2 = __sector_to_block(ec, bio_end_sector(bio)); if (__block_mod(bio->bi_iter.bi_sector, ec->u_bs)) - dm_bufio_prefetch(ec->bufio, block1, 1); + dm_bufio_prefetch(ec->bufio, block1, 1, IOPRIO_DEFAULT); if (__block_mod(bio_end_sector(bio), ec->u_bs) && block2 != block1) - dm_bufio_prefetch(ec->bufio, block2, 1); + dm_bufio_prefetch(ec->bufio, block2, 1, IOPRIO_DEFAULT); } } diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index 7cba183abdce..a2853c24a259 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -1421,7 +1421,7 @@ static int dm_integrity_rw_tag(struct dm_integrity_c *ic, unsigned char *tag, se if (unlikely(r)) return r; - data = dm_bufio_read(ic->bufio, *metadata_block, &b); + data = dm_bufio_read(ic->bufio, *metadata_block, &b, IOPRIO_DEFAULT); if (IS_ERR(data)) return PTR_ERR(data); diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c index 4aa70b71f1da..eb6943fc7024 100644 --- a/drivers/md/dm-snap-persistent.c +++ b/drivers/md/dm-snap-persistent.c @@ -525,7 +525,7 @@ static int read_exceptions(struct pstore *ps, if (unlikely(pf_chunk >= dm_bufio_get_device_size(client))) break; - dm_bufio_prefetch(client, pf_chunk, 1); + dm_bufio_prefetch(client, pf_chunk, 1, IOPRIO_DEFAULT); prefetch_area++; if (unlikely(!prefetch_area)) break; @@ -534,7 +534,7 @@ static int read_exceptions(struct pstore *ps, chunk = area_location(ps, ps->current_area); - area = dm_bufio_read(client, chunk, &bp); + area = dm_bufio_read(client, chunk, &bp, IOPRIO_DEFAULT); if (IS_ERR(area)) { r = PTR_ERR(area); goto ret_destroy_bufio; diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 3ef9f018da60..715173cbf0ee 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -68,7 +68,7 @@ static u8 *fec_read_parity(struct dm_verity *v, u64 rsb, int index, block = div64_u64_rem(position, v->fec->io_size, &rem); *offset = (unsigned int)rem; - res = dm_bufio_read(v->fec->bufio, block, buf); + res = dm_bufio_read(v->fec->bufio, block, buf, IOPRIO_DEFAULT); if (IS_ERR(res)) { DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", v->data_dev->name, (unsigned long long)rsb, @@ -247,7 +247,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf); + bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 26adcfea0302..0038e168f3d7 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -307,7 +307,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf); + data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); if (IS_ERR(data)) return PTR_ERR(data); @@ -717,7 +717,7 @@ static void verity_prefetch_io(struct work_struct *work) } no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, - hash_block_end - hash_block_start + 1); + hash_block_end - hash_block_start + 1, IOPRIO_DEFAULT); } kfree(pw); diff --git a/drivers/md/persistent-data/dm-block-manager.c b/drivers/md/persistent-data/dm-block-manager.c index 0e010e1204aa..86a4f73d2f3d 100644 --- a/drivers/md/persistent-data/dm-block-manager.c +++ b/drivers/md/persistent-data/dm-block-manager.c @@ -474,7 +474,7 @@ int dm_bm_read_lock(struct dm_block_manager *bm, dm_block_t b, void *p; int r; - p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result); + p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result, IOPRIO_DEFAULT); if (IS_ERR(p)) return PTR_ERR(p); @@ -510,7 +510,7 @@ int dm_bm_write_lock(struct dm_block_manager *bm, if (dm_bm_is_read_only(bm)) return -EPERM; - p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result); + p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result, IOPRIO_DEFAULT); if (IS_ERR(p)) return PTR_ERR(p); @@ -624,7 +624,7 @@ EXPORT_SYMBOL_GPL(dm_bm_flush); void dm_bm_prefetch(struct dm_block_manager *bm, dm_block_t b) { - dm_bufio_prefetch(bm->bufio, b, 1); + dm_bufio_prefetch(bm->bufio, b, 1, IOPRIO_DEFAULT); } bool dm_bm_is_read_only(struct dm_block_manager *bm) diff --git a/include/linux/dm-bufio.h b/include/linux/dm-bufio.h index 75e7d8cbb532..d270d48891f7 100644 --- a/include/linux/dm-bufio.h +++ b/include/linux/dm-bufio.h @@ -11,6 +11,7 @@ #define _LINUX_DM_BUFIO_H #include +#include #include /*----------------------------------------------------------------*/ @@ -62,7 +63,7 @@ void dm_bufio_set_sector_offset(struct dm_bufio_client *c, sector_t start); * it dirty. */ void *dm_bufio_read(struct dm_bufio_client *c, sector_t block, - struct dm_buffer **bp); + struct dm_buffer **bp, unsigned short ioprio); /* * Like dm_bufio_read, but return buffer from cache, don't read @@ -84,8 +85,7 @@ void *dm_bufio_new(struct dm_bufio_client *c, sector_t block, * I/O to finish. */ void dm_bufio_prefetch(struct dm_bufio_client *c, - sector_t block, unsigned int n_blocks); - + sector_t block, unsigned int n_blocks, unsigned short ioprio); /* * Release a reference obtained with dm_bufio_{read,get,new}. The data * pointer and dm_buffer pointer is no longer valid after this call. From patchwork Tue Dec 12 11:11:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13489000 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EAvhWgQd" Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FD962695; Tue, 12 Dec 2023 03:12:30 -0800 (PST) Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-5c701bd98f3so1055941a12.1; Tue, 12 Dec 2023 03:12:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379549; x=1702984349; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vN+RlAGZeGH2zMVmYb6ghj5GDWJcjTyD9/i+J4cSzo8=; b=EAvhWgQdAcKkYca7iH3EIeWNvw1BqsZ51l6UWqoLW9u0HmEwCSY04xLdWYx1Hrfkrr zmeYu3itNjMAXRbt5L3R+j5KwjHuxKV9ZW166wmDknc5oL9b2nNLPi0tzcDs79SycgW0 q7yMcfgregHzDeVFPa3pqlXzzfpPowRx1ZXGISvM+IFy9g5mL7FuTIflN7OWB4HBp/rS kZzO1Nwc4ZOtYuAO2PO2fB2C2/RkQowmU44+q43Q4MOZfFbranwEePe1yACFA5QQMIct uYlLT4aA2c2sOn8vJauFRbnORotXTjYh3RbG5I9THGy9SLYY9E4Ve1hctG4ZCubnAcf5 TEYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379549; x=1702984349; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vN+RlAGZeGH2zMVmYb6ghj5GDWJcjTyD9/i+J4cSzo8=; b=TASBvnp+zxs/t/uaFCW09UKo+HcVvZhjTtKUum6TVgOdBEoM/n/APCcE0f4qhnPQZa H+ADqVKdc/4nxOwV/0rNcZdsHKDGl9d0eOBpOWG471jMYXKbMDQ5Bp4np10s2aCeoJPl FTskbzzqIl1Mfq3HQZiI+pdy738LiXFLjd6zCMhJnCsT70QDZ3juacCXzL7KmGcGRn4s fp9XUkKWUKVOERXcAcz9y+9J7qWDQDxKcLiR2TE+2iNaMXfEoJMQvgLhzHWpfy2NI1oc jHMHjwA8Wc3kIZBuerEsNp9GRkn0SFLms+TFscjDeQ1OMkmpvhHiJy0H8n+aiAv6bLOv hjRA== X-Gm-Message-State: AOJu0YyU0nHUuzHWK+4GH7MeqeHyebrhZZdTWV/19NJL5WkdyIVqaQrn 3qkN643HYrS/9MhddWllDh0= X-Google-Smtp-Source: AGHT+IH8HknLDzaW0Q9TDSRdg3SQu4DIMs5eNiFa7zAa7YAtIDMAqdyYIiPkKRRbitlR1WMMmVqmNQ== X-Received: by 2002:a05:6a20:4287:b0:186:e53:b64b with SMTP id o7-20020a056a20428700b001860e53b64bmr3221043pzj.47.1702379549391; Tue, 12 Dec 2023 03:12:29 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:29 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 4/5] dm verity: Fix I/O priority lost when read FEC and hash Date: Tue, 12 Dec 2023 19:11:49 +0800 Message-Id: <20231212111150.18155-5-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin To fix this problem, when read FEC and hash from disk, I/O priority are inconsistent with data block and blocked by other I/O with low I/O priority. Make I/O for FEC and hash has same I/O priority with original data I/O. Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-verity-fec.c | 3 ++- drivers/md/dm-verity-target.c | 8 ++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 715173cbf0ee..6a5a679e7e8a 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -209,6 +209,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, u8 *bbuf, *rs_block; u8 want_digest[HASH_MAX_DIGESTSIZE]; unsigned int n, k; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (neras) *neras = 0; @@ -247,7 +248,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); + bbuf = dm_bufio_read(bufio, block, &buf, bio_prio(bio)); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 0038e168f3d7..8c911b6722ce 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -51,6 +51,7 @@ static DEFINE_STATIC_KEY_FALSE(use_tasklet_enabled); struct dm_verity_prefetch_work { struct work_struct work; struct dm_verity *v; + unsigned short ioprio; sector_t block; unsigned int n_blocks; }; @@ -293,6 +294,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, int r; sector_t hash_block; unsigned int offset; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); verity_hash_at_level(v, block, level, &hash_block, &offset); @@ -307,7 +309,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); + data = dm_bufio_read(v->bufio, hash_block, &buf, bio_prio(bio)); if (IS_ERR(data)) return PTR_ERR(data); @@ -717,7 +719,7 @@ static void verity_prefetch_io(struct work_struct *work) } no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, - hash_block_end - hash_block_start + 1, IOPRIO_DEFAULT); + hash_block_end - hash_block_start + 1, pw->ioprio); } kfree(pw); @@ -728,6 +730,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) sector_t block = io->block; unsigned int n_blocks = io->n_blocks; struct dm_verity_prefetch_work *pw; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (v->validated_blocks) { while (n_blocks && test_bit(block, v->validated_blocks)) { @@ -751,6 +754,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) pw->v = v; pw->block = block; pw->n_blocks = n_blocks; + pw->ioprio = bio_prio(bio); queue_work(v->verify_wq, &pw->work); } From patchwork Tue Dec 12 11:11:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13488951 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c2RHzZKk" Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A15E71710; Tue, 12 Dec 2023 03:12:34 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1d0aaa979f0so31514645ad.0; Tue, 12 Dec 2023 03:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702379554; x=1702984354; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fRWJvmM8+hpVsGJoyvu2YuvVcQVJUfQcuTmMveSE8/s=; b=c2RHzZKkum9NRONbaeT4jXwKCCVFHUqe5Q41RMCOVKCoyQRbyeen+mfbfqXb90MhHQ OE2pUZnsH3fHowyc8v6LxD3T84t9Uo40t9aqIf2iy++6OtEcwRrmDXNaYtHHqnk08HD6 DerSRz91i6ZpDKMXHFKGHpvfww4ypTHPZoQiV9AOgWnBsedwaV/JcXk2vEQ3IGfzmrlJ NwV0NnylvJM4WIkBLw17bhsouuBuccKW1zFej5wBrs0zxXqqVzdbiKjJSnZ6SjUKi2vW +yCJ2mESMchde54tZgnHydGt6bwjCBqkbc8ROScJnOYPji1SK7WDGzwWm4Ex42thBZms 4PXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702379554; x=1702984354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fRWJvmM8+hpVsGJoyvu2YuvVcQVJUfQcuTmMveSE8/s=; b=hx3/PmBitkBd2/sLg4JodXUIGi3v62m8BMUGJb11XAO11kuVnm2c268A7A2Lc/oZSP IeW5day9W7YFVO0jjD+rmSnfNHE4w7UzqfD6xev8pmvFfQjGnf9QomdGd79JcAkNysFE XA+ZHujVnF+bxm0wXWeN+5K/L4W59OeiS6EXDAWetr7AXl6OLb+eSNXxbYOOPg2vkIQX bsKX7wABiHdYpQEJwmmOYeg0EuFS5QZJKRfQ739qITAl2m4Oz3UrBgl09Gmv81nQ21On 0voLz6Hooi05IBGgLjgGjaTBnHYHkcTrYNNpXFVLWo9Vbd2msCwGxSrV8XPgbsUazWxF A88g== X-Gm-Message-State: AOJu0Yx/dw4eCAWONhgafgx83OqEv2zKlzqjFdnMte4aA3qgCtN5INVM 3fFhMLV4L/dJobMEGBON01o= X-Google-Smtp-Source: AGHT+IFlcy+h/DMb4yTLMEPWYaQdXpLguXXRtdGTaHrTLhSYejeIi4auC/X/p0G23rVz46hOssocww== X-Received: by 2002:a17:902:780a:b0:1d3:3876:bfe with SMTP id p10-20020a170902780a00b001d338760bfemr169241pll.90.1702379553803; Tue, 12 Dec 2023 03:12:33 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id 21-20020a170902ee5500b001d0511c990csm8345453plo.237.2023.12.12.03.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Dec 2023 03:12:33 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v4 5/5] dm-crypt: Fix lost ioprio when queuing write bios Date: Tue, 12 Dec 2023 19:11:50 +0800 Message-Id: <20231212111150.18155-6-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> References: <20231212111150.18155-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin The original submitting bio->bi_ioprio setting can be retained by struct dm_crypt_io::base_bio, we set the original bio's ioprio to the cloned bio for write. Link: https://lore.kernel.org/dm-devel/alpine.LRH.2.11.1612141049250.13402@mail.ewheeler.net Signed-off-by: Hongyu Jin --- drivers/md/dm-crypt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 6de107aff331..7149da6555b8 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -1683,6 +1683,7 @@ static struct bio *crypt_alloc_buffer(struct dm_crypt_io *io, unsigned int size) GFP_NOIO, &cc->bs); clone->bi_private = io; clone->bi_end_io = crypt_endio; + clone->bi_ioprio = io->base_bio->bi_ioprio; remaining_size = size;