From patchwork Mon Dec 11 08:59:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13486848 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QS4m+ryR" Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA857101; Mon, 11 Dec 2023 01:00:48 -0800 (PST) Received: by mail-il1-x12c.google.com with SMTP id e9e14a558f8ab-35d3b220b64so17157655ab.2; Mon, 11 Dec 2023 01:00:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702285248; x=1702890048; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=czq/4nF48yftDC4HVsuA9AE8CVBtJqYm6rZADq2HVZo=; b=QS4m+ryRa9v39QRMBV0FZZwlBoPjLzLecfYF2XkwRojLx9UZ+f9EVUzeU96btWQthN EP7aF5AyWV0qtHVHEzKFWNoVyPJp6AYF11vshyfhgePOY/Bf7L3NxXp0WOxOLGt0VNYX z2yukxmsa52nj9lHES2c2z+vyTsT5pN3n+DFnZpdsq35JY6BYg9JUFZi7zZDstz+KBFx bUXZRsRsgPEdOt7Fw2dGe7LWES0zsySk6bCQVU22GMxQgZa0k42oJ002g8pTpvCZca4e 6xg/hIbQfy4/WZmqCPE3uh0R4wuLc0DzyTpDUAKzAonv54eskklyFZ08hGzy8e8m6VWH xeSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702285248; x=1702890048; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=czq/4nF48yftDC4HVsuA9AE8CVBtJqYm6rZADq2HVZo=; b=Z3EZ0DZAYS3P/kEctcmaNVKWysbbVXFUsbFNdXN09riDv6TpzT++ebWzqmig/WOP40 RfGs+kxH/HsSa631vQ7mxZeXmi0v19B4O85hAzPYvKfNQy8pyMLtiRRKRBhiwAXfQI5L b1zEQwwUmxLi0/r++VGh4lO6exhLSbgdJ/ppt/X+T4IhvcRmMF5gNhMmJnlY7fsVlsHN o+cYwxLlOB1I4r3Zo7YaMY5KHeDGPUTsMmUkDuvNU0GxN//ef6htzyGGwAKnlTsYtoWQ a6y8mvr/nBxgu3Lc20itiCbdEQA5pAm/S77c56RCm+41rT9rFjkbUi+ri2BJRzbc8GIB q7fg== X-Gm-Message-State: AOJu0YxJ0lUF77q2MmkCQt6ZPH/gErg3+oh5CvmJaP10M3Dd1eON1mwk SMbKPt57jg4wckRunTuMIW0= X-Google-Smtp-Source: AGHT+IEwn0/n8T+SvHlYhFYyRFqYBOgn7wrNpFzhJLamLhEf4Eabi+8VursYqb2tcOSwbmwGTCML4A== X-Received: by 2002:a05:6e02:1587:b0:35d:59a2:6464 with SMTP id m7-20020a056e02158700b0035d59a26464mr6212743ilu.55.1702285247890; Mon, 11 Dec 2023 01:00:47 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id f14-20020a056e020b4e00b0035b0b05189bsm2211216ilu.38.2023.12.11.01.00.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Dec 2023 01:00:47 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v3 1/5] block: Optimize bio io priority setting Date: Mon, 11 Dec 2023 16:59:56 +0800 Message-Id: <20231211090000.9578-2-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> References: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Current call bio_set_ioprio() for each cloned bio and splited bio, and the io priority can't be passed to module that implement struct gendisk::fops::submit_bio, such as device-mapper. Move bio_set_ioprio() into submit_bio(), only call bio_set_ioprio() once set the priority of origin bio, cloned and splited bio auto inherit the priority of origin bio in clone process. Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin Reviewed-by: Mike Snitzer --- block/blk-core.c | 10 ++++++++++ block/blk-mq.c | 11 ----------- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index fdf25b8d6e78..68158c327aea 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -49,6 +49,7 @@ #include "blk-pm.h" #include "blk-cgroup.h" #include "blk-throttle.h" +#include "blk-ioprio.h" struct dentry *blk_debugfs_root; @@ -809,6 +810,14 @@ void submit_bio_noacct(struct bio *bio) } EXPORT_SYMBOL(submit_bio_noacct); +static void bio_set_ioprio(struct bio *bio) +{ + /* Nobody set ioprio so far? Initialize it based on task's nice value */ + if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) + bio->bi_ioprio = get_current_ioprio(); + blkcg_set_ioprio(bio); +} + /** * submit_bio - submit a bio to the block device layer for I/O * @bio: The &struct bio which describes the I/O @@ -831,6 +840,7 @@ void submit_bio(struct bio *bio) count_vm_events(PGPGOUT, bio_sectors(bio)); } + bio_set_ioprio(bio); submit_bio_noacct(bio); } EXPORT_SYMBOL(submit_bio); diff --git a/block/blk-mq.c b/block/blk-mq.c index e2d11183f62e..a6e2609df9c9 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -40,7 +40,6 @@ #include "blk-stat.h" #include "blk-mq-sched.h" #include "blk-rq-qos.h" -#include "blk-ioprio.h" static DEFINE_PER_CPU(struct llist_head, blk_cpu_done); static DEFINE_PER_CPU(call_single_data_t, blk_cpu_csd); @@ -2922,14 +2921,6 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, return rq; } -static void bio_set_ioprio(struct bio *bio) -{ - /* Nobody set ioprio so far? Initialize it based on task's nice value */ - if (IOPRIO_PRIO_CLASS(bio->bi_ioprio) == IOPRIO_CLASS_NONE) - bio->bi_ioprio = get_current_ioprio(); - blkcg_set_ioprio(bio); -} - /** * blk_mq_submit_bio - Create and send a request to block device. * @bio: Bio pointer. @@ -2963,8 +2954,6 @@ void blk_mq_submit_bio(struct bio *bio) if (!bio_integrity_prep(bio)) return; - bio_set_ioprio(bio); - rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs); if (!rq) { if (!bio) From patchwork Mon Dec 11 08:59:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13486849 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B6c/+Ac/" Received: from mail-il1-x12f.google.com (mail-il1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94C56F1; Mon, 11 Dec 2023 01:00:52 -0800 (PST) Received: by mail-il1-x12f.google.com with SMTP id e9e14a558f8ab-35d64ad4188so19751165ab.3; Mon, 11 Dec 2023 01:00:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702285252; x=1702890052; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3C4XGNOSQh2qdBXT10pq8W1umCRdPJt9fc3zqaA6RvE=; b=B6c/+Ac/eaHHn0JXTnLGcIauJsiztidAQEh8Tdy4VdUC/YbImwy2siiuUAy1XPCqW4 THVpUlsTpP2kZMpYAy0gKa3bwWyHcKs4gMjQp2C52OJJEg5tHV+MWQXOEujAu/d6VPgP vhd5vea1DjBeiUzSbuPZsa92HZBZWZ8GaRrh9pbLiGqTe8ntyai9mpBunKYcFqD5EaBy 4BeKQuM6yqRva33M4+Ciy9wavuIt2tVtG6Zng6i8J0xGtuRH+FimBVyJmFximuONlIrG LdQTUtaF8/naear6uGxK6PYhrQYI0/XOCFz+66aNOWXku7smLRqkzbXBtpBaHMOJ+FYB o9RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702285252; x=1702890052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3C4XGNOSQh2qdBXT10pq8W1umCRdPJt9fc3zqaA6RvE=; b=QFpgV36xUCGg7e9LiQPtShow5QVrliqJxt1FTSfmW32UY6yoeNYDM6LL++z0cU3Ehz CjXU/i1dft3YejAhz8tQxVLbt28/2hAgSQ007JixmzlLrElQKkKRDIVbXYXa8cJFe3ub x7pdDDVKBiIOoT3HmC6rM6C6MspwPXXfjjG8wj1lUMagWEjY7EAEwoOEaMmeF7mcmhq4 lhckP7osEP59eVNNfgKtXkUESM9EwikZSsHEKVXp5PUGgfF1heKXskUeDfINjuiRDVId MNSoy2mBBV41azjkTY7PrRxLBo3F0h0I1SO+txZ9HMMu8K6kayfKI1uTDV/3jEbNjyYk GqjA== X-Gm-Message-State: AOJu0YxCm1FYz/y5DKpyC/2pPp43rGrx5g0FyrfaOEF62l6UK8q76hxL 7wSQCS8a0Bu2qPF4id+3m/8= X-Google-Smtp-Source: AGHT+IEyCY3ErGWmI8LoIrEOhonz6hBK3ptUyoziGLGE/0Z406ge/Sd7ExzYhLFDiYo6/zx9zBAXTw== X-Received: by 2002:a92:d150:0:b0:35d:6220:ac8b with SMTP id t16-20020a92d150000000b0035d6220ac8bmr5172666ilg.58.1702285251875; Mon, 11 Dec 2023 01:00:51 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id f14-20020a056e020b4e00b0035b0b05189bsm2211216ilu.38.2023.12.11.01.00.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Dec 2023 01:00:51 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v3 2/5] dm: Support I/O priority for dm_io() Date: Mon, 11 Dec 2023 16:59:57 +0800 Message-Id: <20231211090000.9578-3-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> References: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Add ioprio field in struct dm_io_region, by this field specific I/O priority when call dm_io(). Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-bufio.c | 3 +++ drivers/md/dm-integrity.c | 5 +++++ drivers/md/dm-io.c | 1 + drivers/md/dm-log.c | 1 + drivers/md/dm-raid1.c | 2 ++ drivers/md/dm-snap-persistent.c | 1 + drivers/md/dm-writecache.c | 4 ++++ include/linux/dm-io.h | 2 ++ 8 files changed, 19 insertions(+) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 62eb27639c9b..7f82262aed54 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -1269,6 +1269,7 @@ static void use_dmio(struct dm_buffer *b, enum req_op op, sector_t sector, .bdev = b->c->bdev, .sector = sector, .count = n_sectors, + .ioprio = IOPRIO_DEFAULT, }; if (b->data_mode != DATA_MODE_VMALLOC) { @@ -2125,6 +2126,7 @@ int dm_bufio_issue_flush(struct dm_bufio_client *c) .bdev = c->bdev, .sector = 0, .count = 0, + .ioprio = IOPRIO_DEFAULT, }; if (WARN_ON_ONCE(dm_bufio_in_request())) @@ -2149,6 +2151,7 @@ int dm_bufio_issue_discard(struct dm_bufio_client *c, sector_t block, sector_t c .bdev = c->bdev, .sector = block_to_sector(c, block), .count = block_to_sector(c, count), + .ioprio = IOPRIO_DEFAULT, }; if (WARN_ON_ONCE(dm_bufio_in_request())) diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index e85c688fd91e..7cba183abdce 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -543,6 +543,7 @@ static int sync_rw_sb(struct dm_integrity_c *ic, blk_opf_t opf) io_loc.bdev = ic->meta_dev ? ic->meta_dev->bdev : ic->dev->bdev; io_loc.sector = ic->start; io_loc.count = SB_SECTORS; + io_loc.ioprio = IOPRIO_DEFAULT; if (op == REQ_OP_WRITE) { sb_set_version(ic); @@ -1070,6 +1071,7 @@ static void rw_journal_sectors(struct dm_integrity_c *ic, blk_opf_t opf, io_loc.bdev = ic->meta_dev ? ic->meta_dev->bdev : ic->dev->bdev; io_loc.sector = ic->start + SB_SECTORS + sector; io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { @@ -1187,6 +1189,7 @@ static void copy_from_journal(struct dm_integrity_c *ic, unsigned int section, u io_loc.bdev = ic->dev->bdev; io_loc.sector = target; io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { @@ -1515,6 +1518,7 @@ static void dm_integrity_flush_buffers(struct dm_integrity_c *ic, bool flush_dat fr.io_reg.bdev = ic->dev->bdev, fr.io_reg.sector = 0, fr.io_reg.count = 0, + fr.io_reg.ioprio = IOPRIO_DEFAULT, fr.ic = ic; init_completion(&fr.comp); r = dm_io(&fr.io_req, 1, &fr.io_reg, NULL); @@ -2738,6 +2742,7 @@ static void integrity_recalc(struct work_struct *w) io_loc.bdev = ic->dev->bdev; io_loc.sector = get_data_sector(ic, area, offset); io_loc.count = n_sectors; + io_loc.ioprio = IOPRIO_DEFAULT; r = dm_io(&io_req, 1, &io_loc, NULL); if (unlikely(r)) { diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index f053ce245814..b40f0a432981 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -354,6 +354,7 @@ static void do_region(const blk_opf_t opf, unsigned int region, &io->client->bios); bio->bi_iter.bi_sector = where->sector + (where->count - remaining); bio->bi_end_io = endio; + bio->bi_ioprio = where->ioprio; store_io_and_region_in_bio(bio, io, region); if (op == REQ_OP_DISCARD || op == REQ_OP_WRITE_ZEROES) { diff --git a/drivers/md/dm-log.c b/drivers/md/dm-log.c index f9f84236dfcd..e0dacdcd94f1 100644 --- a/drivers/md/dm-log.c +++ b/drivers/md/dm-log.c @@ -309,6 +309,7 @@ static int flush_header(struct log_c *lc) .bdev = lc->header_location.bdev, .sector = 0, .count = 0, + .ioprio = IOPRIO_DEFAULT, }; lc->io_req.bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; diff --git a/drivers/md/dm-raid1.c b/drivers/md/dm-raid1.c index ddcb2bc4a617..2de9b1377de3 100644 --- a/drivers/md/dm-raid1.c +++ b/drivers/md/dm-raid1.c @@ -275,6 +275,7 @@ static int mirror_flush(struct dm_target *ti) io[i].bdev = m->dev->bdev; io[i].sector = 0; io[i].count = 0; + io[i].ioprio = IOPRIO_DEFAULT; } error_bits = -1; @@ -475,6 +476,7 @@ static void map_region(struct dm_io_region *io, struct mirror *m, io->bdev = m->dev->bdev; io->sector = map_sector(m, bio); io->count = bio_sectors(bio); + io->ioprio = bio_prio(bio); } static void hold_bio(struct mirror_set *ms, struct bio *bio) diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c index 15649921f2a9..4aa70b71f1da 100644 --- a/drivers/md/dm-snap-persistent.c +++ b/drivers/md/dm-snap-persistent.c @@ -236,6 +236,7 @@ static int chunk_io(struct pstore *ps, void *area, chunk_t chunk, blk_opf_t opf, .bdev = dm_snap_cow(ps->store->snap)->bdev, .sector = ps->store->chunk_size * chunk, .count = ps->store->chunk_size, + .ioprio = IOPRIO_DEFAULT, }; struct dm_io_request io_req = { .bi_opf = opf, diff --git a/drivers/md/dm-writecache.c b/drivers/md/dm-writecache.c index 074cb785eafc..135d1268246f 100644 --- a/drivers/md/dm-writecache.c +++ b/drivers/md/dm-writecache.c @@ -515,6 +515,7 @@ static void ssd_commit_flushed(struct dm_writecache *wc, bool wait_for_ios) region.bdev = wc->ssd_dev->bdev; region.sector = (sector_t)i * (BITMAP_GRANULARITY >> SECTOR_SHIFT); region.count = (sector_t)(j - i) * (BITMAP_GRANULARITY >> SECTOR_SHIFT); + region.ioprio = IOPRIO_DEFAULT; if (unlikely(region.sector >= wc->metadata_sectors)) break; @@ -555,6 +556,7 @@ static void ssd_commit_superblock(struct dm_writecache *wc) region.bdev = wc->ssd_dev->bdev; region.sector = 0; region.count = max(4096U, wc->block_size) >> SECTOR_SHIFT; + region.ioprio = IOPRIO_DEFAULT; if (unlikely(region.sector + region.count > wc->metadata_sectors)) region.count = wc->metadata_sectors - region.sector; @@ -590,6 +592,7 @@ static void writecache_disk_flush(struct dm_writecache *wc, struct dm_dev *dev) region.bdev = dev->bdev; region.sector = 0; region.count = 0; + region.ioprio = IOPRIO_DEFAULT; req.bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; req.mem.type = DM_IO_KMEM; req.mem.ptr.addr = NULL; @@ -984,6 +987,7 @@ static int writecache_read_metadata(struct dm_writecache *wc, sector_t n_sectors region.bdev = wc->ssd_dev->bdev; region.sector = wc->start_sector; region.count = n_sectors; + region.ioprio = IOPRIO_DEFAULT; req.bi_opf = REQ_OP_READ | REQ_SYNC; req.mem.type = DM_IO_VMA; req.mem.ptr.vma = (char *)wc->memory_map; diff --git a/include/linux/dm-io.h b/include/linux/dm-io.h index 7595142f3fc5..227ee6d77c70 100644 --- a/include/linux/dm-io.h +++ b/include/linux/dm-io.h @@ -20,6 +20,8 @@ struct dm_io_region { struct block_device *bdev; sector_t sector; sector_t count; /* If this is zero the region is ignored. */ + /* Set it to IOPRIO_DEFAULT if you don't know what value to set */ + unsigned short ioprio; }; struct page_list { From patchwork Mon Dec 11 08:59:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13486850 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="anrL8DIi" Received: from mail-il1-x131.google.com (mail-il1-x131.google.com [IPv6:2607:f8b0:4864:20::131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12D47100; Mon, 11 Dec 2023 01:00:57 -0800 (PST) Received: by mail-il1-x131.google.com with SMTP id e9e14a558f8ab-35d72b72ff7so17184075ab.0; Mon, 11 Dec 2023 01:00:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702285256; x=1702890056; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4yqIhLm4LzKCMhpYyEZh+xPSurdrvcNEreCvT9oRThc=; b=anrL8DIiJ8ldkcqE/N+E4N/0NLpCJlXqQhKK0KhWUzF3cRFpXhf1gOhKKE7r6RKdrB c83St+vE4ynLJwlOmJRlaOsA6Y0mzB+y22ngzfRntOM4ayLgmqaLAsn5+RZ53vZeBg7z tUHFAuvayxMdnoNE3TXeQAN3UpaBz8BONiO9s0DyYpBo/LjcIRK4ur9pmim26J0DLmHf 5xLBtuVn2HYvpFzrhtVYIsiL3QNpMZWaK+rj/6DpG99retQxXAqkFHwplFCamIu7k4qW mD4zoji5Rik3C+nCVawNLPjgzAVryagKv9qJhjJp1ueoQfbo2xWLs2THyJhnjsQyfYsc aRiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702285256; x=1702890056; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4yqIhLm4LzKCMhpYyEZh+xPSurdrvcNEreCvT9oRThc=; b=Jtr1wVaS3UdcYyLhCtmC+9a+/V9DULSmU6b12rZYMkUBcEyQF1dy9AlbRAeqDIKsN/ 47pqUepY5ojsDR+4CFQWtgd9NTOoYY1hv4QSSaZiltEmb+tJoNSPmbmy9V8CMh2zFECy YFOUvJgp1XTOb902eJrnppryRpTJu+9dgNfMMXUCVCHDFjR2pA59K/Hvz3wODfam2CkQ eXkdb8CnWA5VnCrkQVxhcCu0H0tW51pJFBnoAbOoNC0WBioBSZvzbUWJ2x+EGsGF581S ZBLqsFhkfe0uRLaHoopHrs7ZxJEN9AxsF14ulgGdv7dexAim4jOhl4uBd1ynzdeeMab5 rdLw== X-Gm-Message-State: AOJu0YzDXShxV4FIJvTU05E2VdYM3H87a9alZaOzxW/W99aiDH7xLPjy V/ae1FQTpuNujW0AxBKpXlZfp+R0SJo= X-Google-Smtp-Source: AGHT+IFwEDZIKodMbswqRk8YWhMk1XUsFsZaJNQFEdpMh5j0sMlcPKbMyJdlZTyWQtYCDxweVqhjog== X-Received: by 2002:a05:6e02:1786:b0:35d:59a2:6451 with SMTP id y6-20020a056e02178600b0035d59a26451mr5488058ilu.36.1702285255800; Mon, 11 Dec 2023 01:00:55 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id f14-20020a056e020b4e00b0035b0b05189bsm2211216ilu.38.2023.12.11.01.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Dec 2023 01:00:55 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v3 3/5] dm-bufio: Support I/O priority Date: Mon, 11 Dec 2023 16:59:58 +0800 Message-Id: <20231211090000.9578-4-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> References: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin Add I/O priority parameter for dm_bufio_read() and dm_bufio_prefetch(). Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-bufio.c | 33 ++++++++++--------- drivers/md/dm-ebs-target.c | 8 ++--- drivers/md/dm-integrity.c | 2 +- drivers/md/dm-snap-persistent.c | 4 +-- drivers/md/dm-verity-fec.c | 4 +-- drivers/md/dm-verity-target.c | 4 +-- drivers/md/persistent-data/dm-block-manager.c | 6 ++-- include/linux/dm-bufio.h | 6 ++-- 8 files changed, 34 insertions(+), 33 deletions(-) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index 7f82262aed54..739f5dc52432 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -1256,7 +1256,7 @@ static void dmio_complete(unsigned long error, void *context) } static void use_dmio(struct dm_buffer *b, enum req_op op, sector_t sector, - unsigned int n_sectors, unsigned int offset) + unsigned int n_sectors, unsigned int offset, unsigned short ioprio) { int r; struct dm_io_request io_req = { @@ -1296,7 +1296,7 @@ static void bio_complete(struct bio *bio) } static void use_bio(struct dm_buffer *b, enum req_op op, sector_t sector, - unsigned int n_sectors, unsigned int offset) + unsigned int n_sectors, unsigned int offset, unsigned short ioprio) { struct bio *bio; char *ptr; @@ -1304,13 +1304,14 @@ static void use_bio(struct dm_buffer *b, enum req_op op, sector_t sector, bio = bio_kmalloc(1, GFP_NOWAIT | __GFP_NORETRY | __GFP_NOWARN); if (!bio) { - use_dmio(b, op, sector, n_sectors, offset); + use_dmio(b, op, sector, n_sectors, offset, ioprio); return; } bio_init(bio, b->c->bdev, bio->bi_inline_vecs, 1, op); bio->bi_iter.bi_sector = sector; bio->bi_end_io = bio_complete; bio->bi_private = b; + bio->bi_ioprio = ioprio; ptr = (char *)b->data + offset; len = n_sectors << SECTOR_SHIFT; @@ -1333,7 +1334,7 @@ static inline sector_t block_to_sector(struct dm_bufio_client *c, sector_t block return sector; } -static void submit_io(struct dm_buffer *b, enum req_op op, +static void submit_io(struct dm_buffer *b, enum req_op op, unsigned short ioprio, void (*end_io)(struct dm_buffer *, blk_status_t)) { unsigned int n_sectors; @@ -1363,9 +1364,9 @@ static void submit_io(struct dm_buffer *b, enum req_op op, } if (b->data_mode != DATA_MODE_VMALLOC) - use_bio(b, op, sector, n_sectors, offset); + use_bio(b, op, sector, n_sectors, offset, ioprio); else - use_dmio(b, op, sector, n_sectors, offset); + use_dmio(b, op, sector, n_sectors, offset, ioprio); } /* @@ -1421,7 +1422,7 @@ static void __write_dirty_buffer(struct dm_buffer *b, b->write_end = b->dirty_end; if (!write_list) - submit_io(b, REQ_OP_WRITE, write_endio); + submit_io(b, REQ_OP_WRITE, IOPRIO_DEFAULT, write_endio); else list_add_tail(&b->write_list, write_list); } @@ -1435,7 +1436,7 @@ static void __flush_write_list(struct list_head *write_list) struct dm_buffer *b = list_entry(write_list->next, struct dm_buffer, write_list); list_del(&b->write_list); - submit_io(b, REQ_OP_WRITE, write_endio); + submit_io(b, REQ_OP_WRITE, IOPRIO_DEFAULT, write_endio); cond_resched(); } blk_finish_plug(&plug); @@ -1817,7 +1818,7 @@ static void read_endio(struct dm_buffer *b, blk_status_t status) * and uses dm_bufio_mark_buffer_dirty to write new data back). */ static void *new_read(struct dm_bufio_client *c, sector_t block, - enum new_flag nf, struct dm_buffer **bp) + enum new_flag nf, struct dm_buffer **bp, unsigned short ioprio) { int need_submit = 0; struct dm_buffer *b; @@ -1870,7 +1871,7 @@ static void *new_read(struct dm_bufio_client *c, sector_t block, return NULL; if (need_submit) - submit_io(b, REQ_OP_READ, read_endio); + submit_io(b, REQ_OP_READ, ioprio, read_endio); wait_on_bit_io(&b->state, B_READING, TASK_UNINTERRUPTIBLE); @@ -1890,17 +1891,17 @@ static void *new_read(struct dm_bufio_client *c, sector_t block, void *dm_bufio_get(struct dm_bufio_client *c, sector_t block, struct dm_buffer **bp) { - return new_read(c, block, NF_GET, bp); + return new_read(c, block, NF_GET, bp, IOPRIO_DEFAULT); } EXPORT_SYMBOL_GPL(dm_bufio_get); void *dm_bufio_read(struct dm_bufio_client *c, sector_t block, - struct dm_buffer **bp) + struct dm_buffer **bp, unsigned short ioprio) { if (WARN_ON_ONCE(dm_bufio_in_request())) return ERR_PTR(-EINVAL); - return new_read(c, block, NF_READ, bp); + return new_read(c, block, NF_READ, bp, ioprio); } EXPORT_SYMBOL_GPL(dm_bufio_read); @@ -1910,12 +1911,12 @@ void *dm_bufio_new(struct dm_bufio_client *c, sector_t block, if (WARN_ON_ONCE(dm_bufio_in_request())) return ERR_PTR(-EINVAL); - return new_read(c, block, NF_FRESH, bp); + return new_read(c, block, NF_FRESH, bp, IOPRIO_DEFAULT); } EXPORT_SYMBOL_GPL(dm_bufio_new); void dm_bufio_prefetch(struct dm_bufio_client *c, - sector_t block, unsigned int n_blocks) + sector_t block, unsigned int n_blocks, unsigned short ioprio) { struct blk_plug plug; @@ -1951,7 +1952,7 @@ void dm_bufio_prefetch(struct dm_bufio_client *c, dm_bufio_unlock(c); if (need_submit) - submit_io(b, REQ_OP_READ, read_endio); + submit_io(b, REQ_OP_READ, ioprio, read_endio); dm_bufio_release(b); cond_resched(); diff --git a/drivers/md/dm-ebs-target.c b/drivers/md/dm-ebs-target.c index 435b45201f4d..8198c8a7b416 100644 --- a/drivers/md/dm-ebs-target.c +++ b/drivers/md/dm-ebs-target.c @@ -84,7 +84,7 @@ static int __ebs_rw_bvec(struct ebs_c *ec, enum req_op op, struct bio_vec *bv, /* Avoid reading for writes in case bio vector's page overwrites block completely. */ if (op == REQ_OP_READ || buf_off || bv_len < dm_bufio_get_block_size(ec->bufio)) - ba = dm_bufio_read(ec->bufio, block, &b); + ba = dm_bufio_read(ec->bufio, block, &b, IOPRIO_DEFAULT); else ba = dm_bufio_new(ec->bufio, block, &b); @@ -194,13 +194,13 @@ static void __ebs_process_bios(struct work_struct *ws) bio_list_for_each(bio, &bios) { block1 = __sector_to_block(ec, bio->bi_iter.bi_sector); if (bio_op(bio) == REQ_OP_READ) - dm_bufio_prefetch(ec->bufio, block1, __nr_blocks(ec, bio)); + dm_bufio_prefetch(ec->bufio, block1, __nr_blocks(ec, bio), IOPRIO_DEFAULT); else if (bio_op(bio) == REQ_OP_WRITE && !(bio->bi_opf & REQ_PREFLUSH)) { block2 = __sector_to_block(ec, bio_end_sector(bio)); if (__block_mod(bio->bi_iter.bi_sector, ec->u_bs)) - dm_bufio_prefetch(ec->bufio, block1, 1); + dm_bufio_prefetch(ec->bufio, block1, 1, IOPRIO_DEFAULT); if (__block_mod(bio_end_sector(bio), ec->u_bs) && block2 != block1) - dm_bufio_prefetch(ec->bufio, block2, 1); + dm_bufio_prefetch(ec->bufio, block2, 1, IOPRIO_DEFAULT); } } diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index 7cba183abdce..a2853c24a259 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -1421,7 +1421,7 @@ static int dm_integrity_rw_tag(struct dm_integrity_c *ic, unsigned char *tag, se if (unlikely(r)) return r; - data = dm_bufio_read(ic->bufio, *metadata_block, &b); + data = dm_bufio_read(ic->bufio, *metadata_block, &b, IOPRIO_DEFAULT); if (IS_ERR(data)) return PTR_ERR(data); diff --git a/drivers/md/dm-snap-persistent.c b/drivers/md/dm-snap-persistent.c index 4aa70b71f1da..eb6943fc7024 100644 --- a/drivers/md/dm-snap-persistent.c +++ b/drivers/md/dm-snap-persistent.c @@ -525,7 +525,7 @@ static int read_exceptions(struct pstore *ps, if (unlikely(pf_chunk >= dm_bufio_get_device_size(client))) break; - dm_bufio_prefetch(client, pf_chunk, 1); + dm_bufio_prefetch(client, pf_chunk, 1, IOPRIO_DEFAULT); prefetch_area++; if (unlikely(!prefetch_area)) break; @@ -534,7 +534,7 @@ static int read_exceptions(struct pstore *ps, chunk = area_location(ps, ps->current_area); - area = dm_bufio_read(client, chunk, &bp); + area = dm_bufio_read(client, chunk, &bp, IOPRIO_DEFAULT); if (IS_ERR(area)) { r = PTR_ERR(area); goto ret_destroy_bufio; diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 3ef9f018da60..715173cbf0ee 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -68,7 +68,7 @@ static u8 *fec_read_parity(struct dm_verity *v, u64 rsb, int index, block = div64_u64_rem(position, v->fec->io_size, &rem); *offset = (unsigned int)rem; - res = dm_bufio_read(v->fec->bufio, block, buf); + res = dm_bufio_read(v->fec->bufio, block, buf, IOPRIO_DEFAULT); if (IS_ERR(res)) { DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", v->data_dev->name, (unsigned long long)rsb, @@ -247,7 +247,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf); + bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 26adcfea0302..0038e168f3d7 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -307,7 +307,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf); + data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); if (IS_ERR(data)) return PTR_ERR(data); @@ -717,7 +717,7 @@ static void verity_prefetch_io(struct work_struct *work) } no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, - hash_block_end - hash_block_start + 1); + hash_block_end - hash_block_start + 1, IOPRIO_DEFAULT); } kfree(pw); diff --git a/drivers/md/persistent-data/dm-block-manager.c b/drivers/md/persistent-data/dm-block-manager.c index 0e010e1204aa..86a4f73d2f3d 100644 --- a/drivers/md/persistent-data/dm-block-manager.c +++ b/drivers/md/persistent-data/dm-block-manager.c @@ -474,7 +474,7 @@ int dm_bm_read_lock(struct dm_block_manager *bm, dm_block_t b, void *p; int r; - p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result); + p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result, IOPRIO_DEFAULT); if (IS_ERR(p)) return PTR_ERR(p); @@ -510,7 +510,7 @@ int dm_bm_write_lock(struct dm_block_manager *bm, if (dm_bm_is_read_only(bm)) return -EPERM; - p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result); + p = dm_bufio_read(bm->bufio, b, (struct dm_buffer **) result, IOPRIO_DEFAULT); if (IS_ERR(p)) return PTR_ERR(p); @@ -624,7 +624,7 @@ EXPORT_SYMBOL_GPL(dm_bm_flush); void dm_bm_prefetch(struct dm_block_manager *bm, dm_block_t b) { - dm_bufio_prefetch(bm->bufio, b, 1); + dm_bufio_prefetch(bm->bufio, b, 1, IOPRIO_DEFAULT); } bool dm_bm_is_read_only(struct dm_block_manager *bm) diff --git a/include/linux/dm-bufio.h b/include/linux/dm-bufio.h index 75e7d8cbb532..d270d48891f7 100644 --- a/include/linux/dm-bufio.h +++ b/include/linux/dm-bufio.h @@ -11,6 +11,7 @@ #define _LINUX_DM_BUFIO_H #include +#include #include /*----------------------------------------------------------------*/ @@ -62,7 +63,7 @@ void dm_bufio_set_sector_offset(struct dm_bufio_client *c, sector_t start); * it dirty. */ void *dm_bufio_read(struct dm_bufio_client *c, sector_t block, - struct dm_buffer **bp); + struct dm_buffer **bp, unsigned short ioprio); /* * Like dm_bufio_read, but return buffer from cache, don't read @@ -84,8 +85,7 @@ void *dm_bufio_new(struct dm_bufio_client *c, sector_t block, * I/O to finish. */ void dm_bufio_prefetch(struct dm_bufio_client *c, - sector_t block, unsigned int n_blocks); - + sector_t block, unsigned int n_blocks, unsigned short ioprio); /* * Release a reference obtained with dm_bufio_{read,get,new}. The data * pointer and dm_buffer pointer is no longer valid after this call. From patchwork Mon Dec 11 08:59:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13486851 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JxyHvGnk" Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F5E01A7; Mon, 11 Dec 2023 01:01:00 -0800 (PST) Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-7b7020f03c9so173648839f.3; Mon, 11 Dec 2023 01:01:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702285260; x=1702890060; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vN+RlAGZeGH2zMVmYb6ghj5GDWJcjTyD9/i+J4cSzo8=; b=JxyHvGnkNpqPpJH89mUj2mpRaDAiHAj5RLxPmUz3SXtlP41HUZvyN7n5oFXvwk7yfp XD0gwxJkA49EXjv5Da4Ad397HV8eYPVs5tkA/DGbQavQ//SybErtsSpHpsI3PPra62l6 bhHOAuWAbl930nd2niN61l6e/b1wCqTur3chWnVCcDqpEp4/Jc466j3RqdYjCpAYBM6+ cylOxFTelxQBlitJhvAmh1yV7u397qSYgi/k63VCIyWbgKBktM2gXT+xzfqBLogK1Mw3 I5Ft9siyiVd1itUGRxSYQiJnqo5TMoNcvepDahEfBoR/596+G9+ivHxN8SXYS1dzFZia Q3EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702285260; x=1702890060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vN+RlAGZeGH2zMVmYb6ghj5GDWJcjTyD9/i+J4cSzo8=; b=lbHCefBdvB5IHVE67D4dYLSfWEWEB6eFzcMSub0hJuJIiBPJiMZFbDGCZYUIjiWdnp jQKbFPgh+AY8GjtJNBP1rkZzZttNtty1rIgeWomI4HPZIEo5sirKpwOoRQGLsrdjgx4K YMnouUCTA8WxAJ52PygXcP59tOkRfTjanQKtRJoZD2yhc/c9eki1xhtkDis/0CootGdr +LpyFT2+oN6PmUsrkbERpIVhjpvadiXCRwUNilr4GaWMHk7z6rLR5d2PAbr5L6CGpSM8 Ka1cVLKVplo0OpfSyMypcElFpaPIxPAIg14zi5sJadTqYD2hGaRbYsEaatxL1a6KPMhr z76w== X-Gm-Message-State: AOJu0YyhFqW7Dy81smnC60yD9/Rq7IWpsmxKdS8MQUZeFYLYgZO/Lr86 8SVZvozFJ9nVILu50G/PQN/2PjDo7Ec= X-Google-Smtp-Source: AGHT+IE1sKCuGQStpjgZz24q1F+6gUFjFzwfnx1Z68zljQrTtBbgl+OeUjzPURrTCayBGeirVktHfg== X-Received: by 2002:a05:6e02:16ce:b0:35d:6c24:f5f6 with SMTP id 14-20020a056e0216ce00b0035d6c24f5f6mr7023715ilx.14.1702285259773; Mon, 11 Dec 2023 01:00:59 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id f14-20020a056e020b4e00b0035b0b05189bsm2211216ilu.38.2023.12.11.01.00.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Dec 2023 01:00:59 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v3 4/5] dm verity: Fix I/O priority lost when read FEC and hash Date: Mon, 11 Dec 2023 16:59:59 +0800 Message-Id: <20231211090000.9578-5-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> References: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin To fix this problem, when read FEC and hash from disk, I/O priority are inconsistent with data block and blocked by other I/O with low I/O priority. Make I/O for FEC and hash has same I/O priority with original data I/O. Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin --- drivers/md/dm-verity-fec.c | 3 ++- drivers/md/dm-verity-target.c | 8 ++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 715173cbf0ee..6a5a679e7e8a 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -209,6 +209,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, u8 *bbuf, *rs_block; u8 want_digest[HASH_MAX_DIGESTSIZE]; unsigned int n, k; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (neras) *neras = 0; @@ -247,7 +248,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); + bbuf = dm_bufio_read(bufio, block, &buf, bio_prio(bio)); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 0038e168f3d7..8c911b6722ce 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -51,6 +51,7 @@ static DEFINE_STATIC_KEY_FALSE(use_tasklet_enabled); struct dm_verity_prefetch_work { struct work_struct work; struct dm_verity *v; + unsigned short ioprio; sector_t block; unsigned int n_blocks; }; @@ -293,6 +294,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, int r; sector_t hash_block; unsigned int offset; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); verity_hash_at_level(v, block, level, &hash_block, &offset); @@ -307,7 +309,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); + data = dm_bufio_read(v->bufio, hash_block, &buf, bio_prio(bio)); if (IS_ERR(data)) return PTR_ERR(data); @@ -717,7 +719,7 @@ static void verity_prefetch_io(struct work_struct *work) } no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, - hash_block_end - hash_block_start + 1, IOPRIO_DEFAULT); + hash_block_end - hash_block_start + 1, pw->ioprio); } kfree(pw); @@ -728,6 +730,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) sector_t block = io->block; unsigned int n_blocks = io->n_blocks; struct dm_verity_prefetch_work *pw; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (v->validated_blocks) { while (n_blocks && test_bit(block, v->validated_blocks)) { @@ -751,6 +754,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) pw->v = v; pw->block = block; pw->n_blocks = n_blocks; + pw->ioprio = bio_prio(bio); queue_work(v->verify_wq, &pw->work); } From patchwork Mon Dec 11 09:00:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hongyu Jin X-Patchwork-Id: 13486852 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d+b+STac" Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A773D116; Mon, 11 Dec 2023 01:01:05 -0800 (PST) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-7b3a8366e13so185616839f.1; Mon, 11 Dec 2023 01:01:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702285264; x=1702890064; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=imqsqSeoCG7iiefqcy/059zT5ygF+aA0rzXCKWGVYeM=; b=d+b+STacr4OADxY5G1+HFNa2arpSrTwfEX8oPrUokrBJyFPinATho1limh+0D1dI+O kuRaNSka6jkIwge4utkxUXcg8IGyZBP9zwtBJfqAWnphATjWd0Rrqj6gNHiXeKCi+TA/ PZ84gD/5dFIkz2D1vSQvNgcHL7JboVXMn9Qn4uwkwA2CqdhzA554bKzHB5OYj9Zipuzh C+sZwfIawpiH60V1fMkSQDWTDEEBp87lmgm+xuTSipNpDRbUN1nwAAxy4w7BpMzwjrjv /7w7osf4SiLDJw2pjChRJCEaAUnLnJbD4XP28gdFdYUSKmQKSn7cC2YOb33sI76llAMD vItw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702285264; x=1702890064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=imqsqSeoCG7iiefqcy/059zT5ygF+aA0rzXCKWGVYeM=; b=qrKdkrxWs2IcV8Z9t77xa0iFeXrjbo6XchSZQIA20LtGf5fTkZz/uWDGf6Wzv3BQQ6 /rt/hdIo31othCm0FMfXhhP/IiLTHnMCmbU0Wpon8PfFensGwtwDqcxGcua81Wo0jy3R n0aspVtz2yp5QzlQ11IctcvWafpoYbKB0tIse94gVtfVN9nz4Bs9Y8bjgPWAxTX8INkf FHdP8b1IT6ZUQ6vvhyNaVcTJNSXC8zsHtXF9QH9CaY/ZD/XHwr0MWaleA9Dq+URZWjJe noN6q26Z9EZG+qKRt2txI+VS9If5hFExgliN/Nm0SoBOKnDs7z5Utc8Pwfr9FkspBWew OHvA== X-Gm-Message-State: AOJu0YwuQcA3UT6dMkNKg4CKy+XxMcKt72BWyjQrECQUuP7MzDIwLLFW wpCcqOSCmQs8erdV+z1MH04Aj0z3ugg= X-Google-Smtp-Source: AGHT+IGLCG8im5lLD3WEgqK1syGR1Nyf2Q+dj/rBbukrCX6r/qUF7cEgvaRF1uPm4mONhDN/QIwBbw== X-Received: by 2002:a92:ca48:0:b0:35d:59a2:3329 with SMTP id q8-20020a92ca48000000b0035d59a23329mr7723508ilo.45.1702285263896; Mon, 11 Dec 2023 01:01:03 -0800 (PST) Received: from ubuntu.. ([117.18.48.102]) by smtp.gmail.com with ESMTPSA id f14-20020a056e020b4e00b0035b0b05189bsm2211216ilu.38.2023.12.11.01.01.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Dec 2023 01:01:03 -0800 (PST) From: Hongyu Jin To: agk@redhat.com, snitzer@kernel.org, mpatocka@redhat.com, axboe@kernel.dk, ebiggers@kernel.org Cc: zhiguo.niu@unisoc.com, ke.wang@unisoc.com, yibin.ding@unisoc.com, hongyu.jin@unisoc.com, linux-kernel@vger.kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org Subject: [PATCH v3 5/5] dm-crypt: Fix lost ioprio when queuing write bios Date: Mon, 11 Dec 2023 17:00:00 +0800 Message-Id: <20231211090000.9578-6-hongyu.jin.cn@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> References: <20231211090000.9578-1-hongyu.jin.cn@gmail.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin The original submitting bio->bi_ioprio setting can be retained by struct dm_crypt_io::base_bio, we set the original bio's ioprio to the cloned bio for write. Signed-off-by: Hongyu Jin Reviewed-by: Eric Wheeler --- drivers/md/dm-crypt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 6de107aff331..b67fec865f00 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -1683,6 +1683,7 @@ static struct bio *crypt_alloc_buffer(struct dm_crypt_io *io, unsigned int size) GFP_NOIO, &cc->bs); clone->bi_private = io; clone->bi_end_io = crypt_endio; + clone->bi_ioprio = bio_prio(io->base_bio); remaining_size = size;