From patchwork Tue Jan 30 20:26:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 13537977 Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D060471B5C for ; Tue, 30 Jan 2024 20:27:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706646422; cv=none; b=ef0EVcjjfTx0lg586RvTH5zI6lMdb6MM8RtYzwXE0IwVFaz7d6fGiYnis/1kpsf0tPIu29LAz+Y6o0QSNZgou177KQOiiscI0Mx2vv1zSL8lD2QwNLFbXfTRjl4bL/BcECnXU49qAKRCRaVI+6teUnh/Ap8O+IxRBizls0+VhUM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706646422; c=relaxed/simple; bh=uGDio7c6nNduR6FeZARIul2B2RdX/xUgYwDX+TfUrRE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DDDVEBuYEMWAcCSdBzGpR1lppsCSn3g9aDc/70S0/J3DUChcQxezoOuld4g6fqlyJXg2O+XFHvwuvTNTGI+ZJ8y3ySD9hO9CqW9biWg5yA+m78pQW9nybohw1kxegMhefdDlnGyU3sp9kRpjrjR1BF0Qy83V6c10YYqrKzxhBUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=redhat.com; arc=none smtp.client-ip=209.85.222.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-783dc658bd9so273331385a.1 for ; Tue, 30 Jan 2024 12:27:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706646419; x=1707251219; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9lIbB0ZZu5PD4YTDs2kMAy1ta/PON4PyQfNw4rxc6Fg=; b=XxYbVmotW8daGuepsoyAyoSQaFKCoFaICWRlPUWV0Yf/zhVNQzG3ik+8AMbokSR9oD l1WLLYX4TbDVl4OW9e1mBpXYFNU+MgiwGW9sWK748v7YUVQp3kPN8Py/w1P+Ghb9g94d lnu29N2KZpJjOWFjbojq2w4SK2GzyD0poFekMzqFBoIcbWFRAZQ3ErRsLRlxYGSZHSQz aVHG1bqY9iSHR/yT/MXEpoOsx3UojLqeVI7hEJqENPTKSInuT21ZVO8dXJKz8hZ7nAes Nhw6pduQp0tQ4CjCHESimgJH7ZIwSViy+17Vn63cxOn/BAx06H7kaNkMWXKZPOc/KIt7 +biw== X-Gm-Message-State: AOJu0YxRuh7QCzqe6Girdi6GHHfVEckijQm6nQqkTIFv9gTVvesOl/uk NP18oj0xz45S0w3t18FEgP4LdsMowixIcwBhb0n1lKropMPg+G/xRS0lORsFHQ== X-Google-Smtp-Source: AGHT+IHe/+90px8Z86qorT42OcTABVqY3egSchOZQAMNMkvm1jqP4WiUoujuu6XQ/u9wne9Kzexdew== X-Received: by 2002:a05:6214:21ad:b0:68c:4df3:a12f with SMTP id t13-20020a05621421ad00b0068c4df3a12fmr6726832qvc.3.1706646419650; Tue, 30 Jan 2024 12:26:59 -0800 (PST) Received: from localhost (pool-68-160-141-91.bstnma.fios.verizon.net. [68.160.141.91]) by smtp.gmail.com with ESMTPSA id kd8-20020a056214400800b00686ac3c9db4sm4732431qvb.98.2024.01.30.12.26.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jan 2024 12:26:59 -0800 (PST) From: Mike Snitzer To: axboe@kernel.dk, hongyu.jin.cn@gmail.com Cc: ebiggers@kernel.org, dm-devel@lists.linux.dev, linux-block@vger.kernel.org, Hongyu Jin , Yibin Ding , Eric Biggers , Mikulas Patocka , Mike Snitzer Subject: [PATCH v9 4/5] dm verity: Fix IO priority lost when reading FEC and hash Date: Tue, 30 Jan 2024 15:26:37 -0500 Message-Id: <20240130202638.62600-5-snitzer@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20240130202638.62600-1-snitzer@kernel.org> References: <20240130202638.62600-1-snitzer@kernel.org> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Hongyu Jin After obtaining the data, verification or error correction process may trigger a new IO that loses the priority of the original IO, that is, the verification of the higher priority IO may be blocked by the lower priority IO. Make the IO used for verification and error correction follow the priority of the original IO. Co-developed-by: Yibin Ding Signed-off-by: Yibin Ding Signed-off-by: Hongyu Jin Reviewed-by: Eric Biggers Reviewed-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-verity-fec.c | 21 ++++++++++++--------- drivers/md/dm-verity-target.c | 12 ++++++++---- 2 files changed, 20 insertions(+), 13 deletions(-) diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 49db19e537f9..066521de08da 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -60,7 +60,8 @@ static int fec_decode_rs8(struct dm_verity *v, struct dm_verity_fec_io *fio, * to the data block. Caller is responsible for releasing buf. */ static u8 *fec_read_parity(struct dm_verity *v, u64 rsb, int index, - unsigned int *offset, struct dm_buffer **buf) + unsigned int *offset, struct dm_buffer **buf, + unsigned short ioprio) { u64 position, block, rem; u8 *res; @@ -69,7 +70,7 @@ static u8 *fec_read_parity(struct dm_verity *v, u64 rsb, int index, block = div64_u64_rem(position, v->fec->io_size, &rem); *offset = (unsigned int)rem; - res = dm_bufio_read(v->fec->bufio, block, buf, IOPRIO_DEFAULT); + res = dm_bufio_read(v->fec->bufio, block, buf, ioprio); if (IS_ERR(res)) { DMERR("%s: FEC %llu: parity read failed (block %llu): %ld", v->data_dev->name, (unsigned long long)rsb, @@ -121,16 +122,17 @@ static inline unsigned int fec_buffer_rs_index(unsigned int i, unsigned int j) * Decode all RS blocks from buffers and copy corrected bytes into fio->output * starting from block_offset. */ -static int fec_decode_bufs(struct dm_verity *v, struct dm_verity_fec_io *fio, - u64 rsb, int byte_index, unsigned int block_offset, - int neras) +static int fec_decode_bufs(struct dm_verity *v, struct dm_verity_io *io, + struct dm_verity_fec_io *fio, u64 rsb, int byte_index, + unsigned int block_offset, int neras) { int r, corrected = 0, res; struct dm_buffer *buf; unsigned int n, i, offset; u8 *par, *block; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); - par = fec_read_parity(v, rsb, block_offset, &offset, &buf); + par = fec_read_parity(v, rsb, block_offset, &offset, &buf, bio_prio(bio)); if (IS_ERR(par)) return PTR_ERR(par); @@ -158,7 +160,7 @@ static int fec_decode_bufs(struct dm_verity *v, struct dm_verity_fec_io *fio, if (offset >= v->fec->io_size) { dm_bufio_release(buf); - par = fec_read_parity(v, rsb, block_offset, &offset, &buf); + par = fec_read_parity(v, rsb, block_offset, &offset, &buf, bio_prio(bio)); if (IS_ERR(par)) return PTR_ERR(par); } @@ -210,6 +212,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, u8 *bbuf, *rs_block; u8 want_digest[HASH_MAX_DIGESTSIZE]; unsigned int n, k; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); if (neras) *neras = 0; @@ -248,7 +251,7 @@ static int fec_read_bufs(struct dm_verity *v, struct dm_verity_io *io, bufio = v->bufio; } - bbuf = dm_bufio_read(bufio, block, &buf, IOPRIO_DEFAULT); + bbuf = dm_bufio_read(bufio, block, &buf, bio_prio(bio)); if (IS_ERR(bbuf)) { DMWARN_LIMIT("%s: FEC %llu: read failed (%llu): %ld", v->data_dev->name, @@ -377,7 +380,7 @@ static int fec_decode_rsb(struct dm_verity *v, struct dm_verity_io *io, if (unlikely(r < 0)) return r; - r = fec_decode_bufs(v, fio, rsb, r, pos, neras); + r = fec_decode_bufs(v, io, fio, rsb, r, pos, neras); if (r < 0) return r; diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 4758bfe2c156..8cbf81fc0031 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -51,6 +51,7 @@ static DEFINE_STATIC_KEY_FALSE(use_tasklet_enabled); struct dm_verity_prefetch_work { struct work_struct work; struct dm_verity *v; + unsigned short ioprio; sector_t block; unsigned int n_blocks; }; @@ -294,6 +295,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, int r; sector_t hash_block; unsigned int offset; + struct bio *bio = dm_bio_from_per_bio_data(io, v->ti->per_io_data_size); verity_hash_at_level(v, block, level, &hash_block, &offset); @@ -308,7 +310,7 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io, return -EAGAIN; } } else - data = dm_bufio_read(v->bufio, hash_block, &buf, IOPRIO_DEFAULT); + data = dm_bufio_read(v->bufio, hash_block, &buf, bio_prio(bio)); if (IS_ERR(data)) return PTR_ERR(data); @@ -720,13 +722,14 @@ static void verity_prefetch_io(struct work_struct *work) no_prefetch_cluster: dm_bufio_prefetch(v->bufio, hash_block_start, hash_block_end - hash_block_start + 1, - IOPRIO_DEFAULT); + pw->ioprio); } kfree(pw); } -static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) +static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io, + unsigned short ioprio) { sector_t block = io->block; unsigned int n_blocks = io->n_blocks; @@ -754,6 +757,7 @@ static void verity_submit_prefetch(struct dm_verity *v, struct dm_verity_io *io) pw->v = v; pw->block = block; pw->n_blocks = n_blocks; + pw->ioprio = ioprio; queue_work(v->verify_wq, &pw->work); } @@ -796,7 +800,7 @@ static int verity_map(struct dm_target *ti, struct bio *bio) verity_fec_init_io(io); - verity_submit_prefetch(v, io); + verity_submit_prefetch(v, io, bio_prio(bio)); submit_bio_noacct(bio);