From patchwork Tue Jul 2 15:10:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13719904 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 564D91DFFC for ; Tue, 2 Jul 2024 15:10:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933055; cv=none; b=EtOMFgTMMW77chab4dxLSJpLjZV0ZrbTzpyUkh+YsRAH9IqD94eXLRzcUMj7dBHMS1WJnYptxMU0L27ef2Q/5OJzMx69+PwPt3WZy1Eu68OfAfMSyxPAAt+RC+ij6GPakkUTVSDHPfNl4tw38r33xJjv8oRkja/7pMb7nwyfwlk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933055; c=relaxed/simple; bh=e5VS8zSf15PDrnJHNZNh+0jCSDDOOaKcfd2K0UcDqY8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jHKtZ1adWh7TtzubhStWyC6uIz6zMkoCsY0nDsiYWj96yBSkxFxdJ9Eo5DVcTGZnOsVWCPzFpKdMJRQubIHE10PCF4d7JTvsW0zUkpgi3tG7tV5PEYQIspkNHxy2uPt5hmFmfG5+FXXAGMZnhqm1z8QwGBOzqNXsQW/+QmnKrRA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=yU0t26U7; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="yU0t26U7" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=2c2b2ppIq9h2U710pgvb9Zx5HsoY970Tsy5/d8QZ+dI=; b=yU0t26U7abUhxcDAyAtcVF8xrl Rgf2359BZSnuVjxLEKVHOLuAKGKRAg4xpdwbxNYirE/liebIB/dgGwvc/iHOzPvoP/w09jq+395IB AxXvjuwXsFlr/ueMsdfXw2kSBpOatzAt5seq7XedHb0l/090zJluwNkURr/SsyPenyTa7H1WYn46J 6DVgnDjI7P5emhEjcfiuFMTzmjYZR1bFVrQuKox8mORDYZxtkoWMN/Vw0OyDjcC/crW2OqW0u08Bs qp/LA1loftiehTkISwqTsb6RXQmBvdFVmNbcXHojPfkKStbAIwuFsb1WKUxu6oUcqMSTunHR9ITxn Cgha0N7Q==; Received: from 2a02-8389-2341-5b80-4c69-cf21-4832-bbca.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:4c69:cf21:4832:bbca] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux)) id 1sOf9g-000000079Tl-15FA; Tue, 02 Jul 2024 15:10:52 +0000 From: Christoph Hellwig To: Jens Axboe Cc: "Martin K . Petersen" , Anuj Gupta , Kanchan Joshi , linux-block@vger.kernel.org Subject: [PATCH 1/6] block: split integrity support out of bio.h Date: Tue, 2 Jul 2024 17:10:19 +0200 Message-ID: <20240702151047.1746127-2-hch@lst.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240702151047.1746127-1-hch@lst.de> References: <20240702151047.1746127-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Split struct bio_integrity_payload and the related prototypes out of bio.h into a separate bio-integrity.h header so that it is only pulled in by the few places that need it. Signed-off-by: Christoph Hellwig Reviewed-by: Anuj Gupta Reviewed-by: Kanchan Joshi Reviewed-by: Johannes Thumshirn --- block/bio.c | 2 +- block/blk.h | 1 + block/bounce.c | 2 +- drivers/md/dm.c | 1 + drivers/nvme/host/ioctl.c | 1 + drivers/scsi/sd.c | 3 +- include/linux/bio-integrity.h | 153 +++++++++++++++++++++++++++++++++ include/linux/bio.h | 156 ---------------------------------- include/linux/blk-integrity.h | 1 + 9 files changed, 161 insertions(+), 159 deletions(-) create mode 100644 include/linux/bio-integrity.h diff --git a/block/bio.c b/block/bio.c index e9e809a63c5975..4ca3f31ce45fb5 100644 --- a/block/bio.c +++ b/block/bio.c @@ -4,7 +4,7 @@ */ #include #include -#include +#include #include #include #include diff --git a/block/blk.h b/block/blk.h index 47dadd2439b1ca..401e604f35d2cf 100644 --- a/block/blk.h +++ b/block/blk.h @@ -2,6 +2,7 @@ #ifndef BLK_INTERNAL_H #define BLK_INTERNAL_H +#include #include #include /* for max_pfn/max_low_pfn */ #include diff --git a/block/bounce.c b/block/bounce.c index d6a5219f29dd53..0d898cd5ec497f 100644 --- a/block/bounce.c +++ b/block/bounce.c @@ -10,7 +10,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 7d107ae06e1ae1..92d6eeb0a59327 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -11,6 +11,7 @@ #include "dm-uevent.h" #include "dm-ima.h" +#include #include #include #include diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 8b69427a44762a..fb46f55f8b2894 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -3,6 +3,7 @@ * Copyright (c) 2011-2014, Intel Corporation. * Copyright (c) 2017-2021 Christoph Hellwig. */ +#include #include /* for force_successful_syscall_return */ #include #include diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index 933074f8af4e96..f0f4fd7f20024a 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -33,11 +33,12 @@ * than the level indicated above to trigger output. */ +#include #include #include #include #include -#include +#include #include #include #include diff --git a/include/linux/bio-integrity.h b/include/linux/bio-integrity.h new file mode 100644 index 00000000000000..70ef19a0dc7e8b --- /dev/null +++ b/include/linux/bio-integrity.h @@ -0,0 +1,153 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_BIO_INTEGRITY_H +#define _LINUX_BIO_INTEGRITY_H + +#include + +enum bip_flags { + BIP_BLOCK_INTEGRITY = 1 << 0, /* block layer owns integrity data */ + BIP_MAPPED_INTEGRITY = 1 << 1, /* ref tag has been remapped */ + BIP_CTRL_NOCHECK = 1 << 2, /* disable HBA integrity checking */ + BIP_DISK_NOCHECK = 1 << 3, /* disable disk integrity checking */ + BIP_IP_CHECKSUM = 1 << 4, /* IP checksum */ + BIP_INTEGRITY_USER = 1 << 5, /* Integrity payload is user address */ + BIP_COPY_USER = 1 << 6, /* Kernel bounce buffer in use */ +}; + +struct bio_integrity_payload { + struct bio *bip_bio; /* parent bio */ + + struct bvec_iter bip_iter; + + unsigned short bip_vcnt; /* # of integrity bio_vecs */ + unsigned short bip_max_vcnt; /* integrity bio_vec slots */ + unsigned short bip_flags; /* control flags */ + + struct bvec_iter bio_iter; /* for rewinding parent bio */ + + struct work_struct bip_work; /* I/O completion */ + + struct bio_vec *bip_vec; + struct bio_vec bip_inline_vecs[];/* embedded bvec array */ +}; + +#ifdef CONFIG_BLK_DEV_INTEGRITY + +#define bip_for_each_vec(bvl, bip, iter) \ + for_each_bvec(bvl, (bip)->bip_vec, iter, (bip)->bip_iter) + +#define bio_for_each_integrity_vec(_bvl, _bio, _iter) \ + for_each_bio(_bio) \ + bip_for_each_vec(_bvl, _bio->bi_integrity, _iter) + +static inline struct bio_integrity_payload *bio_integrity(struct bio *bio) +{ + if (bio->bi_opf & REQ_INTEGRITY) + return bio->bi_integrity; + + return NULL; +} + +static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag) +{ + struct bio_integrity_payload *bip = bio_integrity(bio); + + if (bip) + return bip->bip_flags & flag; + + return false; +} + +static inline sector_t bip_get_seed(struct bio_integrity_payload *bip) +{ + return bip->bip_iter.bi_sector; +} + +static inline void bip_set_seed(struct bio_integrity_payload *bip, + sector_t seed) +{ + bip->bip_iter.bi_sector = seed; +} + +struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, gfp_t gfp, + unsigned int nr); +int bio_integrity_add_page(struct bio *bio, struct page *page, unsigned int len, + unsigned int offset); +int bio_integrity_map_user(struct bio *bio, void __user *ubuf, ssize_t len, u32 seed); +void bio_integrity_unmap_free_user(struct bio *bio); +bool bio_integrity_prep(struct bio *bio); +void bio_integrity_advance(struct bio *bio, unsigned int bytes_done); +void bio_integrity_trim(struct bio *bio); +int bio_integrity_clone(struct bio *bio, struct bio *bio_src, gfp_t gfp_mask); +int bioset_integrity_create(struct bio_set *bs, int pool_size); +void bioset_integrity_free(struct bio_set *bs); +void bio_integrity_init(void); + +#else /* CONFIG_BLK_DEV_INTEGRITY */ + +static inline void *bio_integrity(struct bio *bio) +{ + return NULL; +} + +static inline int bioset_integrity_create(struct bio_set *bs, int pool_size) +{ + return 0; +} + +static inline void bioset_integrity_free(struct bio_set *bs) +{ +} + +static inline int bio_integrity_map_user(struct bio *bio, void __user *ubuf, + ssize_t len, u32 seed) +{ + return -EINVAL; +} + +static inline void bio_integrity_unmap_free_user(struct bio *bio) +{ +} + +static inline bool bio_integrity_prep(struct bio *bio) +{ + return true; +} + +static inline int bio_integrity_clone(struct bio *bio, struct bio *bio_src, + gfp_t gfp_mask) +{ + return 0; +} + +static inline void bio_integrity_advance(struct bio *bio, + unsigned int bytes_done) +{ +} + +static inline void bio_integrity_trim(struct bio *bio) +{ +} + +static inline void bio_integrity_init(void) +{ +} + +static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag) +{ + return false; +} + +static inline void *bio_integrity_alloc(struct bio *bio, gfp_t gfp, + unsigned int nr) +{ + return ERR_PTR(-EINVAL); +} + +static inline int bio_integrity_add_page(struct bio *bio, struct page *page, + unsigned int len, unsigned int offset) +{ + return 0; +} +#endif /* CONFIG_BLK_DEV_INTEGRITY */ +#endif /* _LINUX_BIO_INTEGRITY_H */ diff --git a/include/linux/bio.h b/include/linux/bio.h index 818e9361294781..a46e2047bea4d2 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -321,69 +321,6 @@ static inline void bio_next_folio(struct folio_iter *fi, struct bio *bio) #define bio_for_each_folio_all(fi, bio) \ for (bio_first_folio(&fi, bio, 0); fi.folio; bio_next_folio(&fi, bio)) -enum bip_flags { - BIP_BLOCK_INTEGRITY = 1 << 0, /* block layer owns integrity data */ - BIP_MAPPED_INTEGRITY = 1 << 1, /* ref tag has been remapped */ - BIP_CTRL_NOCHECK = 1 << 2, /* disable HBA integrity checking */ - BIP_DISK_NOCHECK = 1 << 3, /* disable disk integrity checking */ - BIP_IP_CHECKSUM = 1 << 4, /* IP checksum */ - BIP_INTEGRITY_USER = 1 << 5, /* Integrity payload is user address */ - BIP_COPY_USER = 1 << 6, /* Kernel bounce buffer in use */ -}; - -/* - * bio integrity payload - */ -struct bio_integrity_payload { - struct bio *bip_bio; /* parent bio */ - - struct bvec_iter bip_iter; - - unsigned short bip_vcnt; /* # of integrity bio_vecs */ - unsigned short bip_max_vcnt; /* integrity bio_vec slots */ - unsigned short bip_flags; /* control flags */ - - struct bvec_iter bio_iter; /* for rewinding parent bio */ - - struct work_struct bip_work; /* I/O completion */ - - struct bio_vec *bip_vec; - struct bio_vec bip_inline_vecs[];/* embedded bvec array */ -}; - -#if defined(CONFIG_BLK_DEV_INTEGRITY) - -static inline struct bio_integrity_payload *bio_integrity(struct bio *bio) -{ - if (bio->bi_opf & REQ_INTEGRITY) - return bio->bi_integrity; - - return NULL; -} - -static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag) -{ - struct bio_integrity_payload *bip = bio_integrity(bio); - - if (bip) - return bip->bip_flags & flag; - - return false; -} - -static inline sector_t bip_get_seed(struct bio_integrity_payload *bip) -{ - return bip->bip_iter.bi_sector; -} - -static inline void bip_set_seed(struct bio_integrity_payload *bip, - sector_t seed) -{ - bip->bip_iter.bi_sector = seed; -} - -#endif /* CONFIG_BLK_DEV_INTEGRITY */ - void bio_trim(struct bio *bio, sector_t offset, sector_t size); extern struct bio *bio_split(struct bio *bio, int sectors, gfp_t gfp, struct bio_set *bs); @@ -721,99 +658,6 @@ static inline bool bioset_initialized(struct bio_set *bs) return bs->bio_slab != NULL; } -#if defined(CONFIG_BLK_DEV_INTEGRITY) - -#define bip_for_each_vec(bvl, bip, iter) \ - for_each_bvec(bvl, (bip)->bip_vec, iter, (bip)->bip_iter) - -#define bio_for_each_integrity_vec(_bvl, _bio, _iter) \ - for_each_bio(_bio) \ - bip_for_each_vec(_bvl, _bio->bi_integrity, _iter) - -int bio_integrity_map_user(struct bio *bio, void __user *ubuf, ssize_t len, u32 seed); -void bio_integrity_unmap_free_user(struct bio *bio); -extern struct bio_integrity_payload *bio_integrity_alloc(struct bio *, gfp_t, unsigned int); -extern int bio_integrity_add_page(struct bio *, struct page *, unsigned int, unsigned int); -extern bool bio_integrity_prep(struct bio *); -extern void bio_integrity_advance(struct bio *, unsigned int); -extern void bio_integrity_trim(struct bio *); -extern int bio_integrity_clone(struct bio *, struct bio *, gfp_t); -extern int bioset_integrity_create(struct bio_set *, int); -extern void bioset_integrity_free(struct bio_set *); -extern void bio_integrity_init(void); - -#else /* CONFIG_BLK_DEV_INTEGRITY */ - -static inline void *bio_integrity(struct bio *bio) -{ - return NULL; -} - -static inline int bioset_integrity_create(struct bio_set *bs, int pool_size) -{ - return 0; -} - -static inline void bioset_integrity_free (struct bio_set *bs) -{ - return; -} - -static inline bool bio_integrity_prep(struct bio *bio) -{ - return true; -} - -static inline int bio_integrity_clone(struct bio *bio, struct bio *bio_src, - gfp_t gfp_mask) -{ - return 0; -} - -static inline void bio_integrity_advance(struct bio *bio, - unsigned int bytes_done) -{ - return; -} - -static inline void bio_integrity_trim(struct bio *bio) -{ - return; -} - -static inline void bio_integrity_init(void) -{ - return; -} - -static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag) -{ - return false; -} - -static inline void *bio_integrity_alloc(struct bio * bio, gfp_t gfp, - unsigned int nr) -{ - return ERR_PTR(-EINVAL); -} - -static inline int bio_integrity_add_page(struct bio *bio, struct page *page, - unsigned int len, unsigned int offset) -{ - return 0; -} - -static inline int bio_integrity_map_user(struct bio *bio, void __user *ubuf, - ssize_t len, u32 seed) -{ - return -EINVAL; -} -static inline void bio_integrity_unmap_free_user(struct bio *bio) -{ -} - -#endif /* CONFIG_BLK_DEV_INTEGRITY */ - /* * Mark a bio as polled. Note that for async polled IO, the caller must * expect -EWOULDBLOCK if we cannot allocate a request (or other resources). diff --git a/include/linux/blk-integrity.h b/include/linux/blk-integrity.h index 804f856ed3e571..de98049b7ded91 100644 --- a/include/linux/blk-integrity.h +++ b/include/linux/blk-integrity.h @@ -3,6 +3,7 @@ #define _LINUX_BLK_INTEGRITY_H #include +#include struct request; From patchwork Tue Jul 2 15:10:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13719905 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E60D81BB6A0 for ; Tue, 2 Jul 2024 15:10:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933057; cv=none; b=pVKCw/SSCKn0/O7CFZjmhlZgFCO5k2h/B+eHP7YWkhMHvPKdMUsCfhphejkcbwD/ND0CqMPZWKtg+lW1EU5Fa61s3dHsSsQiVCp0mvxQ9H9Ry36n/cRRJGy0SuBQEJRBnyif5jnvWUAGOrV+0lG1V9/W+oGK9iULAF6Jx25ATLU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933057; c=relaxed/simple; bh=M5GQ7BCJq4PXak1TLgKx3Zae1ajJkHsYeoobxN3xZNM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gIpJnhVTLwdGi/HgLxvjX1j5pDiytYnVxmcGSyp8i1OT1CoxDWevGFHgOFqzWKVEPLMnitwZa7PnRfquq119raw1w7wEEywnecIyj8jFXklnHTe0UQdDekcmD0fsf973duK6cfp+R/Ixa3TR618Gb3QLas+RX5sFDvmnwhrUnho= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=mxd6i/eH; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mxd6i/eH" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=1XuL9DY/pU8TnRQZjZdiKw2eF70rYsniNjTqrte+D7k=; b=mxd6i/eHXxcGoB1rEUqF/jbBuQ 9tYfdvQquB+sql5dOmLfda8nw23AvEG2sb8RkLqPGDsHlnW4cTtGJaGGzFD4w9MaDbUlxGb3RBb8h 4sCZHFfwrIwq4wKOM/ACg02qpLBwyStq2ftQqIBINmDrRvxFUqz6H+8XuuBkMKmk8eW4WTu4L8ZZE 3dnyhsPuThPsNHJ6ZWW2nXVxHFjqXdIkj4+yoAk8PuF2PCtGnQ50JYQ1N967o/ZH1dY+XDdeYtad/ G1GSoV0wyiGFjRYhQRekIuIlo7fAZCgY0uVTAH/quaVLMOmLv31SANtVRtBUVBOgqCAz3+NlFRuKO RYOLo2iw==; Received: from 2a02-8389-2341-5b80-4c69-cf21-4832-bbca.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:4c69:cf21:4832:bbca] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux)) id 1sOf9j-000000079UD-08tU; Tue, 02 Jul 2024 15:10:55 +0000 From: Christoph Hellwig To: Jens Axboe Cc: "Martin K . Petersen" , Anuj Gupta , Kanchan Joshi , linux-block@vger.kernel.org Subject: [PATCH 2/6] block: also return bio_integrity_payload * from stubs Date: Tue, 2 Jul 2024 17:10:20 +0200 Message-ID: <20240702151047.1746127-3-hch@lst.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240702151047.1746127-1-hch@lst.de> References: <20240702151047.1746127-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html struct bio_integrity_payload is defined unconditionally. No need to return void * from bio_integrity() and bio_integrity_alloc(). Signed-off-by: Christoph Hellwig Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta Reviewed-by: Johannes Thumshirn --- include/linux/bio-integrity.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/bio-integrity.h b/include/linux/bio-integrity.h index 70ef19a0dc7e8b..cac24dac06fff0 100644 --- a/include/linux/bio-integrity.h +++ b/include/linux/bio-integrity.h @@ -85,7 +85,7 @@ void bio_integrity_init(void); #else /* CONFIG_BLK_DEV_INTEGRITY */ -static inline void *bio_integrity(struct bio *bio) +static inline struct bio_integrity_payload *bio_integrity(struct bio *bio) { return NULL; } @@ -138,8 +138,8 @@ static inline bool bio_integrity_flagged(struct bio *bio, enum bip_flags flag) return false; } -static inline void *bio_integrity_alloc(struct bio *bio, gfp_t gfp, - unsigned int nr) +static inline struct bio_integrity_payload * +bio_integrity_alloc(struct bio *bio, gfp_t gfp, unsigned int nr) { return ERR_PTR(-EINVAL); } From patchwork Tue Jul 2 15:10:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13719906 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DA471BB6A0 for ; Tue, 2 Jul 2024 15:10:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933059; cv=none; b=h1yytyjApQG/xTSn7EWyfhaUhSrBRlBq1OGrEJwTPXh8uE5B18XdN+QYjrT1OQBtpVohpJba6meLPD7WRo0XnXWJG6CKd4DmPg/0Gre5h9gQfI5hHdrOZ30RTkKnkJ3KZ5jzRW58M1bwTagPr5C/JJHd9QosuTMHvKV+QHgEFmU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933059; c=relaxed/simple; bh=nvTkX70sZacyzNpQ7y51WYTL/FNQJMpq5ifElsunDkU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qBJy8lyQWISxGqzEmnvK+DesQrF16/RhSt9EGMyBKpCEk23kvFIbJf92AIP9dsRWFZVL07nkSdhZ7XYxkrD87s783XDk9EiYf2mUDnGt6duQmSGYhjqn83b7OaVDIKrxjZkvMPrsAdsOzRSXegD4nP9KPhLMJE87XKt1VYP9trk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=ptokIS4T; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ptokIS4T" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=hIGUP22CMXhQJHh5zIIR2ZxDvxkUYfILjr/RfURYB+Q=; b=ptokIS4Tdvv/G+4BVtlKjM74Re k3E9UsWNfgF2fUYSl5K3A6uQ9GAvWxfHrIqdpeL7CQw1JcTTTmv7Fi3vwTOdW+7Za/5KCFhMO20un ASROFhFFtLzlaE9Cu9UreyelINQuedWwAiLrc1Z4C8bNFmx9kaE3+kAI4A5xyCiYLGBecxvXc122m zGPSV4/ZN0qYSl5n08wlzoUl+5/6cG16Z5S+ZX98o/Viy7QNG6IyV0fEntllzCt92L/Q8fMZmXUli r8KdRg8TPa8tvOdtZ4KLpqAqP6FsHUtHVxJScDoPmpaM8Gic3RQzqXZ7FFe0v8bLA7MHuDlYeQLJu c730grtw==; Received: from 2a02-8389-2341-5b80-4c69-cf21-4832-bbca.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:4c69:cf21:4832:bbca] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux)) id 1sOf9l-000000079UR-1Xkq; Tue, 02 Jul 2024 15:10:57 +0000 From: Christoph Hellwig To: Jens Axboe Cc: "Martin K . Petersen" , Anuj Gupta , Kanchan Joshi , linux-block@vger.kernel.org Subject: [PATCH 3/6] block: don't call bio_uninit from bio_endio Date: Tue, 2 Jul 2024 17:10:21 +0200 Message-ID: <20240702151047.1746127-4-hch@lst.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240702151047.1746127-1-hch@lst.de> References: <20240702151047.1746127-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Commit b222dd2fdd53 ("block: call bio_uninit in bio_endio") added a call to bio_uninit in bio_endio to work around callers that use bio_init but fail to call bio_uninit after they are done to release the resources. While this is an abuse of the bio_init API we still have quite a few of those left. But this early uninit causes a problem for integrity data, as at least some users need the bio_integrity_payload. Right now the only one is the NVMe passthrough which archives this by adding a special case to skip the freeing if the BIP_INTEGRITY_USER flag is set. Sort this out by only putting bi_blkg in bio_endio as that is the cause of the actual leaks - the few users of the crypto context and integrity data all properly call bio_uninit, usually through bio_put for dynamically allocated bios. Signed-off-by: Christoph Hellwig --- block/bio.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/block/bio.c b/block/bio.c index 4ca3f31ce45fb5..68ce75fd9b7c89 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1630,8 +1630,18 @@ void bio_endio(struct bio *bio) goto again; } - /* release cgroup info */ - bio_uninit(bio); +#ifdef CONFIG_BLK_CGROUP + /* + * Release cgroup info. We shouldn't have to do this here, but quite + * a few callers of bio_init fail to call bio_uninit, so we cover up + * for that here at least for now. + */ + if (bio->bi_blkg) { + blkg_put(bio->bi_blkg); + bio->bi_blkg = NULL; + } +#endif + if (bio->bi_end_io) bio->bi_end_io(bio); } From patchwork Tue Jul 2 15:10:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13719907 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 719AA1BB6A0 for ; Tue, 2 Jul 2024 15:11:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933061; cv=none; b=tVaRNFeutrSV6SVlToZn9TCvbjncPqf6VBy0IhvJASj8SomzFuGEjhbUlOKidlUUuE2SAlmV2gjSzdhebkfAak7msvMviaqE/Xjv5PgQ3Cjt0yTdmVHhvYcpX0Tn1S4ooLzMO424RrELsdu994O04tDcoeozekKbrbFCOhECDh0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933061; c=relaxed/simple; bh=pQnMNxzTz6sx53YbdMFbqXEB1iZOq/9Qr+ggkfpnZi4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kwl42wniIMhDh6LoVmjx+VKUsajPC8jr+KIaloW9JRV2+jmbT28NGi4pb6sTy6Tx28K0axQmCf/0mR6y2YQ9PgkSg3rMIL9xI3jJIbmeVEAn0vAZLnphwoeGgQyqHAHw0umZ1CUgjXE65bNjY5WzXhL4XjRzjVLtXymMWS1n6EU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=IUXMtlLS; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="IUXMtlLS" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=njpzKyRI9Qvkux/MnNAf6eEOBeyv2fOl7BBE+/NdCFs=; b=IUXMtlLS57/WjX8KZUcfC6s5GB xx3G/Ro8UshlKbUpDfgcPj9GuO4aPu81Eok7YL1171xkuO1+6DTVXPyVyP020zNNzJRXPbwydKY4x x2m4pNQP7GiIyrMFXhos67W6xXuCiYBU+BgMjoXZJxIEYRrE5I+mh8ETnIJ4SOzsL6mSG7JotjKFR yzYfkLssVyOBWy7j4s0FLzanf9fjSpwbWReU4TRKxf4TzouYN2VaqxXbXnIiQiLhWEZZ4bVlOis/g d9CJdqNxN+VU5ICf5wg5Eps52GdrnG9tywRbqTKno0sLDSy6hI7uc2zbQdLVoioa5au2OCzyqU+33 u4iKRdOw==; Received: from 2a02-8389-2341-5b80-4c69-cf21-4832-bbca.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:4c69:cf21:4832:bbca] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux)) id 1sOf9n-000000079Uk-3Lqn; Tue, 02 Jul 2024 15:11:00 +0000 From: Christoph Hellwig To: Jens Axboe Cc: "Martin K . Petersen" , Anuj Gupta , Kanchan Joshi , linux-block@vger.kernel.org Subject: [PATCH 4/6] block: call bio_integrity_unmap_free_user from blk_rq_unmap_user Date: Tue, 2 Jul 2024 17:10:22 +0200 Message-ID: <20240702151047.1746127-5-hch@lst.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240702151047.1746127-1-hch@lst.de> References: <20240702151047.1746127-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html blk_rq_unmap_user always unmaps user space pass-through request. If such a request has integrity data attached it must come from a user mapping as well. Call bio_integrity_unmap_free_user from blk_rq_unmap_user and remove the nvme_unmap_bio wrapper in the nvme driver. Signed-off-by: Christoph Hellwig Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta Reviewed-by: Johannes Thumshirn --- block/bio-integrity.c | 1 - block/blk-map.c | 3 +++ drivers/nvme/host/ioctl.c | 15 ++++----------- 3 files changed, 7 insertions(+), 12 deletions(-) diff --git a/block/bio-integrity.c b/block/bio-integrity.c index c4aed1dfa497a3..c8757d47e0ef62 100644 --- a/block/bio-integrity.c +++ b/block/bio-integrity.c @@ -174,7 +174,6 @@ void bio_integrity_unmap_free_user(struct bio *bio) bio->bi_integrity = NULL; bio->bi_opf &= ~REQ_INTEGRITY; } -EXPORT_SYMBOL(bio_integrity_unmap_free_user); /** * bio_integrity_add_page - Attach integrity metadata diff --git a/block/blk-map.c b/block/blk-map.c index bce144091128f6..df5f82d114720f 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -757,6 +757,9 @@ int blk_rq_unmap_user(struct bio *bio) bio_release_pages(bio, bio_data_dir(bio) == READ); } + if (bio_integrity(bio)) + bio_integrity_unmap_free_user(bio); + next_bio = bio; bio = bio->bi_next; blk_mq_map_bio_put(next_bio); diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index fb46f55f8b2894..f1d58e70933f54 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -112,13 +112,6 @@ static struct request *nvme_alloc_user_request(struct request_queue *q, return req; } -static void nvme_unmap_bio(struct bio *bio) -{ - if (bio_integrity(bio)) - bio_integrity_unmap_free_user(bio); - blk_rq_unmap_user(bio); -} - static int nvme_map_user_request(struct request *req, u64 ubuffer, unsigned bufflen, void __user *meta_buffer, unsigned meta_len, u32 meta_seed, struct io_uring_cmd *ioucmd, unsigned int flags) @@ -165,7 +158,7 @@ static int nvme_map_user_request(struct request *req, u64 ubuffer, out_unmap: if (bio) - nvme_unmap_bio(bio); + blk_rq_unmap_user(bio); out: blk_mq_free_request(req); return ret; @@ -203,7 +196,7 @@ static int nvme_submit_user_cmd(struct request_queue *q, if (result) *result = le64_to_cpu(nvme_req(req)->result.u64); if (bio) - nvme_unmap_bio(bio); + blk_rq_unmap_user(bio); blk_mq_free_request(req); if (effects) @@ -414,7 +407,7 @@ static void nvme_uring_task_cb(struct io_uring_cmd *ioucmd, struct nvme_uring_cmd_pdu *pdu = nvme_uring_cmd_pdu(ioucmd); if (pdu->bio) - nvme_unmap_bio(pdu->bio); + blk_rq_unmap_user(pdu->bio); io_uring_cmd_done(ioucmd, pdu->status, pdu->result, issue_flags); } @@ -440,7 +433,7 @@ static enum rq_end_io_ret nvme_uring_cmd_end_io(struct request *req, */ if (blk_rq_is_poll(req)) { if (pdu->bio) - nvme_unmap_bio(pdu->bio); + blk_rq_unmap_user(pdu->bio); io_uring_cmd_iopoll_done(ioucmd, pdu->result, pdu->status); } else { io_uring_cmd_do_in_task_lazy(ioucmd, nvme_uring_task_cb); From patchwork Tue Jul 2 15:10:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13719908 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2AF21BB695 for ; Tue, 2 Jul 2024 15:11:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933064; cv=none; b=EMKRY+dRhe/Yhdgx+HRGbIDbgce7isq6BRyg8AF9XwkmDBXhURVdbk71/GZPJ+4cLbKny6awfRZ58iv2OR8a9XXAQ2X9t4Pw7IqdLRxcAwtcEHv0YDGlTjel1l3aRT1NdL9csmXJEWNDes2HB5ul+joBrhdzPlcivzhlDa6X3dk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933064; c=relaxed/simple; bh=bJZFcwmZb/ow39OQ1nA3teqy0nDQ5YgxWmIBix3Nc8M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EAwdxCr1ULrf134Zl3v5ATS+dv4YCqUpv52MRYt/Luj6onekBPv99L0ZeHce3RcjbYy2GinitEWh3lR54IqB0Sp1vo6JJgcZQdUThHQCpPmOin+LhSunNokgDspboL4g7og+PDmHqXWG7lLatN6aDctM86KFfn75t39Zd8XQtmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=KEbrNGIG; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="KEbrNGIG" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=PV92L0B9im4Kj3OftYV0nbZwAZGJpIrUAX90xBRCuGQ=; b=KEbrNGIG2vVzZt0LTXDuLjUBvc b//tIWKudYIVgR3O3/8O7AERnY2IWcEvEX8Hsffqgm8JIMMw1+rzOhkzLr+Z77aRbFnv9agMZ1edI WeRQuuZRuaFnGHb28V/0q2kw3CqObOwirpFeTxsEoSX7LfgvTIR9K0WURHZcdCUsHL9PMAi2lARbL 1pB76J7KFs7dLJonhXwehPGpO+aNKLRQILwg89V4LRserdNw3eDlihn7HpZS9a1voI9WT/O2lOf8S u2OKJ+AnGCLfgdtDnvsjTPxtlxvbXRWelhJwmTJ95/bK2VvIT+w6jqfTSviTUkhAfa2lVm5VwBQ/W Vgx1dFdA==; Received: from 2a02-8389-2341-5b80-4c69-cf21-4832-bbca.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:4c69:cf21:4832:bbca] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux)) id 1sOf9q-000000079Ux-1Bhg; Tue, 02 Jul 2024 15:11:02 +0000 From: Christoph Hellwig To: Jens Axboe Cc: "Martin K . Petersen" , Anuj Gupta , Kanchan Joshi , linux-block@vger.kernel.org Subject: [PATCH 5/6] block: don't free submitter owned integrity payload on I/O completion Date: Tue, 2 Jul 2024 17:10:23 +0200 Message-ID: <20240702151047.1746127-6-hch@lst.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240702151047.1746127-1-hch@lst.de> References: <20240702151047.1746127-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Currently __bio_integrity_endio frees the integrity payload unless it is explicitly marked as user-mapped. This means in-kernel callers that allocate their own integrity payload never get to see it on I/O completion. The current two users don't need it as they just pre-mapped PI tuples received over the network, but this limits uses of integrity data lot. Change bio_integrity_endio to call __bio_integrity_endio for block layer generated integrity data only, and leave freeing of submitter allocated integrity data to bio_uninit which also gets called from the final bio_put. This requires that unmapping user mapped or copied integrity data is now always done by the caller, and the special BIP_INTEGRITY_USER flag can go away. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Kanchan Joshi --- block/bio-integrity.c | 57 ++++++++++++++--------------------- block/blk.h | 13 ++++++-- include/linux/bio-integrity.h | 3 +- 3 files changed, 34 insertions(+), 39 deletions(-) diff --git a/block/bio-integrity.c b/block/bio-integrity.c index c8757d47e0ef62..4aa836d603fb23 100644 --- a/block/bio-integrity.c +++ b/block/bio-integrity.c @@ -22,9 +22,17 @@ void blk_flush_integrity(void) flush_workqueue(kintegrityd_wq); } -static void __bio_integrity_free(struct bio_set *bs, - struct bio_integrity_payload *bip) +/** + * bio_integrity_free - Free bio integrity payload + * @bio: bio containing bip to be freed + * + * Description: Free the integrity portion of a bio. + */ +void bio_integrity_free(struct bio *bio) { + struct bio_integrity_payload *bip = bio_integrity(bio); + struct bio_set *bs = bio->bi_pool; + if (bs && mempool_initialized(&bs->bio_integrity_pool)) { if (bip->bip_vec) bvec_free(&bs->bvec_integrity_pool, bip->bip_vec, @@ -33,6 +41,8 @@ static void __bio_integrity_free(struct bio_set *bs, } else { kfree(bip); } + bio->bi_integrity = NULL; + bio->bi_opf &= ~REQ_INTEGRITY; } /** @@ -86,7 +96,10 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, return bip; err: - __bio_integrity_free(bs, bip); + if (bs && mempool_initialized(&bs->bio_integrity_pool)) + mempool_free(bip, &bs->bio_integrity_pool); + else + kfree(bip); return ERR_PTR(-ENOMEM); } EXPORT_SYMBOL(bio_integrity_alloc); @@ -132,28 +145,6 @@ static void bio_integrity_unmap_user(struct bio_integrity_payload *bip) bio_integrity_unpin_bvec(bip->bip_vec, bip->bip_max_vcnt, dirty); } -/** - * bio_integrity_free - Free bio integrity payload - * @bio: bio containing bip to be freed - * - * Description: Used to free the integrity portion of a bio. Usually - * called from bio_free(). - */ -void bio_integrity_free(struct bio *bio) -{ - struct bio_integrity_payload *bip = bio_integrity(bio); - struct bio_set *bs = bio->bi_pool; - - if (bip->bip_flags & BIP_INTEGRITY_USER) - return; - if (bip->bip_flags & BIP_BLOCK_INTEGRITY) - kfree(bvec_virt(bip->bip_vec)); - - __bio_integrity_free(bs, bip); - bio->bi_integrity = NULL; - bio->bi_opf &= ~REQ_INTEGRITY; -} - /** * bio_integrity_unmap_free_user - Unmap and free bio user integrity payload * @bio: bio containing bip to be unmapped and freed @@ -165,14 +156,9 @@ void bio_integrity_free(struct bio *bio) void bio_integrity_unmap_free_user(struct bio *bio) { struct bio_integrity_payload *bip = bio_integrity(bio); - struct bio_set *bs = bio->bi_pool; - if (WARN_ON_ONCE(!(bip->bip_flags & BIP_INTEGRITY_USER))) - return; bio_integrity_unmap_user(bip); - __bio_integrity_free(bs, bip); - bio->bi_integrity = NULL; - bio->bi_opf &= ~REQ_INTEGRITY; + bio_integrity_free(bio); } /** @@ -273,7 +259,7 @@ static int bio_integrity_copy_user(struct bio *bio, struct bio_vec *bvec, goto free_bip; } - bip->bip_flags |= BIP_INTEGRITY_USER | BIP_COPY_USER; + bip->bip_flags |= BIP_COPY_USER; bip->bip_iter.bi_sector = seed; bip->bip_vcnt = nr_vecs; return 0; @@ -294,7 +280,6 @@ static int bio_integrity_init_user(struct bio *bio, struct bio_vec *bvec, return PTR_ERR(bip); memcpy(bip->bip_vec, bvec, nr_vecs * sizeof(*bvec)); - bip->bip_flags |= BIP_INTEGRITY_USER; bip->bip_iter.bi_sector = seed; bip->bip_iter.bi_size = len; bip->bip_vcnt = nr_vecs; @@ -502,6 +487,8 @@ static void bio_integrity_verify_fn(struct work_struct *work) struct bio *bio = bip->bip_bio; blk_integrity_verify(bio); + + kfree(bvec_virt(bip->bip_vec)); bio_integrity_free(bio); bio_endio(bio); } @@ -522,13 +509,13 @@ bool __bio_integrity_endio(struct bio *bio) struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); struct bio_integrity_payload *bip = bio_integrity(bio); - if (bio_op(bio) == REQ_OP_READ && !bio->bi_status && - (bip->bip_flags & BIP_BLOCK_INTEGRITY) && bi->csum_type) { + if (bio_op(bio) == REQ_OP_READ && !bio->bi_status && bi->csum_type) { INIT_WORK(&bip->bip_work, bio_integrity_verify_fn); queue_work(kintegrityd_wq, &bip->bip_work); return false; } + kfree(bvec_virt(bip->bip_vec)); bio_integrity_free(bio); return true; } diff --git a/block/blk.h b/block/blk.h index 401e604f35d2cf..2233dc8d36b82a 100644 --- a/block/blk.h +++ b/block/blk.h @@ -202,11 +202,20 @@ static inline unsigned int blk_queue_get_max_sectors(struct request *rq) #ifdef CONFIG_BLK_DEV_INTEGRITY void blk_flush_integrity(void); -bool __bio_integrity_endio(struct bio *); void bio_integrity_free(struct bio *bio); + +/* + * Integrity payloads can either be owned by the submitter, in which case + * bio_uninit will free them, or owned and generated by the block layer, + * in which case we'll verify them here (for reads) and free them before + * the bio is handed back to the submitted. + */ +bool __bio_integrity_endio(struct bio *bio); static inline bool bio_integrity_endio(struct bio *bio) { - if (bio_integrity(bio)) + struct bio_integrity_payload *bip = bio_integrity(bio); + + if (bip && (bip->bip_flags & BIP_BLOCK_INTEGRITY)) return __bio_integrity_endio(bio); return true; } diff --git a/include/linux/bio-integrity.h b/include/linux/bio-integrity.h index cac24dac06fff0..3823d9be0d0790 100644 --- a/include/linux/bio-integrity.h +++ b/include/linux/bio-integrity.h @@ -10,8 +10,7 @@ enum bip_flags { BIP_CTRL_NOCHECK = 1 << 2, /* disable HBA integrity checking */ BIP_DISK_NOCHECK = 1 << 3, /* disable disk integrity checking */ BIP_IP_CHECKSUM = 1 << 4, /* IP checksum */ - BIP_INTEGRITY_USER = 1 << 5, /* Integrity payload is user address */ - BIP_COPY_USER = 1 << 6, /* Kernel bounce buffer in use */ + BIP_COPY_USER = 1 << 5, /* Kernel bounce buffer in use */ }; struct bio_integrity_payload { From patchwork Tue Jul 2 15:10:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13719909 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FE231BBBC8 for ; Tue, 2 Jul 2024 15:11:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933067; cv=none; b=ulyrxAL8YGhqYqtZWw/+UXvjllXxw/d5/C3YghkA7aVruajfgP97hiltVRZQh/3pntUjqhaJHl9xev6OOB7ayUgvcU1uqUEX6cFLttMMf/Y9hPZe+znIzkv9Py7uD6cUvHqTZb+3Jtd7/DWE5FX7yRjzlTCAAtOcI2Fxfw8l0C0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719933067; c=relaxed/simple; bh=6qzbI9W0dIjuMs0tR6+1cR/DQHZG+GO9fhr825YOT+U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j1RciIopJGXiE1ZOCUOEthsdtDPxmJTSPIJjqvg0w/mfP6bQ/4xozx7eD7iEJ4PcsxsMNVEGSsY0LmsRIs5eWCT8dB/DLhwK1Wro7YHrI2/FdR3FhEXSmlGpkP5yUYi9R55D7UFNCvsVQrHD0LwkabvlFqyKv7fbs2OzsGSs3C8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de; spf=none smtp.mailfrom=bombadil.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=2y3NEA2K; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=bombadil.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="2y3NEA2K" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=qnVQ0Zsyk5yLVE9nCi8pUTdqtQJmDy7FRD4pSkUnbOQ=; b=2y3NEA2KrF66gmYcsX8VktbPFu 57x5XDmmfewaL86pRo/QgA1ls8ctiMAXpumLkl/OUK5cKjoq4dVcMJTWySqBdtuFFbTOWgK4oLSCR sGFHssxUixGf0v6A408eBsgN3gy8VrviENztG9K2FPJMCjsviImL5rCFr02rgckJ/cegcpaBPH1I5 hPvKSqkysGxrwORhXYb/AQk1oDZiMsERYPY1QGJm9VFO47vm6qSL1IiWJUjMfNsMUOCt0d+nhf9Tp VrHhacYf+PDANxMzkr1HTzx3KJpkF/vB/9Nl/KXf516BdyuZEbCoEiLOY+h2EyY0LOWAi+jk4zQM2 N0f0RsKw==; Received: from 2a02-8389-2341-5b80-4c69-cf21-4832-bbca.cable.dynamic.v6.surfer.at ([2a02:8389:2341:5b80:4c69:cf21:4832:bbca] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.97.1 #2 (Red Hat Linux)) id 1sOf9s-000000079VF-3378; Tue, 02 Jul 2024 15:11:05 +0000 From: Christoph Hellwig To: Jens Axboe Cc: "Martin K . Petersen" , Anuj Gupta , Kanchan Joshi , linux-block@vger.kernel.org Subject: [PATCH 6/6] block: don't free the integrity payload in bio_integrity_unmap_free_user Date: Tue, 2 Jul 2024 17:10:24 +0200 Message-ID: <20240702151047.1746127-7-hch@lst.de> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240702151047.1746127-1-hch@lst.de> References: <20240702151047.1746127-1-hch@lst.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Now that the integrity payload is always freed in bio_uninit, don't bother freeing it a little earlier in bio_integrity_unmap_free_user. With that the separate bio_integrity_unmap_free_user can go away by just passing the bio to bio_integrity_unmap_user. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Kanchan Joshi --- block/bio-integrity.c | 31 +++++++++++-------------------- block/blk-map.c | 2 +- include/linux/bio-integrity.h | 4 ++-- 3 files changed, 14 insertions(+), 23 deletions(-) diff --git a/block/bio-integrity.c b/block/bio-integrity.c index 4aa836d603fb23..4b5c604585561e 100644 --- a/block/bio-integrity.c +++ b/block/bio-integrity.c @@ -131,34 +131,25 @@ static void bio_integrity_uncopy_user(struct bio_integrity_payload *bip) bio_integrity_unpin_bvec(copy, nr_vecs, true); } -static void bio_integrity_unmap_user(struct bio_integrity_payload *bip) +/** + * bio_integrity_unmap_user - Unmap user integrity payload + * @bio: bio containing bip to be unmapped + * + * Unmap the user mapped integrity portion of a bio. + */ +void bio_integrity_unmap_user(struct bio *bio) { - bool dirty = bio_data_dir(bip->bip_bio) == READ; + struct bio_integrity_payload *bip = bio_integrity(bio); if (bip->bip_flags & BIP_COPY_USER) { - if (dirty) + if (bio_data_dir(bio) == READ) bio_integrity_uncopy_user(bip); kfree(bvec_virt(bip->bip_vec)); return; } - bio_integrity_unpin_bvec(bip->bip_vec, bip->bip_max_vcnt, dirty); -} - -/** - * bio_integrity_unmap_free_user - Unmap and free bio user integrity payload - * @bio: bio containing bip to be unmapped and freed - * - * Description: Used to unmap and free the user mapped integrity portion of a - * bio. Submitter attaching the user integrity buffer is responsible for - * unmapping and freeing it during completion. - */ -void bio_integrity_unmap_free_user(struct bio *bio) -{ - struct bio_integrity_payload *bip = bio_integrity(bio); - - bio_integrity_unmap_user(bip); - bio_integrity_free(bio); + bio_integrity_unpin_bvec(bip->bip_vec, bip->bip_max_vcnt, + bio_data_dir(bio) == READ); } /** diff --git a/block/blk-map.c b/block/blk-map.c index df5f82d114720f..0e1167b239342f 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -758,7 +758,7 @@ int blk_rq_unmap_user(struct bio *bio) } if (bio_integrity(bio)) - bio_integrity_unmap_free_user(bio); + bio_integrity_unmap_user(bio); next_bio = bio; bio = bio->bi_next; diff --git a/include/linux/bio-integrity.h b/include/linux/bio-integrity.h index 3823d9be0d0790..dd831c269e9948 100644 --- a/include/linux/bio-integrity.h +++ b/include/linux/bio-integrity.h @@ -73,7 +73,7 @@ struct bio_integrity_payload *bio_integrity_alloc(struct bio *bio, gfp_t gfp, int bio_integrity_add_page(struct bio *bio, struct page *page, unsigned int len, unsigned int offset); int bio_integrity_map_user(struct bio *bio, void __user *ubuf, ssize_t len, u32 seed); -void bio_integrity_unmap_free_user(struct bio *bio); +void bio_integrity_unmap_user(struct bio *bio); bool bio_integrity_prep(struct bio *bio); void bio_integrity_advance(struct bio *bio, unsigned int bytes_done); void bio_integrity_trim(struct bio *bio); @@ -104,7 +104,7 @@ static inline int bio_integrity_map_user(struct bio *bio, void __user *ubuf, return -EINVAL; } -static inline void bio_integrity_unmap_free_user(struct bio *bio) +static inline void bio_integrity_unmap_user(struct bio *bio) { }