From patchwork Thu May 5 20:11:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840164 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64BD9C4167D for ; Thu, 5 May 2022 20:11:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385712AbiEEUPF (ORCPT ); Thu, 5 May 2022 16:15:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241971AbiEEUPD (ORCPT ); Thu, 5 May 2022 16:15:03 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A4A55F26F; Thu, 5 May 2022 13:11:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=fBVrbe6Bwv56qkohYH6YihFUc4A7+M1nHINpHwxi6xU=; b=0k+473pDQ4z4f6kipwABLtNjYh AL+GUDVi7MDmHQlUKoBIhZi8TIt4fC+TOMejQZRlh84TkzL9uQZKjEnp/MFW6gq6bPd6qFJoM6iLX iLaUJHCYZlWMIEUX8FD/SB2Fi4AAMClHsLrOpqTtMro+3Yqf+KEIAEgOcOsvGJ8NGj3ZMn1UbvD6N G94IPLKHFCFB7fuw+H3lQR1dR/WWOpVLIzOE+UrOdV6lF/fMaKSZxehmYlYIAShuHrTacdI0/3Ak2 76liOR/Mb5npROsejlA4vPob1/ols2kQ8tqKDwFVcpEN2ogEr36jVad3xKwZPbaMczmP17d6WyMjb YgQpO6KA==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhok-0006hv-9H; Thu, 05 May 2022 20:11:18 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 1/7] btrfs: add a btrfs_dio_rw wrapper Date: Thu, 5 May 2022 15:11:09 -0500 Message-Id: <20220505201115.937837-2-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Add a wrapper around iomap_dio_rw that keeps the direct I/O internals isolated in inode.c. Signed-off-by: Christoph Hellwig --- fs/btrfs/ctree.h | 5 +++-- fs/btrfs/file.c | 6 ++---- fs/btrfs/inode.c | 11 +++++++++-- 3 files changed, 14 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 6e939bf01dcc3..aa6e71fdc72b9 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3358,9 +3358,10 @@ ssize_t btrfs_encoded_read(struct kiocb *iocb, struct iov_iter *iter, ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, const struct btrfs_ioctl_encoded_io_args *encoded); +ssize_t btrfs_dio_rw(struct kiocb *iocb, struct iov_iter *iter, + size_t done_before); + extern const struct dentry_operations btrfs_dentry_operations; -extern const struct iomap_ops btrfs_dio_iomap_ops; -extern const struct iomap_dio_ops btrfs_dio_ops; /* Inode locking type flags, by default the exclusive lock is taken */ #define BTRFS_ILOCK_SHARED (1U << 0) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index b64fb93d90469..46c2baa8fdf54 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1929,8 +1929,7 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) */ again: from->nofault = true; - err = iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - IOMAP_DIO_PARTIAL, written); + err = btrfs_dio_rw(iocb, from, written); from->nofault = false; /* No increment (+=) because iomap returns a cumulative value. */ @@ -3693,8 +3692,7 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to) */ pagefault_disable(); to->nofault = true; - ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - IOMAP_DIO_PARTIAL, read); + ret = btrfs_dio_rw(iocb, to, read); to->nofault = false; pagefault_enable(); diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b42d6e7e4049f..cdf96a2472821 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -8155,15 +8155,22 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, btrfs_dio_private_put(dip); } -const struct iomap_ops btrfs_dio_iomap_ops = { +static const struct iomap_ops btrfs_dio_iomap_ops = { .iomap_begin = btrfs_dio_iomap_begin, .iomap_end = btrfs_dio_iomap_end, }; -const struct iomap_dio_ops btrfs_dio_ops = { +static const struct iomap_dio_ops btrfs_dio_ops = { .submit_io = btrfs_submit_direct, }; +ssize_t btrfs_dio_rw(struct kiocb *iocb, struct iov_iter *iter, + size_t done_before) +{ + return iomap_dio_rw(iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops, + IOMAP_DIO_PARTIAL, done_before); +} + static int btrfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, u64 start, u64 len) { From patchwork Thu May 5 20:11:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840162 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B73D7C433EF for ; Thu, 5 May 2022 20:11:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385727AbiEEUPF (ORCPT ); Thu, 5 May 2022 16:15:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385691AbiEEUPE (ORCPT ); Thu, 5 May 2022 16:15:04 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04AD235DC3; Thu, 5 May 2022 13:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=8deOESBnsyOQ7JadvF2zZ/Cdu2JHPhu0r/iumY6xPD0=; b=ix4I59EZ7p0dx7aHeA3+FKPbIk SxlhMzoCGSoRjXAYrTxcmDy0FBFT5QBkfI5Q5EKFph2wuKNcPoRsQjyYJwUDVaEl4D2QddCgcjodI zZHBWuDBjoyXS1xbil+qUsCK20oHn0XuDsYnV+HC24HG5iBp7PYZiA/NoaWFtMQc9OYvjwGvB/HrQ +cBqPEI9Nt+bHrhZg9kQYnaWDw0i5U3XpCKnGHXlxHjqbobGRZvha9mjw2MHRvQ1NCun0d3qmdmj3 azk5lROgO6XwCC6GWfLNoFzYqVPJ33VAsCG+v/CFdcfCJ9halsro0pKCqmhA1t6DWpFPKqmuhs1kC +2tVlyOA==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhol-0006iJ-5E; Thu, 05 May 2022 20:11:19 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 2/7] iomap: allow the file system to provide a bio_set for direct I/O Date: Thu, 5 May 2022 15:11:10 -0500 Message-Id: <20220505201115.937837-3-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Allow the file system to provide a specific bio_set for allocating direct I/O bios. This will allow file systems that use the ->submit_io hook to stash away additional information for file system use. To make use of this additional space for information in the completion path, the file system needs to override the ->bi_end_io callback and then call back into iomap, so export iomap_dio_bio_end_io for that. Signed-off-by: Christoph Hellwig Reviewed-by: Darrick J. Wong --- fs/iomap/direct-io.c | 18 ++++++++++++++---- include/linux/iomap.h | 11 +++++++++++ 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index b08f5dc31780d..15929690d89e3 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -51,6 +51,15 @@ struct iomap_dio { }; }; +static struct bio *iomap_dio_alloc_bio(const struct iomap_iter *iter, + struct iomap_dio *dio, unsigned short nr_vecs, unsigned int opf) +{ + if (dio->dops && dio->dops->bio_set) + return bio_alloc_bioset(iter->iomap.bdev, nr_vecs, opf, + GFP_KERNEL, dio->dops->bio_set); + return bio_alloc(iter->iomap.bdev, nr_vecs, opf, GFP_KERNEL); +} + static void iomap_dio_submit_bio(const struct iomap_iter *iter, struct iomap_dio *dio, struct bio *bio, loff_t pos) { @@ -144,7 +153,7 @@ static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret) cmpxchg(&dio->error, 0, ret); } -static void iomap_dio_bio_end_io(struct bio *bio) +void iomap_dio_bio_end_io(struct bio *bio) { struct iomap_dio *dio = bio->bi_private; bool should_dirty = (dio->flags & IOMAP_DIO_DIRTY); @@ -176,16 +185,17 @@ static void iomap_dio_bio_end_io(struct bio *bio) bio_put(bio); } } +EXPORT_SYMBOL_GPL(iomap_dio_bio_end_io); static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, loff_t pos, unsigned len) { struct inode *inode = file_inode(dio->iocb->ki_filp); struct page *page = ZERO_PAGE(0); - int flags = REQ_SYNC | REQ_IDLE; struct bio *bio; - bio = bio_alloc(iter->iomap.bdev, 1, REQ_OP_WRITE | flags, GFP_KERNEL); + bio = iomap_dio_alloc_bio(iter, dio, 1, + REQ_OP_WRITE | REQ_SYNC | REQ_IDLE); fscrypt_set_bio_crypt_ctx(bio, inode, pos >> inode->i_blkbits, GFP_KERNEL); bio->bi_iter.bi_sector = iomap_sector(&iter->iomap, pos); @@ -311,7 +321,7 @@ static loff_t iomap_dio_bio_iter(const struct iomap_iter *iter, goto out; } - bio = bio_alloc(iomap->bdev, nr_pages, bio_opf, GFP_KERNEL); + bio = iomap_dio_alloc_bio(iter, dio, nr_pages, bio_opf); fscrypt_set_bio_crypt_ctx(bio, inode, pos >> inode->i_blkbits, GFP_KERNEL); bio->bi_iter.bi_sector = iomap_sector(iomap, pos); diff --git a/include/linux/iomap.h b/include/linux/iomap.h index b76f0dd149fb4..526c9e7f2eaf8 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -320,6 +320,16 @@ struct iomap_dio_ops { unsigned flags); void (*submit_io)(const struct iomap_iter *iter, struct bio *bio, loff_t file_offset); + + /* + * Filesystems wishing to attach private information to a directio bio + * must provide a ->submit_io method that attaches the additional + * information to the bio and changes the ->bi_end_io callback to a + * custom function. This function should, at a minimum, perform any + * relevant post-processing of the bio and end with a call to + * iomap_dio_bio_end_io. + */ + struct bio_set *bio_set; }; /* @@ -349,6 +359,7 @@ struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, unsigned int dio_flags, size_t done_before); ssize_t iomap_dio_complete(struct iomap_dio *dio); +void iomap_dio_bio_end_io(struct bio *bio); #ifdef CONFIG_SWAP struct file; From patchwork Thu May 5 20:11:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53D81C433F5 for ; Thu, 5 May 2022 20:11:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385735AbiEEUPI (ORCPT ); Thu, 5 May 2022 16:15:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385705AbiEEUPE (ORCPT ); Thu, 5 May 2022 16:15:04 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1CE95F8EA; Thu, 5 May 2022 13:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Me76yjQq3T1VS7so/j4YdFeOuO3BbSiZIVa7gQnyKLs=; b=0/ikGgOjESiwheZm4E1C9xdJf3 BKKZUI5qHSZloHemx1/QE5JceoUVj5I6AHTCCH6333xYEd8dh4cpYMAdSDQ4ouq8uOSrHbRuQJX0+ RGAKf+YEt5y0M//1Ev9NUncaGTSzDvvX7MksSlEMc2YuNzGzwqTGsFutJnWDMHvf4YVexgrA12aJV 9TWgSN8LwcnXjpefkFzaC9RU92B0bDmwY4XYvUkK98u/9JS3sECOd4mQ582EbXawvZUZUcri6mRlF lTUgDaPcwMKD3dTkCNcNvCW82km1SPxL/SG6BV2hqG+uuWXTwsk3s4wpTVANDCnq6tfU+zK+97VLy YIbswo8A==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhom-0006ip-A1; Thu, 05 May 2022 20:11:20 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 3/7] iomap: add per-iomap_iter private data Date: Thu, 5 May 2022 15:11:11 -0500 Message-Id: <20220505201115.937837-4-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Allow the file system to keep state for all iterations. For now only wire it up for direct I/O as there is an immediate need for it there. Signed-off-by: Christoph Hellwig Reviewed-by: Darrick J. Wong --- fs/btrfs/inode.c | 2 +- fs/erofs/data.c | 2 +- fs/ext4/file.c | 4 ++-- fs/f2fs/file.c | 4 ++-- fs/gfs2/file.c | 4 ++-- fs/iomap/direct-io.c | 8 +++++--- fs/xfs/xfs_file.c | 6 +++--- fs/zonefs/super.c | 4 ++-- include/linux/iomap.h | 5 +++-- 9 files changed, 21 insertions(+), 18 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index cdf96a2472821..88e617e9bf5df 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -8168,7 +8168,7 @@ ssize_t btrfs_dio_rw(struct kiocb *iocb, struct iov_iter *iter, size_t done_before) { return iomap_dio_rw(iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - IOMAP_DIO_PARTIAL, done_before); + IOMAP_DIO_PARTIAL, NULL, done_before); } static int btrfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, diff --git a/fs/erofs/data.c b/fs/erofs/data.c index 780db1e5f4b72..91c11d5bb9990 100644 --- a/fs/erofs/data.c +++ b/fs/erofs/data.c @@ -385,7 +385,7 @@ static ssize_t erofs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) if (!err) return iomap_dio_rw(iocb, to, &erofs_iomap_ops, - NULL, 0, 0); + NULL, 0, NULL, 0); if (err < 0) return err; } diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 6feb07e3e1eb5..109d07629f81f 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -76,7 +76,7 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) return generic_file_read_iter(iocb, to); } - ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, 0); + ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, NULL, 0); inode_unlock_shared(inode); file_accessed(iocb->ki_filp); @@ -565,7 +565,7 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) iomap_ops = &ext4_iomap_overwrite_ops; ret = iomap_dio_rw(iocb, from, iomap_ops, &ext4_dio_write_ops, (unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0, - 0); + NULL, 0); if (ret == -ENOTBLK) ret = 0; diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index 5b89af0f27f05..04bc8709314bf 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -4309,7 +4309,7 @@ static ssize_t f2fs_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) */ inc_page_count(sbi, F2FS_DIO_READ); dio = __iomap_dio_rw(iocb, to, &f2fs_iomap_ops, - &f2fs_iomap_dio_read_ops, 0, 0); + &f2fs_iomap_dio_read_ops, 0, NULL, 0); if (IS_ERR_OR_NULL(dio)) { ret = PTR_ERR_OR_ZERO(dio); if (ret != -EIOCBQUEUED) @@ -4527,7 +4527,7 @@ static ssize_t f2fs_dio_write_iter(struct kiocb *iocb, struct iov_iter *from, if (pos + count > inode->i_size) dio_flags |= IOMAP_DIO_FORCE_WAIT; dio = __iomap_dio_rw(iocb, from, &f2fs_iomap_ops, - &f2fs_iomap_dio_write_ops, dio_flags, 0); + &f2fs_iomap_dio_write_ops, dio_flags, NULL, 0); if (IS_ERR_OR_NULL(dio)) { ret = PTR_ERR_OR_ZERO(dio); if (ret == -ENOTBLK) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 48f01323c37c1..76307a90bf81f 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -839,7 +839,7 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to, pagefault_disable(); to->nofault = true; ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, - IOMAP_DIO_PARTIAL, written); + IOMAP_DIO_PARTIAL, NULL, written); to->nofault = false; pagefault_enable(); if (ret > 0) @@ -906,7 +906,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, from->nofault = true; ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, - IOMAP_DIO_PARTIAL, read); + IOMAP_DIO_PARTIAL, NULL, read); from->nofault = false; if (ret == -ENOTBLK) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 15929690d89e3..145b2668478d0 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -484,7 +484,7 @@ static loff_t iomap_dio_iter(const struct iomap_iter *iter, struct iomap_dio * __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags, size_t done_before) + unsigned int dio_flags, void *private, size_t done_before) { struct address_space *mapping = iocb->ki_filp->f_mapping; struct inode *inode = file_inode(iocb->ki_filp); @@ -493,6 +493,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, .pos = iocb->ki_pos, .len = iov_iter_count(iter), .flags = IOMAP_DIRECT, + .private = private, }; loff_t end = iomi.pos + iomi.len - 1, ret = 0; bool wait_for_completion = @@ -684,11 +685,12 @@ EXPORT_SYMBOL_GPL(__iomap_dio_rw); ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags, size_t done_before) + unsigned int dio_flags, void *private, size_t done_before) { struct iomap_dio *dio; - dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags, done_before); + dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags, private, + done_before); if (IS_ERR_OR_NULL(dio)) return PTR_ERR_OR_ZERO(dio); return iomap_dio_complete(dio); diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 5bddb1e9e0b3e..85c412107a100 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -225,7 +225,7 @@ xfs_file_dio_read( ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); if (ret) return ret; - ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0, 0); + ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0, NULL, 0); xfs_iunlock(ip, XFS_IOLOCK_SHARED); return ret; @@ -534,7 +534,7 @@ xfs_file_dio_write_aligned( } trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, 0, 0); + &xfs_dio_write_ops, 0, NULL, 0); out_unlock: if (iolock) xfs_iunlock(ip, iolock); @@ -612,7 +612,7 @@ xfs_file_dio_write_unaligned( trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, flags, 0); + &xfs_dio_write_ops, flags, NULL, 0); /* * Retry unaligned I/O with exclusive blocking semantics if the DIO diff --git a/fs/zonefs/super.c b/fs/zonefs/super.c index e20e7c8414896..777fe626c2b38 100644 --- a/fs/zonefs/super.c +++ b/fs/zonefs/super.c @@ -861,7 +861,7 @@ static ssize_t zonefs_file_dio_write(struct kiocb *iocb, struct iov_iter *from) ret = zonefs_file_dio_append(iocb, from); else ret = iomap_dio_rw(iocb, from, &zonefs_iomap_ops, - &zonefs_write_dio_ops, 0, 0); + &zonefs_write_dio_ops, 0, NULL, 0); if (zi->i_ztype == ZONEFS_ZTYPE_SEQ && (ret > 0 || ret == -EIOCBQUEUED)) { if (ret > 0) @@ -996,7 +996,7 @@ static ssize_t zonefs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) } file_accessed(iocb->ki_filp); ret = iomap_dio_rw(iocb, to, &zonefs_iomap_ops, - &zonefs_read_dio_ops, 0, 0); + &zonefs_read_dio_ops, 0, NULL, 0); } else { ret = generic_file_read_iter(iocb, to); if (ret == -EIO) diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 526c9e7f2eaf8..ea72fa58c06c3 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -188,6 +188,7 @@ struct iomap_iter { unsigned flags; struct iomap iomap; struct iomap srcmap; + void *private; }; int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops); @@ -354,10 +355,10 @@ struct iomap_dio_ops { ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags, size_t done_before); + unsigned int dio_flags, void *private, size_t done_before); struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags, size_t done_before); + unsigned int dio_flags, void *private, size_t done_before); ssize_t iomap_dio_complete(struct iomap_dio *dio); void iomap_dio_bio_end_io(struct bio *bio); From patchwork Thu May 5 20:11:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DC47C433EF for ; Thu, 5 May 2022 20:11:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385740AbiEEUPL (ORCPT ); Thu, 5 May 2022 16:15:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385707AbiEEUPE (ORCPT ); Thu, 5 May 2022 16:15:04 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DFA205F8EB; Thu, 5 May 2022 13:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=nDH3ymgBZY1Z7qV7318Eaghbp5uJ1ymMWyGzPtDu5pc=; b=PwwUZ2g+MY7lREtPp5wsJt4QRM J/o0ox27aj251PgbmEPFoz8fYpgfKi7tnrjg8gF/l35VMmjUzU4E4GsE5k1M/kt28zETll2Gpxv3u YxoBvLanPCzJfwA6oyZfr29szeekPVwxV0OnRzwaDZ76lzQ6KKJP7mrlQeB+0LCvVUeAfvhbFkbHQ MKq8Dg8TrQ0P7ZtSAg4GDY8F7xcwo520283PrBx/FhYvJRj41xCj6wQR5DybRQS1+6hi38d0YS9Y/ VIRhmM5OyzIQew7v1xaI44Rck6YTFRsw2W9b+6uKJn+8+zNT1I+8NHoPg9+IJaDThrqIhCbZBplbk J+id1dIA==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhon-0006j2-69; Thu, 05 May 2022 20:11:21 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 4/7] btrfs: allocate dio_data on stack Date: Thu, 5 May 2022 15:11:12 -0500 Message-Id: <20220505201115.937837-5-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Make use of the new iomap_iter->private field to avoid a memory allocation per iomap range. Signed-off-by: Christoph Hellwig --- fs/btrfs/inode.c | 34 +++++++++++----------------------- 1 file changed, 11 insertions(+), 23 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 88e617e9bf5df..9686f123bf4e3 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7546,10 +7546,11 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start, loff_t length, unsigned int flags, struct iomap *iomap, struct iomap *srcmap) { + struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap); struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); struct extent_map *em; struct extent_state *cached_state = NULL; - struct btrfs_dio_data *dio_data = NULL; + struct btrfs_dio_data *dio_data = iter->private; u64 lockstart, lockend; const bool write = !!(flags & IOMAP_WRITE); int ret = 0; @@ -7595,17 +7596,7 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start, } } - if (flags & IOMAP_NOWAIT) { - dio_data = kzalloc(sizeof(*dio_data), GFP_NOWAIT); - if (!dio_data) - return -EAGAIN; - } else { - dio_data = kzalloc(sizeof(*dio_data), GFP_NOFS); - if (!dio_data) - return -ENOMEM; - } - - iomap->private = dio_data; + memset(dio_data, 0, sizeof(*dio_data)); /* * We always try to allocate data space and must do it before locking @@ -7769,23 +7760,22 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start, extent_changeset_free(dio_data->data_reserved); } - kfree(dio_data); - return ret; } static int btrfs_dio_iomap_end(struct inode *inode, loff_t pos, loff_t length, ssize_t written, unsigned int flags, struct iomap *iomap) { - int ret = 0; - struct btrfs_dio_data *dio_data = iomap->private; + struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap); + struct btrfs_dio_data *dio_data = iter->private; size_t submitted = dio_data->submitted; const bool write = !!(flags & IOMAP_WRITE); + int ret = 0; if (!write && (iomap->type == IOMAP_HOLE)) { /* If reading from a hole, unlock and return */ unlock_extent(&BTRFS_I(inode)->io_tree, pos, pos + length - 1); - goto out; + return 0; } if (submitted < length) { @@ -7802,10 +7792,6 @@ static int btrfs_dio_iomap_end(struct inode *inode, loff_t pos, loff_t length, if (write) extent_changeset_free(dio_data->data_reserved); -out: - kfree(dio_data); - iomap->private = NULL; - return ret; } @@ -8041,7 +8027,7 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, int ret; blk_status_t status; struct btrfs_io_geometry geom; - struct btrfs_dio_data *dio_data = iter->iomap.private; + struct btrfs_dio_data *dio_data = iter->private; struct extent_map *em = NULL; dip = btrfs_create_dio_private(dio_bio, inode, file_offset); @@ -8167,8 +8153,10 @@ static const struct iomap_dio_ops btrfs_dio_ops = { ssize_t btrfs_dio_rw(struct kiocb *iocb, struct iov_iter *iter, size_t done_before) { + struct btrfs_dio_data data; + return iomap_dio_rw(iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - IOMAP_DIO_PARTIAL, NULL, done_before); + IOMAP_DIO_PARTIAL, &data, done_before); } static int btrfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, From patchwork Thu May 5 20:11:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840166 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6751FC433EF for ; Thu, 5 May 2022 20:11:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385802AbiEEUPd (ORCPT ); Thu, 5 May 2022 16:15:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1359685AbiEEUPa (ORCPT ); Thu, 5 May 2022 16:15:30 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79BF15F8F2; Thu, 5 May 2022 13:11:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=xSu6q6f1HTdS9tkh59ToOB06wn8WXvnLrBuqUIUpBQo=; b=0TLefhLWv3kcuM81/ytmlUYWVn nQSA8jDnan6MGB3VEFB9R1zfRHwGPWQBpcYtXy/taO5u8AL+hH1BGaq+uIOfKteC5DikVaIaLWY+m CWobTXe0T1cmcFuWkNutX4yUKwp4QjcXF4zM9HT70b6fng+Sap5j+qOkI64X1dyU3OjqSIbCz2Jzc LuvTHQvR4/xF7yFt7TU2CQSuYZefkEGTsRg/5c4H5MIrZ90ZlDUcZtL6kCCiFXvxBb9ObR8wLSbSO NA5iDqOx46cDqz5CI2aQy2F47UXFts9+BWeFEd7HlTpwOUA+KVy7zNWdzCejatsiCoaZk0uh2Wd33 d+ODURlw==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhoo-0006jX-Eh; Thu, 05 May 2022 20:11:22 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 5/7] btrfs: remove the disk_bytenr in struct btrfs_dio_private Date: Thu, 5 May 2022 15:11:13 -0500 Message-Id: <20220505201115.937837-6-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This field is never used, so remove it. Signed-off-by: Christoph Hellwig --- fs/btrfs/btrfs_inode.h | 1 - fs/btrfs/inode.c | 1 - 2 files changed, 2 deletions(-) diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h index 32131a5d321b3..14c28213ca0d3 100644 --- a/fs/btrfs/btrfs_inode.h +++ b/fs/btrfs/btrfs_inode.h @@ -403,7 +403,6 @@ struct btrfs_dio_private { * grab the file offset, thus need a dedicated member for file offset. */ u64 file_offset; - u64 disk_bytenr; /* Used for bio::bi_size */ u32 bytes; diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 9686f123bf4e3..b1c0c7da6411c 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -8002,7 +8002,6 @@ static struct btrfs_dio_private *btrfs_create_dio_private(struct bio *dio_bio, dip->inode = inode; dip->file_offset = file_offset; dip->bytes = dio_bio->bi_iter.bi_size; - dip->disk_bytenr = dio_bio->bi_iter.bi_sector << 9; dip->dio_bio = dio_bio; refcount_set(&dip->refs, 1); return dip; From patchwork Thu May 5 20:11:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32F95C433EF for ; Thu, 5 May 2022 20:12:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385736AbiEEUPj (ORCPT ); Thu, 5 May 2022 16:15:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385733AbiEEUPb (ORCPT ); Thu, 5 May 2022 16:15:31 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD7255F8EB; Thu, 5 May 2022 13:11:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Fo858EZ24RtCWgP5iiuyw7PCY4Ku4LcL8vFcHP9Ktg4=; b=PdzWesLjuyOXtqCS17xCBEazqs FUKQH0yVQSXMnPE2ZQpPtgm4xx41/qyMl6Rg7X2ApKU4SJlKr3Zbw2Ighrfd3zgpl14vEnSPkzSvJ 8HWq62YZphEU40ive083QPsqQY348LXJ6V/0/hiK50+vXr+rwR79u4yzTwLff9vk5H7d9r/HliHS6 Q4ZHpPw/BGLBz7KsKi72MjnwvdHeem48CV6nAKhW3H/e8yFeREzY40UrhzQEQ5O3/lbjJLM8Xa19Y 6mcN2TS1Dgg1ovtSf15GEy1BMAkg/ArQI77RVamnOSV6BJer+pZijJqVs1mAMPCVRHlWKiafzaAwu YLb1u6GA==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhou-0006ln-Oc; Thu, 05 May 2022 20:11:28 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 6/7] btrfs: move struct btrfs_dio_private to inode.c Date: Thu, 5 May 2022 15:11:14 -0500 Message-Id: <20220505201115.937837-7-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The btrfs_dio_private structure is only used in inode.c, so move the definition there. Signed-off-by: Christoph Hellwig --- fs/btrfs/btrfs_inode.h | 24 ------------------------ fs/btrfs/ctree.h | 1 - fs/btrfs/inode.c | 24 ++++++++++++++++++++++++ 3 files changed, 24 insertions(+), 25 deletions(-) diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h index 14c28213ca0d3..33811e896623f 100644 --- a/fs/btrfs/btrfs_inode.h +++ b/fs/btrfs/btrfs_inode.h @@ -395,30 +395,6 @@ static inline bool btrfs_inode_can_compress(const struct btrfs_inode *inode) return true; } -struct btrfs_dio_private { - struct inode *inode; - - /* - * Since DIO can use anonymous page, we cannot use page_offset() to - * grab the file offset, thus need a dedicated member for file offset. - */ - u64 file_offset; - /* Used for bio::bi_size */ - u32 bytes; - - /* - * References to this structure. There is one reference per in-flight - * bio plus one while we're still setting up. - */ - refcount_t refs; - - /* dio_bio came from fs/direct-io.c */ - struct bio *dio_bio; - - /* Array of checksums */ - u8 csums[]; -}; - /* * btrfs_inode_item stores flags in a u64, btrfs_inode stores them in two * separate u32s. These two functions convert between the two representations. diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index aa6e71fdc72b9..fa64323c453f5 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3217,7 +3217,6 @@ int btrfs_del_orphan_item(struct btrfs_trans_handle *trans, int btrfs_find_orphan_item(struct btrfs_root *root, u64 offset); /* file-item.c */ -struct btrfs_dio_private; int btrfs_del_csums(struct btrfs_trans_handle *trans, struct btrfs_root *root, u64 bytenr, u64 len); blk_status_t btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio, u8 *dst); diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b1c0c7da6411c..edccfc5889e6c 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -68,6 +68,30 @@ struct btrfs_dio_data { bool nocow_done; }; +struct btrfs_dio_private { + struct inode *inode; + + /* + * Since DIO can use anonymous page, we cannot use page_offset() to + * grab the file offset, thus need a dedicated member for file offset. + */ + u64 file_offset; + /* Used for bio::bi_size */ + u32 bytes; + + /* + * References to this structure. There is one reference per in-flight + * bio plus one while we're still setting up. + */ + refcount_t refs; + + /* dio_bio came from fs/direct-io.c */ + struct bio *dio_bio; + + /* Array of checksums */ + u8 csums[]; +}; + struct btrfs_rename_ctx { /* Output field. Stores the index number of the old directory entry. */ u64 index; From patchwork Thu May 5 20:11:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "hch@lst.de" X-Patchwork-Id: 12840168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3D87C4332F for ; Thu, 5 May 2022 20:12:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385757AbiEEUPm (ORCPT ); Thu, 5 May 2022 16:15:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1385755AbiEEUPb (ORCPT ); Thu, 5 May 2022 16:15:31 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0D355FF06; Thu, 5 May 2022 13:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=WtbEYwJtR2Fjgp7y4ng65BxJF9ljaWs0cq/rZODAVK8=; b=XoijPpKgtO61VW8la2vd1yPvD9 pynphNPyALfHjjodvU1bSllyacSGe6zIJZv67PJ6IajoTgR5iAyojawEOL1SB5Cto9Zfkt7EX8Amx VMw2dSMKv5Lza8HjLaG1tqHIaRSvqiGziJFp3u27TdhnQYaZxVfJzfvXhpg94kFPOFfUF6QNXNXeo CyubzBIKwJZ1y7SAXikhJsw3+dKWBENpof4e1yM8dxoL8F9iMrIhK0Fabmzd04wUB7gxflnQVJG7U RCHE07MkK3kc7Pp1qpal6ZGmDu9IOxu3LnL+ZTSU4gUS0OwWIi0X8OyKlaWMQR9p9Bspeq/qaRYyv zv2pMK1w==; Received: from 65-114-90-19.dia.static.qwest.net ([65.114.90.19] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nmhov-0006mN-Po; Thu, 05 May 2022 20:11:29 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba , "Darrick J. Wong" Cc: linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 7/7] btrfs: allocate the btrfs_dio_private as part of the iomap dio bio Date: Thu, 5 May 2022 15:11:15 -0500 Message-Id: <20220505201115.937837-8-hch@lst.de> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505201115.937837-1-hch@lst.de> References: <20220505201115.937837-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Create a new bio_set that contains all the per-bio private data needed by btrfs for direct I/O and tell the iomap code to use that instead of separately allocation the btrfs_dio_private structure. Signed-off-by: Christoph Hellwig --- fs/btrfs/inode.c | 92 ++++++++++++++++++------------------------------ 1 file changed, 34 insertions(+), 58 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index edccfc5889e6c..9443f9cef2b05 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -85,13 +85,14 @@ struct btrfs_dio_private { */ refcount_t refs; - /* dio_bio came from fs/direct-io.c */ - struct bio *dio_bio; - /* Array of checksums */ - u8 csums[]; + u8 *csums; + + struct bio bio; }; +static struct bio_set btrfs_dio_bioset; + struct btrfs_rename_ctx { /* Output field. Stores the index number of the old directory entry. */ u64 index; @@ -7828,19 +7829,19 @@ static void btrfs_dio_private_put(struct btrfs_dio_private *dip) if (!refcount_dec_and_test(&dip->refs)) return; - if (btrfs_op(dip->dio_bio) == BTRFS_MAP_WRITE) { + if (btrfs_op(&dip->bio) == BTRFS_MAP_WRITE) { __endio_write_update_ordered(BTRFS_I(dip->inode), dip->file_offset, dip->bytes, - !dip->dio_bio->bi_status); + !dip->bio.bi_status); } else { unlock_extent(&BTRFS_I(dip->inode)->io_tree, dip->file_offset, dip->file_offset + dip->bytes - 1); } - bio_endio(dip->dio_bio); - kfree(dip); + kfree(dip->csums); + bio_endio(&dip->bio); } static void submit_dio_repair_bio(struct inode *inode, struct bio *bio, @@ -7942,7 +7943,7 @@ static void btrfs_end_dio_bio(struct bio *bio) err = btrfs_check_read_dio_bio(dip, bbio, !err); if (err) - dip->dio_bio->bi_status = err; + dip->bio.bi_status = err; btrfs_record_physical_zoned(dip->inode, bbio->file_offset, bio); @@ -7997,49 +7998,16 @@ static inline blk_status_t btrfs_submit_dio_bio(struct bio *bio, return ret; } -/* - * If this succeeds, the btrfs_dio_private is responsible for cleaning up locked - * or ordered extents whether or not we submit any bios. - */ -static struct btrfs_dio_private *btrfs_create_dio_private(struct bio *dio_bio, - struct inode *inode, - loff_t file_offset) -{ - const bool write = (btrfs_op(dio_bio) == BTRFS_MAP_WRITE); - const bool csum = !(BTRFS_I(inode)->flags & BTRFS_INODE_NODATASUM); - size_t dip_size; - struct btrfs_dio_private *dip; - - dip_size = sizeof(*dip); - if (!write && csum) { - struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); - size_t nblocks; - - nblocks = dio_bio->bi_iter.bi_size >> fs_info->sectorsize_bits; - dip_size += fs_info->csum_size * nblocks; - } - - dip = kzalloc(dip_size, GFP_NOFS); - if (!dip) - return NULL; - - dip->inode = inode; - dip->file_offset = file_offset; - dip->bytes = dio_bio->bi_iter.bi_size; - dip->dio_bio = dio_bio; - refcount_set(&dip->refs, 1); - return dip; -} - static void btrfs_submit_direct(const struct iomap_iter *iter, struct bio *dio_bio, loff_t file_offset) { + struct btrfs_dio_private *dip = + container_of(dio_bio, struct btrfs_dio_private, bio); struct inode *inode = iter->inode; const bool write = (btrfs_op(dio_bio) == BTRFS_MAP_WRITE); struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); const bool raid56 = (btrfs_data_alloc_profile(fs_info) & BTRFS_BLOCK_GROUP_RAID56_MASK); - struct btrfs_dio_private *dip; struct bio *bio; u64 start_sector; int async_submit = 0; @@ -8053,24 +8021,25 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, struct btrfs_dio_data *dio_data = iter->private; struct extent_map *em = NULL; - dip = btrfs_create_dio_private(dio_bio, inode, file_offset); - if (!dip) { - if (!write) { - unlock_extent(&BTRFS_I(inode)->io_tree, file_offset, - file_offset + dio_bio->bi_iter.bi_size - 1); - } - dio_bio->bi_status = BLK_STS_RESOURCE; - bio_endio(dio_bio); - return; - } + dip->inode = inode; + dip->file_offset = file_offset; + dip->bytes = dio_bio->bi_iter.bi_size; + refcount_set(&dip->refs, 1); + dip->csums = NULL; + + if (!write && !(BTRFS_I(inode)->flags & BTRFS_INODE_NODATASUM)) { + unsigned int nr_sectors = + (dio_bio->bi_iter.bi_size >> fs_info->sectorsize_bits); - if (!write) { /* * Load the csums up front to reduce csum tree searches and * contention when submitting bios. - * - * If we have csums disabled this will do nothing. */ + status = BLK_STS_RESOURCE; + dip->csums = kcalloc(nr_sectors, fs_info->csum_size, GFP_NOFS); + if (!dip) + goto out_err; + status = btrfs_lookup_bio_sums(inode, dio_bio, dip->csums); if (status != BLK_STS_OK) goto out_err; @@ -8160,7 +8129,7 @@ static void btrfs_submit_direct(const struct iomap_iter *iter, out_err_em: free_extent_map(em); out_err: - dip->dio_bio->bi_status = status; + dio_bio->bi_status = status; btrfs_dio_private_put(dip); } @@ -8171,6 +8140,7 @@ static const struct iomap_ops btrfs_dio_iomap_ops = { static const struct iomap_dio_ops btrfs_dio_ops = { .submit_io = btrfs_submit_direct, + .bio_set = &btrfs_dio_bioset, }; ssize_t btrfs_dio_rw(struct kiocb *iocb, struct iov_iter *iter, @@ -8992,6 +8962,7 @@ void __cold btrfs_destroy_cachep(void) * destroy cache. */ rcu_barrier(); + bioset_exit(&btrfs_dio_bioset); kmem_cache_destroy(btrfs_inode_cachep); kmem_cache_destroy(btrfs_trans_handle_cachep); kmem_cache_destroy(btrfs_path_cachep); @@ -9032,6 +9003,11 @@ int __init btrfs_init_cachep(void) if (!btrfs_free_space_bitmap_cachep) goto fail; + if (bioset_init(&btrfs_dio_bioset, BIO_POOL_SIZE, + offsetof(struct btrfs_dio_private, bio), + BIOSET_NEED_BVECS)) + goto fail; + return 0; fail: btrfs_destroy_cachep();