From patchwork Tue Jan 30 05:54:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13536910 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8190381B9; Tue, 30 Jan 2024 05:54:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706594062; cv=none; b=PE06QfeYFepMwKW9iIBg0VgNPoIl7MC0NOeKyaN73IIT4As3u3kPYaSrfMVQ8vaVPCfHNVFkqIXSh35uiqPTrOYOPDkvClMEMWXKsLBbEuCIispgkjEJMuB4AqALFeobFQ6yRMCNfEsAxlnyyMQ9kLGrE3ELzK8xZ4k6v5iqGi8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706594062; c=relaxed/simple; bh=dkuYEzYpITu1HQ0h5gF4Cg9LPRIjo2PFmBa7ZMeaq/A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cOKMy5xocfPf6KH/vPQn5Uge9XRXZUGR9+su7TaqN2TA3VkOACHUmcWmgcx9QjW0w0tUO+bZxdFHnJJcxHDLuQdTTDqjeiAYeOj5Q74aEJfKy7pZ6ZvLzutlMv/xLc8tsGbyvhpYRQWnLv1YBsVE6YCF8WBabi41DsEyQTZR418= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=MuJjlDIw; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="MuJjlDIw" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=c6QBcK6hZm/AsW8HURTe3yrT8n/4oDbgtP25uaPmGHc=; b=MuJjlDIw9GLIdwLHzp5MOU6s9D tJPMCn7glheGLvoTY5JbW7t4WcUhr+IMq8vUsY4bcNJmyB4W4RH8ewt7C4Lhbn/fzdNXdXkdwFf5j qzj0kiSf4yHCymfXiwmSFcLzdLnFQr3PGRTSBSKERufs7fL0Ao4BCxH8FBY1vNeLE8ZTpVmg2ga9J gPJoTwnmGOLnYFSfmS+jvv1l2RWo2SIhrLeVmN2rXJ/C1omgbSWRxDGyBCpYfBZsEVKkOM5QXujBP bei3UUOMUX2tpP/MEg6aNSMbWFFE8uLdtOmUvBfCFfT39V8RI/eZXBXQLqQKrpXbDX5dl+Ef6fxGc WzM5CGmQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rUh4a-00000008zkh-0eKg; Tue, 30 Jan 2024 05:54:16 +0000 From: "Matthew Wilcox (Oracle)" To: Christoph Hellwig Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 1/3] fs: Introduce buffered_write_operations Date: Tue, 30 Jan 2024 05:54:11 +0000 Message-ID: <20240130055414.2143959-2-willy@infradead.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130055414.2143959-1-willy@infradead.org> References: <20240130055414.2143959-1-willy@infradead.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Start the process of moving write_begin and write_end out from the address_space_operations to their own struct. Make them take a folio instead of a page, and pass a pointer to the fsdata to write_end (so that we don't need to initialise it). Pass an optional buffered_write_operations pointer to various functions in filemap.c. The old names are available as macros for now, except for generic_file_write_iter() which is used as a function pointer by many filesystems. Signed-off-by: Matthew Wilcox (Oracle) --- fs/jfs/file.c | 3 +- fs/ramfs/file-mmu.c | 3 +- fs/ufs/file.c | 2 +- include/linux/fs.h | 3 -- include/linux/pagemap.h | 22 +++++++++++++ mm/filemap.c | 70 +++++++++++++++++++++++++++-------------- 6 files changed, 71 insertions(+), 32 deletions(-) diff --git a/fs/jfs/file.c b/fs/jfs/file.c index 01b6912e60f8..9c62445ea6be 100644 --- a/fs/jfs/file.c +++ b/fs/jfs/file.c @@ -4,8 +4,7 @@ * Portions Copyright (C) Christoph Hellwig, 2001-2002 */ -#include -#include +#include #include #include #include "jfs_incore.h" diff --git a/fs/ramfs/file-mmu.c b/fs/ramfs/file-mmu.c index c7a1aa3c882b..a621b08b0235 100644 --- a/fs/ramfs/file-mmu.c +++ b/fs/ramfs/file-mmu.c @@ -24,8 +24,7 @@ * caches is sufficient. */ -#include -#include +#include #include #include diff --git a/fs/ufs/file.c b/fs/ufs/file.c index 6558882a89ef..b557d4a14143 100644 --- a/fs/ufs/file.c +++ b/fs/ufs/file.c @@ -24,7 +24,7 @@ * ext2 fs regular file handling primitives */ -#include +#include #include "ufs_fs.h" #include "ufs.h" diff --git a/include/linux/fs.h b/include/linux/fs.h index ed5966a70495..92dc0cf08b1f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -3048,10 +3048,7 @@ extern int generic_file_rw_checks(struct file *file_in, struct file *file_out); ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *to, ssize_t already_read); extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *); -extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *); -extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *); extern ssize_t generic_file_direct_write(struct kiocb *, struct iov_iter *); -ssize_t generic_perform_write(struct kiocb *, struct iov_iter *); ssize_t direct_write_fallback(struct kiocb *iocb, struct iov_iter *iter, ssize_t direct_written, ssize_t buffered_written); diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 2df35e65557d..a5c474ad230e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -18,6 +18,28 @@ struct folio_batch; +struct buffered_write_operations { + int (*write_begin)(struct file *, struct address_space *mapping, + loff_t pos, size_t len, struct folio **foliop, + void **fsdata); + int (*write_end)(struct file *, struct address_space *mapping, + loff_t pos, size_t len, size_t copied, + struct folio *folio, void **fsdata); +}; + +ssize_t filemap_write_iter(struct kiocb *, struct iov_iter *, + const struct buffered_write_operations *); +ssize_t __filemap_write_iter(struct kiocb *, struct iov_iter *, + const struct buffered_write_operations *); +ssize_t filemap_perform_write(struct kiocb *, struct iov_iter *, + const struct buffered_write_operations *); + +ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *); +#define generic_perform_write(kiocb, iter) \ + filemap_perform_write(kiocb, iter, NULL) +#define __generic_file_write_iter(kiocb, iter) \ + __filemap_write_iter(kiocb, iter, NULL) + unsigned long invalidate_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t end); diff --git a/mm/filemap.c b/mm/filemap.c index 750e779c23db..66d779b787c8 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -95,7 +95,7 @@ * ->invalidate_lock (filemap_fault) * ->lock_page (filemap_fault, access_process_vm) * - * ->i_rwsem (generic_perform_write) + * ->i_rwsem (filemap_perform_write) * ->mmap_lock (fault_in_readable->do_page_fault) * * bdi->wb.list_lock @@ -3890,7 +3890,8 @@ generic_file_direct_write(struct kiocb *iocb, struct iov_iter *from) } EXPORT_SYMBOL(generic_file_direct_write); -ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) +ssize_t filemap_perform_write(struct kiocb *iocb, struct iov_iter *i, + const struct buffered_write_operations *ops) { struct file *file = iocb->ki_filp; loff_t pos = iocb->ki_pos; @@ -3900,9 +3901,9 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) ssize_t written = 0; do { - struct page *page; - unsigned long offset; /* Offset into pagecache page */ - unsigned long bytes; /* Bytes to write to page */ + struct folio *folio; + size_t offset; /* Offset into pagecache folio */ + size_t bytes; /* Bytes to write to page */ size_t copied; /* Bytes copied from user */ void *fsdata = NULL; @@ -3927,19 +3928,31 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) break; } - status = a_ops->write_begin(file, mapping, pos, bytes, + if (ops) + status = ops->write_begin(file, mapping, pos, bytes, + &folio, &fsdata); + else { + struct page *page; + status = a_ops->write_begin(file, mapping, pos, bytes, &page, &fsdata); + if (status >= 0) + folio = page_folio(page); + } if (unlikely(status < 0)) break; if (mapping_writably_mapped(mapping)) - flush_dcache_page(page); + flush_dcache_folio(folio); - copied = copy_page_from_iter_atomic(page, offset, bytes, i); - flush_dcache_page(page); + copied = copy_folio_from_iter_atomic(folio, offset, bytes, i); + flush_dcache_folio(folio); - status = a_ops->write_end(file, mapping, pos, bytes, copied, - page, fsdata); + if (ops) + status = ops->write_end(file, mapping, pos, bytes, + copied, folio, &fsdata); + else + status = a_ops->write_end(file, mapping, pos, bytes, + copied, &folio->page, fsdata); if (unlikely(status != copied)) { iov_iter_revert(i, copied - max(status, 0L)); if (unlikely(status < 0)) @@ -3969,12 +3982,13 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) iocb->ki_pos += written; return written; } -EXPORT_SYMBOL(generic_perform_write); +EXPORT_SYMBOL(filemap_perform_write); /** - * __generic_file_write_iter - write data to a file - * @iocb: IO state structure (file, offset, etc.) - * @from: iov_iter with data to write + * __filemap_write_iter - write data to a file + * @iocb: IO state structure (file, offset, etc.) + * @from: iov_iter with data to write + * @ops: How to inform the filesystem that a write is starting/finishing. * * This function does all the work needed for actually writing data to a * file. It does all basic checks, removes SUID from the file, updates @@ -3992,7 +4006,8 @@ EXPORT_SYMBOL(generic_perform_write); * * number of bytes written, even for truncated writes * * negative error code if no data has been written at all */ -ssize_t __generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) +ssize_t __filemap_write_iter(struct kiocb *iocb, struct iov_iter *from, + const struct buffered_write_operations *ops) { struct file *file = iocb->ki_filp; struct address_space *mapping = file->f_mapping; @@ -4019,27 +4034,29 @@ ssize_t __generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) if (ret < 0 || !iov_iter_count(from) || IS_DAX(inode)) return ret; return direct_write_fallback(iocb, from, ret, - generic_perform_write(iocb, from)); + filemap_perform_write(iocb, from, ops)); } - return generic_perform_write(iocb, from); + return filemap_perform_write(iocb, from, ops); } -EXPORT_SYMBOL(__generic_file_write_iter); +EXPORT_SYMBOL(__filemap_write_iter); /** - * generic_file_write_iter - write data to a file + * filemap_write_iter - write data to a file * @iocb: IO state structure * @from: iov_iter with data to write * - * This is a wrapper around __generic_file_write_iter() to be used by most + * This is a wrapper around __filemap_write_iter() to be used by most * filesystems. It takes care of syncing the file in case of O_SYNC file * and acquires i_rwsem as needed. + * * Return: - * * negative error code if no data has been written at all of + * * negative error code if no data has been written at all or if * vfs_fsync_range() failed for a synchronous write * * number of bytes written, even for truncated writes */ -ssize_t generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) +ssize_t filemap_write_iter(struct kiocb *iocb, struct iov_iter *from, + const struct buffered_write_operations *ops) { struct file *file = iocb->ki_filp; struct inode *inode = file->f_mapping->host; @@ -4048,13 +4065,18 @@ ssize_t generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) inode_lock(inode); ret = generic_write_checks(iocb, from); if (ret > 0) - ret = __generic_file_write_iter(iocb, from); + ret = __filemap_write_iter(iocb, from, ops); inode_unlock(inode); if (ret > 0) ret = generic_write_sync(iocb, ret); return ret; } + +ssize_t generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) +{ + return filemap_write_iter(iocb, from, NULL); +} EXPORT_SYMBOL(generic_file_write_iter); /** From patchwork Tue Jan 30 05:54:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13536911 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D27C381B9; Tue, 30 Jan 2024 05:54:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706594065; cv=none; b=IKqwN7FwNfPIjQWCVkMQzW2EBm07vXv5zapeSZT4D0o3TK5giW8enS5NjWK2ftT+yyzX8RuqSfYv2gj09pwecTQ48bzJERt/wBwTBzSU8XUBrrxyUx0mW/MbWN4ih5gWab8KlPJQLSnvIs+Ah6zojF+wasaolofuVJZtR9cMv7k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706594065; c=relaxed/simple; bh=JMGNgA5p77L4yCmgHLC5rGqvzm4lD2Au+5vm0jaJPkg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KjPqbnrvNbBGiiuTjlYfpnsvt1smYSlHlnRRCjlJh8baAhfO7f4PO3cfXIkr4+m6boRw+2rTf3ZClgwMZ6nObOL0KraA1hAd/3Ox3TWWXK9iimv+QWVVtv1j4woj0e7Hi8WO9pUFB4DLwgdfX1m5zfYzA7TpD4UcOJe6D1LPQO8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=lvX7yeCB; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lvX7yeCB" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=1Ef/D0lEas0yp4KPjSxjRO+7105j66U6WTJQm7R4Kn8=; b=lvX7yeCBJlayQsdK8PUlA0HuhG G5nIQwPrqg5Nr5CgLaofgotf/cYjScz6efp7vuNSpVwXzJ4/3hXRiAslNsbl6O663ILOPkgQyD3cK kDxlr4g2EJlVKFMxf18altMxeF9hMpWcHRfQsG604zLXLaEPcaivh9oyUGZ0rus5JLybfhkfpPFLN Se0FsSeLZE5LcUqhaIKBeIFXdpgew7sUGNLd2DLykvyhzSAdigLBFqQUqNLSvzRpiTpFnFzZSXvg8 ipXacqLFu6Wxjo5LdlHkgqvAC5oqMjRxfeiY3/CiZwn49t0rYk1l5r9uDWrAPKH6o9STDPoDfCNKv ASYOi/nQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rUh4a-00000008zkj-0zzP; Tue, 30 Jan 2024 05:54:16 +0000 From: "Matthew Wilcox (Oracle)" To: Christoph Hellwig Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 2/3] fs: Supply optional buffered_write_operations in buffer.c Date: Tue, 30 Jan 2024 05:54:12 +0000 Message-ID: <20240130055414.2143959-3-willy@infradead.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130055414.2143959-1-willy@infradead.org> References: <20240130055414.2143959-1-willy@infradead.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 generic_cont_expand_simple() and cont_expand_zero() currently call ->write_begin and ->write_end, so will also need to be converted to use buffered_write_operations. Use macro magic again to pass in optional buffered_write_operations. Signed-off-by: Matthew Wilcox (Oracle) --- fs/buffer.c | 62 +++++++++++++++++++++++++++---------- include/linux/buffer_head.h | 22 ++++++++++--- 2 files changed, 64 insertions(+), 20 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index d3bcf601d3e5..8ed76fc6cff0 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2441,11 +2441,13 @@ EXPORT_SYMBOL(block_read_full_folio); * truncates. Uses filesystem pagecache writes to allow the filesystem to * deal with the hole. */ -int generic_cont_expand_simple(struct inode *inode, loff_t size) +int generic_cont_expand_simple(struct inode *inode, loff_t size, + const struct buffered_write_operations *ops) { struct address_space *mapping = inode->i_mapping; const struct address_space_operations *aops = mapping->a_ops; struct page *page; + struct folio *folio; void *fsdata = NULL; int err; @@ -2453,11 +2455,17 @@ int generic_cont_expand_simple(struct inode *inode, loff_t size) if (err) goto out; - err = aops->write_begin(NULL, mapping, size, 0, &page, &fsdata); + if (ops) + err = ops->write_begin(NULL, mapping, size, 0, &folio, &fsdata); + else + err = aops->write_begin(NULL, mapping, size, 0, &page, &fsdata); if (err) goto out; - err = aops->write_end(NULL, mapping, size, 0, 0, page, fsdata); + if (ops) + err = ops->write_end(NULL, mapping, size, 0, 0, folio, &fsdata); + else + err = aops->write_end(NULL, mapping, size, 0, 0, page, fsdata); BUG_ON(err > 0); out: @@ -2466,12 +2474,14 @@ int generic_cont_expand_simple(struct inode *inode, loff_t size) EXPORT_SYMBOL(generic_cont_expand_simple); static int cont_expand_zero(struct file *file, struct address_space *mapping, - loff_t pos, loff_t *bytes) + loff_t pos, loff_t *bytes, + const struct buffered_write_operations *ops) { struct inode *inode = mapping->host; const struct address_space_operations *aops = mapping->a_ops; unsigned int blocksize = i_blocksize(inode); struct page *page; + struct folio *folio; void *fsdata = NULL; pgoff_t index, curidx; loff_t curpos; @@ -2489,13 +2499,23 @@ static int cont_expand_zero(struct file *file, struct address_space *mapping, } len = PAGE_SIZE - zerofrom; - err = aops->write_begin(file, mapping, curpos, len, - &page, &fsdata); + if (ops) { + err = ops->write_begin(file, mapping, curpos, len, + &folio, &fsdata); + page = &folio->page; + } else { + err = aops->write_begin(file, mapping, curpos, len, + &page, &fsdata); + } if (err) goto out; zero_user(page, zerofrom, len); - err = aops->write_end(file, mapping, curpos, len, len, - page, fsdata); + if (ops) + err = ops->write_end(file, mapping, curpos, len, len, + folio, &fsdata); + else + err = aops->write_end(file, mapping, curpos, len, len, + page, fsdata); if (err < 0) goto out; BUG_ON(err != len); @@ -2522,13 +2542,23 @@ static int cont_expand_zero(struct file *file, struct address_space *mapping, } len = offset - zerofrom; - err = aops->write_begin(file, mapping, curpos, len, - &page, &fsdata); + if (ops) { + err = ops->write_begin(file, mapping, curpos, len, + &folio, &fsdata); + page = &folio->page; + } else { + err = aops->write_begin(file, mapping, curpos, len, + &page, &fsdata); + } if (err) goto out; zero_user(page, zerofrom, len); - err = aops->write_end(file, mapping, curpos, len, len, - page, fsdata); + if (ops) + err = ops->write_end(file, mapping, curpos, len, len, + folio, &fsdata); + else + err = aops->write_end(file, mapping, curpos, len, len, + page, fsdata); if (err < 0) goto out; BUG_ON(err != len); @@ -2543,16 +2573,16 @@ static int cont_expand_zero(struct file *file, struct address_space *mapping, * We may have to extend the file. */ int cont_write_begin(struct file *file, struct address_space *mapping, - loff_t pos, unsigned len, - struct page **pagep, void **fsdata, - get_block_t *get_block, loff_t *bytes) + loff_t pos, unsigned len, struct page **pagep, void **fsdata, + get_block_t *get_block, loff_t *bytes, + const struct buffered_write_operations *ops) { struct inode *inode = mapping->host; unsigned int blocksize = i_blocksize(inode); unsigned int zerofrom; int err; - err = cont_expand_zero(file, mapping, pos, bytes); + err = cont_expand_zero(file, mapping, pos, bytes, ops); if (err) return err; diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index d78454a4dd1f..80de88c12d23 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -268,16 +268,30 @@ int generic_write_end(struct file *, struct address_space *, loff_t, unsigned, unsigned, struct page *, void *); void folio_zero_new_buffers(struct folio *folio, size_t from, size_t to); -int cont_write_begin(struct file *, struct address_space *, loff_t, - unsigned, struct page **, void **, - get_block_t *, loff_t *); -int generic_cont_expand_simple(struct inode *inode, loff_t size); +int cont_write_begin(struct file *, struct address_space *, loff_t pos, + unsigned len, struct page **, void **fsdata, get_block_t *, + loff_t *bytes, const struct buffered_write_operations *); +int generic_cont_expand_simple(struct inode *inode, loff_t size, + const struct buffered_write_operations *ops); void block_commit_write(struct page *page, unsigned int from, unsigned int to); int block_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf, get_block_t get_block); sector_t generic_block_bmap(struct address_space *, sector_t, get_block_t *); int block_truncate_page(struct address_space *, loff_t, get_block_t *); +#define _cont_write_begin(file, mapping, pos, len, pagep, fsdata, \ + getblk, bytes, ops, extra...) \ + cont_write_begin(file, mapping, pos, len, pagep, fsdata, \ + getblk,bytes, ops) +#define cont_write_begin(file, mapping, pos, len, pagep, fsdata, \ + getblk, bytes, ops...) \ + _cont_write_begin(file, mapping, pos, len, pagep, fsdata, \ + getblk, bytes, ## ops, NULL) +#define _generic_cont_expand_simple(inode, size, ops, extra...) \ + generic_cont_expand_simple(inode, size, ops) +#define generic_cont_expand_simple(inode, size, ops...) \ + _generic_cont_expand_simple(inode, size, ## ops, NULL) + #ifdef CONFIG_MIGRATION extern int buffer_migrate_folio(struct address_space *, struct folio *dst, struct folio *src, enum migrate_mode); From patchwork Tue Jan 30 05:54:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13536909 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 532FD381B4; Tue, 30 Jan 2024 05:54:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706594060; cv=none; b=FbxRCEDboURYNTXp8zROS8Y/xyRy0/scema71JBEQReBNUZ7t31M9veg7RnJYJnUv7FPo62Fu8xDIn8SHJ1lHnOvr9x8+7qhjP8PprZq3Tsk4oy8ZI8XVdjfwkEjpuVj8bjs7/86UmUb4eKv9/iOu8HnddjL/NjmXufPnNhQ2g8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706594060; c=relaxed/simple; bh=c0fD+MAYXNAXa5oNaXtYEsoBD240F+BZn6nT/koVWEA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IUT5JwsNXpvLpSrSUcWF2ql/2TAClU35WAizUDh+QNF3eNMBhgcxTZfP53aF6X2/RqseLwwWOP45ydR/rtuO39GaFdUEdulYSlfRI9Dm1se+7lz97Q1ed9kZnQ4W3DunsIQZIPd7TNvPd8Fkq6Tn7DHes1YQ1vbR29ZhcYAnlko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=CuR47Eo2; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="CuR47Eo2" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=V7/IQmodZEVVOURk79W2QOsKgj01svX706ULbJwppUc=; b=CuR47Eo2LLxYsdZaF02nqwACcc 7hjWROyRhzcYaJKKZND9rB/m6cxLGcqzO7nG9rBJBqrVClJRvbG7fu2WYpdi/pTsg2vBFsykXv+sf kz6pjrAg5W3yhSDCsj7bkv556m1oHlQua+NNhV1YMCe5rr4bmMkUipCSsMFpeuG/2ELGn1rJUEggL CgPJRAuSgQLd9SIHq+TsE2K3Y+o/wf5/pfAI4n7MpdN1xtmyhBpRi4e1mySBdKw3NaYjyGhcLEoXN Q8ruV52v5SoUD+w8rbEZBK2C0VlmfhtTYIpVj+ifWPqi0q7aLFafSyy77nqU0wZowDSGb3RGCERXL IH6EMzTQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rUh4a-00000008zkn-1MYY; Tue, 30 Jan 2024 05:54:16 +0000 From: "Matthew Wilcox (Oracle)" To: Christoph Hellwig Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 3/3] ext4: Convert to buffered_write_operations Date: Tue, 30 Jan 2024 05:54:13 +0000 Message-ID: <20240130055414.2143959-4-willy@infradead.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130055414.2143959-1-willy@infradead.org> References: <20240130055414.2143959-1-willy@infradead.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Pass the appropriate buffered_write_operations to filemap_perform_write(). Saves a lot of page<->folio conversions. Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/ext4.h | 8 ++++-- fs/ext4/file.c | 10 ++++++- fs/ext4/inline.c | 15 ++++------ fs/ext4/inode.c | 73 ++++++++++++++++++++++++------------------------ 4 files changed, 58 insertions(+), 48 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index a5d784872303..3491d5c279c7 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -3021,6 +3021,10 @@ extern void ext4_da_update_reserve_space(struct inode *inode, extern int ext4_issue_zeroout(struct inode *inode, ext4_lblk_t lblk, ext4_fsblk_t pblk, ext4_lblk_t len); +extern const struct buffered_write_operations ext4_bw_ops; +extern const struct buffered_write_operations ext4_journalled_bw_ops; +extern const struct buffered_write_operations ext4_da_bw_ops; + /* indirect.c */ extern int ext4_ind_map_blocks(handle_t *handle, struct inode *inode, struct ext4_map_blocks *map, int flags); @@ -3549,13 +3553,13 @@ int ext4_readpage_inline(struct inode *inode, struct folio *folio); extern int ext4_try_to_write_inline_data(struct address_space *mapping, struct inode *inode, loff_t pos, unsigned len, - struct page **pagep); + struct folio **foliop); int ext4_write_inline_data_end(struct inode *inode, loff_t pos, unsigned len, unsigned copied, struct folio *folio); extern int ext4_da_write_inline_data_begin(struct address_space *mapping, struct inode *inode, loff_t pos, unsigned len, - struct page **pagep, + struct folio **foliop, void **fsdata); extern int ext4_try_add_inline_entry(handle_t *handle, struct ext4_filename *fname, diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 6aa15dafc677..1bca5f47cb5f 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -287,16 +287,24 @@ static ssize_t ext4_buffered_write_iter(struct kiocb *iocb, { ssize_t ret; struct inode *inode = file_inode(iocb->ki_filp); + const struct buffered_write_operations *ops; if (iocb->ki_flags & IOCB_NOWAIT) return -EOPNOTSUPP; + if (ext4_inode_journal_mode(inode)) + ops = &ext4_journalled_bw_ops; + else if (test_opt(inode->i_sb, DELALLOC)) + ops = &ext4_da_bw_ops; + else + ops = &ext4_bw_ops; + inode_lock(inode); ret = ext4_write_checks(iocb, from); if (ret <= 0) goto out; - ret = generic_perform_write(iocb, from); + ret = filemap_perform_write(iocb, from, ops); out: inode_unlock(inode); diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c index d5bd1e3a5d36..0d3fc5c14ad5 100644 --- a/fs/ext4/inline.c +++ b/fs/ext4/inline.c @@ -658,9 +658,8 @@ static int ext4_convert_inline_data_to_extent(struct address_space *mapping, * to the page make it update and let the later codes create extent for it. */ int ext4_try_to_write_inline_data(struct address_space *mapping, - struct inode *inode, - loff_t pos, unsigned len, - struct page **pagep) + struct inode *inode, loff_t pos, unsigned len, + struct folio **foliop) { int ret; handle_t *handle; @@ -708,7 +707,7 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, goto out; } - *pagep = &folio->page; + *foliop = folio; down_read(&EXT4_I(inode)->xattr_sem); if (!ext4_has_inline_data(inode)) { ret = 0; @@ -889,10 +888,8 @@ static int ext4_da_convert_inline_data_to_extent(struct address_space *mapping, * normal ext4_da_write_end). */ int ext4_da_write_inline_data_begin(struct address_space *mapping, - struct inode *inode, - loff_t pos, unsigned len, - struct page **pagep, - void **fsdata) + struct inode *inode, loff_t pos, unsigned len, + struct folio **foliop, void **fsdata) { int ret; handle_t *handle; @@ -954,7 +951,7 @@ int ext4_da_write_inline_data_begin(struct address_space *mapping, goto out_release_page; up_read(&EXT4_I(inode)->xattr_sem); - *pagep = &folio->page; + *foliop = folio; brelse(iloc.bh); return 1; out_release_page: diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 5af1b0b8680e..4f532870c388 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -1113,8 +1113,7 @@ static int ext4_block_write_begin(struct folio *folio, loff_t pos, unsigned len, * ext4_write_begin() is the right place. */ static int ext4_write_begin(struct file *file, struct address_space *mapping, - loff_t pos, unsigned len, - struct page **pagep, void **fsdata) + loff_t pos, size_t len, struct folio **foliop, void **fsdata) { struct inode *inode = mapping->host; int ret, needed_blocks; @@ -1139,7 +1138,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, if (ext4_test_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA)) { ret = ext4_try_to_write_inline_data(mapping, inode, pos, len, - pagep); + foliop); if (ret < 0) return ret; if (ret == 1) @@ -1239,7 +1238,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, folio_put(folio); return ret; } - *pagep = &folio->page; + *foliop = folio; return ret; } @@ -1264,12 +1263,10 @@ static int write_end_fn(handle_t *handle, struct inode *inode, * ext4 never places buffers on inode->i_mapping->i_private_list. metadata * buffers are managed internally. */ -static int ext4_write_end(struct file *file, - struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct page *page, void *fsdata) +static int ext4_write_end(struct file *file, struct address_space *mapping, + loff_t pos, size_t len, size_t copied, struct folio *folio, + void **fsdata) { - struct folio *folio = page_folio(page); handle_t *handle = ext4_journal_current_handle(); struct inode *inode = mapping->host; loff_t old_size = inode->i_size; @@ -1284,7 +1281,7 @@ static int ext4_write_end(struct file *file, return ext4_write_inline_data_end(inode, pos, len, copied, folio); - copied = block_write_end(file, mapping, pos, len, copied, page, fsdata); + copied = block_write_end(file, mapping, pos, len, copied, &folio->page, *fsdata); /* * it's important to update i_size while still holding folio lock: * page writeout could otherwise come in and zero beyond i_size. @@ -1369,11 +1366,9 @@ static void ext4_journalled_zero_new_buffers(handle_t *handle, } static int ext4_journalled_write_end(struct file *file, - struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct page *page, void *fsdata) + struct address_space *mapping, loff_t pos, size_t len, + size_t copied, struct folio *folio, void **fsdata) { - struct folio *folio = page_folio(page); handle_t *handle = ext4_journal_current_handle(); struct inode *inode = mapping->host; loff_t old_size = inode->i_size; @@ -2856,9 +2851,9 @@ static int ext4_nonda_switch(struct super_block *sb) return 0; } -static int ext4_da_write_begin(struct file *file, struct address_space *mapping, - loff_t pos, unsigned len, - struct page **pagep, void **fsdata) +static int ext4_da_write_begin(struct file *file, + struct address_space *mapping, loff_t pos, size_t len, + struct folio **foliop, void **fsdata) { int ret, retries = 0; struct folio *folio; @@ -2873,14 +2868,14 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, if (ext4_nonda_switch(inode->i_sb) || ext4_verity_in_progress(inode)) { *fsdata = (void *)FALL_BACK_TO_NONDELALLOC; return ext4_write_begin(file, mapping, pos, - len, pagep, fsdata); + len, foliop, fsdata); } *fsdata = (void *)0; trace_ext4_da_write_begin(inode, pos, len); if (ext4_test_inode_state(inode, EXT4_STATE_MAY_INLINE_DATA)) { ret = ext4_da_write_inline_data_begin(mapping, inode, pos, len, - pagep, fsdata); + foliop, fsdata); if (ret < 0) return ret; if (ret == 1) @@ -2918,7 +2913,7 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, return ret; } - *pagep = &folio->page; + *foliop = folio; return ret; } @@ -2945,9 +2940,8 @@ static int ext4_da_should_update_i_disksize(struct folio *folio, return 1; } -static int ext4_da_do_write_end(struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct folio *folio) +static int ext4_da_do_write_end(struct address_space *mapping, loff_t pos, + size_t len, size_t copied, struct folio *folio) { struct inode *inode = mapping->host; loff_t old_size = inode->i_size; @@ -3007,18 +3001,16 @@ static int ext4_da_do_write_end(struct address_space *mapping, return copied; } -static int ext4_da_write_end(struct file *file, - struct address_space *mapping, - loff_t pos, unsigned len, unsigned copied, - struct page *page, void *fsdata) +static int ext4_da_write_end(struct file *file, struct address_space *mapping, + loff_t pos, size_t len, size_t copied, struct folio *folio, + void **fsdata) { struct inode *inode = mapping->host; - int write_mode = (int)(unsigned long)fsdata; - struct folio *folio = page_folio(page); + int write_mode = (int)(unsigned long)*fsdata; if (write_mode == FALL_BACK_TO_NONDELALLOC) return ext4_write_end(file, mapping, pos, - len, copied, &folio->page, fsdata); + len, copied, folio, fsdata); trace_ext4_da_write_end(inode, pos, len, copied); @@ -3556,8 +3548,6 @@ static const struct address_space_operations ext4_aops = { .read_folio = ext4_read_folio, .readahead = ext4_readahead, .writepages = ext4_writepages, - .write_begin = ext4_write_begin, - .write_end = ext4_write_end, .dirty_folio = ext4_dirty_folio, .bmap = ext4_bmap, .invalidate_folio = ext4_invalidate_folio, @@ -3573,8 +3563,6 @@ static const struct address_space_operations ext4_journalled_aops = { .read_folio = ext4_read_folio, .readahead = ext4_readahead, .writepages = ext4_writepages, - .write_begin = ext4_write_begin, - .write_end = ext4_journalled_write_end, .dirty_folio = ext4_journalled_dirty_folio, .bmap = ext4_bmap, .invalidate_folio = ext4_journalled_invalidate_folio, @@ -3590,8 +3578,6 @@ static const struct address_space_operations ext4_da_aops = { .read_folio = ext4_read_folio, .readahead = ext4_readahead, .writepages = ext4_writepages, - .write_begin = ext4_da_write_begin, - .write_end = ext4_da_write_end, .dirty_folio = ext4_dirty_folio, .bmap = ext4_bmap, .invalidate_folio = ext4_invalidate_folio, @@ -3611,6 +3597,21 @@ static const struct address_space_operations ext4_dax_aops = { .swap_activate = ext4_iomap_swap_activate, }; +const struct buffered_write_operations ext4_bw_ops = { + .write_begin = ext4_write_begin, + .write_end = ext4_write_end, +}; + +const struct buffered_write_operations ext4_journalled_bw_ops = { + .write_begin = ext4_write_begin, + .write_end = ext4_journalled_write_end, +}; + +const struct buffered_write_operations ext4_da_bw_ops = { + .write_begin = ext4_da_write_begin, + .write_end = ext4_da_write_end, +}; + void ext4_set_aops(struct inode *inode) { switch (ext4_inode_journal_mode(inode)) {