The following changes since commit 5b8d6e8539498e8b2fa67fbcce3fe87834d44a7a:
Merge tag 'xtensa-20230716' of https://github.com/jcmvbkbc/linux-xtensa (2023-07-16 14:12:49 -0700)
are available in the Git repository at:
git://git.infradead.org/users/willy/pagecache.git tags/large-folio-writes
for you to fetch changes up to b684c2e69df4883c2d00045bdd159b6a94d28c98:
iomap: Copy larger chunks from userspace (2023-07-16 17:53:18 -0400)
----------------------------------------------------------------
Compared to the last version I sent out by email, I split the
copy_page_from_iter_atomic() change into two pieces based on Luis &
Christoph's feedback. There should be no other changes. I also added
the Reviewed-by tags that were offered.
----------------------------------------------------------------
Create large folios in iomap buffered write path
Commit ebb7fb1557b1 limited the length of ioend chains to 4096 entries
to improve worst-case latency. Unfortunately, this had the effect of
limiting the performance of:
fio -name write-bandwidth -rw=write -bs=1024Ki -size=32Gi -runtime=30 \
-iodepth 1 -ioengine sync -zero_buffers=1 -direct=0 -end_fsync=1 \
-numjobs=4 -directory=/mnt/test
The problem ends up being lock contention on the i_pages spinlock as we
clear the writeback bit on each folio (and propagate that up through
the tree). By using larger folios, we decrease the number of folios
to be processed by a factor of 256 for this benchmark, eliminating the
lock contention.
Creating large folios in the buffered write path is also the right
thing to do. It's a project that has been on the back burner for years,
it just hasn't been important enough to do before now.
----------------------------------------------------------------
Matthew Wilcox (Oracle) (10):
iov_iter: Map the page later in copy_page_from_iter_atomic()
iov_iter: Handle compound highmem pages in copy_page_from_iter_atomic()
iov_iter: Add copy_folio_from_iter_atomic()
iomap: Remove large folio handling in iomap_invalidate_folio()
doc: Correct the description of ->release_folio
iomap: Remove unnecessary test from iomap_release_folio()
filemap: Add fgf_t typedef
filemap: Allow __filemap_get_folio to allocate large folios
iomap: Create large folios in the buffered write path
iomap: Copy larger chunks from userspace
Documentation/filesystems/locking.rst | 15 +++++--
fs/btrfs/file.c | 6 +--
fs/f2fs/compress.c | 2 +-
fs/f2fs/f2fs.h | 2 +-
fs/gfs2/bmap.c | 2 +-
fs/iomap/buffered-io.c | 54 +++++++++++------------
include/linux/iomap.h | 2 +-
include/linux/pagemap.h | 82 ++++++++++++++++++++++++++++++-----
include/linux/uio.h | 9 +++-
lib/iov_iter.c | 43 +++++++++++-------
mm/filemap.c | 65 ++++++++++++++-------------
mm/folio-compat.c | 2 +-
mm/readahead.c | 13 ------
13 files changed, 187 insertions(+), 110 deletions(-)
The following changes since commit 5b8d6e8539498e8b2fa67fbcce3fe87834d44a7a: Merge tag 'xtensa-20230716' of https://github.com/jcmvbkbc/linux-xtensa (2023-07-16 14:12:49 -0700) are available in the Git repository at: git://git.infradead.org/users/willy/pagecache.git tags/large-folio-writes for you to fetch changes up to b684c2e69df4883c2d00045bdd159b6a94d28c98: iomap: Copy larger chunks from userspace (2023-07-16 17:53:18 -0400) ---------------------------------------------------------------- Compared to the last version I sent out by email, I split the copy_page_from_iter_atomic() change into two pieces based on Luis & Christoph's feedback. There should be no other changes. I also added the Reviewed-by tags that were offered. ---------------------------------------------------------------- Create large folios in iomap buffered write path Commit ebb7fb1557b1 limited the length of ioend chains to 4096 entries to improve worst-case latency. Unfortunately, this had the effect of limiting the performance of: fio -name write-bandwidth -rw=write -bs=1024Ki -size=32Gi -runtime=30 \ -iodepth 1 -ioengine sync -zero_buffers=1 -direct=0 -end_fsync=1 \ -numjobs=4 -directory=/mnt/test The problem ends up being lock contention on the i_pages spinlock as we clear the writeback bit on each folio (and propagate that up through the tree). By using larger folios, we decrease the number of folios to be processed by a factor of 256 for this benchmark, eliminating the lock contention. Creating large folios in the buffered write path is also the right thing to do. It's a project that has been on the back burner for years, it just hasn't been important enough to do before now. ---------------------------------------------------------------- Matthew Wilcox (Oracle) (10): iov_iter: Map the page later in copy_page_from_iter_atomic() iov_iter: Handle compound highmem pages in copy_page_from_iter_atomic() iov_iter: Add copy_folio_from_iter_atomic() iomap: Remove large folio handling in iomap_invalidate_folio() doc: Correct the description of ->release_folio iomap: Remove unnecessary test from iomap_release_folio() filemap: Add fgf_t typedef filemap: Allow __filemap_get_folio to allocate large folios iomap: Create large folios in the buffered write path iomap: Copy larger chunks from userspace Documentation/filesystems/locking.rst | 15 +++++-- fs/btrfs/file.c | 6 +-- fs/f2fs/compress.c | 2 +- fs/f2fs/f2fs.h | 2 +- fs/gfs2/bmap.c | 2 +- fs/iomap/buffered-io.c | 54 +++++++++++------------ include/linux/iomap.h | 2 +- include/linux/pagemap.h | 82 ++++++++++++++++++++++++++++++----- include/linux/uio.h | 9 +++- lib/iov_iter.c | 43 +++++++++++------- mm/filemap.c | 65 ++++++++++++++------------- mm/folio-compat.c | 2 +- mm/readahead.c | 13 ------ 13 files changed, 187 insertions(+), 110 deletions(-)