| Message ID | 20241110152906.1747545-10-axboe@kernel.dk (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Uncached buffered IO | expand |
On Sun, Nov 10, 2024 at 08:28:01AM -0700, Jens Axboe wrote: > If the folio is marked as uncached, drop pages when writeback completes. > Intended to be used with RWF_UNCACHED, to avoid needing sync writes for > uncached IO. > > Signed-off-by: Jens Axboe <axboe@kernel.dk> > --- > mm/filemap.c | 23 +++++++++++++++++++++++ > 1 file changed, 23 insertions(+) > > diff --git a/mm/filemap.c b/mm/filemap.c > index bd698340ef24..efd02b047541 100644 > --- a/mm/filemap.c > +++ b/mm/filemap.c > @@ -1600,6 +1600,23 @@ int folio_wait_private_2_killable(struct folio *folio) > } > EXPORT_SYMBOL(folio_wait_private_2_killable); > > +/* > + * If folio was marked as uncached, then pages should be dropped when writeback > + * completes. Do that now. If we fail, it's likely because of a big folio - > + * just reset uncached for that case and latter completions should invalidate. > + */ > +static void folio_end_uncached(struct folio *folio) > +{ > + bool reset = true; > + > + if (folio_trylock(folio)) { > + reset = !invalidate_complete_folio2(folio->mapping, folio, 0); > + folio_unlock(folio); The same problem with folio_mapped() here. > + } > + if (reset) > + folio_set_uncached(folio); > +} > + > /** > * folio_end_writeback - End writeback against a folio. > * @folio: The folio.
diff --git a/mm/filemap.c b/mm/filemap.c index bd698340ef24..efd02b047541 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1600,6 +1600,23 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as uncached, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset uncached for that case and latter completions should invalidate. + */ +static void folio_end_uncached(struct folio *folio) +{ + bool reset = true; + + if (folio_trylock(folio)) { + reset = !invalidate_complete_folio2(folio->mapping, folio, 0); + folio_unlock(folio); + } + if (reset) + folio_set_uncached(folio); +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1610,6 +1627,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1631,9 +1650,13 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + folio_uncached = folio_test_clear_uncached(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_uncached) + folio_end_uncached(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback);
If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe <axboe@kernel.dk> --- mm/filemap.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)