| Message ID | 20250117163001.2326672-2-tabba@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | KVM: Restricted mapping of guest_memfd at the host and arm64 support | expand |
On Fri, Jan 17, 2025 at 04:29:47PM +0000, Fuad Tabba wrote: > Some folio types, such as hugetlb, handle freeing their own > folios. Moreover, guest_memfd will require being notified once a > folio's reference count reaches 0 to facilitate shared to private > folio conversion, without the folio actually being freed at that > point. > > As a first step towards that, this patch consolidates freeing > folios that have a type. The first user is hugetlb folios. Later > in this patch series, guest_memfd will become the second user of > this. > > Suggested-by: David Hildenbrand <david@redhat.com> > Signed-off-by: Fuad Tabba <tabba@google.com> > --- > include/linux/page-flags.h | 15 +++++++++++++++ > mm/swap.c | 24 +++++++++++++++++++----- > 2 files changed, 34 insertions(+), 5 deletions(-) > > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > index 691506bdf2c5..6615f2f59144 100644 > --- a/include/linux/page-flags.h > +++ b/include/linux/page-flags.h > @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) > return page_mapcount_is_type(data_race(page->page_type)); > } > > +static inline int page_get_type(const struct page *page) > +{ > + return page->page_type >> 24; > +} > + > +static inline bool folio_has_type(const struct folio *folio) > +{ > + return page_has_type(&folio->page); > +} > + > +static inline int folio_get_type(const struct folio *folio) > +{ > + return page_get_type(&folio->page); > +} > + > #define FOLIO_TYPE_OPS(lname, fname) \ > static __always_inline bool folio_test_##fname(const struct folio *folio) \ > { \ > diff --git a/mm/swap.c b/mm/swap.c > index 10decd9dffa1..6f01b56bce13 100644 > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) > unlock_page_lruvec_irqrestore(lruvec, flags); > } > > +static void free_typed_folio(struct folio *folio) > +{ > + switch (folio_get_type(folio)) { I think you need: +#if IS_ENABLED(CONFIG_HUGETLBFS) > + case PGTY_hugetlb: > + free_huge_folio(folio); > + return; +#endif I think this worked before because folio_test_hugetlb was defined by: FOLIO_TEST_FLAG_FALSE(hugetlb) and evidently compiler optimizes out the free_huge_folio(folio) before linking. You'll probably want to do the same for the PGTY_guestmem in the later patch! > + case PGTY_offline: > + /* Nothing to do, it's offline. */ > + return; > + default: > + WARN_ON_ONCE(1); > + } > +} > + > void __folio_put(struct folio *folio) > { > if (unlikely(folio_is_zone_device(folio))) { > @@ -101,8 +115,8 @@ void __folio_put(struct folio *folio) > return; > } > > - if (folio_test_hugetlb(folio)) { > - free_huge_folio(folio); > + if (unlikely(folio_has_type(folio))) { > + free_typed_folio(folio); > return; > } > > @@ -934,13 +948,13 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) > if (!folio_ref_sub_and_test(folio, nr_refs)) > continue; > > - /* hugetlb has its own memcg */ > - if (folio_test_hugetlb(folio)) { > + if (unlikely(folio_has_type(folio))) { > + /* typed folios have their own memcg, if any */ > if (lruvec) { > unlock_page_lruvec_irqrestore(lruvec, flags); > lruvec = NULL; > } > - free_huge_folio(folio); > + free_typed_folio(folio); > continue; > } > folio_unqueue_deferred_split(folio); > -- > 2.48.0.rc2.279.g1de40edade-goog >
On Fri, 17 Jan 2025 at 22:05, Elliot Berman <elliot.berman@oss.qualcomm.com> wrote: > > On Fri, Jan 17, 2025 at 04:29:47PM +0000, Fuad Tabba wrote: > > Some folio types, such as hugetlb, handle freeing their own > > folios. Moreover, guest_memfd will require being notified once a > > folio's reference count reaches 0 to facilitate shared to private > > folio conversion, without the folio actually being freed at that > > point. > > > > As a first step towards that, this patch consolidates freeing > > folios that have a type. The first user is hugetlb folios. Later > > in this patch series, guest_memfd will become the second user of > > this. > > > > Suggested-by: David Hildenbrand <david@redhat.com> > > Signed-off-by: Fuad Tabba <tabba@google.com> > > --- > > include/linux/page-flags.h | 15 +++++++++++++++ > > mm/swap.c | 24 +++++++++++++++++++----- > > 2 files changed, 34 insertions(+), 5 deletions(-) > > > > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > > index 691506bdf2c5..6615f2f59144 100644 > > --- a/include/linux/page-flags.h > > +++ b/include/linux/page-flags.h > > @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) > > return page_mapcount_is_type(data_race(page->page_type)); > > } > > > > +static inline int page_get_type(const struct page *page) > > +{ > > + return page->page_type >> 24; > > +} > > + > > +static inline bool folio_has_type(const struct folio *folio) > > +{ > > + return page_has_type(&folio->page); > > +} > > + > > +static inline int folio_get_type(const struct folio *folio) > > +{ > > + return page_get_type(&folio->page); > > +} > > + > > #define FOLIO_TYPE_OPS(lname, fname) \ > > static __always_inline bool folio_test_##fname(const struct folio *folio) \ > > { \ > > diff --git a/mm/swap.c b/mm/swap.c > > index 10decd9dffa1..6f01b56bce13 100644 > > --- a/mm/swap.c > > +++ b/mm/swap.c > > @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) > > unlock_page_lruvec_irqrestore(lruvec, flags); > > } > > > > +static void free_typed_folio(struct folio *folio) > > +{ > > + switch (folio_get_type(folio)) { > > I think you need: > > +#if IS_ENABLED(CONFIG_HUGETLBFS) > > + case PGTY_hugetlb: > > + free_huge_folio(folio); > > + return; > +#endif > > I think this worked before because folio_test_hugetlb was defined by: > FOLIO_TEST_FLAG_FALSE(hugetlb) > and evidently compiler optimizes out the free_huge_folio(folio) before > linking. > > You'll probably want to do the same for the PGTY_guestmem in the later > patch! Thanks Elliot. This will keep the kernel test robot happy when I respin. Cheers, /fuad > > > + case PGTY_offline: > > + /* Nothing to do, it's offline. */ > > + return; > > + default: > > + WARN_ON_ONCE(1); > > + } > > +} > > + > > void __folio_put(struct folio *folio) > > { > > if (unlikely(folio_is_zone_device(folio))) { > > @@ -101,8 +115,8 @@ void __folio_put(struct folio *folio) > > return; > > } > > > > - if (folio_test_hugetlb(folio)) { > > - free_huge_folio(folio); > > + if (unlikely(folio_has_type(folio))) { > > + free_typed_folio(folio); > > return; > > } > > > > @@ -934,13 +948,13 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) > > if (!folio_ref_sub_and_test(folio, nr_refs)) > > continue; > > > > - /* hugetlb has its own memcg */ > > - if (folio_test_hugetlb(folio)) { > > + if (unlikely(folio_has_type(folio))) { > > + /* typed folios have their own memcg, if any */ > > if (lruvec) { > > unlock_page_lruvec_irqrestore(lruvec, flags); > > lruvec = NULL; > > } > > - free_huge_folio(folio); > > + free_typed_folio(folio); > > continue; > > } > > folio_unqueue_deferred_split(folio); > > -- > > 2.48.0.rc2.279.g1de40edade-goog > >
On 17.01.25 17:29, Fuad Tabba wrote: > Some folio types, such as hugetlb, handle freeing their own > folios. Moreover, guest_memfd will require being notified once a > folio's reference count reaches 0 to facilitate shared to private > folio conversion, without the folio actually being freed at that > point. > > As a first step towards that, this patch consolidates freeing > folios that have a type. The first user is hugetlb folios. Later > in this patch series, guest_memfd will become the second user of > this. > > Suggested-by: David Hildenbrand <david@redhat.com> > Signed-off-by: Fuad Tabba <tabba@google.com> > --- > include/linux/page-flags.h | 15 +++++++++++++++ > mm/swap.c | 24 +++++++++++++++++++----- > 2 files changed, 34 insertions(+), 5 deletions(-) > > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > index 691506bdf2c5..6615f2f59144 100644 > --- a/include/linux/page-flags.h > +++ b/include/linux/page-flags.h > @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) > return page_mapcount_is_type(data_race(page->page_type)); > } > > +static inline int page_get_type(const struct page *page) > +{ > + return page->page_type >> 24; > +} > + > +static inline bool folio_has_type(const struct folio *folio) > +{ > + return page_has_type(&folio->page); > +} > + > +static inline int folio_get_type(const struct folio *folio) > +{ > + return page_get_type(&folio->page); > +} > + > #define FOLIO_TYPE_OPS(lname, fname) \ > static __always_inline bool folio_test_##fname(const struct folio *folio) \ > { \ > diff --git a/mm/swap.c b/mm/swap.c > index 10decd9dffa1..6f01b56bce13 100644 > --- a/mm/swap.c > +++ b/mm/swap.c > @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) > unlock_page_lruvec_irqrestore(lruvec, flags); > } > > +static void free_typed_folio(struct folio *folio) > +{ > + switch (folio_get_type(folio)) { > + case PGTY_hugetlb: > + free_huge_folio(folio); > + return; > + case PGTY_offline: > + /* Nothing to do, it's offline. */ > + return; Please drop the PGTY_offline part for now, it was rather to highlight what could be done. But the real goal will be to not make offline pages use the refcount at all (frozen). If we really want the temporary PGTY_offline change, it should be introduced separately. Apart from that LGTM!
On 17.01.25 23:05, Elliot Berman wrote: > On Fri, Jan 17, 2025 at 04:29:47PM +0000, Fuad Tabba wrote: >> Some folio types, such as hugetlb, handle freeing their own >> folios. Moreover, guest_memfd will require being notified once a >> folio's reference count reaches 0 to facilitate shared to private >> folio conversion, without the folio actually being freed at that >> point. >> >> As a first step towards that, this patch consolidates freeing >> folios that have a type. The first user is hugetlb folios. Later >> in this patch series, guest_memfd will become the second user of >> this. >> >> Suggested-by: David Hildenbrand <david@redhat.com> >> Signed-off-by: Fuad Tabba <tabba@google.com> >> --- >> include/linux/page-flags.h | 15 +++++++++++++++ >> mm/swap.c | 24 +++++++++++++++++++----- >> 2 files changed, 34 insertions(+), 5 deletions(-) >> >> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h >> index 691506bdf2c5..6615f2f59144 100644 >> --- a/include/linux/page-flags.h >> +++ b/include/linux/page-flags.h >> @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) >> return page_mapcount_is_type(data_race(page->page_type)); >> } >> >> +static inline int page_get_type(const struct page *page) >> +{ >> + return page->page_type >> 24; >> +} >> + >> +static inline bool folio_has_type(const struct folio *folio) >> +{ >> + return page_has_type(&folio->page); >> +} >> + >> +static inline int folio_get_type(const struct folio *folio) >> +{ >> + return page_get_type(&folio->page); >> +} >> + >> #define FOLIO_TYPE_OPS(lname, fname) \ >> static __always_inline bool folio_test_##fname(const struct folio *folio) \ >> { \ >> diff --git a/mm/swap.c b/mm/swap.c >> index 10decd9dffa1..6f01b56bce13 100644 >> --- a/mm/swap.c >> +++ b/mm/swap.c >> @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) >> unlock_page_lruvec_irqrestore(lruvec, flags); >> } >> >> +static void free_typed_folio(struct folio *folio) >> +{ >> + switch (folio_get_type(folio)) { > > I think you need: > > +#if IS_ENABLED(CONFIG_HUGETLBFS) >> + case PGTY_hugetlb: >> + free_huge_folio(folio); >> + return; > +#endif > > I think this worked before because folio_test_hugetlb was defined by: > FOLIO_TEST_FLAG_FALSE(hugetlb) > and evidently compiler optimizes out the free_huge_folio(folio) before > linking. Likely, we should be using case PGTY_hugetlb: if(IF_ENABLED(CONFIG_HUGETLBFS)) free_huge_folio(folio); return: if possible (I assume so).
On Mon, 20 Jan 2025 at 10:39, David Hildenbrand <david@redhat.com> wrote: > > On 17.01.25 17:29, Fuad Tabba wrote: > > Some folio types, such as hugetlb, handle freeing their own > > folios. Moreover, guest_memfd will require being notified once a > > folio's reference count reaches 0 to facilitate shared to private > > folio conversion, without the folio actually being freed at that > > point. > > > > As a first step towards that, this patch consolidates freeing > > folios that have a type. The first user is hugetlb folios. Later > > in this patch series, guest_memfd will become the second user of > > this. > > > > Suggested-by: David Hildenbrand <david@redhat.com> > > Signed-off-by: Fuad Tabba <tabba@google.com> > > --- > > include/linux/page-flags.h | 15 +++++++++++++++ > > mm/swap.c | 24 +++++++++++++++++++----- > > 2 files changed, 34 insertions(+), 5 deletions(-) > > > > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > > index 691506bdf2c5..6615f2f59144 100644 > > --- a/include/linux/page-flags.h > > +++ b/include/linux/page-flags.h > > @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) > > return page_mapcount_is_type(data_race(page->page_type)); > > } > > > > +static inline int page_get_type(const struct page *page) > > +{ > > + return page->page_type >> 24; > > +} > > + > > +static inline bool folio_has_type(const struct folio *folio) > > +{ > > + return page_has_type(&folio->page); > > +} > > + > > +static inline int folio_get_type(const struct folio *folio) > > +{ > > + return page_get_type(&folio->page); > > +} > > + > > #define FOLIO_TYPE_OPS(lname, fname) \ > > static __always_inline bool folio_test_##fname(const struct folio *folio) \ > > { \ > > diff --git a/mm/swap.c b/mm/swap.c > > index 10decd9dffa1..6f01b56bce13 100644 > > --- a/mm/swap.c > > +++ b/mm/swap.c > > @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) > > unlock_page_lruvec_irqrestore(lruvec, flags); > > } > > > > +static void free_typed_folio(struct folio *folio) > > +{ > > + switch (folio_get_type(folio)) { > > + case PGTY_hugetlb: > > + free_huge_folio(folio); > > + return; > > + case PGTY_offline: > > + /* Nothing to do, it's offline. */ > > + return; > > Please drop the PGTY_offline part for now, it was rather to highlight > what could be done. Will do. Thanks, /fuad > > But the real goal will be to not make offline pages > use the refcount at all (frozen). > > If we really want the temporary PGTY_offline change, it should be > introduced separately. > > Apart from that LGTM! > > -- > Cheers, > > David / dhildenb >
On 1/20/25 11:39, David Hildenbrand wrote: > On 17.01.25 17:29, Fuad Tabba wrote: >> diff --git a/mm/swap.c b/mm/swap.c >> index 10decd9dffa1..6f01b56bce13 100644 >> --- a/mm/swap.c >> +++ b/mm/swap.c >> @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) >> unlock_page_lruvec_irqrestore(lruvec, flags); >> } >> >> +static void free_typed_folio(struct folio *folio) >> +{ >> + switch (folio_get_type(folio)) { >> + case PGTY_hugetlb: >> + free_huge_folio(folio); >> + return; >> + case PGTY_offline: >> + /* Nothing to do, it's offline. */ >> + return; > > Please drop the PGTY_offline part for now, it was rather to highlight > what could be done. > > But the real goal will be to not make offline pages > use the refcount at all (frozen). > > If we really want the temporary PGTY_offline change, it should be > introduced separately. > > Apart from that LGTM! I gues you mean the WARN_ON_ONCE(1) should be dropped from the default: handler as well, right? IIUC offline pages are not not yet frozen so there will be warnings otherwise. And I haven't check if the other types are frozen (I know slab is, very recently :)
On Mon, 20 Jan 2025 at 10:39, David Hildenbrand <david@redhat.com> wrote: > > On 17.01.25 23:05, Elliot Berman wrote: > > On Fri, Jan 17, 2025 at 04:29:47PM +0000, Fuad Tabba wrote: > >> Some folio types, such as hugetlb, handle freeing their own > >> folios. Moreover, guest_memfd will require being notified once a > >> folio's reference count reaches 0 to facilitate shared to private > >> folio conversion, without the folio actually being freed at that > >> point. > >> > >> As a first step towards that, this patch consolidates freeing > >> folios that have a type. The first user is hugetlb folios. Later > >> in this patch series, guest_memfd will become the second user of > >> this. > >> > >> Suggested-by: David Hildenbrand <david@redhat.com> > >> Signed-off-by: Fuad Tabba <tabba@google.com> > >> --- > >> include/linux/page-flags.h | 15 +++++++++++++++ > >> mm/swap.c | 24 +++++++++++++++++++----- > >> 2 files changed, 34 insertions(+), 5 deletions(-) > >> > >> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > >> index 691506bdf2c5..6615f2f59144 100644 > >> --- a/include/linux/page-flags.h > >> +++ b/include/linux/page-flags.h > >> @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) > >> return page_mapcount_is_type(data_race(page->page_type)); > >> } > >> > >> +static inline int page_get_type(const struct page *page) > >> +{ > >> + return page->page_type >> 24; > >> +} > >> + > >> +static inline bool folio_has_type(const struct folio *folio) > >> +{ > >> + return page_has_type(&folio->page); > >> +} > >> + > >> +static inline int folio_get_type(const struct folio *folio) > >> +{ > >> + return page_get_type(&folio->page); > >> +} > >> + > >> #define FOLIO_TYPE_OPS(lname, fname) \ > >> static __always_inline bool folio_test_##fname(const struct folio *folio) \ > >> { \ > >> diff --git a/mm/swap.c b/mm/swap.c > >> index 10decd9dffa1..6f01b56bce13 100644 > >> --- a/mm/swap.c > >> +++ b/mm/swap.c > >> @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) > >> unlock_page_lruvec_irqrestore(lruvec, flags); > >> } > >> > >> +static void free_typed_folio(struct folio *folio) > >> +{ > >> + switch (folio_get_type(folio)) { > > > > I think you need: > > > > +#if IS_ENABLED(CONFIG_HUGETLBFS) > >> + case PGTY_hugetlb: > >> + free_huge_folio(folio); > >> + return; > > +#endif > > > > I think this worked before because folio_test_hugetlb was defined by: > > FOLIO_TEST_FLAG_FALSE(hugetlb) > > and evidently compiler optimizes out the free_huge_folio(folio) before > > linking. > > Likely, we should be using > > case PGTY_hugetlb: > if(IF_ENABLED(CONFIG_HUGETLBFS)) > free_huge_folio(folio); > return: > > if possible (I assume so). Yes it does. I'll fix it. Cheers, /fuad > -- > Cheers, > > David / dhildenb >
On 1/20/25 11:43, Vlastimil Babka wrote: > On 1/20/25 11:39, David Hildenbrand wrote: >> On 17.01.25 17:29, Fuad Tabba wrote: >>> diff --git a/mm/swap.c b/mm/swap.c >>> index 10decd9dffa1..6f01b56bce13 100644 >>> --- a/mm/swap.c >>> +++ b/mm/swap.c >>> @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) >>> unlock_page_lruvec_irqrestore(lruvec, flags); >>> } >>> >>> +static void free_typed_folio(struct folio *folio) >>> +{ >>> + switch (folio_get_type(folio)) { >>> + case PGTY_hugetlb: >>> + free_huge_folio(folio); >>> + return; >>> + case PGTY_offline: >>> + /* Nothing to do, it's offline. */ >>> + return; >> >> Please drop the PGTY_offline part for now, it was rather to highlight >> what could be done. >> >> But the real goal will be to not make offline pages >> use the refcount at all (frozen). >> >> If we really want the temporary PGTY_offline change, it should be >> introduced separately. >> >> Apart from that LGTM! > > I gues you mean the WARN_ON_ONCE(1) should be dropped from the default: > handler as well, right? IIUC offline pages are not not yet frozen so there > will be warnings otherwise. And I haven't check if the other types are > frozen (I know slab is, very recently :) Oh and also free_typed_folio() would have to become bool and if it returns false, the normal freeing proceeds?
On 20.01.25 11:43, Vlastimil Babka wrote: > On 1/20/25 11:39, David Hildenbrand wrote: >> On 17.01.25 17:29, Fuad Tabba wrote: >>> diff --git a/mm/swap.c b/mm/swap.c >>> index 10decd9dffa1..6f01b56bce13 100644 >>> --- a/mm/swap.c >>> +++ b/mm/swap.c >>> @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) >>> unlock_page_lruvec_irqrestore(lruvec, flags); >>> } >>> >>> +static void free_typed_folio(struct folio *folio) >>> +{ >>> + switch (folio_get_type(folio)) { >>> + case PGTY_hugetlb: >>> + free_huge_folio(folio); >>> + return; >>> + case PGTY_offline: >>> + /* Nothing to do, it's offline. */ >>> + return; >> >> Please drop the PGTY_offline part for now, it was rather to highlight >> what could be done. >> >> But the real goal will be to not make offline pages >> use the refcount at all (frozen). >> >> If we really want the temporary PGTY_offline change, it should be >> introduced separately. >> >> Apart from that LGTM! > > I gues you mean the WARN_ON_ONCE(1) should be dropped from the default: > handler as well, right? IIUC offline pages are not not yet frozen so there > will be warnings otherwise. If we get offline pages here, it is unexpected and wrong. All users clear PG_offline before handing them back to the buddy. There is one nasty race in virtio-mem code for handling memory offlining with PG_offline pages, which we haven't seen so far in practice. See virtio_mem_fake_offline_going_offline()->page_ref_dec_and_test() for the nasty details. It would only trigger with some weird speculative references. The proper fix will be do leave the refcount frozen for them, so speculative refcount users will just fail. If we want to tackle that before we can do that, we should do it in a separate patch (and not buried in this series). And I haven't check if the other types are > frozen (I know slab is, very recently :) I think if we would have that, we would end up triggering the free_pages_prepare()->free_page_is_bad(), because page->_mapcount == folio->mapcount would not be -1 for typed folios. Right now we don't expect folios of specific types to ever get freed (in the future, these won't be folios anymore at all -- only guest_memfd and hugetlb would be folios, that need special care to be handed back to their pool).
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 691506bdf2c5..6615f2f59144 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -962,6 +962,21 @@ static inline bool page_has_type(const struct page *page) return page_mapcount_is_type(data_race(page->page_type)); } +static inline int page_get_type(const struct page *page) +{ + return page->page_type >> 24; +} + +static inline bool folio_has_type(const struct folio *folio) +{ + return page_has_type(&folio->page); +} + +static inline int folio_get_type(const struct folio *folio) +{ + return page_get_type(&folio->page); +} + #define FOLIO_TYPE_OPS(lname, fname) \ static __always_inline bool folio_test_##fname(const struct folio *folio) \ { \ diff --git a/mm/swap.c b/mm/swap.c index 10decd9dffa1..6f01b56bce13 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) unlock_page_lruvec_irqrestore(lruvec, flags); } +static void free_typed_folio(struct folio *folio) +{ + switch (folio_get_type(folio)) { + case PGTY_hugetlb: + free_huge_folio(folio); + return; + case PGTY_offline: + /* Nothing to do, it's offline. */ + return; + default: + WARN_ON_ONCE(1); + } +} + void __folio_put(struct folio *folio) { if (unlikely(folio_is_zone_device(folio))) { @@ -101,8 +115,8 @@ void __folio_put(struct folio *folio) return; } - if (folio_test_hugetlb(folio)) { - free_huge_folio(folio); + if (unlikely(folio_has_type(folio))) { + free_typed_folio(folio); return; } @@ -934,13 +948,13 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) if (!folio_ref_sub_and_test(folio, nr_refs)) continue; - /* hugetlb has its own memcg */ - if (folio_test_hugetlb(folio)) { + if (unlikely(folio_has_type(folio))) { + /* typed folios have their own memcg, if any */ if (lruvec) { unlock_page_lruvec_irqrestore(lruvec, flags); lruvec = NULL; } - free_huge_folio(folio); + free_typed_folio(folio); continue; } folio_unqueue_deferred_split(folio);
Some folio types, such as hugetlb, handle freeing their own folios. Moreover, guest_memfd will require being notified once a folio's reference count reaches 0 to facilitate shared to private folio conversion, without the folio actually being freed at that point. As a first step towards that, this patch consolidates freeing folios that have a type. The first user is hugetlb folios. Later in this patch series, guest_memfd will become the second user of this. Suggested-by: David Hildenbrand <david@redhat.com> Signed-off-by: Fuad Tabba <tabba@google.com> --- include/linux/page-flags.h | 15 +++++++++++++++ mm/swap.c | 24 +++++++++++++++++++----- 2 files changed, 34 insertions(+), 5 deletions(-)