| Message ID | 20250404140657.29285-1-kernel@pankajraghav.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v3] docs: update THP admin guide about non-tmpfs filesystem support | expand |
On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: > From: Pankaj Raghav <p.raghav@samsung.com> > > THP support for non-tmpfs filesystem has been around for some time now. > Update the admin guide to reflect it. > > While we are at it, move FilePmdMapped to previous paragraph for clarity, > and clarify ShmemPmdMapped & ShmemHugePage. > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > Acked-by: David Hildenbrand <david@redhat.com> > --- > > Changes since v2: > - Address comment from Bagas Sanjaya > - Squash commits and Ack from David > > Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- > 1 file changed, 15 insertions(+), 7 deletions(-) > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > index dff8d5985f0f..f8aae64e38d0 100644 > --- a/Documentation/admin-guide/mm/transhuge.rst > +++ b/Documentation/admin-guide/mm/transhuge.rst > @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages > that supports the automatic promotion and demotion of page sizes and > without the shortcomings of hugetlbfs. > > -Currently THP only works for anonymous memory mappings and tmpfs/shmem. > -But in the future it can expand to other filesystems. > +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and > +filesystems that support large folios. That seems to allude that THP can be supported on filesystems that suppor large folios. I don't think we want to call that THP and that can confuse folks. Leaving "currently" also seems to indicate that there is more work to be done for THP for filesystems but that's not true as well. So how about something like: THP only works for anonymous memory mappings, and the tmpfs/shmem is the only filesystem to support it. The alternative to THP for other filesystems is to support large folios and with it you can end up using huge pages opportunistically. Luis
On 04.04.25 17:32, Luis Chamberlain wrote: > On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: >> From: Pankaj Raghav <p.raghav@samsung.com> >> >> THP support for non-tmpfs filesystem has been around for some time now. >> Update the admin guide to reflect it. >> >> While we are at it, move FilePmdMapped to previous paragraph for clarity, >> and clarify ShmemPmdMapped & ShmemHugePage. >> >> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> >> Acked-by: David Hildenbrand <david@redhat.com> >> --- >> >> Changes since v2: >> - Address comment from Bagas Sanjaya >> - Squash commits and Ack from David >> >> Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- >> 1 file changed, 15 insertions(+), 7 deletions(-) >> >> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst >> index dff8d5985f0f..f8aae64e38d0 100644 >> --- a/Documentation/admin-guide/mm/transhuge.rst >> +++ b/Documentation/admin-guide/mm/transhuge.rst >> @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages >> that supports the automatic promotion and demotion of page sizes and >> without the shortcomings of hugetlbfs. >> >> -Currently THP only works for anonymous memory mappings and tmpfs/shmem. >> -But in the future it can expand to other filesystems. >> +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and >> +filesystems that support large folios. > > That seems to allude that THP can be supported on filesystems > that suppor large folios. I don't think we want to call that THP > and that can confuse folks. Leaving "currently" also seems to > indicate that there is more work to be done for THP for filesystems > but that's not true as well. So how about something like: > > THP only works for anonymous memory mappings, and the tmpfs/shmem is the only > filesystem to support it. The alternative to THP for other filesystems is to > support large folios and with it you can end up using huge pages That makes things more complicated without a good reason. See CONFIG_READ_ONLY_THP_FOR_FS as an early usage of the term "THP" for stuff we have in the pagecache. (with large folios we now properly implement this concept, and support more than only PMD size)
On Fri, Apr 04, 2025 at 08:32:15AM -0700, Luis Chamberlain wrote: > On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: > > From: Pankaj Raghav <p.raghav@samsung.com> > > > > THP support for non-tmpfs filesystem has been around for some time now. > > Update the admin guide to reflect it. > > > > While we are at it, move FilePmdMapped to previous paragraph for clarity, > > and clarify ShmemPmdMapped & ShmemHugePage. > > > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > > Acked-by: David Hildenbrand <david@redhat.com> > > --- > > > > Changes since v2: > > - Address comment from Bagas Sanjaya > > - Squash commits and Ack from David > > > > Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- > > 1 file changed, 15 insertions(+), 7 deletions(-) > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > > index dff8d5985f0f..f8aae64e38d0 100644 > > --- a/Documentation/admin-guide/mm/transhuge.rst > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages > > that supports the automatic promotion and demotion of page sizes and > > without the shortcomings of hugetlbfs. > > > > -Currently THP only works for anonymous memory mappings and tmpfs/shmem. > > -But in the future it can expand to other filesystems. > > +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and > > +filesystems that support large folios. > > That seems to allude that THP can be supported on filesystems > that suppor large folios. I don't think we want to call that THP But we do allocate a THP in the page cache if we support large folios. See [1] where THP was supported through page cache. From what I understand, THP support was added first to the page cache and then large folios (orders in between) support came later. For example, you can mount XFS and do a simple dd as follows: $ dd if=/dev/random of=/mnt/hello bs=1G count=2 And if you monitor `FileHugePages` after dd is done, it would have filled up if THP is enabled. > and that can confuse folks. Leaving "currently" also seems to > indicate that there is more work to be done for THP for filesystems > but that's not true as well. So how about something like: There is more work to be done in the filesystems right? Adding large folio support is the prerequisite here to get the THP support. > > THP only works for anonymous memory mappings, and the tmpfs/shmem is the only > filesystem to support it. The alternative to THP for other filesystems is to > support large folios and with it you can end up using huge pages > opportunistically. > > Luis [1] https://lore.kernel.org/linux-mm/20201029193405.29125-1-willy@infradead.org/ -- Pankaj
On Fri, Apr 04, 2025 at 06:18:12PM +0200, David Hildenbrand wrote: > On 04.04.25 17:32, Luis Chamberlain wrote: > > On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: > > > From: Pankaj Raghav <p.raghav@samsung.com> > > > > > > THP support for non-tmpfs filesystem has been around for some time now. > > > Update the admin guide to reflect it. > > > > > > While we are at it, move FilePmdMapped to previous paragraph for clarity, > > > and clarify ShmemPmdMapped & ShmemHugePage. > > > > > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > > > Acked-by: David Hildenbrand <david@redhat.com> > > > --- > > > > > > Changes since v2: > > > - Address comment from Bagas Sanjaya > > > - Squash commits and Ack from David > > > > > > Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- > > > 1 file changed, 15 insertions(+), 7 deletions(-) > > > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > > > index dff8d5985f0f..f8aae64e38d0 100644 > > > --- a/Documentation/admin-guide/mm/transhuge.rst > > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > > @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages > > > that supports the automatic promotion and demotion of page sizes and > > > without the shortcomings of hugetlbfs. > > > -Currently THP only works for anonymous memory mappings and tmpfs/shmem. > > > -But in the future it can expand to other filesystems. > > > +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and > > > +filesystems that support large folios. > > > > That seems to allude that THP can be supported on filesystems > > that suppor large folios. I don't think we want to call that THP > > and that can confuse folks. Leaving "currently" also seems to > > indicate that there is more work to be done for THP for filesystems > > but that's not true as well. So how about something like: > > > > THP only works for anonymous memory mappings, and the tmpfs/shmem is the only > > filesystem to support it. The alternative to THP for other filesystems is to > > support large folios and with it you can end up using huge pages > > That makes things more complicated without a good reason. > > See CONFIG_READ_ONLY_THP_FOR_FS as an early usage of the term "THP" for > stuff we have in the pagecache. OK. > (with large folios we now properly implement > this concept, and support more than only PMD size) Do we really want to call large folio support on filesystems THP? Luis
On Fri, Apr 04, 2025 at 06:31:16PM +0200, Pankaj Raghav (Samsung) wrote: > On Fri, Apr 04, 2025 at 08:32:15AM -0700, Luis Chamberlain wrote: > > On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: > > > From: Pankaj Raghav <p.raghav@samsung.com> > > > > > > THP support for non-tmpfs filesystem has been around for some time now. > > > Update the admin guide to reflect it. > > > > > > While we are at it, move FilePmdMapped to previous paragraph for clarity, > > > and clarify ShmemPmdMapped & ShmemHugePage. > > > > > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > > > Acked-by: David Hildenbrand <david@redhat.com> > > > --- > > > > > > Changes since v2: > > > - Address comment from Bagas Sanjaya > > > - Squash commits and Ack from David > > > > > > Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- > > > 1 file changed, 15 insertions(+), 7 deletions(-) > > > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > > > index dff8d5985f0f..f8aae64e38d0 100644 > > > --- a/Documentation/admin-guide/mm/transhuge.rst > > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > > @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages > > > that supports the automatic promotion and demotion of page sizes and > > > without the shortcomings of hugetlbfs. > > > > > > -Currently THP only works for anonymous memory mappings and tmpfs/shmem. > > > -But in the future it can expand to other filesystems. > > > +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and > > > +filesystems that support large folios. > > > > That seems to allude that THP can be supported on filesystems > > that suppor large folios. I don't think we want to call that THP > > But we do allocate a THP in the page cache if we support large folios. > > See [1] where THP was supported through page cache. From what I > understand, THP support was added first to the page cache and then large > folios (orders in between) support came later. I see, yes Do we want to clarify this further? > For example, you can mount XFS and do a simple dd as follows: > > $ dd if=/dev/random of=/mnt/hello bs=1G count=2 > > And if you monitor `FileHugePages` after dd is done, it would have filled > up if THP is enabled. > > > and that can confuse folks. Leaving "currently" also seems to > > indicate that there is more work to be done for THP for filesystems > > but that's not true as well. So how about something like: > > There is more work to be done in the filesystems right? Adding large > folio support is the prerequisite here to get the THP support. This would be good to clarify as well. Luis
On 04.04.25 19:58, Luis Chamberlain wrote: > On Fri, Apr 04, 2025 at 06:18:12PM +0200, David Hildenbrand wrote: >> On 04.04.25 17:32, Luis Chamberlain wrote: >>> On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: >>>> From: Pankaj Raghav <p.raghav@samsung.com> >>>> >>>> THP support for non-tmpfs filesystem has been around for some time now. >>>> Update the admin guide to reflect it. >>>> >>>> While we are at it, move FilePmdMapped to previous paragraph for clarity, >>>> and clarify ShmemPmdMapped & ShmemHugePage. >>>> >>>> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> >>>> Acked-by: David Hildenbrand <david@redhat.com> >>>> --- >>>> >>>> Changes since v2: >>>> - Address comment from Bagas Sanjaya >>>> - Squash commits and Ack from David >>>> >>>> Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- >>>> 1 file changed, 15 insertions(+), 7 deletions(-) >>>> >>>> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst >>>> index dff8d5985f0f..f8aae64e38d0 100644 >>>> --- a/Documentation/admin-guide/mm/transhuge.rst >>>> +++ b/Documentation/admin-guide/mm/transhuge.rst >>>> @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages >>>> that supports the automatic promotion and demotion of page sizes and >>>> without the shortcomings of hugetlbfs. >>>> -Currently THP only works for anonymous memory mappings and tmpfs/shmem. >>>> -But in the future it can expand to other filesystems. >>>> +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and >>>> +filesystems that support large folios. >>> >>> That seems to allude that THP can be supported on filesystems >>> that suppor large folios. I don't think we want to call that THP >>> and that can confuse folks. Leaving "currently" also seems to >>> indicate that there is more work to be done for THP for filesystems >>> but that's not true as well. So how about something like: >>> >>> THP only works for anonymous memory mappings, and the tmpfs/shmem is the only >>> filesystem to support it. The alternative to THP for other filesystems is to >>> support large folios and with it you can end up using huge pages >> >> That makes things more complicated without a good reason. >> >> See CONFIG_READ_ONLY_THP_FOR_FS as an early usage of the term "THP" for >> stuff we have in the pagecache. > > OK. > >> (with large folios we now properly implement >> this concept, and support more than only PMD size) > > Do we really want to call large folio support on filesystems THP? Good question. "folio" is just the metadata we currently use to manage a chunk of memory, and a "large folio" is one that spans more than a single page -- huge page, large page, super page, ... in the past the metadata for that used to be a complicated piece of "compound page". In the future, we might call it differently (struct file_mem ?), who knows. So "large folio" support in a fs allows for the usage of these larger chunks of memory (huge pages). The filesystem tries to transparently make use of huge pages (of various sizes); in contrast to hugetlb, that is "transparent" in a way that these can be split, partially mapped, partially truncated -- just like for anon or shmem. (Well, the "transparent" part was never really 100% true, but that's a different story) I see more similarity with THP than with hugetlb in that sense. In the end I don't think the terminology matters that much -- note that we are talking about an admin guide that resides in transhuge.rst -- as long as people don't confuse it with "ordinary" huge pages as in hugetlb. "file huge page" is rather misleading. I'd be curious in which sense these file thingies are different than what we have for anon/shmem/... except that some toggles don't apply to them and the pagecache mostly "more transparently" makes use of them. (which makes them even more transparent? ;) )
On 04.04.25 16:06, Pankaj Raghav (Samsung) wrote: > From: Pankaj Raghav <p.raghav@samsung.com> > > THP support for non-tmpfs filesystem has been around for some time now. > Update the admin guide to reflect it. > > While we are at it, move FilePmdMapped to previous paragraph for clarity, > and clarify ShmemPmdMapped & ShmemHugePage. > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > Acked-by: David Hildenbrand <david@redhat.com> > --- BTW, we should go over the whole document and bring it up to date. Interestingly, it starts with "Performance critical computing applications dealing with large memory working sets are already running on top of libhugetlbfs and in turn hugetlbfs. Transparent HugePage Support (THP) is an alternative mean of using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. Currently THP only works for anonymous memory mappings and tmpfs/shmem. But in the future it can expand to other filesystems."
On Fri, Apr 04, 2025 at 09:07:23PM +0100, David Hildenbrand wrote: > On 04.04.25 19:58, Luis Chamberlain wrote: > > On Fri, Apr 04, 2025 at 06:18:12PM +0200, David Hildenbrand wrote: > > > On 04.04.25 17:32, Luis Chamberlain wrote: > > > > On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: > > > > > From: Pankaj Raghav <p.raghav@samsung.com> > > > > > > > > > > THP support for non-tmpfs filesystem has been around for some time now. > > > > > Update the admin guide to reflect it. > > > > > > > > > > While we are at it, move FilePmdMapped to previous paragraph for clarity, > > > > > and clarify ShmemPmdMapped & ShmemHugePage. > > > > > > > > > > Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> > > > > > Acked-by: David Hildenbrand <david@redhat.com> > > > > > --- > > > > > > > > > > Changes since v2: > > > > > - Address comment from Bagas Sanjaya > > > > > - Squash commits and Ack from David > > > > > > > > > > Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- > > > > > 1 file changed, 15 insertions(+), 7 deletions(-) > > > > > > > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > > > > > index dff8d5985f0f..f8aae64e38d0 100644 > > > > > --- a/Documentation/admin-guide/mm/transhuge.rst > > > > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > > > > @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages > > > > > that supports the automatic promotion and demotion of page sizes and > > > > > without the shortcomings of hugetlbfs. > > > > > -Currently THP only works for anonymous memory mappings and tmpfs/shmem. > > > > > -But in the future it can expand to other filesystems. > > > > > +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and > > > > > +filesystems that support large folios. > > > > > > > > That seems to allude that THP can be supported on filesystems > > > > that suppor large folios. I don't think we want to call that THP > > > > and that can confuse folks. Leaving "currently" also seems to > > > > indicate that there is more work to be done for THP for filesystems > > > > but that's not true as well. So how about something like: > > > > > > > > THP only works for anonymous memory mappings, and the tmpfs/shmem is the only > > > > filesystem to support it. The alternative to THP for other filesystems is to > > > > support large folios and with it you can end up using huge pages > > > > > > That makes things more complicated without a good reason. > > > > > > See CONFIG_READ_ONLY_THP_FOR_FS as an early usage of the term "THP" for > > > stuff we have in the pagecache. > > > > OK. > > > > > (with large folios we now properly implement > > > this concept, and support more than only PMD size) > > > > Do we really want to call large folio support on filesystems THP? > > Good question. > > "folio" is just the metadata we currently use to manage a chunk of memory, > and a "large folio" is one that spans more than a single page -- huge page, > large page, super page, ... in the past the metadata for that used to be a > complicated piece of "compound page". In the future, we might call it > differently (struct file_mem ?), who knows. > > So "large folio" support in a fs allows for the usage of these larger chunks > of memory (huge pages). I'm a bit confused here. I thought the term 'huge pages' referred to specific page table level like PMD, PUD, or PGD (and not PTE). And "large folio" term can span up to anything (currently up to PMD). Could you clarify if I'm misunderstanding something?
On 04.04.25 21:44, Daniel Gomez wrote: > On Fri, Apr 04, 2025 at 09:07:23PM +0100, David Hildenbrand wrote: >> On 04.04.25 19:58, Luis Chamberlain wrote: >>> On Fri, Apr 04, 2025 at 06:18:12PM +0200, David Hildenbrand wrote: >>>> On 04.04.25 17:32, Luis Chamberlain wrote: >>>>> On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: >>>>>> From: Pankaj Raghav <p.raghav@samsung.com> >>>>>> >>>>>> THP support for non-tmpfs filesystem has been around for some time now. >>>>>> Update the admin guide to reflect it. >>>>>> >>>>>> While we are at it, move FilePmdMapped to previous paragraph for clarity, >>>>>> and clarify ShmemPmdMapped & ShmemHugePage. >>>>>> >>>>>> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com> >>>>>> Acked-by: David Hildenbrand <david@redhat.com> >>>>>> --- >>>>>> >>>>>> Changes since v2: >>>>>> - Address comment from Bagas Sanjaya >>>>>> - Squash commits and Ack from David >>>>>> >>>>>> Documentation/admin-guide/mm/transhuge.rst | 22 +++++++++++++++------- >>>>>> 1 file changed, 15 insertions(+), 7 deletions(-) >>>>>> >>>>>> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst >>>>>> index dff8d5985f0f..f8aae64e38d0 100644 >>>>>> --- a/Documentation/admin-guide/mm/transhuge.rst >>>>>> +++ b/Documentation/admin-guide/mm/transhuge.rst >>>>>> @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages >>>>>> that supports the automatic promotion and demotion of page sizes and >>>>>> without the shortcomings of hugetlbfs. >>>>>> -Currently THP only works for anonymous memory mappings and tmpfs/shmem. >>>>>> -But in the future it can expand to other filesystems. >>>>>> +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and >>>>>> +filesystems that support large folios. >>>>> >>>>> That seems to allude that THP can be supported on filesystems >>>>> that suppor large folios. I don't think we want to call that THP >>>>> and that can confuse folks. Leaving "currently" also seems to >>>>> indicate that there is more work to be done for THP for filesystems >>>>> but that's not true as well. So how about something like: >>>>> >>>>> THP only works for anonymous memory mappings, and the tmpfs/shmem is the only >>>>> filesystem to support it. The alternative to THP for other filesystems is to >>>>> support large folios and with it you can end up using huge pages >>>> >>>> That makes things more complicated without a good reason. >>>> >>>> See CONFIG_READ_ONLY_THP_FOR_FS as an early usage of the term "THP" for >>>> stuff we have in the pagecache. >>> >>> OK. >>> >>>> (with large folios we now properly implement >>>> this concept, and support more than only PMD size) >>> >>> Do we really want to call large folio support on filesystems THP? >> >> Good question. >> >> "folio" is just the metadata we currently use to manage a chunk of memory, >> and a "large folio" is one that spans more than a single page -- huge page, >> large page, super page, ... in the past the metadata for that used to be a >> complicated piece of "compound page". In the future, we might call it >> differently (struct file_mem ?), who knows. >> >> So "large folio" support in a fs allows for the usage of these larger chunks >> of memory (huge pages). > > I'm a bit confused here. I thought the term 'huge pages' referred to specific > page table level like PMD, PUD, or PGD (and not PTE). And "large folio" term can > span up to anything (currently up to PMD). Could you clarify if I'm > misunderstanding something? The transhuge.rst is still a bit inconsistent and confusing, but we started generalizing the concept to special-case "Traditional THPs" to be PMD-sized THPs. See the description about mTHP in the document. If a small folio represents a single page, then a large folio represents multiple pages (large/huge/super page). But again, "folio" is the metadata we use to manage this chunk of memory. Just like we didn't call a "huge page" a "compound page" back in the days; the "compound page" was a way to manage the metadata assigned to a "larger chunk of memory covering multiple base pages" as one unit. I once had a longer writeup about all that (including how hugetlb supports different huge page sizes, how freebsd calls these thing super pages etc). This topic seems to pop every 6 months, likely we really should do a better job at documenting all that.
On Fri, Apr 04, 2025 at 04:06:57PM +0200, Pankaj Raghav (Samsung) wrote: > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > index dff8d5985f0f..f8aae64e38d0 100644 > --- a/Documentation/admin-guide/mm/transhuge.rst > +++ b/Documentation/admin-guide/mm/transhuge.rst > @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages > that supports the automatic promotion and demotion of page sizes and > without the shortcomings of hugetlbfs. > > -Currently THP only works for anonymous memory mappings and tmpfs/shmem. > -But in the future it can expand to other filesystems. > +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and > +filesystems that support large folios. > > .. note:: > in the examples below we presume that the basic page size is 4K and > @@ -463,11 +463,19 @@ fields for each mapping. (Note that AnonHugePages only applies to traditional > PMD-sized THP for historical reasons and should have been called > AnonHugePmdMapped). > > -The number of file transparent huge pages mapped to userspace is available > -by reading ShmemPmdMapped and ShmemHugePages fields in ``/proc/meminfo``. > -To identify what applications are mapping file transparent huge pages, it > -is necessary to read ``/proc/PID/smaps`` and count the FilePmdMapped fields > -for each mapping. > +The number of PMD-sized transparent huge pages currently used by > +filesystem data (page cache) is available by reading the FileHugePages field > +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace > +is available by reading FilePmdMapped field in ``proc/meminfo``. To identify > +what applications are mapping these huge pages, it is necessary to read > +``/proc/PID/smaps`` and count the FilePmdMapped fields for each mapping. > + > +In similar fashion, the number of PMD-sized transparent huge pages currently > +used by tmpfs/shmem is available by reading the ShmemHugePages field > +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace > +is available by reading ShmemPmdMapped field in ``proc/meminfo``. To identify > +what applications are mapping these huge pages, it is necessary to read > +``/proc/PID/smaps`` and count the ShmemPmdMapped fields for each mapping. > > Note that reading the smaps file is expensive and reading it > frequently will incur overhead. > Looks good, thanks! Reviewed-by: Bagas Sanjaya <bagasdotme@gmail.com>
diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index dff8d5985f0f..f8aae64e38d0 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -12,8 +12,8 @@ using huge pages for the backing of virtual memory with huge pages that supports the automatic promotion and demotion of page sizes and without the shortcomings of hugetlbfs. -Currently THP only works for anonymous memory mappings and tmpfs/shmem. -But in the future it can expand to other filesystems. +Currently, THP only works for anonymous memory mappings, tmpfs/shmem and +filesystems that support large folios. .. note:: in the examples below we presume that the basic page size is 4K and @@ -463,11 +463,19 @@ fields for each mapping. (Note that AnonHugePages only applies to traditional PMD-sized THP for historical reasons and should have been called AnonHugePmdMapped). -The number of file transparent huge pages mapped to userspace is available -by reading ShmemPmdMapped and ShmemHugePages fields in ``/proc/meminfo``. -To identify what applications are mapping file transparent huge pages, it -is necessary to read ``/proc/PID/smaps`` and count the FilePmdMapped fields -for each mapping. +The number of PMD-sized transparent huge pages currently used by +filesystem data (page cache) is available by reading the FileHugePages field +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace +is available by reading FilePmdMapped field in ``proc/meminfo``. To identify +what applications are mapping these huge pages, it is necessary to read +``/proc/PID/smaps`` and count the FilePmdMapped fields for each mapping. + +In similar fashion, the number of PMD-sized transparent huge pages currently +used by tmpfs/shmem is available by reading the ShmemHugePages field +in ``/proc/meminfo``. The number of these huge pages that are mapped to userspace +is available by reading ShmemPmdMapped field in ``proc/meminfo``. To identify +what applications are mapping these huge pages, it is necessary to read +``/proc/PID/smaps`` and count the ShmemPmdMapped fields for each mapping. Note that reading the smaps file is expensive and reading it frequently will incur overhead.