| Message ID | 20210128164533.18566-1-david@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] mm/page_alloc: count CMA pages per zone and print them in /proc/zoneinfo | expand |
On Thu, Jan 28, 2021 at 05:45:33PM +0100, David Hildenbrand wrote: > Let's count the number of CMA pages per zone and print them in > /proc/zoneinfo. > > Having access to the total number of CMA pages per zone is helpful for > debugging purposes to know where exactly the CMA pages ended up, and to > figure out how many pages of a zone might behave differently, even after > some of these pages might already have been allocated. > > As one example, CMA pages part of a kernel zone cannot be used for > ordinary kernel allocations but instead behave more like ZONE_MOVABLE. > > For now, we are only able to get the global nr+free cma pages from > /proc/meminfo and the free cma pages per zone from /proc/zoneinfo. > > Example after this patch when booting a 6 GiB QEMU VM with > "hugetlb_cma=2G": > # cat /proc/zoneinfo | grep cma > cma 0 > nr_free_cma 0 > cma 0 > nr_free_cma 0 > cma 524288 > nr_free_cma 493016 > cma 0 > cma 0 > # cat /proc/meminfo | grep Cma > CmaTotal: 2097152 kB > CmaFree: 1972064 kB > > Note: We track/print only with CONFIG_CMA; "nr_free_cma" in /proc/zoneinfo > is currently also printed without CONFIG_CMA. > > Cc: Andrew Morton <akpm@linux-foundation.org> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org> > Cc: Mike Rapoport <rppt@kernel.org> > Cc: Oscar Salvador <osalvador@suse.de> > Cc: Michal Hocko <mhocko@kernel.org> > Cc: Wei Yang <richard.weiyang@linux.alibaba.com> > Cc: linux-api@vger.kernel.org > Signed-off-by: David Hildenbrand <david@redhat.com> IMHO looks better to me, thanks: Reviewed-by: Oscar Salvador <osalvador@suse.de> > --- > > v1 -> v2: > - Print/track only with CONFIG_CMA > - Extend patch description > > --- > include/linux/mmzone.h | 6 ++++++ > mm/page_alloc.c | 1 + > mm/vmstat.c | 5 +++++ > 3 files changed, 12 insertions(+) > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index ae588b2f87ef..27d22fb22e05 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -503,6 +503,9 @@ struct zone { > * bootmem allocator): > * managed_pages = present_pages - reserved_pages; > * > + * cma pages is present pages that are assigned for CMA use > + * (MIGRATE_CMA). > + * > * So present_pages may be used by memory hotplug or memory power > * management logic to figure out unmanaged pages by checking > * (present_pages - managed_pages). And managed_pages should be used > @@ -527,6 +530,9 @@ struct zone { > atomic_long_t managed_pages; > unsigned long spanned_pages; > unsigned long present_pages; > +#ifdef CONFIG_CMA > + unsigned long cma_pages; > +#endif > > const char *name; > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index b031a5ae0bd5..9a82375bbcb2 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -2168,6 +2168,7 @@ void __init init_cma_reserved_pageblock(struct page *page) > } > > adjust_managed_page_count(page, pageblock_nr_pages); > + page_zone(page)->cma_pages += pageblock_nr_pages; > } > #endif > > diff --git a/mm/vmstat.c b/mm/vmstat.c > index 7758486097f9..957680db41fa 100644 > --- a/mm/vmstat.c > +++ b/mm/vmstat.c > @@ -1650,6 +1650,11 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, > zone->spanned_pages, > zone->present_pages, > zone_managed_pages(zone)); > +#ifdef CONFIG_CMA > + seq_printf(m, > + "\n cma %lu", > + zone->cma_pages); > +#endif > > seq_printf(m, > "\n protection: (%ld", > -- > 2.29.2 >
On Thu, 28 Jan 2021, David Hildenbrand wrote: > diff --git a/mm/vmstat.c b/mm/vmstat.c > index 7758486097f9..957680db41fa 100644 > --- a/mm/vmstat.c > +++ b/mm/vmstat.c > @@ -1650,6 +1650,11 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, > zone->spanned_pages, > zone->present_pages, > zone_managed_pages(zone)); > +#ifdef CONFIG_CMA > + seq_printf(m, > + "\n cma %lu", > + zone->cma_pages); > +#endif > > seq_printf(m, > "\n protection: (%ld", Hmm, not sure about this. If cma is only printed for CONFIG_CMA, we can't distinguish between (1) a kernel without your patch without including some version checking and (2) a kernel without CONFIG_CMA enabled. IOW, "cma 0" carries value: we know immediately that we do not have any CMA pages on this zone, period. /proc/zoneinfo is also not known for its conciseness so I think printing "cma 0" even for !CONFIG_CMA is helpful :) I think this #ifdef should be removed and it should call into a zone_cma_pages(struct zone *zone) which returns 0UL if disabled.
> Am 28.01.2021 um 22:54 schrieb David Rientjes <rientjes@google.com>: > > On Thu, 28 Jan 2021, David Hildenbrand wrote: > >> diff --git a/mm/vmstat.c b/mm/vmstat.c >> index 7758486097f9..957680db41fa 100644 >> --- a/mm/vmstat.c >> +++ b/mm/vmstat.c >> @@ -1650,6 +1650,11 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, >> zone->spanned_pages, >> zone->present_pages, >> zone_managed_pages(zone)); >> +#ifdef CONFIG_CMA >> + seq_printf(m, >> + "\n cma %lu", >> + zone->cma_pages); >> +#endif >> >> seq_printf(m, >> "\n protection: (%ld", > > Hmm, not sure about this. If cma is only printed for CONFIG_CMA, we can't > distinguish between (1) a kernel without your patch without including some > version checking and (2) a kernel without CONFIG_CMA enabled. IOW, > "cma 0" carries value: we know immediately that we do not have any CMA > pages on this zone, period. > > /proc/zoneinfo is also not known for its conciseness so I think printing > "cma 0" even for !CONFIG_CMA is helpful :) > > I think this #ifdef should be removed and it should call into a > zone_cma_pages(struct zone *zone) which returns 0UL if disabled. > Yeah, that’s also what I proposed in a sub-thread here. The last option would be going the full mile and not printing nr_free_cma. Code might get a bit uglier though, but we could also remove that stats counter ;) I don‘t particularly care, while printing „0“ might be easier, removing nr_free_cma might be cleaner. But then, maybe there are tools that expect that value to be around on any kernel? Thoughts? Thanks
On Thu, 28 Jan 2021, David Hildenbrand wrote: > > On Thu, 28 Jan 2021, David Hildenbrand wrote: > > > >> diff --git a/mm/vmstat.c b/mm/vmstat.c > >> index 7758486097f9..957680db41fa 100644 > >> --- a/mm/vmstat.c > >> +++ b/mm/vmstat.c > >> @@ -1650,6 +1650,11 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, > >> zone->spanned_pages, > >> zone->present_pages, > >> zone_managed_pages(zone)); > >> +#ifdef CONFIG_CMA > >> + seq_printf(m, > >> + "\n cma %lu", > >> + zone->cma_pages); > >> +#endif > >> > >> seq_printf(m, > >> "\n protection: (%ld", > > > > Hmm, not sure about this. If cma is only printed for CONFIG_CMA, we can't > > distinguish between (1) a kernel without your patch without including some > > version checking and (2) a kernel without CONFIG_CMA enabled. IOW, > > "cma 0" carries value: we know immediately that we do not have any CMA > > pages on this zone, period. > > > > /proc/zoneinfo is also not known for its conciseness so I think printing > > "cma 0" even for !CONFIG_CMA is helpful :) > > > > I think this #ifdef should be removed and it should call into a > > zone_cma_pages(struct zone *zone) which returns 0UL if disabled. > > > > Yeah, that’s also what I proposed in a sub-thread here. > Ah, I certainly think your original intuition was correct. > The last option would be going the full mile and not printing nr_free_cma. Code might get a bit uglier though, but we could also remove that stats counter ;) > > I don‘t particularly care, while printing „0“ might be easier, removing nr_free_cma might be cleaner. > > But then, maybe there are tools that expect that value to be around on any kernel? > Yeah, that's probably undue risk, the ship has sailed and there's no significant upside. I still think "cma 0" in /proc/zoneinfo carries value, though, especially for NUMA and it looks like this is how it's done in linux-next. With a single read of the file, userspace can make the determination what CMA pages exist on this node. In general, I think the rule-of-thumb is that the fewer ifdefs in /proc/zoneinfo, the easier it is for userspace to parse it. (I made that change to /proc/zoneinfo to even print non-existant zones for each node because otherwise you cannot determine what the indices of things like vm.lowmem_reserve_ratio represent.)
> Am 28.01.2021 um 23:28 schrieb David Rientjes <rientjes@google.com>: > > On Thu, 28 Jan 2021, David Hildenbrand wrote: > >>> On Thu, 28 Jan 2021, David Hildenbrand wrote: >>> >>>> diff --git a/mm/vmstat.c b/mm/vmstat.c >>>> index 7758486097f9..957680db41fa 100644 >>>> --- a/mm/vmstat.c >>>> +++ b/mm/vmstat.c >>>> @@ -1650,6 +1650,11 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, >>>> zone->spanned_pages, >>>> zone->present_pages, >>>> zone_managed_pages(zone)); >>>> +#ifdef CONFIG_CMA >>>> + seq_printf(m, >>>> + "\n cma %lu", >>>> + zone->cma_pages); >>>> +#endif >>>> >>>> seq_printf(m, >>>> "\n protection: (%ld", >>> >>> Hmm, not sure about this. If cma is only printed for CONFIG_CMA, we can't >>> distinguish between (1) a kernel without your patch without including some >>> version checking and (2) a kernel without CONFIG_CMA enabled. IOW, >>> "cma 0" carries value: we know immediately that we do not have any CMA >>> pages on this zone, period. >>> >>> /proc/zoneinfo is also not known for its conciseness so I think printing >>> "cma 0" even for !CONFIG_CMA is helpful :) >>> >>> I think this #ifdef should be removed and it should call into a >>> zone_cma_pages(struct zone *zone) which returns 0UL if disabled. >>> >> >> Yeah, that’s also what I proposed in a sub-thread here. >> > > Ah, I certainly think your original intuition was correct. > >> The last option would be going the full mile and not printing nr_free_cma. Code might get a bit uglier though, but we could also remove that stats counter ;) >> >> I don‘t particularly care, while printing „0“ might be easier, removing nr_free_cma might be cleaner. >> >> But then, maybe there are tools that expect that value to be around on any kernel? >> > > Yeah, that's probably undue risk, the ship has sailed and there's no > significant upside. > > I still think "cma 0" in /proc/zoneinfo carries value, though, especially > for NUMA and it looks like this is how it's done in linux-next. With a > single read of the file, userspace can make the determination what CMA > pages exist on this node. > > In general, I think the rule-of-thumb is that the fewer ifdefs in > /proc/zoneinfo, the easier it is for userspace to parse it. Makes sense, I‘ll send an updated version tomorrow - thanks! > > (I made that change to /proc/zoneinfo to even print non-existant zones for > each node because otherwise you cannot determine what the indices of > things like vm.lowmem_reserve_ratio represent.)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index ae588b2f87ef..27d22fb22e05 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -503,6 +503,9 @@ struct zone { * bootmem allocator): * managed_pages = present_pages - reserved_pages; * + * cma pages is present pages that are assigned for CMA use + * (MIGRATE_CMA). + * * So present_pages may be used by memory hotplug or memory power * management logic to figure out unmanaged pages by checking * (present_pages - managed_pages). And managed_pages should be used @@ -527,6 +530,9 @@ struct zone { atomic_long_t managed_pages; unsigned long spanned_pages; unsigned long present_pages; +#ifdef CONFIG_CMA + unsigned long cma_pages; +#endif const char *name; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b031a5ae0bd5..9a82375bbcb2 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2168,6 +2168,7 @@ void __init init_cma_reserved_pageblock(struct page *page) } adjust_managed_page_count(page, pageblock_nr_pages); + page_zone(page)->cma_pages += pageblock_nr_pages; } #endif diff --git a/mm/vmstat.c b/mm/vmstat.c index 7758486097f9..957680db41fa 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1650,6 +1650,11 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, zone->spanned_pages, zone->present_pages, zone_managed_pages(zone)); +#ifdef CONFIG_CMA + seq_printf(m, + "\n cma %lu", + zone->cma_pages); +#endif seq_printf(m, "\n protection: (%ld",
Let's count the number of CMA pages per zone and print them in /proc/zoneinfo. Having access to the total number of CMA pages per zone is helpful for debugging purposes to know where exactly the CMA pages ended up, and to figure out how many pages of a zone might behave differently, even after some of these pages might already have been allocated. As one example, CMA pages part of a kernel zone cannot be used for ordinary kernel allocations but instead behave more like ZONE_MOVABLE. For now, we are only able to get the global nr+free cma pages from /proc/meminfo and the free cma pages per zone from /proc/zoneinfo. Example after this patch when booting a 6 GiB QEMU VM with "hugetlb_cma=2G": # cat /proc/zoneinfo | grep cma cma 0 nr_free_cma 0 cma 0 nr_free_cma 0 cma 524288 nr_free_cma 493016 cma 0 cma 0 # cat /proc/meminfo | grep Cma CmaTotal: 2097152 kB CmaFree: 1972064 kB Note: We track/print only with CONFIG_CMA; "nr_free_cma" in /proc/zoneinfo is currently also printed without CONFIG_CMA. Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org> Cc: Mike Rapoport <rppt@kernel.org> Cc: Oscar Salvador <osalvador@suse.de> Cc: Michal Hocko <mhocko@kernel.org> Cc: Wei Yang <richard.weiyang@linux.alibaba.com> Cc: linux-api@vger.kernel.org Signed-off-by: David Hildenbrand <david@redhat.com> --- v1 -> v2: - Print/track only with CONFIG_CMA - Extend patch description --- include/linux/mmzone.h | 6 ++++++ mm/page_alloc.c | 1 + mm/vmstat.c | 5 +++++ 3 files changed, 12 insertions(+)