| Message ID | 20220303213252.28593-6-joao.m.martins@oracle.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | sparse-vmemmap: memory savings for compound devmaps (device-dax) | expand |
On Fri, Mar 4, 2022 at 5:33 AM Joao Martins <joao.m.martins@oracle.com> wrote: > > Currently memmap_init_zone_device() ends up initializing 32768 pages > when it only needs to initialize 128 given tail page reuse. That > number is worse with 1GB compound pages, 262144 instead of 128. Update > memmap_init_zone_device() to skip redundant initialization, detailed > below. > > When a pgmap @vmemmap_shift is set, all pages are mapped at a given > huge page alignment and use compound pages to describe them as opposed > to a struct per 4K. > > With @vmemmap_shift > 0 and when struct pages are stored in ram > (!altmap) most tail pages are reused. Consequently, the amount of > unique struct pages is a lot smaller than the total amount of struct > pages being mapped. > > The altmap path is left alone since it does not support memory savings > based on compound pages devmap. > > Signed-off-by: Joao Martins <joao.m.martins@oracle.com> Reviewed-by: Muchun Song <songmuchun@bytedance.com> But a nit below. > --- > mm/page_alloc.c | 17 ++++++++++++++++- > 1 file changed, 16 insertions(+), 1 deletion(-) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index e0c1e6bb09dd..e9282d043cca 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -6653,6 +6653,21 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, > } > } > > +/* > + * With compound page geometry and when struct pages are stored in ram most > + * tail pages are reused. Consequently, the amount of unique struct pages to > + * initialize is a lot smaller that the total amount of struct pages being > + * mapped. This is a paired / mild layering violation with explicit knowledge > + * of how the sparse_vmemmap internals handle compound pages in the lack > + * of an altmap. See vmemmap_populate_compound_pages(). > + */ > +static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap, > + unsigned long nr_pages) > +{ > + return is_power_of_2(sizeof(struct page)) && > + !altmap ? 2 * (PAGE_SIZE/sizeof(struct page)) : nr_pages; It is better to add spaces around that '/'.
On 3/4/22 03:27, Muchun Song wrote: > On Fri, Mar 4, 2022 at 5:33 AM Joao Martins <joao.m.martins@oracle.com> wrote: >> >> Currently memmap_init_zone_device() ends up initializing 32768 pages >> when it only needs to initialize 128 given tail page reuse. That >> number is worse with 1GB compound pages, 262144 instead of 128. Update >> memmap_init_zone_device() to skip redundant initialization, detailed >> below. >> >> When a pgmap @vmemmap_shift is set, all pages are mapped at a given >> huge page alignment and use compound pages to describe them as opposed >> to a struct per 4K. >> >> With @vmemmap_shift > 0 and when struct pages are stored in ram >> (!altmap) most tail pages are reused. Consequently, the amount of >> unique struct pages is a lot smaller than the total amount of struct >> pages being mapped. >> >> The altmap path is left alone since it does not support memory savings >> based on compound pages devmap. >> >> Signed-off-by: Joao Martins <joao.m.martins@oracle.com> > > Reviewed-by: Muchun Song <songmuchun@bytedance.com> > Thanks! > But a nit below. > >> --- >> mm/page_alloc.c | 17 ++++++++++++++++- >> 1 file changed, 16 insertions(+), 1 deletion(-) >> >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c >> index e0c1e6bb09dd..e9282d043cca 100644 >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -6653,6 +6653,21 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, >> } >> } >> >> +/* >> + * With compound page geometry and when struct pages are stored in ram most >> + * tail pages are reused. Consequently, the amount of unique struct pages to >> + * initialize is a lot smaller that the total amount of struct pages being >> + * mapped. This is a paired / mild layering violation with explicit knowledge >> + * of how the sparse_vmemmap internals handle compound pages in the lack >> + * of an altmap. See vmemmap_populate_compound_pages(). >> + */ >> +static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap, >> + unsigned long nr_pages) >> +{ >> + return is_power_of_2(sizeof(struct page)) && >> + !altmap ? 2 * (PAGE_SIZE/sizeof(struct page)) : nr_pages; > > It is better to add spaces around that '/'. /me nods
diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e0c1e6bb09dd..e9282d043cca 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6653,6 +6653,21 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, } } +/* + * With compound page geometry and when struct pages are stored in ram most + * tail pages are reused. Consequently, the amount of unique struct pages to + * initialize is a lot smaller that the total amount of struct pages being + * mapped. This is a paired / mild layering violation with explicit knowledge + * of how the sparse_vmemmap internals handle compound pages in the lack + * of an altmap. See vmemmap_populate_compound_pages(). + */ +static inline unsigned long compound_nr_pages(struct vmem_altmap *altmap, + unsigned long nr_pages) +{ + return is_power_of_2(sizeof(struct page)) && + !altmap ? 2 * (PAGE_SIZE/sizeof(struct page)) : nr_pages; +} + static void __ref memmap_init_compound(struct page *head, unsigned long head_pfn, unsigned long zone_idx, int nid, @@ -6717,7 +6732,7 @@ void __ref memmap_init_zone_device(struct zone *zone, continue; memmap_init_compound(page, pfn, zone_idx, nid, pgmap, - pfns_per_compound); + compound_nr_pages(altmap, pfns_per_compound)); } pr_info("%s initialised %lu pages in %ums\n", __func__,
Currently memmap_init_zone_device() ends up initializing 32768 pages when it only needs to initialize 128 given tail page reuse. That number is worse with 1GB compound pages, 262144 instead of 128. Update memmap_init_zone_device() to skip redundant initialization, detailed below. When a pgmap @vmemmap_shift is set, all pages are mapped at a given huge page alignment and use compound pages to describe them as opposed to a struct per 4K. With @vmemmap_shift > 0 and when struct pages are stored in ram (!altmap) most tail pages are reused. Consequently, the amount of unique struct pages is a lot smaller than the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pages devmap. Signed-off-by: Joao Martins <joao.m.martins@oracle.com> --- mm/page_alloc.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)