[RFC,0/4] mm/memblock: Skip prep and initialization of struct pages freed later by HVO

Message

Usama Arif July 24, 2023, 1:46 p.m. UTC

If the region is for gigantic hugepages and if HVO is enabled, then those
struct pages which will be freed later by HVO don't need to be prepared and
initialized. This can save significant time when a large number of hugepages
are allocated at boot time.

For a 1G hugepage, this series avoid initialization and preparation of
262144 - 64 = 262080 struct pages per hugepage.

When tested on a 512G system (which can allocate max 500 1G hugepages), the
kexec-boot time with HVO and DEFERRED_STRUCT_PAGE_INIT enabled without this
patchseries to running init is 3.9 seconds. With this patch it is 1.2 seconds.
This represents an approximately 70% reduction in boot time and will
significantly reduce server downtime when using a large number of
gigantic pages.

Thanks,
Usama

Usama Arif (4):
  mm/hugetlb: Skip prep of tail pages when HVO is enabled
  mm/memblock: Add hugepage_size member to struct memblock_region
  mm/hugetlb_vmemmap: Use nid of the head page to reallocate it
  mm/memblock: Skip initialization of struct pages freed later by HVO

 arch/arm64/mm/kasan_init.c                   |  2 +-
 arch/powerpc/platforms/pasemi/iommu.c        |  2 +-
 arch/powerpc/platforms/pseries/setup.c       |  4 +-
 arch/powerpc/sysdev/dart_iommu.c             |  2 +-
 include/linux/memblock.h                     |  8 +-
 mm/cma.c                                     |  4 +-
 mm/hugetlb.c                                 | 36 +++++---
 mm/hugetlb_vmemmap.c                         |  6 +-
 mm/hugetlb_vmemmap.h                         |  4 +
 mm/memblock.c                                | 87 +++++++++++++-------
 mm/mm_init.c                                 |  2 +-
 mm/sparse-vmemmap.c                          |  2 +-
 tools/testing/memblock/tests/alloc_nid_api.c |  2 +-
 13 files changed, 106 insertions(+), 55 deletions(-)

Comments

Usama Arif July 26, 2023, 10:34 a.m. UTC | #1

On 24/07/2023 14:46, Usama Arif wrote:
> If the region is for gigantic hugepages and if HVO is enabled, then those
> struct pages which will be freed later by HVO don't need to be prepared and
> initialized. This can save significant time when a large number of hugepages
> are allocated at boot time.
> 
> For a 1G hugepage, this series avoid initialization and preparation of
> 262144 - 64 = 262080 struct pages per hugepage.
> 
> When tested on a 512G system (which can allocate max 500 1G hugepages), the
> kexec-boot time with HVO and DEFERRED_STRUCT_PAGE_INIT enabled without this
> patchseries to running init is 3.9 seconds. With this patch it is 1.2 seconds.
> This represents an approximately 70% reduction in boot time and will
> significantly reduce server downtime when using a large number of
> gigantic pages.

There were a few errors reported by kernel-bot if different config 
options were changed (CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP disabled, 
CONFIG_CMA enabled). I will fix these in the next revision if the 
general approach in the patches looks good to the community to start review.

Thanks,
Usama

> 
> Thanks,
> Usama
> 
> Usama Arif (4):
>    mm/hugetlb: Skip prep of tail pages when HVO is enabled
>    mm/memblock: Add hugepage_size member to struct memblock_region
>    mm/hugetlb_vmemmap: Use nid of the head page to reallocate it
>    mm/memblock: Skip initialization of struct pages freed later by HVO
> 
>   arch/arm64/mm/kasan_init.c                   |  2 +-
>   arch/powerpc/platforms/pasemi/iommu.c        |  2 +-
>   arch/powerpc/platforms/pseries/setup.c       |  4 +-
>   arch/powerpc/sysdev/dart_iommu.c             |  2 +-
>   include/linux/memblock.h                     |  8 +-
>   mm/cma.c                                     |  4 +-
>   mm/hugetlb.c                                 | 36 +++++---
>   mm/hugetlb_vmemmap.c                         |  6 +-
>   mm/hugetlb_vmemmap.h                         |  4 +
>   mm/memblock.c                                | 87 +++++++++++++-------
>   mm/mm_init.c                                 |  2 +-
>   mm/sparse-vmemmap.c                          |  2 +-
>   tools/testing/memblock/tests/alloc_nid_api.c |  2 +-
>   13 files changed, 106 insertions(+), 55 deletions(-)
>