| Message ID | 20240202161554.565023-1-zi.yan@sent.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Enable >0 order folio memory compaction | expand |
On Fri, Feb 02, 2024 at 11:15:51AM -0500, Zi Yan wrote: > From: Zi Yan <ziy@nvidia.com> > > Hi all, > > This patchset enables >0 order folio memory compaction, which is one of > the prerequisitions for large folio support[1]. > > [1] https://lore.kernel.org/linux-mm/20230912162815.440749-1-zi.yan@sent.com/ This URL started being referenced to your patch series instead of the rationale as to why this is important, and that is that compaction today skips pages with order > 0 and that this is already a problem for the page cache. The correct URL which you had in your *first* cover letter is: https://lore.kernel.org/linux-mm/f8d47176-03a8-99bf-a813-b5942830fd73@arm.com/ Luis
On 2 Feb 2024, at 14:55, Luis Chamberlain wrote: > On Fri, Feb 02, 2024 at 11:15:51AM -0500, Zi Yan wrote: >> From: Zi Yan <ziy@nvidia.com> >> >> Hi all, >> >> This patchset enables >0 order folio memory compaction, which is one of >> the prerequisitions for large folio support[1]. >> >> [1] https://lore.kernel.org/linux-mm/20230912162815.440749-1-zi.yan@sent.com/ > > This URL started being referenced to your patch series instead of the > rationale as to why this is important, and that is that compaction today > skips pages with order > 0 and that this is already a problem for the > page cache. The correct URL which you had in your *first* cover letter > is: > > https://lore.kernel.org/linux-mm/f8d47176-03a8-99bf-a813-b5942830fd73@arm.com/ You are right. Thank you for correcting it. -- Best Regards, Yan, Zi
On 2/3/2024 12:15 AM, Zi Yan wrote: > From: Zi Yan <ziy@nvidia.com> > > Hi all, > > This patchset enables >0 order folio memory compaction, which is one of > the prerequisitions for large folio support[1]. It includes the fix[4] for > V2 and is on top of mm-everything-2024-01-29-07-19. > > I am aware of that split free pages is necessary for folio > migration in compaction, since if >0 order free pages are never split > and no order-0 free page is scanned, compaction will end prematurely due > to migration returns -ENOMEM. Free page split becomes a must instead of > an optimization. > > lkp ncompare results for default LRU (-no-mglru) and CONFIG_LRU_GEN are > shown at the bottom (on a 8-CPU (Intel Xeon E5-2650 v4 @ 2.20GHz) 16G VM). > In sum, most of vm-scalability applications do not see performance change, > and the others see ~4% to ~26% performance boost under default LRU and > ~2% to ~6% performance boost under CONFIG_LRU_GEN. For the whole series, looks good to me. And I did not find any regression after running thpcompact. So feel free to add: Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Tested-by: Baolin Wang <baolin.wang@linux.alibaba.com>
On 5 Feb 2024, at 3:16, Baolin Wang wrote: > On 2/3/2024 12:15 AM, Zi Yan wrote: >> From: Zi Yan <ziy@nvidia.com> >> >> Hi all, >> >> This patchset enables >0 order folio memory compaction, which is one of >> the prerequisitions for large folio support[1]. It includes the fix[4] for >> V2 and is on top of mm-everything-2024-01-29-07-19. >> >> I am aware of that split free pages is necessary for folio >> migration in compaction, since if >0 order free pages are never split >> and no order-0 free page is scanned, compaction will end prematurely due >> to migration returns -ENOMEM. Free page split becomes a must instead of >> an optimization. >> >> lkp ncompare results for default LRU (-no-mglru) and CONFIG_LRU_GEN are >> shown at the bottom (on a 8-CPU (Intel Xeon E5-2650 v4 @ 2.20GHz) 16G VM). >> In sum, most of vm-scalability applications do not see performance change, >> and the others see ~4% to ~26% performance boost under default LRU and >> ~2% to ~6% performance boost under CONFIG_LRU_GEN. > > For the whole series, looks good to me. And I did not find any regression after running thpcompact. So feel free to add: > Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> > Tested-by: Baolin Wang <baolin.wang@linux.alibaba.com> Thank you for the review and testing. -- Best Regards, Yan, Zi
From: Zi Yan <ziy@nvidia.com> Hi all, This patchset enables >0 order folio memory compaction, which is one of the prerequisitions for large folio support[1]. It includes the fix[4] for V2 and is on top of mm-everything-2024-01-29-07-19. I am aware of that split free pages is necessary for folio migration in compaction, since if >0 order free pages are never split and no order-0 free page is scanned, compaction will end prematurely due to migration returns -ENOMEM. Free page split becomes a must instead of an optimization. lkp ncompare results for default LRU (-no-mglru) and CONFIG_LRU_GEN are shown at the bottom (on a 8-CPU (Intel Xeon E5-2650 v4 @ 2.20GHz) 16G VM). In sum, most of vm-scalability applications do not see performance change, and the others see ~4% to ~26% performance boost under default LRU and ~2% to ~6% performance boost under CONFIG_LRU_GEN. Changelog === From V2 [3]: 1. Added missing free page count in fast isolation path. This fixed the weird performance outcome. From V1 [2]: 1. Used folio_test_large() instead of folio_order() > 0. (per Matthew Wilcox) 2. Fixed code rebase error. (per Baolin Wang) 3. Used list_split_init() instead of list_split(). (per Ryan Boberts) 4. Added free_pages_prepare_fpi_none() to avoid duplicate free page code in compaction_free(). 5. Dropped source page order sorting patch. From RFC [1]: 1. Enabled >0 order folio compaction in the first patch by splitting all to-be-migrated folios. (per Huang, Ying) 2. Stopped isolating compound pages with order greater than cc->order to avoid wasting effort, since cc->order gives a hint that no free pages with order greater than it exist, thus migrating the compound pages will fail. (per Baolin Wang) 3. Retained the folio check within lru lock. (per Baolin Wang) 4. Made isolate_freepages_block() generate order-sorted multi lists. (per Johannes Weiner) Overview === To support >0 order folio compaction, the patchset changes how free pages used for migration are kept during compaction. Free pages used to be split into order-0 pages that are post allocation processed (i.e., PageBuddy flag cleared, page order stored in page->private is zeroed, and page reference is set to 1). Now all free pages are kept in a MAX_ORDER+1 array of page lists based on their order without post allocation process. When migrate_pages() asks for a new page, one of the free pages, based on the requested page order, is then processed and given out. Feel free to give comments and ask questions. Thanks. vm-scalability results on CONFIG_LRU_GEN === ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-xread-seq-mt/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 15107616 +3.2% 15590339 +1.3% 15297619 +3.0% 15567998 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-pread-seq/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 12611785 +1.8% 12832919 +0.9% 12724223 +1.6% 12812682 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-readtwice/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 9833393 +5.7% 10390190 +3.0% 10126606 +5.9% 10408804 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-mmap-read/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 7034709 ± 3% +2.9% 7241429 +3.2% 7256680 ± 2% +3.9% 7308375 vm-scalability.throughput vm-scalability results on default LRU (with -no-mglru suffix) === ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-xread-seq-mt/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 14401491 +3.7% 14940270 +2.4% 14748626 +4.0% 14975716 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-pread-seq/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 11407497 +5.1% 11989632 -0.5% 11349272 +4.8% 11957423 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-pread-seq-mt/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 11348474 +3.3% 11719453 -1.2% 11208759 +3.7% 11771926 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-readtwice/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 8065614 ± 3% +7.7% 8686626 ± 2% +5.0% 8467577 ± 4% +11.8% 9016077 ± 2% vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-mmap-read/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 6438422 ± 2% +27.5% 8206734 ± 2% +10.6% 7118390 +26.2% 8127192 ± 4% vm-scalability.throughput [1] https://lore.kernel.org/linux-mm/20230912162815.440749-1-zi.yan@sent.com/ [2] https://lore.kernel.org/linux-mm/20231113170157.280181-1-zi.yan@sent.com/ [3] https://lore.kernel.org/linux-mm/20240123034636.1095672-1-zi.yan@sent.com/ [4] https://lore.kernel.org/linux-mm/23BA8CC1-1014-4D09-9C33-938638E13C01@nvidia.com/ Zi Yan (3): mm/compaction: enable compacting >0 order folios. mm/compaction: add support for >0 order folio memory compaction. mm/compaction: optimize >0 order folio compaction with free page split. mm/compaction.c | 219 ++++++++++++++++++++++++++++++++++-------------- mm/internal.h | 9 +- mm/page_alloc.c | 6 ++ 3 files changed, 170 insertions(+), 64 deletions(-)