| Message ID | 20230403201839.4097845-1-zi.yan@sent.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Split a folio to any lower order folios | expand |
On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: > File folio supports any order and people would like to support flexible orders > for anonymous folio[1] too. Currently, split_huge_page() only splits a huge > page to order-0 pages, but splitting to orders higher than 0 is also useful. > This patchset adds support for splitting a huge page to any lower order pages > and uses it during file folio truncate operations. This series (and its v1 & v2) don't appear to have much in the way of detailed review. As it's at v3 and has been fairly stable I'll queue it up for some testing now, but I do ask that some reviewers go through it please.
On Tue, 4 Apr 2023, Andrew Morton wrote: > On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: > > > File folio supports any order and people would like to support flexible orders > > for anonymous folio[1] too. Currently, split_huge_page() only splits a huge > > page to order-0 pages, but splitting to orders higher than 0 is also useful. > > This patchset adds support for splitting a huge page to any lower order pages > > and uses it during file folio truncate operations. > > This series (and its v1 & v2) don't appear to have much in the way of > detailed review. As it's at v3 and has been fairly stable I'll queue > it up for some testing now, but I do ask that some reviewers go through > it please. Andrew, please don't let this series drift into 6.4-rc1. I've seen a bug or two (I'll point out in response to those patches), but overall I don't see what the justification for the series is: done because it could be done, it seems to me, but liable to add surprises. The cover letter says "splitting to orders higher than 0 is also useful", but it's not clear why; and the infrastructure provided seems unsuited to the one use provided - I'll say more on that truncation patch. Thanks, Hugh
On Sun, 16 Apr 2023 11:11:49 -0700 (PDT) Hugh Dickins <hughd@google.com> wrote: > On Tue, 4 Apr 2023, Andrew Morton wrote: > > On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: > > > > > File folio supports any order and people would like to support flexible orders > > > for anonymous folio[1] too. Currently, split_huge_page() only splits a huge > > > page to order-0 pages, but splitting to orders higher than 0 is also useful. > > > This patchset adds support for splitting a huge page to any lower order pages > > > and uses it during file folio truncate operations. > > > > This series (and its v1 & v2) don't appear to have much in the way of > > detailed review. As it's at v3 and has been fairly stable I'll queue > > it up for some testing now, but I do ask that some reviewers go through > > it please. > > Andrew, please don't let this series drift into 6.4-rc1. I have it still parked awaiting some reviewer input. > I've seen a bug or two (I'll point out in response to those patches), > but overall I don't see what the justification for the series is: done > because it could be done, it seems to me, but liable to add surprises. > > The cover letter says "splitting to orders higher than 0 is also useful", > but it's not clear why; and the infrastructure provided seems unsuited > to the one use provided - I'll say more on that truncation patch. OK, I'll drop the series for this cycle.
On 16.04.23 20:11, Hugh Dickins wrote: > On Tue, 4 Apr 2023, Andrew Morton wrote: >> On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: >> >>> File folio supports any order and people would like to support flexible orders >>> for anonymous folio[1] too. Currently, split_huge_page() only splits a huge >>> page to order-0 pages, but splitting to orders higher than 0 is also useful. >>> This patchset adds support for splitting a huge page to any lower order pages >>> and uses it during file folio truncate operations. >> >> This series (and its v1 & v2) don't appear to have much in the way of >> detailed review. As it's at v3 and has been fairly stable I'll queue >> it up for some testing now, but I do ask that some reviewers go through >> it please. > > Andrew, please don't let this series drift into 6.4-rc1. > > I've seen a bug or two (I'll point out in response to those patches), > but overall I don't see what the justification for the series is: done > because it could be done, it seems to me, but liable to add surprises. > > The cover letter says "splitting to orders higher than 0 is also useful", > but it's not clear why; and the infrastructure provided seems unsuited > to the one use provided - I'll say more on that truncation patch. I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without. Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation.
On 17 Apr 2023, at 10:20, David Hildenbrand wrote: > On 16.04.23 20:11, Hugh Dickins wrote: >> On Tue, 4 Apr 2023, Andrew Morton wrote: >>> On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: >>> >>>> File folio supports any order and people would like to support flexible orders >>>> for anonymous folio[1] too. Currently, split_huge_page() only splits a huge >>>> page to order-0 pages, but splitting to orders higher than 0 is also useful. >>>> This patchset adds support for splitting a huge page to any lower order pages >>>> and uses it during file folio truncate operations. >>> >>> This series (and its v1 & v2) don't appear to have much in the way of >>> detailed review. As it's at v3 and has been fairly stable I'll queue >>> it up for some testing now, but I do ask that some reviewers go through >>> it please. >> >> Andrew, please don't let this series drift into 6.4-rc1. >> >> I've seen a bug or two (I'll point out in response to those patches), >> but overall I don't see what the justification for the series is: done >> because it could be done, it seems to me, but liable to add surprises. >> >> The cover letter says "splitting to orders higher than 0 is also useful", >> but it's not clear why; and the infrastructure provided seems unsuited >> to the one use provided - I'll say more on that truncation patch. > > I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without. Ryan is working on large folio for anonymous pages and has shown promising performance results[1]. This patchset would avoid getting base pages during split if possible. > > Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation. Come on. 225 out of 365 insertions are self tests code. We need motivation to add testing code? [1] https://lore.kernel.org/linux-mm/20230414130303.2345383-1-ryan.roberts@arm.com/ -- Best Regards, Yan, Zi
On 17.04.23 21:26, Zi Yan wrote: > On 17 Apr 2023, at 10:20, David Hildenbrand wrote: > >> On 16.04.23 20:11, Hugh Dickins wrote: >>> On Tue, 4 Apr 2023, Andrew Morton wrote: >>>> On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: >>>> >>>>> File folio supports any order and people would like to support flexible orders >>>>> for anonymous folio[1] too. Currently, split_huge_page() only splits a huge >>>>> page to order-0 pages, but splitting to orders higher than 0 is also useful. >>>>> This patchset adds support for splitting a huge page to any lower order pages >>>>> and uses it during file folio truncate operations. >>>> >>>> This series (and its v1 & v2) don't appear to have much in the way of >>>> detailed review. As it's at v3 and has been fairly stable I'll queue >>>> it up for some testing now, but I do ask that some reviewers go through >>>> it please. >>> >>> Andrew, please don't let this series drift into 6.4-rc1. >>> >>> I've seen a bug or two (I'll point out in response to those patches), >>> but overall I don't see what the justification for the series is: done >>> because it could be done, it seems to me, but liable to add surprises. >>> >>> The cover letter says "splitting to orders higher than 0 is also useful", >>> but it's not clear why; and the infrastructure provided seems unsuited >>> to the one use provided - I'll say more on that truncation patch. >> >> I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without. > > Ryan is working on large folio for anonymous pages and has shown promising performance > results[1]. This patchset would avoid getting base pages during split if possible. > Yes, I know. And it would be great to get some actual numbers with/without your patches to show that this optimization actually matters in practice. Unrelated, your cover letter mentions "file folio truncate operations.". Would it also apply to FALLOC_FL_PUNCH_HOLE, when partially zapping a THP? >> >> Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation. > > Come on. 225 out of 365 insertions are self tests code. We need motivation to add > testing code? Well, if you add feature X and the tests target feature X, then certainly having the tests require the same motivation as feature X. But yeah, the actual kernel code change is smaller than it looks at first sight.
On 18 Apr 2023, at 6:29, David Hildenbrand wrote: > On 17.04.23 21:26, Zi Yan wrote: >> On 17 Apr 2023, at 10:20, David Hildenbrand wrote: >> >>> On 16.04.23 20:11, Hugh Dickins wrote: >>>> On Tue, 4 Apr 2023, Andrew Morton wrote: >>>>> On Mon, 3 Apr 2023 16:18:32 -0400 Zi Yan <zi.yan@sent.com> wrote: >>>>> >>>>>> File folio supports any order and people would like to support flexible orders >>>>>> for anonymous folio[1] too. Currently, split_huge_page() only splits a huge >>>>>> page to order-0 pages, but splitting to orders higher than 0 is also useful. >>>>>> This patchset adds support for splitting a huge page to any lower order pages >>>>>> and uses it during file folio truncate operations. >>>>> >>>>> This series (and its v1 & v2) don't appear to have much in the way of >>>>> detailed review. As it's at v3 and has been fairly stable I'll queue >>>>> it up for some testing now, but I do ask that some reviewers go through >>>>> it please. >>>> >>>> Andrew, please don't let this series drift into 6.4-rc1. >>>> >>>> I've seen a bug or two (I'll point out in response to those patches), >>>> but overall I don't see what the justification for the series is: done >>>> because it could be done, it seems to me, but liable to add surprises. >>>> >>>> The cover letter says "splitting to orders higher than 0 is also useful", >>>> but it's not clear why; and the infrastructure provided seems unsuited >>>> to the one use provided - I'll say more on that truncation patch. >>> >>> I agree. Maybe this patch set is something we want to have in the future once actual consumers that can benefit are in place, such that we can show actual performance numbers with/without. >> >> Ryan is working on large folio for anonymous pages and has shown promising performance >> results[1]. This patchset would avoid getting base pages during split if possible. >> > Yes, I know. And it would be great to get some actual numbers with/without your patches to show that this optimization actually matters in practice. Sure. Will try to add perf numbers in the next version. > > Unrelated, your cover letter mentions "file folio truncate operations.". Would it also apply to FALLOC_FL_PUNCH_HOLE, when partially zapping a THP? Yes. In the self tests, I have fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, offset[j], len[j]); and it uses the truncate operation I updated in patch 6. > >>> >>> Until then, "365 insertions(+), 68 deletions(-)" certainly needs some reasonable motivation. >> >> Come on. 225 out of 365 insertions are self tests code. We need motivation to add >> testing code? > > Well, if you add feature X and the tests target feature X, then certainly having the tests require the same motivation as feature X. But yeah, the actual kernel code change is smaller than it looks at first sight. > > -- > Thanks, > > David / dhildenb -- Best Regards, Yan, Zi
Hi Zi yan, > From: Zi Yan <ziy@nvidia.com> > > Hi all, > > File folio supports any order and people would like to support flexible orders > for anonymous folio[1] too. Currently, split_huge_page() only splits a huge > page to order-0 pages, but splitting to orders higher than 0 is also useful. > This patchset adds support for splitting a huge page to any lower order pages > and uses it during file folio truncate operations. > I recently posted patches to enable block size > page size(Large Block Sizes) in XFS[1]. The main idea of LBS is to have a notion of minimum order in the page cache that corresponds to the filesystem block size. Ability to split a folio based on a given order is something that would definitely optimize the LBS implementation. The current implementation refuses to split a large folio if it has a minimum order set in the page cache [2]. What we would like to have instead is to split it based on the minimum order. The main use is of course being able to free some folios during partial truncate operation. Your patch was also suggested by willy during our LPC talk[3]. I tried rebasing your patch and there were a lot of non-trivial conflicts. Is there any plans on sending a new version? [1] https://lore.kernel.org/linux-xfs/20240213093713.1753368-1-kernel@pankajraghav.com/ [2] https://lore.kernel.org/linux-xfs/20240213093713.1753368-9-kernel@pankajraghav.com/ [3] https://youtu.be/ar72r5Xf7x4?si=XDb-g7SSIgS-5TkP&t=1457 -- Pankaj
On 13 Feb 2024, at 7:30, Pankaj Raghav (Samsung) wrote: > Hi Zi yan, > >> From: Zi Yan <ziy@nvidia.com> >> >> Hi all, >> >> File folio supports any order and people would like to support flexible orders >> for anonymous folio[1] too. Currently, split_huge_page() only splits a huge >> page to order-0 pages, but splitting to orders higher than 0 is also useful. >> This patchset adds support for splitting a huge page to any lower order pages >> and uses it during file folio truncate operations. >> > > I recently posted patches to enable block size > page size(Large Block > Sizes) in XFS[1]. > The main idea of LBS is to have a notion of minimum order in the > page cache that corresponds to the filesystem block size. > > Ability to split a folio based on a given order is something that would > definitely optimize the LBS implementation. > > The current implementation refuses to split a large folio if it has a > minimum order set in the page cache [2]. What we would like to have instead > is to split it based on the minimum order. The main use is of course being > able to free some folios during partial truncate operation. > > Your patch was also suggested by willy during our LPC talk[3]. > > I tried rebasing your patch and there were a lot of non-trivial conflicts. > Is there any plans on sending a new version? Sure. I am going to rebase and send a new version out. > > > [1] https://lore.kernel.org/linux-xfs/20240213093713.1753368-1-kernel@pankajraghav.com/ > [2] https://lore.kernel.org/linux-xfs/20240213093713.1753368-9-kernel@pankajraghav.com/ > [3] https://youtu.be/ar72r5Xf7x4?si=XDb-g7SSIgS-5TkP&t=1457 > > -- > Pankaj -- Best Regards, Yan, Zi
From: Zi Yan <ziy@nvidia.com> Hi all, File folio supports any order and people would like to support flexible orders for anonymous folio[1] too. Currently, split_huge_page() only splits a huge page to order-0 pages, but splitting to orders higher than 0 is also useful. This patchset adds support for splitting a huge page to any lower order pages and uses it during file folio truncate operations. The patchset is on top of mm-everything-2023-03-27-21-20. Changelog === Since v2 --- 1. Fixed an issue in __split_page_owner() introduced during my rebase Since v1 --- 1. Changed split_page_memcg() and split_page_owner() parameter to use order 2. Used folio_test_pmd_mappable() in place of the equivalent code Details === * Patch 1 changes split_page_memcg() to use order instead of nr_pages * Patch 2 changes split_page_owner() to use order instead of nr_pages * Patch 3 and 4 add new_order parameter split_page_memcg() and split_page_owner() and prepare for upcoming changes. * Patch 5 adds split_huge_page_to_list_to_order() to split a huge page to any lower order. The original split_huge_page_to_list() calls split_huge_page_to_list_to_order() with new_order = 0. * Patch 6 uses split_huge_page_to_list_to_order() in large pagecache folio truncation instead of split the large folio all the way down to order-0. * Patch 7 adds a test API to debugfs and test cases in split_huge_page_test selftests. Comments and/or suggestions are welcome. [1] https://lore.kernel.org/linux-mm/Y%2FblF0GIunm+pRIC@casper.infradead.org/ Zi Yan (7): mm/memcg: use order instead of nr in split_page_memcg() mm/page_owner: use order instead of nr in split_page_owner() mm: memcg: make memcg huge page split support any order split. mm: page_owner: add support for splitting to any order in split page_owner. mm: thp: split huge page to any lower order pages. mm: truncate: split huge page cache page to a non-zero order if possible. mm: huge_memory: enable debugfs to split huge pages to any order. include/linux/huge_mm.h | 10 +- include/linux/memcontrol.h | 4 +- include/linux/page_owner.h | 10 +- mm/huge_memory.c | 137 ++++++++--- mm/memcontrol.c | 10 +- mm/page_alloc.c | 8 +- mm/page_owner.c | 8 +- mm/truncate.c | 21 +- .../selftests/mm/split_huge_page_test.c | 225 +++++++++++++++++- 9 files changed, 365 insertions(+), 68 deletions(-)