| Message ID | 20240811224940.39876-1-21cnbao@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | mm: collect the number of anon mTHP | expand |
Hi Andrew, David, Usama, I'm attempting to rebase this series on top of Usama's [PATCH v3 0/6] mm: split underutilized THPs[1] However, I feel it is impossible and we might be tackling things in the wrong order. This series should probably come before Usama's one, as the new partially_mapped should be ok to handle both nr_split_deferred and split_deferred. That said, the new partially_mapped still has several unresolved issues blocking my rebase. [1]https://lore.kernel.org/linux-mm/20240813120328.1275952-1-usamaarif642@gmail.com What do you think about this? Thanks Barry
On 18/08/2024 08:58, Barry Song wrote: > Hi Andrew, David, Usama, > > I'm attempting to rebase this series on top of Usama's > [PATCH v3 0/6] mm: split underutilized THPs[1] > > However, I feel it is impossible and we might be tackling things > in the wrong order. > > This series should probably come before Usama's one, as the new > partially_mapped should be ok to handle both nr_split_deferred > and split_deferred. That said, the new partially_mapped still has > several unresolved issues blocking my rebase. Hi, I have sent a v4 [1] that resolves all outstanding issues/comments raised in v3. It has been running a production workload for 12 hours, showing good performance and 0 crashes. Hopefully it helps to solve the existing merge conflicts between my series, yours and Yu Zhaos. [1] https://lore.kernel.org/all/20240819023145.2415299-1-usamaarif642@gmail.com/ Thanks > > [1]https://lore.kernel.org/linux-mm/20240813120328.1275952-1-usamaarif642@gmail.com > > What do you think about this? > > Thanks > Barry
On 18.08.24 09:58, Barry Song wrote: > Hi Andrew, David, Usama, > > I'm attempting to rebase this series on top of Usama's > [PATCH v3 0/6] mm: split underutilized THPs[1] > > However, I feel it is impossible and we might be tackling things > in the wrong order. Is just the ordering suboptimal (which can/will get resolved one way or the other), or is there something fundamental that will make this series here "impossible"?
On Mon, Aug 19, 2024 at 8:28 PM David Hildenbrand <david@redhat.com> wrote: > > On 18.08.24 09:58, Barry Song wrote: > > Hi Andrew, David, Usama, > > > > I'm attempting to rebase this series on top of Usama's > > [PATCH v3 0/6] mm: split underutilized THPs[1] > > > > However, I feel it is impossible and we might be tackling things > > in the wrong order. > > Is just the ordering suboptimal (which can/will get resolved one way or > the other), or is there something fundamental that will make this series > here "impossible"? i think it is just the ordering suboptimal. Ideally, mTHP counters can go first, then the new partially_mapped feature will rebase on top of mTHP counters. > > -- > Cheers, > > David / dhildenb >
On Mon, Aug 19, 2024 at 8:33 PM Barry Song <21cnbao@gmail.com> wrote: > > On Mon, Aug 19, 2024 at 8:28 PM David Hildenbrand <david@redhat.com> wrote: > > > > On 18.08.24 09:58, Barry Song wrote: > > > Hi Andrew, David, Usama, > > > > > > I'm attempting to rebase this series on top of Usama's > > > [PATCH v3 0/6] mm: split underutilized THPs[1] > > > > > > However, I feel it is impossible and we might be tackling things > > > in the wrong order. > > > > Is just the ordering suboptimal (which can/will get resolved one way or > > the other), or is there something fundamental that will make this series > > here "impossible"? > > i think it is just the ordering suboptimal. Ideally, mTHP counters can go > first, then the new partially_mapped feature will rebase on top of > mTHP counters. Sorry, please allow me to ramble a bit more. The nr_split_deferred counter is straightforward and simple without the partially_mapped feature. Each time we enter split_list, we increment by 1, and when we leave, we decrement by 1. With the new partially_mapped feature, we can enter split_list without actually being partially_mapped. If the MTHP counter series is processed first, the partially_mapped series can handle all cases while properly clearing and setting the partially_mapped flag. These flag operations need to be handled carefully. Currently, I notice that Usama's series is clearing the flag unconditionally in all cases. In simple terms, mTHP counters are just a counting mechanism that doesn't introduce new features. However, partially_mapped is a new feature. A better approach might be to handle the counters first, then ensure that the new feature doesn't break the counter. > > > > > -- > > Cheers, > > > > David / dhildenb Thanks Barry
On 19/08/2024 09:52, Barry Song wrote: > On Mon, Aug 19, 2024 at 8:33 PM Barry Song <21cnbao@gmail.com> wrote: >> >> On Mon, Aug 19, 2024 at 8:28 PM David Hildenbrand <david@redhat.com> wrote: >>> >>> On 18.08.24 09:58, Barry Song wrote: >>>> Hi Andrew, David, Usama, >>>> >>>> I'm attempting to rebase this series on top of Usama's >>>> [PATCH v3 0/6] mm: split underutilized THPs[1] >>>> >>>> However, I feel it is impossible and we might be tackling things >>>> in the wrong order. >>> >>> Is just the ordering suboptimal (which can/will get resolved one way or >>> the other), or is there something fundamental that will make this series >>> here "impossible"? >> >> i think it is just the ordering suboptimal. Ideally, mTHP counters can go >> first, then the new partially_mapped feature will rebase on top of >> mTHP counters. > > Sorry, please allow me to ramble a bit more. > > The nr_split_deferred counter is straightforward and simple without the > partially_mapped feature. Each time we enter split_list, we increment by > 1, and when we leave, we decrement by 1. > > With the new partially_mapped feature, we can enter split_list without > actually being partially_mapped. If the MTHP counter series is processed > first, the partially_mapped series can handle all cases while properly > clearing and setting the partially_mapped flag. These flag operations > need to be handled carefully. > Currently, I notice that Usama's series is clearing the flag unconditionally > in all cases. > > In simple terms, mTHP counters are just a counting mechanism that > doesn't introduce new features. However, partially_mapped is a new > feature. A better approach might be to handle the counters first, then > ensure that the new feature doesn't break the counter. > I am ok if the series needs to be reversed, I think the difficulty is for Andrew to tackle my series, yours and Yu Zhaos, which all seem to be touching the same code, so whatever makes it easier for Andrew I am happy with it. >> >>> >>> -- >>> Cheers, >>> >>> David / dhildenb > > Thanks > Barry
From: Barry Song <v-songbaohua@oppo.com> Knowing the number of anon mTHPs in the system is crucial for performance analysis. It helps in understanding the ratio and distribution of mTHPs versus small folios throughout the system. Additionally, partial unmapping by userspace can lead to significant waste of mTHPs over time and increase memory reclamation pressure. We need this information for comprehensive system tuning. -v2: * don't rely on rmap to implement - 1, uses folio_free, split etc. Thanks for David's comment; * rename sys counters and refine doc. Thanks for Ryan's comment; -v1: https://lore.kernel.org/all/20240808010457.228753-1-21cnbao@gmail.com/ Barry Song (2): mm: collect the number of anon large folios mm: collect the number of anon large folios on split_deferred list Documentation/admin-guide/mm/transhuge.rst | 10 ++++++++++ include/linux/huge_mm.h | 16 ++++++++++++++-- mm/huge_memory.c | 19 ++++++++++++++++--- mm/migrate.c | 4 ++++ mm/page_alloc.c | 5 ++++- mm/rmap.c | 1 + 6 files changed, 49 insertions(+), 6 deletions(-)