diff mbox series

[v2] Revert "mm: skip CMA pages when they are not available"

Message ID 20240315081803.2223-1-liuhailong@oppo.com (mailing list archive)
State New
Headers show
Series [v2] Revert "mm: skip CMA pages when they are not available" | expand

Commit Message

刘海龙(LaoLiu) March 15, 2024, 8:18 a.m. UTC
From: "Hailong.Liu" <liuhailong@oppo.com>

This reverts
commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")

skip_cma may cause system not responding. if cma pages is large in lru_list
and system is in lowmemory, many tasks would direct reclaim and waste
cpu time to isolate_lru_pages and return.

Test this patch on android-5.15 8G device
reproducer:
- cma_declare_contiguous 3G pages
- set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
  only.
- run a memleak process in userspace

trace the trace_mm_vmscan_lru_isolate which get
[ 1178.059160] 2825  3405 I UsbFfs-worker   : isolate_lru_pages:1895 total_scan: 24385 skipped: 24382
[ 1178.059699] 2825  3405 I UsbFfs-worker   : isolate_lru_pages:1895 total_scan: 24412 skipped: 24401
[ 1178.061747] 2825  3405 I UsbFfs-worker   : isolate_lru_pages:1895 total_scan: 24412 skipped: 24401
From the log, tasks is waste time to scan lru list and call stack as below.

Task name: UsbFfs-worker [affinity: 0xff] pid: 3374 cpu: 7 prio: 120 start: ffffff8897a35c80
state: 0x0[R] exit_state: 0x0 stack base: 0xffffffc01eaa0000
Last_enqueued_ts:       0.000000000 Last_sleep_ts:       0.000000000
Stack:
[<ffffffd32ee7d910>] __switch_to+0x180
[<ffffffd3302022fc>] __schedule+0x4dc
[<ffffffd330201e08>] preempt_schedule+0x5c
[<ffffffd33020a4d0>] _raw_spin_unlock_irq+0x54
[<ffffffd32f14906c>] shrink_inactive_list+0x1d0
[<ffffffd32f143998>] shrink_lruvec+0x1bc
[<ffffffd32f147c0c>] shrink_node_memcgs+0x184
[<ffffffd32f147414>] shrink_node+0x2d0
[<ffffffd32f146d38>] shrink_zones+0x14c
[<ffffffd32f142e84>] do_try_to_free_pages+0xe8
[<ffffffd32f142b08>] try_to_free_pages+0x2e0
[<ffffffd32f1a8e44>] __alloc_pages_direct_reclaim+0x84
[<ffffffd32f1a2d58>] __alloc_pages_slowpath+0x4d0
[<ffffffd32f1a23bc>] __alloc_pages_nodemask[jt]+0x124
[<ffffffd32f19a220>] __vmalloc_area_node+0x188
[<ffffffd32f19a540>] __vmalloc_node+0x148
[<ffffffd32f19a60c>] vmalloc+0x4c
[<ffffffd32f910218>] ffs_epfile_io+0x258
[<ffffffd330033780>] kretprobe_trampoline[jt]+0x0
[<ffffffd330033780>] kretprobe_trampoline[jt]+0x0
[<ffffffd32f28129c>] __io_submit_one+0x1c0
[<ffffffd32f280e38>] io_submit_one+0x88
[<ffffffd32f280c88>] __do_sys_io_submit+0x178
[<ffffffd32f27eac0>] __arm64_sys_io_submit+0x20
[<ffffffd32eeabb74>] el0_svc_common.llvm.9961749221945255377+0xd0
[<ffffffd32eeaba34>] do_el0_svc+0x28
[<ffffffd32ff21be8>] el0_svc+0x14
[<ffffffd32ff21b70>] el0_sync_handler+0x88
[<ffffffd32ee128b8>] el0_sync+0x1b8

Task name: kthreadd [affinity: 0xff] pid: 2 cpu: 7 prio: 120 start: ffffff87808c0000
state: 0x0[R] exit_state: 0x0 stack base: 0xffffffc008078000
Last_enqueued_ts:       0.000000000 Last_sleep_ts:       0.000000000
Stack:
[<ffffffd32ee7d910>] __switch_to+0x180
[<ffffffd3302022fc>] __schedule+0x4dc
[<ffffffd330201e08>] preempt_schedule+0x5c
[<ffffffd33020a4d0>] _raw_spin_unlock_irq+0x54
[<ffffffd32f149168>] shrink_inactive_list+0x2cc
[<ffffffd32f143998>] shrink_lruvec+0x1bc
[<ffffffd32f147c0c>] shrink_node_memcgs+0x184
[<ffffffd32f147414>] shrink_node+0x2d0
[<ffffffd32f146d38>] shrink_zones+0x14c
[<ffffffd32f142e84>] do_try_to_free_pages+0xe8
[<ffffffd32f142b08>] try_to_free_pages+0x2e0
[<ffffffd32f1a8e44>] __alloc_pages_direct_reclaim+0x84
[<ffffffd32f1a2d58>] __alloc_pages_slowpath+0x4d0
[<ffffffd32f1a23bc>] __alloc_pages_nodemask[jt]+0x124
[<ffffffd32f19a220>] __vmalloc_area_node+0x188
[<ffffffd32f19a044>] __vmalloc_node_range+0x88
[<ffffffd32f0fb430>] scs_alloc+0x1b8
[<ffffffd32f0fb62c>] scs_prepare+0x20
[<ffffffd32ef2ce04>] dup_task_struct+0xd4
[<ffffffd32ef2a77c>] copy_process+0x144
[<ffffffd32ef2bae4>] kernel_clone+0xb4
[<ffffffd32ef2c040>] kernel_thread+0x5c
[<ffffffd32ef618d0>] kthreadd+0x184

Signed-off-by: Hailong.Liu <liuhailong@oppo.com>
---
 mm/vmscan.c | 24 ++----------------------
 1 file changed, 2 insertions(+), 22 deletions(-)

--
Changes in v2:
- add reproducer
- fix build error on v6.6-rc7

Comments

Barry Song March 15, 2024, 8:46 a.m. UTC | #1
On Fri, Mar 15, 2024 at 9:18 PM <liuhailong@oppo.com> wrote:
>
> From: "Hailong.Liu" <liuhailong@oppo.com>
>
> This reverts
> commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")

Reverting these changes seems inappropriate since the original patches
did improve most cases.
While doing direct reclamation without MOVABLE flags, scanning cma
folio doesn't help at all.

>
> skip_cma may cause system not responding. if cma pages is large in lru_list
> and system is in lowmemory, many tasks would direct reclaim and waste
> cpu time to isolate_lru_pages and return.
>
> Test this patch on android-5.15 8G device
> reproducer:
> - cma_declare_contiguous 3G pages
> - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
>   only.
> - run a memleak process in userspace


I assume that scanning cma folio provides additional opportunities to reclaim
anonymous memory, even though it is ineffective for allocating NON-MOVABLE
folios. Consequently, we alleviate pressure for future allocations of anonymous
folios. Moreover, setting swappiness=0 is a rare scenario. Instead of entirely
removing the mechanism, could we detect such corner cases and handle them
differently.

Otherwise, I don't see any chance of this being acknowledged.

>
> trace the trace_mm_vmscan_lru_isolate which get
> [ 1178.059160] 2825  3405 I UsbFfs-worker   : isolate_lru_pages:1895 total_scan: 24385 skipped: 24382
> [ 1178.059699] 2825  3405 I UsbFfs-worker   : isolate_lru_pages:1895 total_scan: 24412 skipped: 24401
> [ 1178.061747] 2825  3405 I UsbFfs-worker   : isolate_lru_pages:1895 total_scan: 24412 skipped: 24401
> From the log, tasks is waste time to scan lru list and call stack as below.
>
> Task name: UsbFfs-worker [affinity: 0xff] pid: 3374 cpu: 7 prio: 120 start: ffffff8897a35c80
> state: 0x0[R] exit_state: 0x0 stack base: 0xffffffc01eaa0000
> Last_enqueued_ts:       0.000000000 Last_sleep_ts:       0.000000000
> Stack:
> [<ffffffd32ee7d910>] __switch_to+0x180
> [<ffffffd3302022fc>] __schedule+0x4dc
> [<ffffffd330201e08>] preempt_schedule+0x5c
> [<ffffffd33020a4d0>] _raw_spin_unlock_irq+0x54
> [<ffffffd32f14906c>] shrink_inactive_list+0x1d0
> [<ffffffd32f143998>] shrink_lruvec+0x1bc
> [<ffffffd32f147c0c>] shrink_node_memcgs+0x184
> [<ffffffd32f147414>] shrink_node+0x2d0
> [<ffffffd32f146d38>] shrink_zones+0x14c
> [<ffffffd32f142e84>] do_try_to_free_pages+0xe8
> [<ffffffd32f142b08>] try_to_free_pages+0x2e0
> [<ffffffd32f1a8e44>] __alloc_pages_direct_reclaim+0x84
> [<ffffffd32f1a2d58>] __alloc_pages_slowpath+0x4d0
> [<ffffffd32f1a23bc>] __alloc_pages_nodemask[jt]+0x124
> [<ffffffd32f19a220>] __vmalloc_area_node+0x188
> [<ffffffd32f19a540>] __vmalloc_node+0x148
> [<ffffffd32f19a60c>] vmalloc+0x4c
> [<ffffffd32f910218>] ffs_epfile_io+0x258
> [<ffffffd330033780>] kretprobe_trampoline[jt]+0x0
> [<ffffffd330033780>] kretprobe_trampoline[jt]+0x0
> [<ffffffd32f28129c>] __io_submit_one+0x1c0
> [<ffffffd32f280e38>] io_submit_one+0x88
> [<ffffffd32f280c88>] __do_sys_io_submit+0x178
> [<ffffffd32f27eac0>] __arm64_sys_io_submit+0x20
> [<ffffffd32eeabb74>] el0_svc_common.llvm.9961749221945255377+0xd0
> [<ffffffd32eeaba34>] do_el0_svc+0x28
> [<ffffffd32ff21be8>] el0_svc+0x14
> [<ffffffd32ff21b70>] el0_sync_handler+0x88
> [<ffffffd32ee128b8>] el0_sync+0x1b8
>
> Task name: kthreadd [affinity: 0xff] pid: 2 cpu: 7 prio: 120 start: ffffff87808c0000
> state: 0x0[R] exit_state: 0x0 stack base: 0xffffffc008078000
> Last_enqueued_ts:       0.000000000 Last_sleep_ts:       0.000000000
> Stack:
> [<ffffffd32ee7d910>] __switch_to+0x180
> [<ffffffd3302022fc>] __schedule+0x4dc
> [<ffffffd330201e08>] preempt_schedule+0x5c
> [<ffffffd33020a4d0>] _raw_spin_unlock_irq+0x54
> [<ffffffd32f149168>] shrink_inactive_list+0x2cc
> [<ffffffd32f143998>] shrink_lruvec+0x1bc
> [<ffffffd32f147c0c>] shrink_node_memcgs+0x184
> [<ffffffd32f147414>] shrink_node+0x2d0
> [<ffffffd32f146d38>] shrink_zones+0x14c
> [<ffffffd32f142e84>] do_try_to_free_pages+0xe8
> [<ffffffd32f142b08>] try_to_free_pages+0x2e0
> [<ffffffd32f1a8e44>] __alloc_pages_direct_reclaim+0x84
> [<ffffffd32f1a2d58>] __alloc_pages_slowpath+0x4d0
> [<ffffffd32f1a23bc>] __alloc_pages_nodemask[jt]+0x124
> [<ffffffd32f19a220>] __vmalloc_area_node+0x188
> [<ffffffd32f19a044>] __vmalloc_node_range+0x88
> [<ffffffd32f0fb430>] scs_alloc+0x1b8
> [<ffffffd32f0fb62c>] scs_prepare+0x20
> [<ffffffd32ef2ce04>] dup_task_struct+0xd4
> [<ffffffd32ef2a77c>] copy_process+0x144
> [<ffffffd32ef2bae4>] kernel_clone+0xb4
> [<ffffffd32ef2c040>] kernel_thread+0x5c
> [<ffffffd32ef618d0>] kthreadd+0x184
>
> Signed-off-by: Hailong.Liu <liuhailong@oppo.com>
> ---
>  mm/vmscan.c | 24 ++----------------------
>  1 file changed, 2 insertions(+), 22 deletions(-)
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 6f13394b112e..29306c29309f 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -2261,25 +2261,6 @@ static __always_inline void update_lru_sizes(struct lruvec *lruvec,
>
>  }
>
> -#ifdef CONFIG_CMA
> -/*
> - * It is waste of effort to scan and reclaim CMA pages if it is not available
> - * for current allocation context. Kswapd can not be enrolled as it can not
> - * distinguish this scenario by using sc->gfp_mask = GFP_KERNEL
> - */
> -static bool skip_cma(struct folio *folio, struct scan_control *sc)
> -{
> -       return !current_is_kswapd() &&
> -                       gfp_migratetype(sc->gfp_mask) != MIGRATE_MOVABLE &&
> -                       get_pageblock_migratetype(&folio->page) == MIGRATE_CMA;
> -}
> -#else
> -static bool skip_cma(struct folio *folio, struct scan_control *sc)
> -{
> -       return false;
> -}
> -#endif
> -
>  /*
>   * Isolating page from the lruvec to fill in @dst list by nr_to_scan times.
>   *
> @@ -2326,8 +2307,7 @@ static unsigned long isolate_lru_folios(unsigned long nr_to_scan,
>                 nr_pages = folio_nr_pages(folio);
>                 total_scan += nr_pages;
>
> -               if (folio_zonenum(folio) > sc->reclaim_idx ||
> -                               skip_cma(folio, sc)) {
> +               if (folio_zonenum(folio) > sc->reclaim_idx) {
>                         nr_skipped[folio_zonenum(folio)] += nr_pages;
>                         move_to = &folios_skipped;
>                         goto move;
> @@ -4945,7 +4925,7 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_c
>         }
>
>         /* ineligible */
> -       if (zone > sc->reclaim_idx || skip_cma(folio, sc)) {
> +       if (zone > sc->reclaim_idx) {
>                 gen = folio_inc_gen(lruvec, folio, false);
>                 list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone]);
>                 return true;
> --
> Changes in v2:
> - add reproducer
> - fix build error on v6.6-rc7
>
>

Thanks
Barry
刘海龙(LaoLiu) March 15, 2024, 10:43 a.m. UTC | #2
On 2024/3/15 16:46, Barry Song wrote:
> On Fri, Mar 15, 2024 at 9:18 PM <liuhailong@oppo.com> wrote:
>>
>> From: "Hailong.Liu" <liuhailong@oppo.com>
>>
>> This reverts
>> commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
>> commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> 
> Reverting these changes seems inappropriate since the original patches
> did improve most cases.
> While doing direct reclamation without MOVABLE flags, scanning cma
> folio doesn't help at all.
> 

Absolutely, but without this patch  it give chance to reclaim cma pages to freelist, 
other tasks without this flag would work normally without holding too much time 
on lru lock and lru size decrease。 it also trigger memory psi event to allow 
admin do something to release memory.

>>
>> skip_cma may cause system not responding. if cma pages is large in lru_list
>> and system is in lowmemory, many tasks would direct reclaim and waste
>> cpu time to isolate_lru_pages and return.
>>
>> Test this patch on android-5.15 8G device
>> reproducer:
>> - cma_declare_contiguous 3G pages
>> - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
>>   only.
>> - run a memleak process in userspace
> 
> 
> I assume that scanning cma folio provides additional opportunities to reclaim
> anonymous memory, even though it is ineffective for allocating NON-MOVABLE
> folios. Consequently, we alleviate pressure for future allocations of anonymous
> folios. Moreover, setting swappiness=0 is a rare scenario. Instead of entirely
> removing the mechanism, could we detect such corner cases and handle them
> differently.
> 
> Otherwise, I don't see any chance of this being acknowledged.

It happens in internal aging test. open camera and record 8K video which use 
more than 2Gib dmabuf pages. without this patch the devices would kill camera process,
but the system hang with it.

In fact, in my opinion the issue of this patch increase the scanning lru time 
and reclaim nothing. Do you have a better solution ?

Brs,
Hailong.
Barry Song March 17, 2024, 5:47 a.m. UTC | #3
On Fri, Mar 15, 2024 at 11:44 PM 刘海龙(LaoLiu) <liuhailong@oppo.com> wrote:
>
> On 2024/3/15 16:46, Barry Song wrote:
> > On Fri, Mar 15, 2024 at 9:18 PM <liuhailong@oppo.com> wrote:
> >>
> >> From: "Hailong.Liu" <liuhailong@oppo.com>
> >>
> >> This reverts
> >> commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> >> commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> >
> > Reverting these changes seems inappropriate since the original patches
> > did improve most cases.
> > While doing direct reclamation without MOVABLE flags, scanning cma
> > folio doesn't help at all.
> >
>
> Absolutely, but without this patch  it give chance to reclaim cma pages to freelist,
> other tasks without this flag would work normally without holding too much time
> on lru lock and lru size decrease。 it also trigger memory psi event to allow
> admin do something to release memory.
>
> >>
> >> skip_cma may cause system not responding. if cma pages is large in lru_list
> >> and system is in lowmemory, many tasks would direct reclaim and waste
> >> cpu time to isolate_lru_pages and return.
> >>
> >> Test this patch on android-5.15 8G device
> >> reproducer:
> >> - cma_declare_contiguous 3G pages
> >> - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
> >>   only.
> >> - run a memleak process in userspace
> >
> >
> > I assume that scanning cma folio provides additional opportunities to reclaim
> > anonymous memory, even though it is ineffective for allocating NON-MOVABLE
> > folios. Consequently, we alleviate pressure for future allocations of anonymous
> > folios. Moreover, setting swappiness=0 is a rare scenario. Instead of entirely
> > removing the mechanism, could we detect such corner cases and handle them
> > differently.
> >
> > Otherwise, I don't see any chance of this being acknowledged.
>
> It happens in internal aging test. open camera and record 8K video which use
> more than 2Gib dmabuf pages. without this patch the devices would kill camera process,
> but the system hang with it.
>
> In fact, in my opinion the issue of this patch increase the scanning lru time
> and reclaim nothing. Do you have a better solution ?

rather than aggressively reverting the original patch which might help lots of
cases, could we just detect the corner cases and fix them , for example, by
something as the belows,

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 96147c4536e4..2f75ad9870c1 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -147,6 +147,9 @@ struct scan_control {
        /* Always discard instead of demoting to lower tier memory */
        unsigned int no_demotion:1;

+       /* no skipping cma */
+       unsigned int no_skip_cma:1;
+
        /* Allocation order */
        s8 order;

@@ -1595,7 +1598,7 @@ static __always_inline void
update_lru_sizes(struct lruvec *lruvec,
  */
 static bool skip_cma(struct folio *folio, struct scan_control *sc)
 {
-       return !current_is_kswapd() &&
+       return !current_is_kswapd() && !sc->no_skip_cma &&
                        gfp_migratetype(sc->gfp_mask) != MIGRATE_MOVABLE &&
                        folio_migratetype(folio) == MIGRATE_CMA;
 }
@@ -1636,7 +1639,7 @@ static unsigned long isolate_lru_folios(unsigned
long nr_to_scan,
        unsigned long nr_taken = 0;
        unsigned long nr_zone_taken[MAX_NR_ZONES] = { 0 };
        unsigned long nr_skipped[MAX_NR_ZONES] = { 0, };
-       unsigned long skipped = 0;
+       unsigned long skipped = 0, skipped_cma = 0;
        unsigned long scan, total_scan, nr_pages;
        LIST_HEAD(folios_skipped);

@@ -1645,6 +1648,7 @@ static unsigned long isolate_lru_folios(unsigned
long nr_to_scan,
        while (scan < nr_to_scan && !list_empty(src)) {
                struct list_head *move_to = src;
                struct folio *folio;
+               bool cma = false;

                folio = lru_to_folio(src);
                prefetchw_prev_lru_folio(folio, src, flags);
@@ -1652,10 +1656,11 @@ static unsigned long
isolate_lru_folios(unsigned long nr_to_scan,
                nr_pages = folio_nr_pages(folio);
                total_scan += nr_pages;

-               if (folio_zonenum(folio) > sc->reclaim_idx ||
-                               skip_cma(folio, sc)) {
+               cma = skip_cma(folio, sc);
+               if (folio_zonenum(folio) > sc->reclaim_idx || cma) {
                        nr_skipped[folio_zonenum(folio)] += nr_pages;
                        move_to = &folios_skipped;
+                       skipped_cma += nr_pages;
                        goto move;
                }

@@ -1716,6 +1721,16 @@ static unsigned long
isolate_lru_folios(unsigned long nr_to_scan,
        *nr_scanned = total_scan;
        trace_mm_vmscan_lru_isolate(sc->reclaim_idx, sc->order, nr_to_scan,
                                    total_scan, skipped, nr_taken, lru);
+
+       /*
+        * If the majority of folios are skipped due to contiguous
memory allocator (CMA),
+        * the least recently used (LRU) list might become overloaded
with CMA folios.
+        * This could result in numerous idle loops for non-movable
allocations. On the
+        * other hand, reclaiming CMA folios can alleviate the strain
on other processes
+        * that require anonymous pages.
+        */
+       sc->no_skip_cma = skipped_cma > total_scan / 2 ? 1 : 0;
+
        update_lru_sizes(lruvec, lru, nr_zone_taken);
        return nr_taken;
 }


This is merely a proof of concept (PoC), and I haven't even built it
yet. Additionally, LRU_GEN remains
unchanged. It will also need similar changes.

>
> Brs,
> Hailong.

Thanks
Barry
Michal Hocko March 19, 2024, 8:55 a.m. UTC | #4
On Fri 15-03-24 16:18:03, liuhailong@oppo.com wrote:
> From: "Hailong.Liu" <liuhailong@oppo.com>
> 
> This reverts
> commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> 
> skip_cma may cause system not responding. if cma pages is large in lru_list
> and system is in lowmemory, many tasks would direct reclaim and waste
> cpu time to isolate_lru_pages and return.
> 
> Test this patch on android-5.15 8G device
> reproducer:
> - cma_declare_contiguous 3G pages
> - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
>   only.
> - run a memleak process in userspace

Does this represent a sane configuration? CMA memory is unusable for
kernel allocations and memleak process is also hard to reclaim due to
swap suppression. Isn't such a system doomed to struggle to reclaim any
memory? Btw. how does the same setup behave with the regular LRU
implementation? My guess would be that it would struggle as well.
Barry Song March 19, 2024, 11:09 a.m. UTC | #5
On Tue, Mar 19, 2024 at 4:56 PM Michal Hocko <mhocko@suse.com> wrote:
>
> On Fri 15-03-24 16:18:03, liuhailong@oppo.com wrote:
> > From: "Hailong.Liu" <liuhailong@oppo.com>
> >
> > This reverts
> > commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> > commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> >
> > skip_cma may cause system not responding. if cma pages is large in lru_list
> > and system is in lowmemory, many tasks would direct reclaim and waste
> > cpu time to isolate_lru_pages and return.
> >
> > Test this patch on android-5.15 8G device
> > reproducer:
> > - cma_declare_contiguous 3G pages
> > - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
> >   only.
> > - run a memleak process in userspace
>
> Does this represent a sane configuration? CMA memory is unusable for
> kernel allocations and memleak process is also hard to reclaim due to
> swap suppression. Isn't such a system doomed to struggle to reclaim any
> memory? Btw. how does the same setup behave with the regular LRU
> implementation? My guess would be that it would struggle as well.

I assume the regular LRU implementation you are talking about is the LRU
without skip_cma()?

I remember Hailong mentioned something like " it also trigger memory psi
event to allow admin do something to release memory" and " without
patch the devices would kill camera process".  So it seems the difference
is if a killing will occur.

Hailong, would you like to provide more detail?

> --
> Michal Hocko
> SUSE Labs
>
刘海龙(LaoLiu) March 19, 2024, 12:27 p.m. UTC | #6
On 2024/3/19 19:09, Barry Song wrote:
> On Tue, Mar 19, 2024 at 4:56 PM Michal Hocko <mhocko@suse.com> wrote:
>>
>> On Fri 15-03-24 16:18:03, liuhailong@oppo.com wrote:
>>> From: "Hailong.Liu" <liuhailong@oppo.com>
>>>
>>> This reverts
>>> commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
>>> commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
>>>
>>> skip_cma may cause system not responding. if cma pages is large in lru_list
>>> and system is in lowmemory, many tasks would direct reclaim and waste
>>> cpu time to isolate_lru_pages and return.
>>>
>>> Test this patch on android-5.15 8G device
>>> reproducer:
>>> - cma_declare_contiguous 3G pages
>>> - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
>>>   only.
>>> - run a memleak process in userspace
>>
>> Does this represent a sane configuration? CMA memory is unusable for
>> kernel allocations and memleak process is also hard to reclaim due to
>> swap suppression. Isn't such a system doomed to struggle to reclaim any
>> memory? 
Yes, All processes in the system are also hard to reclaim. and all processes
enter direct reclaim. with this patch, much of process which should skip_cma
would retry, scan, skipped in the process of isolsate_lru_pages. and system 
process will have high priority, some normal processes (like kswapd) are 
preempted. 


Btw. how does the same setup behave with the regular LRU
>> implementation? My guess would be that it would struggle as well.
> 
> I assume the regular LRU implementation you are talking about is the LRU
> without skip_cma()?
> 
> I remember Hailong mentioned something like " it also trigger memory psi
> event to allow admin do something to release memory" and " without
> patch the devices would kill camera process".  So it seems the difference
> is if a killing will occur.
> 
> Hailong, would you like to provide more detail?

psi_event triggered after psi_memstall_leave. much system processes 
perform_reclaim scan and skipped and leave without reclaim any pages. 
the process is fast, so lmkd (userspace lowmemory killer) could not work
as before.

> 
>> --
>> Michal Hocko
>> SUSE Labs
>>
Michal Hocko March 19, 2024, 1:40 p.m. UTC | #7
On Tue 19-03-24 19:09:18, Barry Song wrote:
> On Tue, Mar 19, 2024 at 4:56 PM Michal Hocko <mhocko@suse.com> wrote:
> >
> > On Fri 15-03-24 16:18:03, liuhailong@oppo.com wrote:
> > > From: "Hailong.Liu" <liuhailong@oppo.com>
> > >
> > > This reverts
> > > commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> > > commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> > >
> > > skip_cma may cause system not responding. if cma pages is large in lru_list
> > > and system is in lowmemory, many tasks would direct reclaim and waste
> > > cpu time to isolate_lru_pages and return.
> > >
> > > Test this patch on android-5.15 8G device
> > > reproducer:
> > > - cma_declare_contiguous 3G pages
> > > - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
> > >   only.
> > > - run a memleak process in userspace
> >
> > Does this represent a sane configuration? CMA memory is unusable for
> > kernel allocations and memleak process is also hard to reclaim due to
> > swap suppression. Isn't such a system doomed to struggle to reclaim any
> > memory? Btw. how does the same setup behave with the regular LRU
> > implementation? My guess would be that it would struggle as well.
> 
> I assume the regular LRU implementation you are talking about is the LRU
> without skip_cma()?

No, I mean standard LRU reclaim implementation rather than MGLRU.
Barry Song March 22, 2024, 1:25 a.m. UTC | #8
On Wed, Mar 20, 2024 at 1:27 AM 刘海龙(LaoLiu) <liuhailong@oppo.com> wrote:
>
> On 2024/3/19 19:09, Barry Song wrote:
> > On Tue, Mar 19, 2024 at 4:56 PM Michal Hocko <mhocko@suse.com> wrote:
> >>
> >> On Fri 15-03-24 16:18:03, liuhailong@oppo.com wrote:
> >>> From: "Hailong.Liu" <liuhailong@oppo.com>
> >>>
> >>> This reverts
> >>> commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> >>> commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> >>>
> >>> skip_cma may cause system not responding. if cma pages is large in lru_list
> >>> and system is in lowmemory, many tasks would direct reclaim and waste
> >>> cpu time to isolate_lru_pages and return.
> >>>
> >>> Test this patch on android-5.15 8G device
> >>> reproducer:
> >>> - cma_declare_contiguous 3G pages
> >>> - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
> >>>   only.
> >>> - run a memleak process in userspace
> >>
> >> Does this represent a sane configuration? CMA memory is unusable for
> >> kernel allocations and memleak process is also hard to reclaim due to
> >> swap suppression. Isn't such a system doomed to struggle to reclaim any
> >> memory?
> Yes, All processes in the system are also hard to reclaim. and all processes
> enter direct reclaim. with this patch, much of process which should skip_cma
> would retry, scan, skipped in the process of isolsate_lru_pages. and system
> process will have high priority, some normal processes (like kswapd) are
> preempted.
>
>
> Btw. how does the same setup behave with the regular LRU
> >> implementation? My guess would be that it would struggle as well.
> >
> > I assume the regular LRU implementation you are talking about is the LRU
> > without skip_cma()?
> >
> > I remember Hailong mentioned something like " it also trigger memory psi
> > event to allow admin do something to release memory" and " without
> > patch the devices would kill camera process".  So it seems the difference
> > is if a killing will occur.
> >
> > Hailong, would you like to provide more detail?
>
> psi_event triggered after psi_memstall_leave. much system processes
> perform_reclaim scan and skipped and leave without reclaim any pages.
> the process is fast, so lmkd (userspace lowmemory killer) could not work
> as before.

Can you provide more details?

Does PSI still give accurate reports when processes are stuck in an idle loop
because most pages in the LRU list are in CMA? Also, in such a case, will
the LMKD not receive CPU resources to execute?

>
> >
> >> --
> >> Michal Hocko
> >> SUSE Labs
> >>
>
Barry Song March 22, 2024, 1:28 a.m. UTC | #9
On Wed, Mar 20, 2024 at 2:40 AM Michal Hocko <mhocko@suse.com> wrote:
>
> On Tue 19-03-24 19:09:18, Barry Song wrote:
> > On Tue, Mar 19, 2024 at 4:56 PM Michal Hocko <mhocko@suse.com> wrote:
> > >
> > > On Fri 15-03-24 16:18:03, liuhailong@oppo.com wrote:
> > > > From: "Hailong.Liu" <liuhailong@oppo.com>
> > > >
> > > > This reverts
> > > > commit b7108d66318a ("Multi-gen LRU: skip CMA pages when they are not eligible")
> > > > commit 5da226dbfce3 ("mm: skip CMA pages when they are not available")
> > > >
> > > > skip_cma may cause system not responding. if cma pages is large in lru_list
> > > > and system is in lowmemory, many tasks would direct reclaim and waste
> > > > cpu time to isolate_lru_pages and return.
> > > >
> > > > Test this patch on android-5.15 8G device
> > > > reproducer:
> > > > - cma_declare_contiguous 3G pages
> > > > - set /proc/sys/vm/swappiness 0 to enable direct_reclaim reclaim file
> > > >   only.
> > > > - run a memleak process in userspace
> > >
> > > Does this represent a sane configuration? CMA memory is unusable for
> > > kernel allocations and memleak process is also hard to reclaim due to
> > > swap suppression. Isn't such a system doomed to struggle to reclaim any
> > > memory? Btw. how does the same setup behave with the regular LRU
> > > implementation? My guess would be that it would struggle as well.
> >
> > I assume the regular LRU implementation you are talking about is the LRU
> > without skip_cma()?
>
> No, I mean standard LRU reclaim implementation rather than MGLRU.

I guess Hailong was running the standard LRU with active/inactive lists
as his v1 even didn't touch MGLRU,
https://lore.kernel.org/linux-mm/20240314141516.31747-1-liuhailong@oppo.com/

Hailong, please correct me if this is not the case.

>
> --
> Michal Hocko
> SUSE Labs
diff mbox series

Patch

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 6f13394b112e..29306c29309f 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2261,25 +2261,6 @@  static __always_inline void update_lru_sizes(struct lruvec *lruvec,

 }

-#ifdef CONFIG_CMA
-/*
- * It is waste of effort to scan and reclaim CMA pages if it is not available
- * for current allocation context. Kswapd can not be enrolled as it can not
- * distinguish this scenario by using sc->gfp_mask = GFP_KERNEL
- */
-static bool skip_cma(struct folio *folio, struct scan_control *sc)
-{
-	return !current_is_kswapd() &&
-			gfp_migratetype(sc->gfp_mask) != MIGRATE_MOVABLE &&
-			get_pageblock_migratetype(&folio->page) == MIGRATE_CMA;
-}
-#else
-static bool skip_cma(struct folio *folio, struct scan_control *sc)
-{
-	return false;
-}
-#endif
-
 /*
  * Isolating page from the lruvec to fill in @dst list by nr_to_scan times.
  *
@@ -2326,8 +2307,7 @@  static unsigned long isolate_lru_folios(unsigned long nr_to_scan,
 		nr_pages = folio_nr_pages(folio);
 		total_scan += nr_pages;

-		if (folio_zonenum(folio) > sc->reclaim_idx ||
-				skip_cma(folio, sc)) {
+		if (folio_zonenum(folio) > sc->reclaim_idx) {
 			nr_skipped[folio_zonenum(folio)] += nr_pages;
 			move_to = &folios_skipped;
 			goto move;
@@ -4945,7 +4925,7 @@  static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_c
 	}

 	/* ineligible */
-	if (zone > sc->reclaim_idx || skip_cma(folio, sc)) {
+	if (zone > sc->reclaim_idx) {
 		gen = folio_inc_gen(lruvec, folio, false);
 		list_move_tail(&folio->lru, &lrugen->folios[gen][type][zone]);
 		return true;