| Message ID | 95d610623363009a71024c7a473d6895f39f3caf.1694219361.git.wangjiexun@tinylab.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm/madvise: add cond_resched() in madvise_cold_or_pageout_pte_range() | expand |
On Sat, 9 Sep 2023 13:33:08 +0800 Jiexun Wang <wangjiexun@tinylab.org> wrote: > Currently the madvise_cold_or_pageout_pte_range() function exhibits > significant latency under memory pressure, which can be effectively > reduced by adding cond_resched() within the loop. > > When the batch_count reaches SWAP_CLUSTER_MAX, we reschedule > the task to ensure fairness and avoid long lock holding times. > > ... > > @@ -441,6 +443,13 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, > arch_enter_lazy_mmu_mode(); > for (; addr < end; pte++, addr += PAGE_SIZE) { > ptent = ptep_get(pte); > + > + if (++batch_count == SWAP_CLUSTER_MAX) { > + pte_unmap_unlock(start_pte, ptl); > + cond_resched(); > + start_pte = pte_offset_map_lock(mm, pmd, addr, &ptl); > + batch_count = 0; > + } > > if (pte_none(ptent)) > continue; I doubt if we can simply drop the lock like this then proceed as if nothing has changed while the lock was released. Could be that something along these lines: @@ -434,6 +436,7 @@ huge_unlock: regular_folio: #endif tlb_change_page_size(tlb, PAGE_SIZE); +restart: start_pte = pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); if (!start_pte) return 0; @@ -441,6 +444,15 @@ regular_folio: arch_enter_lazy_mmu_mode(); for (; addr < end; pte++, addr += PAGE_SIZE) { ptent = ptep_get(pte); + + if (++batch_count == SWAP_CLUSTER_MAX) { + batch_count = 0; + if (need_resched()) { + pte_unmap_unlock(start_pte, ptl); + cond_resched(); + goto restart; + } + } if (pte_none(ptent)) continue; would work, but more analysis would be needed.
Hi Jiexun, kernel test robot noticed the following build warnings: [auto build test WARNING on akpm-mm/mm-everything] url: https://github.com/intel-lab-lkp/linux/commits/Jiexun-Wang/mm-madvise-add-cond_resched-in-madvise_cold_or_pageout_pte_range/20230909-133707 base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything patch link: https://lore.kernel.org/r/95d610623363009a71024c7a473d6895f39f3caf.1694219361.git.wangjiexun%40tinylab.org patch subject: [PATCH 1/1] mm/madvise: add cond_resched() in madvise_cold_or_pageout_pte_range() compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <yujie.liu@intel.com> | Closes: https://lore.kernel.org/r/202309111456.SBqOkY0h-lkp@intel.com/ includecheck warnings: (new ones prefixed by >>) >> mm/madvise.c: linux/swap.h is included more than once. vim +30 mm/madvise.c > 30 #include <linux/swap.h> 31 #include <linux/swapops.h> 32 #include <linux/shmem_fs.h> 33 #include <linux/mmu_notifier.h> > 34 #include <linux/swap.h> 35
diff --git a/mm/madvise.c b/mm/madvise.c index 4dded5d27e7e..df760096ea85 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -31,6 +31,7 @@ #include <linux/swapops.h> #include <linux/shmem_fs.h> #include <linux/mmu_notifier.h> +#include <linux/swap.h> #include <asm/tlb.h> @@ -353,6 +354,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, struct folio *folio = NULL; LIST_HEAD(folio_list); bool pageout_anon_only_filter; + unsigned int batch_count = 0; if (fatal_signal_pending(current)) return -EINTR; @@ -441,6 +443,13 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, arch_enter_lazy_mmu_mode(); for (; addr < end; pte++, addr += PAGE_SIZE) { ptent = ptep_get(pte); + + if (++batch_count == SWAP_CLUSTER_MAX) { + pte_unmap_unlock(start_pte, ptl); + cond_resched(); + start_pte = pte_offset_map_lock(mm, pmd, addr, &ptl); + batch_count = 0; + } if (pte_none(ptent)) continue;
Currently the madvise_cold_or_pageout_pte_range() function exhibits significant latency under memory pressure, which can be effectively reduced by adding cond_resched() within the loop. When the batch_count reaches SWAP_CLUSTER_MAX, we reschedule the task to ensure fairness and avoid long lock holding times. Signed-off-by: Jiexun Wang <wangjiexun@tinylab.org> --- mm/madvise.c | 9 +++++++++ 1 file changed, 9 insertions(+)