[110/131] mm: base LRU balancing on an explicit cost model

Message ID	20200603230253._MC-eONlj%akpm@linux-foundation.org (mailing list archive)
State	New, archived
Headers	show Return-Path: <SRS0=OQel=7Q=kvack.org=owner-linux-mm@kernel.org> DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD1221531 Date: Wed, 03 Jun 2020 16:02:53 -0700 From: Andrew Morton <akpm@linux-foundation.org> To: akpm@linux-foundation.org, hannes@cmpxchg.org, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, riel@surriel.com, torvalds@linux-foundation.org Subject: [patch 110/131] mm: base LRU balancing on an explicit cost model Message-ID: <20200603230253._MC-eONlj%akpm@linux-foundation.org> In-Reply-To: <20200603155549.e041363450869eaae4c7f05b@linux-foundation.org> User-Agent: s-nail v14.8.16 Sender: owner-linux-mm@kvack.org Precedence: bulk
Series	[001/131] mm/slub: fix a memory leak in sysfs_slab_add() \| expand [001/131] mm/slub: fix a memory leak in sysfs_slab_add() [002/131] mm/memcg: optimize memory.numa_stat like memory.stat [003/131] mm/gup: move __get_user_pages_fast() down a few lines in gup.c [004/131] mm/gup: refactor and de-duplicate gup_fast() code [005/131] mm/gup: introduce pin_user_pages_fast_only() [006/131] drm/i915: convert get_user_pages() --> pin_user_pages() [007/131] mm/gup: might_lock_read(mmap_sem) in get_user_pages_fast() [008/131] kasan: stop tests being eliminated as dead code with FORTIFY_SOURCE [009/131] string.h: fix incompatibility between FORTIFY_SOURCE and KASAN [010/131] mm: clarify __GFP_MEMALLOC usage [011/131] mm: memblock: replace dereferences of memblock_region.nid with API calls [012/131] mm: make early_pfn_to_nid() and related defintions close to each other [013/131] mm: remove CONFIG_HAVE_MEMBLOCK_NODE_MAP option [014/131] mm: free_area_init: use maximal zone PFNs rather than zone sizes [015/131] mm: use free_area_init() instead of free_area_init_nodes() [016/131] alpha: simplify detection of memory zone boundaries [017/131] arm: simplify detection of memory zone boundaries [018/131] arm64: simplify detection of memory zone boundaries for UMA configs [019/131] csky: simplify detection of memory zone boundaries [020/131] m68k: mm: simplify detection of memory zone boundaries [021/131] parisc: simplify detection of memory zone boundaries [022/131] sparc32: simplify detection of memory zone boundaries [023/131] unicore32: simplify detection of memory zone boundaries [024/131] xtensa: simplify detection of memory zone boundaries [025/131] mm: memmap_init: iterate over memblock regions rather that check each PFN [026/131] mm: remove early_pfn_in_nid() and CONFIG_NODES_SPAN_OTHER_NODES [027/131] mm: free_area_init: allow defining max_zone_pfn in descending order [028/131] mm: rename free_area_init_node() to free_area_init_memoryless_node() [029/131] mm: clean up free_area_init_node() and its helpers [030/131] mm: simplify find_min_pfn_with_active_regions() [031/131] docs/vm: update memory-models documentation [032/131] mm/page_alloc.c: bad_[reason\|flags] is not necessary when PageHWPoison [033/131] mm/page_alloc.c: bad_flags is not necessary for bad_page() [034/131] mm/page_alloc.c: rename free_pages_check_bad() to check_free_page_bad() [035/131] mm/page_alloc.c: rename free_pages_check() to check_free_page() [036/131] mm/page_alloc.c: extract check_[new\|free]_page_bad() common part to page_bad_reason() [037/131] mm,page_alloc,cma: conditionally prefer cma pageblocks for movable allocations [038/131] mm/page_alloc.c: remove unused free_bootmem_with_active_regions [039/131] mm/page_alloc.c: only tune sysctl_lowmem_reserve_ratio value once when changing it [040/131] mm/page_alloc.c: clear out zone->lowmem_reserve[] if the zone is empty [041/131] mm/vmstat.c: do not show lowmem reserve protection information of empty zone [042/131] mm/page_alloc: use ac->high_zoneidx for classzone_idx [043/131] mm/page_alloc: integrate classzone_idx and high_zoneidx [044/131] mm/page_alloc.c: use NODE_MASK_NONE in build_zonelists() [045/131] mm: rename gfpflags_to_migratetype to gfp_migratetype for same convention [046/131] mm/page_alloc.c: reset numa stats for boot pagesets [047/131] mm, page_alloc: reset the zone->watermark_boost early [048/131] mm/page_alloc: restrict and formalize compound_page_dtors[] [049/131] mm/pagealloc.c: call touch_nmi_watchdog() on max order boundaries in deferred init [050/131] mm: initialize deferred pages with interrupts enabled [051/131] mm: call cond_resched() from deferred_init_memmap() [052/131] padata: remove exit routine [053/131] padata: initialize earlier [054/131] padata: allocate work structures for parallel jobs from a pool [055/131] padata: add basic support for multithreaded jobs [056/131] mm: don't track number of pages during deferred initialization [057/131] mm: parallelize deferred_init_memmap() [058/131] mm: make deferred init's max threads arch-specific [059/131] padata: document multithreaded jobs [060/131] mm/page_alloc.c: add missing newline [061/131] khugepaged: add self test [062/131] khugepaged: do not stop collapse if less than half PTEs are referenced [063/131] khugepaged: drain all LRU caches before scanning pages [064/131] khugepaged: drain LRU add pagevec after swapin [065/131] khugepaged: allow to collapse a page shared across fork [066/131] khugepaged: allow to collapse PTE-mapped compound pages [067/131] thp: change CoW semantics for anon-THP [068/131] khugepaged: introduce 'max_ptes_shared' tunable [069/131] hugetlbfs: add arch_hugetlb_valid_size [070/131] hugetlbfs: move hugepagesz= parsing to arch independent code [071/131] hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate [072/131] hugetlbfs: clean up command line processing [073/131] hugetlbfs: fix changes to command line processing [074/131] mm/hugetlb: avoid unnecessary check on pud and pmd entry in huge_pte_offset [075/131] arm64/mm: drop __HAVE_ARCH_HUGE_PTEP_GET [076/131] mm/hugetlb: define a generic fallback for is_hugepage_only_range() [077/131] mm/hugetlb: define a generic fallback for arch_clear_hugepage_flags() [078/131] mm: simplify calling a compound page destructor [079/131] mm/vmscan.c: use update_lru_size() in update_lru_sizes() [080/131] mm/vmscan: count layzfree pages and fix nr_isolated_* mismatch [081/131] mm/vmscan.c: change prototype for shrink_page_list [082/131] mm/vmscan: update the comment of should_continue_reclaim() [083/131] mm: fix NUMA node file count error in replace_page_cache() [084/131] mm: memcontrol: fix stat-corrupting race in charge moving [085/131] mm: memcontrol: drop @compound parameter from memcg charging API [086/131] mm: shmem: remove rare optimization when swapin races with hole punching [087/131] mm: memcontrol: move out cgroup swaprate throttling [088/131] mm: memcontrol: convert page cache to a new mem_cgroup_charge() API [089/131] mm: memcontrol: prepare uncharging for removal of private page type counters [090/131] mm: memcontrol: prepare move_account for removal of private page type counters [091/131] mm: memcontrol: prepare cgroup vmstat infrastructure for native anon counters [092/131] mm: memcontrol: switch to native NR_FILE_PAGES and NR_SHMEM counters [093/131] mm: memcontrol: switch to native NR_ANON_MAPPED counter [094/131] mm: memcontrol: switch to native NR_ANON_THPS counter [095/131] mm: memcontrol: convert anon and file-thp to new mem_cgroup_charge() API [096/131] mm: memcontrol: drop unused try/commit/cancel charge API [097/131] mm: memcontrol: prepare swap controller setup for integration [098/131] mm: memcontrol: make swap tracking an integral part of memory control [099/131] mm: memcontrol: charge swapin pages on instantiation [100/131] mm: memcontrol: document the new swap control behavior [101/131] mm: memcontrol: delete unused lrucare handling [102/131] mm: memcontrol: update page->mem_cgroup stability rules [103/131] mm: fix LRU balancing effect of new transparent huge pages [104/131] mm: keep separate anon and file statistics on page reclaim activity [105/131] mm: allow swappiness that prefers reclaiming anon over the file workingset [106/131] mm: fold and remove lru_cache_add_anon() and lru_cache_add_file() [107/131] mm: workingset: let cache workingset challenge anon [108/131] mm: remove use-once cache bias from LRU balancing [109/131] mm: vmscan: drop unnecessary div0 avoidance rounding in get_scan_count() [110/131] mm: base LRU balancing on an explicit cost model [111/131] mm: deactivations shouldn't bias the LRU balance [112/131] mm: only count actual rotations as LRU reclaim cost [113/131] mm: balance LRU lists based on relative thrashing [114/131] mm: vmscan: determine anon/file pressure balance at the reclaim root [115/131] mm: vmscan: reclaim writepage is IO cost [116/131] mm: vmscan: limit the range of LRU type balancing [117/131] mm: swap: fix vmstats for huge pages [118/131] mm: swap: memcg: fix memcg stats for huge pages [119/131] tools/vm/page_owner_sort.c: filter out unneeded line [120/131] mm, mempolicy: fix up gup usage in lookup_node [121/131] include/linux/memblock.h: fix minor typo and unclear comment [122/131] sparc32: register memory occupied by kernel as memblock.memory [123/131] hugetlbfs: get unmapped area below TASK_UNMAPPED_BASE for hugetlbfs [124/131] mm: thp: don't need to drain lru cache when splitting and mlocking THP [125/131] powerpc/mm: drop platform defined pmd_mknotpresent() [126/131] mm/thp: rename pmd_mknotpresent() as pmd_mkinvalid() [127/131] drivers/base/memory.c: cache memory blocks in xarray to accelerate lookup [128/131] mm: add DEBUG_WX support [129/131] riscv: support DEBUG_WX [130/131] x86: mm: use ARCH_HAS_DEBUG_WX instead of arch defined [131/131] arm64: mm: use ARCH_HAS_DEBUG_WX instead of arch defined

Message ID

20200603230253._MC-eONlj%akpm@linux-foundation.org (mailing list archive)

State

New, archived

Headers

DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD1221531
Date: Wed, 03 Jun 2020 16:02:53 -0700
From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, hannes@cmpxchg.org,
 iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com,
 minchan@kernel.org, mm-commits@vger.kernel.org, riel@surriel.com,
 torvalds@linux-foundation.org
Subject: [patch 110/131] mm: base LRU balancing on an explicit
 cost model
Message-ID: <20200603230253._MC-eONlj%akpm@linux-foundation.org>
In-Reply-To: <20200603155549.e041363450869eaae4c7f05b@linux-foundation.org>
User-Agent: s-nail v14.8.16
Sender: owner-linux-mm@kvack.org
Precedence: bulk

Series

[001/131] mm/slub: fix a memory leak in sysfs_slab_add() | expand

Commit Message

Andrew Morton June 3, 2020, 11:02 p.m. UTC

From: Johannes Weiner <hannes@cmpxchg.org>
Subject: mm: base LRU balancing on an explicit cost model

Currently, scan pressure between the anon and file LRU lists is balanced
based on a mixture of reclaim efficiency and a somewhat vague notion of
"value" of having certain pages in memory over others.  That concept of
value is problematic, because it has caused us to count any event that
remotely makes one LRU list more or less preferrable for reclaim, even
when these events are not directly comparable and impose very different
costs on the system.  One example is referenced file pages that we still
deactivate and referenced anonymous pages that we actually rotate back to
the head of the list.

There is also conceptual overlap with the LRU algorithm itself.  By
rotating recently used pages instead of reclaiming them, the algorithm
already biases the applied scan pressure based on page value.  Thus, when
rebalancing scan pressure due to rotations, we should think of reclaim
cost, and leave assessing the page value to the LRU algorithm.

Lastly, considering both value-increasing as well as value-decreasing
events can sometimes cause the same type of event to be counted twice,
i.e.  how rotating a page increases the LRU value, while reclaiming it
succesfully decreases the value.  In itself this will balance out fine,
but it quietly skews the impact of events that are only recorded once.

The abstract metric of "value", the murky relationship with the LRU
algorithm, and accounting both negative and positive events make the
current pressure balancing model hard to reason about and modify.

This patch switches to a balancing model of accounting the concrete,
actually observed cost of reclaiming one LRU over another.  For now, that
cost includes pages that are scanned but rotated back to the list head. 
Subsequent patches will add consideration for IO caused by refaulting of
recently evicted pages.

Replace struct zone_reclaim_stat with two cost counters in the lruvec, and
make everything that affects cost go through a new lru_note_cost()
function.

Link: http://lkml.kernel.org/r/20200520232525.798933-9-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@surriel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/mmzone.h |   21 ++++++------------
 include/linux/swap.h   |    2 +
 mm/memcontrol.c        |   18 +++++----------
 mm/swap.c              |   21 +++++++-----------
 mm/vmscan.c            |   44 +++++++++++++++++++--------------------
 5 files changed, 46 insertions(+), 60 deletions(-)

--- a/include/linux/mmzone.h~mm-base-lru-balancing-on-an-explicit-cost-model
+++ a/include/linux/mmzone.h
@@ -242,19 +242,6 @@  static inline bool is_active_lru(enum lr
 	return (lru == LRU_ACTIVE_ANON || lru == LRU_ACTIVE_FILE);
 }
 
-struct zone_reclaim_stat {
-	/*
-	 * The pageout code in vmscan.c keeps track of how many of the
-	 * mem/swap backed and file backed pages are referenced.
-	 * The higher the rotated/scanned ratio, the more valuable
-	 * that cache is.
-	 *
-	 * The anon LRU stats live in [0], file LRU stats in [1]
-	 */
-	unsigned long		recent_rotated[2];
-	unsigned long		recent_scanned[2];
-};
-
 enum lruvec_flags {
 	LRUVEC_CONGESTED,		/* lruvec has many dirty pages
 					 * backed by a congested BDI
@@ -263,7 +250,13 @@  enum lruvec_flags {
 
 struct lruvec {
 	struct list_head		lists[NR_LRU_LISTS];
-	struct zone_reclaim_stat	reclaim_stat;
+	/*
+	 * These track the cost of reclaiming one LRU - file or anon -
+	 * over the other. As the observed cost of reclaiming one LRU
+	 * increases, the reclaim scan balance tips toward the other.
+	 */
+	unsigned long			anon_cost;
+	unsigned long			file_cost;
 	/* Evictions & activations on the inactive file list */
 	atomic_long_t			inactive_age;
 	/* Refaults at the time of last reclaim cycle */
--- a/include/linux/swap.h~mm-base-lru-balancing-on-an-explicit-cost-model
+++ a/include/linux/swap.h
@@ -334,6 +334,8 @@  extern unsigned long nr_free_pagecache_p
 
 
 /* linux/mm/swap.c */
+extern void lru_note_cost(struct lruvec *lruvec, bool file,
+			  unsigned int nr_pages);
 extern void lru_cache_add(struct page *);
 extern void lru_add_page_tail(struct page *page, struct page *page_tail,
 			 struct lruvec *lruvec, struct list_head *head);
--- a/mm/memcontrol.c~mm-base-lru-balancing-on-an-explicit-cost-model
+++ a/mm/memcontrol.c
@@ -3853,23 +3853,17 @@  static int memcg_stat_show(struct seq_fi
 	{
 		pg_data_t *pgdat;
 		struct mem_cgroup_per_node *mz;
-		struct zone_reclaim_stat *rstat;
-		unsigned long recent_rotated[2] = {0, 0};
-		unsigned long recent_scanned[2] = {0, 0};
+		unsigned long anon_cost = 0;
+		unsigned long file_cost = 0;
 
 		for_each_online_pgdat(pgdat) {
 			mz = mem_cgroup_nodeinfo(memcg, pgdat->node_id);
-			rstat = &mz->lruvec.reclaim_stat;
 
-			recent_rotated[0] += rstat->recent_rotated[0];
-			recent_rotated[1] += rstat->recent_rotated[1];
-			recent_scanned[0] += rstat->recent_scanned[0];
-			recent_scanned[1] += rstat->recent_scanned[1];
+			anon_cost += mz->lruvec.anon_cost;
+			file_cost += mz->lruvec.file_cost;
 		}
-		seq_printf(m, "recent_rotated_anon %lu\n", recent_rotated[0]);
-		seq_printf(m, "recent_rotated_file %lu\n", recent_rotated[1]);
-		seq_printf(m, "recent_scanned_anon %lu\n", recent_scanned[0]);
-		seq_printf(m, "recent_scanned_file %lu\n", recent_scanned[1]);
+		seq_printf(m, "anon_cost %lu\n", anon_cost);
+		seq_printf(m, "file_cost %lu\n", file_cost);
 	}
 #endif
 
--- a/mm/swap.c~mm-base-lru-balancing-on-an-explicit-cost-model
+++ a/mm/swap.c
@@ -278,15 +278,12 @@  void rotate_reclaimable_page(struct page
 	}
 }
 
-static void update_page_reclaim_stat(struct lruvec *lruvec,
-				     int file, int rotated,
-				     unsigned int nr_pages)
-{
-	struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat;
-
-	reclaim_stat->recent_scanned[file] += nr_pages;
-	if (rotated)
-		reclaim_stat->recent_rotated[file] += nr_pages;
+void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages)
+{
+	if (file)
+		lruvec->file_cost += nr_pages;
+	else
+		lruvec->anon_cost += nr_pages;
 }
 
 static void __activate_page(struct page *page, struct lruvec *lruvec,
@@ -541,7 +538,7 @@  static void lru_deactivate_file_fn(struc
 
 	if (active)
 		__count_vm_event(PGDEACTIVATE);
-	update_page_reclaim_stat(lruvec, file, 0, hpage_nr_pages(page));
+	lru_note_cost(lruvec, !file, hpage_nr_pages(page));
 }
 
 static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec,
@@ -557,7 +554,7 @@  static void lru_deactivate_fn(struct pag
 		add_page_to_lru_list(page, lruvec, lru);
 
 		__count_vm_events(PGDEACTIVATE, hpage_nr_pages(page));
-		update_page_reclaim_stat(lruvec, file, 0, hpage_nr_pages(page));
+		lru_note_cost(lruvec, !file, hpage_nr_pages(page));
 	}
 }
 
@@ -582,7 +579,7 @@  static void lru_lazyfree_fn(struct page
 
 		__count_vm_events(PGLAZYFREE, hpage_nr_pages(page));
 		count_memcg_page_event(page, PGLAZYFREE);
-		update_page_reclaim_stat(lruvec, 1, 0, hpage_nr_pages(page));
+		lru_note_cost(lruvec, 0, hpage_nr_pages(page));
 	}
 }
 
--- a/mm/vmscan.c~mm-base-lru-balancing-on-an-explicit-cost-model
+++ a/mm/vmscan.c
@@ -1916,7 +1916,6 @@  shrink_inactive_list(unsigned long nr_to
 	bool file = is_file_lru(lru);
 	enum vm_event_item item;
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
-	struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat;
 	bool stalled = false;
 
 	while (unlikely(too_many_isolated(pgdat, file, sc))) {
@@ -1940,7 +1939,6 @@  shrink_inactive_list(unsigned long nr_to
 				     &nr_scanned, sc, lru);
 
 	__mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, nr_taken);
-	reclaim_stat->recent_scanned[file] += nr_taken;
 	item = current_is_kswapd() ? PGSCAN_KSWAPD : PGSCAN_DIRECT;
 	if (!cgroup_reclaim(sc))
 		__count_vm_events(item, nr_scanned);
@@ -1960,8 +1958,12 @@  shrink_inactive_list(unsigned long nr_to
 	move_pages_to_lru(lruvec, &page_list);
 
 	__mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, -nr_taken);
-	reclaim_stat->recent_rotated[0] += stat.nr_activate[0];
-	reclaim_stat->recent_rotated[1] += stat.nr_activate[1];
+	/*
+	 * Rotating pages costs CPU without actually
+	 * progressing toward the reclaim goal.
+	 */
+	lru_note_cost(lruvec, 0, stat.nr_activate[0]);
+	lru_note_cost(lruvec, 1, stat.nr_activate[1]);
 	item = current_is_kswapd() ? PGSTEAL_KSWAPD : PGSTEAL_DIRECT;
 	if (!cgroup_reclaim(sc))
 		__count_vm_events(item, nr_reclaimed);
@@ -2013,7 +2015,6 @@  static void shrink_active_list(unsigned
 	LIST_HEAD(l_active);
 	LIST_HEAD(l_inactive);
 	struct page *page;
-	struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat;
 	unsigned nr_deactivate, nr_activate;
 	unsigned nr_rotated = 0;
 	int file = is_file_lru(lru);
@@ -2027,7 +2028,6 @@  static void shrink_active_list(unsigned
 				     &nr_scanned, sc, lru);
 
 	__mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, nr_taken);
-	reclaim_stat->recent_scanned[file] += nr_taken;
 
 	__count_vm_events(PGREFILL, nr_scanned);
 	__count_memcg_events(lruvec_memcg(lruvec), PGREFILL, nr_scanned);
@@ -2085,7 +2085,7 @@  static void shrink_active_list(unsigned
 	 * helps balance scan pressure between file and anonymous pages in
 	 * get_scan_count.
 	 */
-	reclaim_stat->recent_rotated[file] += nr_rotated;
+	lru_note_cost(lruvec, file, nr_rotated);
 
 	nr_activate = move_pages_to_lru(lruvec, &l_active);
 	nr_deactivate = move_pages_to_lru(lruvec, &l_inactive);
@@ -2242,13 +2242,13 @@  static void get_scan_count(struct lruvec
 {
 	struct mem_cgroup *memcg = lruvec_memcg(lruvec);
 	int swappiness = mem_cgroup_swappiness(memcg);
-	struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat;
 	u64 fraction[2];
 	u64 denominator = 0;	/* gcc */
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
 	unsigned long anon_prio, file_prio;
 	enum scan_balance scan_balance;
 	unsigned long anon, file;
+	unsigned long totalcost;
 	unsigned long ap, fp;
 	enum lru_list lru;
 
@@ -2324,26 +2324,26 @@  static void get_scan_count(struct lruvec
 		lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, MAX_NR_ZONES);
 
 	spin_lock_irq(&pgdat->lru_lock);
-	if (unlikely(reclaim_stat->recent_scanned[0] > anon / 4)) {
-		reclaim_stat->recent_scanned[0] /= 2;
-		reclaim_stat->recent_rotated[0] /= 2;
-	}
-
-	if (unlikely(reclaim_stat->recent_scanned[1] > file / 4)) {
-		reclaim_stat->recent_scanned[1] /= 2;
-		reclaim_stat->recent_rotated[1] /= 2;
+	totalcost = lruvec->anon_cost + lruvec->file_cost;
+	if (unlikely(totalcost > (anon + file) / 4)) {
+		lruvec->anon_cost /= 2;
+		lruvec->file_cost /= 2;
+		totalcost /= 2;
 	}
 
 	/*
 	 * The amount of pressure on anon vs file pages is inversely
-	 * proportional to the fraction of recently scanned pages on
-	 * each list that were recently referenced and in active use.
+	 * proportional to the assumed cost of reclaiming each list,
+	 * as determined by the share of pages that are likely going
+	 * to refault or rotate on each list (recently referenced),
+	 * times the relative IO cost of bringing back a swapped out
+	 * anonymous page vs reloading a filesystem page (swappiness).
 	 */
-	ap = anon_prio * (reclaim_stat->recent_scanned[0] + 1);
-	ap /= reclaim_stat->recent_rotated[0] + 1;
+	ap = anon_prio * (totalcost + 1);
+	ap /= lruvec->anon_cost + 1;
 
-	fp = file_prio * (reclaim_stat->recent_scanned[1] + 1);
-	fp /= reclaim_stat->recent_rotated[1] + 1;
+	fp = file_prio * (totalcost + 1);
+	fp /= lruvec->file_cost + 1;
 	spin_unlock_irq(&pgdat->lru_lock);
 
 	fraction[0] = ap;

[110/131] mm: base LRU balancing on an explicit cost model

Commit Message

Patch