[010/131] mm: clarify __GFP_MEMALLOC usage

Message ID	20200603225649.lZMovsIIu%akpm@linux-foundation.org (mailing list archive)
State	New, archived
Headers	show Return-Path: <SRS0=OQel=7Q=kvack.org=owner-linux-mm@kernel.org> DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 741BC20659 Date: Wed, 03 Jun 2020 15:56:49 -0700 From: Andrew Morton <akpm@linux-foundation.org> To: akpm@linux-foundation.org, jhubbard@nvidia.com, joel@joelfernandes.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, paulmck@kernel.org, rientjes@google.com, torvalds@linux-foundation.org Subject: [patch 010/131] mm: clarify __GFP_MEMALLOC usage Message-ID: <20200603225649.lZMovsIIu%akpm@linux-foundation.org> In-Reply-To: <20200603155549.e041363450869eaae4c7f05b@linux-foundation.org> User-Agent: s-nail v14.8.16 Sender: owner-linux-mm@kvack.org Precedence: bulk
Series	[001/131] mm/slub: fix a memory leak in sysfs_slab_add() \| expand [001/131] mm/slub: fix a memory leak in sysfs_slab_add() [002/131] mm/memcg: optimize memory.numa_stat like memory.stat [003/131] mm/gup: move __get_user_pages_fast() down a few lines in gup.c [004/131] mm/gup: refactor and de-duplicate gup_fast() code [005/131] mm/gup: introduce pin_user_pages_fast_only() [006/131] drm/i915: convert get_user_pages() --> pin_user_pages() [007/131] mm/gup: might_lock_read(mmap_sem) in get_user_pages_fast() [008/131] kasan: stop tests being eliminated as dead code with FORTIFY_SOURCE [009/131] string.h: fix incompatibility between FORTIFY_SOURCE and KASAN [010/131] mm: clarify __GFP_MEMALLOC usage [011/131] mm: memblock: replace dereferences of memblock_region.nid with API calls [012/131] mm: make early_pfn_to_nid() and related defintions close to each other [013/131] mm: remove CONFIG_HAVE_MEMBLOCK_NODE_MAP option [014/131] mm: free_area_init: use maximal zone PFNs rather than zone sizes [015/131] mm: use free_area_init() instead of free_area_init_nodes() [016/131] alpha: simplify detection of memory zone boundaries [017/131] arm: simplify detection of memory zone boundaries [018/131] arm64: simplify detection of memory zone boundaries for UMA configs [019/131] csky: simplify detection of memory zone boundaries [020/131] m68k: mm: simplify detection of memory zone boundaries [021/131] parisc: simplify detection of memory zone boundaries [022/131] sparc32: simplify detection of memory zone boundaries [023/131] unicore32: simplify detection of memory zone boundaries [024/131] xtensa: simplify detection of memory zone boundaries [025/131] mm: memmap_init: iterate over memblock regions rather that check each PFN [026/131] mm: remove early_pfn_in_nid() and CONFIG_NODES_SPAN_OTHER_NODES [027/131] mm: free_area_init: allow defining max_zone_pfn in descending order [028/131] mm: rename free_area_init_node() to free_area_init_memoryless_node() [029/131] mm: clean up free_area_init_node() and its helpers [030/131] mm: simplify find_min_pfn_with_active_regions() [031/131] docs/vm: update memory-models documentation [032/131] mm/page_alloc.c: bad_[reason\|flags] is not necessary when PageHWPoison [033/131] mm/page_alloc.c: bad_flags is not necessary for bad_page() [034/131] mm/page_alloc.c: rename free_pages_check_bad() to check_free_page_bad() [035/131] mm/page_alloc.c: rename free_pages_check() to check_free_page() [036/131] mm/page_alloc.c: extract check_[new\|free]_page_bad() common part to page_bad_reason() [037/131] mm,page_alloc,cma: conditionally prefer cma pageblocks for movable allocations [038/131] mm/page_alloc.c: remove unused free_bootmem_with_active_regions [039/131] mm/page_alloc.c: only tune sysctl_lowmem_reserve_ratio value once when changing it [040/131] mm/page_alloc.c: clear out zone->lowmem_reserve[] if the zone is empty [041/131] mm/vmstat.c: do not show lowmem reserve protection information of empty zone [042/131] mm/page_alloc: use ac->high_zoneidx for classzone_idx [043/131] mm/page_alloc: integrate classzone_idx and high_zoneidx [044/131] mm/page_alloc.c: use NODE_MASK_NONE in build_zonelists() [045/131] mm: rename gfpflags_to_migratetype to gfp_migratetype for same convention [046/131] mm/page_alloc.c: reset numa stats for boot pagesets [047/131] mm, page_alloc: reset the zone->watermark_boost early [048/131] mm/page_alloc: restrict and formalize compound_page_dtors[] [049/131] mm/pagealloc.c: call touch_nmi_watchdog() on max order boundaries in deferred init [050/131] mm: initialize deferred pages with interrupts enabled [051/131] mm: call cond_resched() from deferred_init_memmap() [052/131] padata: remove exit routine [053/131] padata: initialize earlier [054/131] padata: allocate work structures for parallel jobs from a pool [055/131] padata: add basic support for multithreaded jobs [056/131] mm: don't track number of pages during deferred initialization [057/131] mm: parallelize deferred_init_memmap() [058/131] mm: make deferred init's max threads arch-specific [059/131] padata: document multithreaded jobs [060/131] mm/page_alloc.c: add missing newline [061/131] khugepaged: add self test [062/131] khugepaged: do not stop collapse if less than half PTEs are referenced [063/131] khugepaged: drain all LRU caches before scanning pages [064/131] khugepaged: drain LRU add pagevec after swapin [065/131] khugepaged: allow to collapse a page shared across fork [066/131] khugepaged: allow to collapse PTE-mapped compound pages [067/131] thp: change CoW semantics for anon-THP [068/131] khugepaged: introduce 'max_ptes_shared' tunable [069/131] hugetlbfs: add arch_hugetlb_valid_size [070/131] hugetlbfs: move hugepagesz= parsing to arch independent code [071/131] hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate [072/131] hugetlbfs: clean up command line processing [073/131] hugetlbfs: fix changes to command line processing [074/131] mm/hugetlb: avoid unnecessary check on pud and pmd entry in huge_pte_offset [075/131] arm64/mm: drop __HAVE_ARCH_HUGE_PTEP_GET [076/131] mm/hugetlb: define a generic fallback for is_hugepage_only_range() [077/131] mm/hugetlb: define a generic fallback for arch_clear_hugepage_flags() [078/131] mm: simplify calling a compound page destructor [079/131] mm/vmscan.c: use update_lru_size() in update_lru_sizes() [080/131] mm/vmscan: count layzfree pages and fix nr_isolated_* mismatch [081/131] mm/vmscan.c: change prototype for shrink_page_list [082/131] mm/vmscan: update the comment of should_continue_reclaim() [083/131] mm: fix NUMA node file count error in replace_page_cache() [084/131] mm: memcontrol: fix stat-corrupting race in charge moving [085/131] mm: memcontrol: drop @compound parameter from memcg charging API [086/131] mm: shmem: remove rare optimization when swapin races with hole punching [087/131] mm: memcontrol: move out cgroup swaprate throttling [088/131] mm: memcontrol: convert page cache to a new mem_cgroup_charge() API [089/131] mm: memcontrol: prepare uncharging for removal of private page type counters [090/131] mm: memcontrol: prepare move_account for removal of private page type counters [091/131] mm: memcontrol: prepare cgroup vmstat infrastructure for native anon counters [092/131] mm: memcontrol: switch to native NR_FILE_PAGES and NR_SHMEM counters [093/131] mm: memcontrol: switch to native NR_ANON_MAPPED counter [094/131] mm: memcontrol: switch to native NR_ANON_THPS counter [095/131] mm: memcontrol: convert anon and file-thp to new mem_cgroup_charge() API [096/131] mm: memcontrol: drop unused try/commit/cancel charge API [097/131] mm: memcontrol: prepare swap controller setup for integration [098/131] mm: memcontrol: make swap tracking an integral part of memory control [099/131] mm: memcontrol: charge swapin pages on instantiation [100/131] mm: memcontrol: document the new swap control behavior [101/131] mm: memcontrol: delete unused lrucare handling [102/131] mm: memcontrol: update page->mem_cgroup stability rules [103/131] mm: fix LRU balancing effect of new transparent huge pages [104/131] mm: keep separate anon and file statistics on page reclaim activity [105/131] mm: allow swappiness that prefers reclaiming anon over the file workingset [106/131] mm: fold and remove lru_cache_add_anon() and lru_cache_add_file() [107/131] mm: workingset: let cache workingset challenge anon [108/131] mm: remove use-once cache bias from LRU balancing [109/131] mm: vmscan: drop unnecessary div0 avoidance rounding in get_scan_count() [110/131] mm: base LRU balancing on an explicit cost model [111/131] mm: deactivations shouldn't bias the LRU balance [112/131] mm: only count actual rotations as LRU reclaim cost [113/131] mm: balance LRU lists based on relative thrashing [114/131] mm: vmscan: determine anon/file pressure balance at the reclaim root [115/131] mm: vmscan: reclaim writepage is IO cost [116/131] mm: vmscan: limit the range of LRU type balancing [117/131] mm: swap: fix vmstats for huge pages [118/131] mm: swap: memcg: fix memcg stats for huge pages [119/131] tools/vm/page_owner_sort.c: filter out unneeded line [120/131] mm, mempolicy: fix up gup usage in lookup_node [121/131] include/linux/memblock.h: fix minor typo and unclear comment [122/131] sparc32: register memory occupied by kernel as memblock.memory [123/131] hugetlbfs: get unmapped area below TASK_UNMAPPED_BASE for hugetlbfs [124/131] mm: thp: don't need to drain lru cache when splitting and mlocking THP [125/131] powerpc/mm: drop platform defined pmd_mknotpresent() [126/131] mm/thp: rename pmd_mknotpresent() as pmd_mkinvalid() [127/131] drivers/base/memory.c: cache memory blocks in xarray to accelerate lookup [128/131] mm: add DEBUG_WX support [129/131] riscv: support DEBUG_WX [130/131] x86: mm: use ARCH_HAS_DEBUG_WX instead of arch defined [131/131] arm64: mm: use ARCH_HAS_DEBUG_WX instead of arch defined

Message ID

20200603225649.lZMovsIIu%akpm@linux-foundation.org (mailing list archive)

State

New, archived

Headers

DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 741BC20659
Date: Wed, 03 Jun 2020 15:56:49 -0700
From: Andrew Morton <akpm@linux-foundation.org>
To: akpm@linux-foundation.org, jhubbard@nvidia.com,
 joel@joelfernandes.org, linux-mm@kvack.org, mhocko@suse.com,
 mm-commits@vger.kernel.org, neilb@suse.de, paulmck@kernel.org,
 rientjes@google.com, torvalds@linux-foundation.org
Subject: [patch 010/131] mm: clarify __GFP_MEMALLOC usage
Message-ID: <20200603225649.lZMovsIIu%akpm@linux-foundation.org>
In-Reply-To: <20200603155549.e041363450869eaae4c7f05b@linux-foundation.org>
User-Agent: s-nail v14.8.16
Sender: owner-linux-mm@kvack.org
Precedence: bulk

Series

[001/131] mm/slub: fix a memory leak in sysfs_slab_add() | expand

Commit Message

Andrew Morton June 3, 2020, 10:56 p.m. UTC

From: Michal Hocko <mhocko@suse.com>
Subject: mm: clarify __GFP_MEMALLOC usage

It seems that the existing documentation is not explicit about the
expected usage and potential risks enough.  While it is calls out that
users have to free memory when using this flag it is not really apparent
that users have to careful to not deplete memory reserves and that they
should implement some sort of throttling wrt.  freeing process.

This is partly based on Neil's explanation [1].

Let's also call out that a pre allocated pool allocator should be
considered.

[1] http://lkml.kernel.org/r/877dz0yxoa.fsf@notabene.neil.brown.name

[akpm@linux-foundation.org: coding style fixes]
[mhocko@kernel.org: update]
  Link: http://lkml.kernel.org/r/20200406070137.GC19426@dhcp22.suse.cz
Link: http://lkml.kernel.org/r/20200403083543.11552-2-mhocko@kernel.org
Signed-off-by: Michal Hocko <mhocko@suse.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Neil Brown <neilb@suse.de>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/gfp.h |    5 +++++
 1 file changed, 5 insertions(+)

--- a/include/linux/gfp.h~mm-clarify-__gfp_memalloc-usage
+++ a/include/linux/gfp.h
@@ -110,6 +110,11 @@  struct vm_area_struct;
  * the caller guarantees the allocation will allow more memory to be freed
  * very shortly e.g. process exiting or swapping. Users either should
  * be the MM or co-ordinating closely with the VM (e.g. swap over NFS).
+ * Users of this flag have to be extremely careful to not deplete the reserve
+ * completely and implement a throttling mechanism which controls the
+ * consumption of the reserve based on the amount of freed memory.
+ * Usage of a pre-allocated pool (e.g. mempool) should be always considered
+ * before using this flag.
  *
  * %__GFP_NOMEMALLOC is used to explicitly forbid access to emergency reserves.
  * This takes precedence over the %__GFP_MEMALLOC flag if both are set.

[010/131] mm: clarify __GFP_MEMALLOC usage

Commit Message

Patch