[052/155] mm/gup: fix omission of check on FOLL_LONGTERM in gup fast path

Commit Message

Andrew Morton April 2, 2020, 4:06 a.m. UTC

From: Pingfan Liu <[e-mail]>
Subject: mm/gup: fix omission of check on FOLL_LONGTERM in gup fast path

FOLL_LONGTERM is a special case of FOLL_PIN.  It suggests a pin which is
going to be given to hardware and can't move.  It would truncate CMA
permanently and should be excluded.

In gup slow path, where
__gup_longterm_locked->check_and_migrate_cma_pages() handles
FOLL_LONGTERM, but in fast path, there lacks such a check, which means a
possible leak of CMA page to longterm pinned.

Place a check in try_grab_compound_head() in the fast path to fix the
leak, and if FOLL_LONGTERM happens on CMA, it will fall back to slow path
to migrate the page.

Some note about the check: Huge page's subpages have the same migrate type
due to either allocation from a free_list[] or alloc_contig_range() with
param MIGRATE_MOVABLE.  So it is enough to check on a single subpage by
is_migrate_cma_page(subpage)

Link: http://lkml.kernel.org/r/[e-mail]
Signed-off-by: Pingfan Liu <[e-mail]>
Reviewed-by: Christoph Hellwig <[e-mail]>
Reviewed-by: Jason Gunthorpe <[e-mail]>
Cc: Ira Weiny <[e-mail]>
Cc: Mike Rapoport <[e-mail]>
Cc: Dan Williams <[e-mail]>
Cc: Matthew Wilcox <[e-mail]>
Cc: John Hubbard <[e-mail]>
Cc: "Aneesh Kumar K.V" <[e-mail]>
Cc: Christoph Hellwig <[e-mail]>
Cc: Shuah Khan <[e-mail]>
Cc: Jason Gunthorpe <[e-mail]>
Signed-off-by: Andrew Morton <[e-mail]>
---

 mm/gup.c |    8 ++++++++
 1 file changed, 8 insertions(+)

Patch

--- a/mm/gup.c~mm-gup-fix-omission-of-check-on-foll_longterm-in-gup-fast-path
+++ a/mm/gup.c
@@ -89,6 +89,14 @@  static __maybe_unused struct page *try_g
 		int orig_refs = refs;
 
 		/*
+		 * Can't do FOLL_LONGTERM + FOLL_PIN with CMA in the gup fast
+		 * path, so fail and let the caller fall back to the slow path.
+		 */
+		if (unlikely(flags & FOLL_LONGTERM) &&
+				is_migrate_cma_page(page))
+			return NULL;
+
+		/*
 		 * When pinning a compound page of order > 1 (which is what
 		 * hpage_pincount_available() checks for), use an exact count to
 		 * track it, via hpage_pincount_add/_sub().