[087/155] mm/gup: fix __get_user_pages() on fault retry of hugetlb

Commit Message

Andrew Morton April 2, 2020, 4:08 a.m. UTC

From: Peter Xu <[e-mail]>
Subject: mm/gup: fix __get_user_pages() on fault retry of hugetlb

When follow_hugetlb_page() returns with *locked==0, it means we've got a
VM_FAULT_RETRY within the fauling process and we've released the mmap_sem.
When that happens, we should stop and bail out.

Link: http://lkml.kernel.org/r/[e-mail]
Signed-off-by: Peter Xu <[e-mail]>
Tested-by: Brian Geffon <[e-mail]>
Cc: Andrea Arcangeli <[e-mail]>
Cc: Bobby Powers <[e-mail]>
Cc: David Hildenbrand <[e-mail]>
Cc: Denis Plotnikov <[e-mail]>
Cc: "Dr . David Alan Gilbert" <[e-mail]>
Cc: Hugh Dickins <[e-mail]>
Cc: Jerome Glisse <[e-mail]>
Cc: Johannes Weiner <[e-mail]>
Cc: "Kirill A . Shutemov" <[e-mail]>
Cc: Linus Torvalds <[e-mail]>
Cc: Martin Cracauer <[e-mail]>
Cc: Marty McFadden <[e-mail]>
Cc: Matthew Wilcox <[e-mail]>
Cc: Maya Gokhale <[e-mail]>
Cc: Mel Gorman <[e-mail]>
Cc: Mike Kravetz <[e-mail]>
Cc: Mike Rapoport <[e-mail]>
Cc: Pavel Emelyanov <[e-mail]>
Signed-off-by: Andrew Morton <[e-mail]>
---

 mm/gup.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

Patch

--- a/mm/gup.c~mm-gup-fix-__get_user_pages-on-fault-retry-of-hugetlb
+++ a/mm/gup.c
@@ -1065,6 +1065,16 @@  static long __get_user_pages(struct task
 				i = follow_hugetlb_page(mm, vma, pages, vmas,
 						&start, &nr_pages, i,
 						gup_flags, locked);
+				if (locked && *locked == 0) {
+					/*
+					 * We've got a VM_FAULT_RETRY
+					 * and we've lost mmap_sem.
+					 * We must stop here.
+					 */
+					BUG_ON(gup_flags & FOLL_NOWAIT);
+					BUG_ON(ret != 0);
+					goto out;
+				}
 				continue;
 			}
 		}