[PATCHv6,2/3] mm/gup: fix omission of check on FOLL_LONGTERM in gup fast path

Commit Message

Pingfan Liu March 16, 2020, 4:34 a.m. UTC

FOLL_LONGTERM is a special case of FOLL_PIN. It suggests a pin which is
going to be given to hardware and can't move. It would truncate CMA
permanently and should be excluded.

In gup slow path, slow path, where
__gup_longterm_locked->check_and_migrate_cma_pages() handles FOLL_LONGTERM,
but in fast path, there lacks such a check, which means a possible leak of
CMA page to longterm pinned.

Place a check in try_grab_compound_head() in the fast path to fix the leak,
and if FOLL_LONGTERM happens on CMA, it will fall back to slow path to
migrate the page.

Some note about the check:
Huge page's subpages have the same migrate type due to either
allocation from a free_list[] or alloc_contig_range() with param
MIGRATE_MOVABLE. So it is enough to check on a single subpage
by is_migrate_cma_page(subpage)

Signed-off-by: Pingfan Liu <[e-mail]>
Cc: Ira Weiny <[e-mail]>
Cc: Andrew Morton <[e-mail]>
Cc: Mike Rapoport <[e-mail]>
Cc: Dan Williams <[e-mail]>
Cc: Matthew Wilcox <[e-mail]>
Cc: John Hubbard <[e-mail]>
Cc: "Aneesh Kumar K.V" <[e-mail]>
Cc: Christoph Hellwig <[e-mail]>
Cc: Shuah Khan <[e-mail]>
Cc: Jason Gunthorpe <[e-mail]>
To: [e-mail]
Cc: [e-mail]
---
 mm/gup.c | 9 +++++++++
 1 file changed, 9 insertions(+)

--
2.7.5

Comments

Christoph Hellwig March 16, 2020, 8:55 a.m. UTC | #1

On Mon, Mar 16, 2020 at 12:34:03PM +0800, Pingfan Liu wrote:
> FOLL_LONGTERM is a special case of FOLL_PIN. It suggests a pin which is
> going to be given to hardware and can't move. It would truncate CMA
> permanently and should be excluded.
> 
> In gup slow path, slow path, where
> __gup_longterm_locked->check_and_migrate_cma_pages() handles FOLL_LONGTERM,
> but in fast path, there lacks such a check, which means a possible leak of
> CMA page to longterm pinned.
> 
> Place a check in try_grab_compound_head() in the fast path to fix the leak,
> and if FOLL_LONGTERM happens on CMA, it will fall back to slow path to
> migrate the page.
> 
> Some note about the check:
> Huge page's subpages have the same migrate type due to either
> allocation from a free_list[] or alloc_contig_range() with param
> MIGRATE_MOVABLE. So it is enough to check on a single subpage
> by is_migrate_cma_page(subpage)
> 
> Signed-off-by: Pingfan Liu <[e-mail]>
> Cc: Ira Weiny <[e-mail]>
> Cc: Andrew Morton <[e-mail]>
> Cc: Mike Rapoport <[e-mail]>
> Cc: Dan Williams <[e-mail]>
> Cc: Matthew Wilcox <[e-mail]>
> Cc: John Hubbard <[e-mail]>
> Cc: "Aneesh Kumar K.V" <[e-mail]>
> Cc: Christoph Hellwig <[e-mail]>
> Cc: Shuah Khan <[e-mail]>
> Cc: Jason Gunthorpe <[e-mail]>
> To: [e-mail]
> Cc: [e-mail]
> ---
>  mm/gup.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/mm/gup.c b/mm/gup.c
> index 9df77b1..78132cf 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -89,6 +89,15 @@ static __maybe_unused struct page *try_grab_compound_head(struct page *page,
>  		int orig_refs = refs;
> 
>  		/*
> +		 * Huge page's subpages have the same migrate type due to either
> +		 * allocation from a free_list[] or alloc_contig_range() with
> +		 * param MIGRATE_MOVABLE. So it is enough to check on a subpage.
> +		 */
> +		if (unlikely(flags & FOLL_LONGTERM) &&
> +			is_migrate_cma_page(page))
> +			return NULL;

Wrong indentation.  We either align two tabs for continuation
statements, or after the opening brace of the previous line.  With a
likely or unlikely thrown in the former IMHO looks much better.  E.g.

		if (unlikely(flags & FOLL_LONGTERM) &&
				is_migrate_cma_page(page))
			return NULL;

Pingfan Liu March 17, 2020, 11:45 a.m. UTC | #2

On Mon, Mar 16, 2020 at 4:55 PM Christoph Hellwig <[e-mail]> wrote:
>
[...]
>
> Wrong indentation.  We either align two tabs for continuation
> statements, or after the opening brace of the previous line.  With a
> likely or unlikely thrown in the former IMHO looks much better.  E.g.
>
>                 if (unlikely(flags & FOLL_LONGTERM) &&
>                                 is_migrate_cma_page(page))
>                         return NULL;
>
OK, thanks. I will send out V7 to fix it.

Regards,
Pingfan

Patch

diff --git a/mm/gup.c b/mm/gup.c
index 9df77b1..78132cf 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -89,6 +89,15 @@  static __maybe_unused struct page *try_grab_compound_head(struct page *page,
 		int orig_refs = refs;

 		/*
+		 * Huge page's subpages have the same migrate type due to either
+		 * allocation from a free_list[] or alloc_contig_range() with
+		 * param MIGRATE_MOVABLE. So it is enough to check on a subpage.
+		 */
+		if (unlikely(flags & FOLL_LONGTERM) &&
+			is_migrate_cma_page(page))
+			return NULL;
+
+		/*
 		 * When pinning a compound page of order > 1 (which is what
 		 * hpage_pincount_available() checks for), use an exact count to
 		 * track it, via hpage_pincount_add/_sub().