mm: Fix warning in move_normal_pmd()
diff mbox series

Message ID 20200715123513.42240-1-kirill.shutemov@linux.intel.com
State New
Headers show
Series
  • mm: Fix warning in move_normal_pmd()
Related show

Commit Message

Kirill A. Shutemov July 15, 2020, 12:35 p.m. UTC
mremap(2) does not allow source and destination regions to overlap, but
shift_arg_pages() calls move_page_tables() directly and in this case the
source and destination overlap often. It confuses move_normal_pmd():

  WARNING: CPU: 3 PID: 27091 at mm/mremap.c:211 move_page_tables+0x6ef/0x720

move_normal_pmd() expects the destination PMD to be empty, but when
ranges overlap nobody removes PTE page tables on source side.
move_ptes() only removes PTE entries, leaving tables behind.
When the source PMD becomes destination and alignment/size is right we
step onto the warning.

The warning is harmless: kernel correctly fallbacks to handle entries on
per-entry basis.

The fix is to avoid move_normal_pmd() if we see that source and
destination ranges overlap.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reported-and-tested-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: William Kucharski <william.kucharski@oracle.com>
---
 mm/mremap.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

Comments

Joel Fernandes July 15, 2020, 1:14 p.m. UTC | #1
On Wed, Jul 15, 2020 at 03:35:13PM +0300, Kirill A. Shutemov wrote:
> mremap(2) does not allow source and destination regions to overlap, but
> shift_arg_pages() calls move_page_tables() directly and in this case the
> source and destination overlap often. It confuses move_normal_pmd():
> 
>   WARNING: CPU: 3 PID: 27091 at mm/mremap.c:211 move_page_tables+0x6ef/0x720
> 
> move_normal_pmd() expects the destination PMD to be empty, but when
> ranges overlap nobody removes PTE page tables on source side.
> move_ptes() only removes PTE entries, leaving tables behind.
> When the source PMD becomes destination and alignment/size is right we
> step onto the warning.
> 
> The warning is harmless: kernel correctly fallbacks to handle entries on
> per-entry basis.

A link to the debugging effort could be added to the change log:
https://lore.kernel.org/lkml/20200713025354.GB3644504@google.com/

> The fix is to avoid move_normal_pmd() if we see that source and
> destination ranges overlap.

Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>

And one thing that bothers me:

>  mm/mremap.c | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/mremap.c b/mm/mremap.c
> index 5dd572d57ca9..e33fcee541fe 100644
> --- a/mm/mremap.c
> +++ b/mm/mremap.c
> @@ -245,6 +245,18 @@ unsigned long move_page_tables(struct vm_area_struct *vma,
>  	unsigned long extent, next, old_end;
>  	struct mmu_notifier_range range;
>  	pmd_t *old_pmd, *new_pmd;
> +	bool overlaps;
> +
> +	/*
> +	 * shift_arg_pages() can call move_page_tables() on overlapping ranges.
> +	 * In this case we cannot use move_normal_pmd() because destination pmd
> +	 * might be established page table: move_ptes() doesn't free page
> +	 * table.
> +	 */
> +	if (old_addr > new_addr)
> +		overlaps = old_addr - new_addr < len;
> +	else
> +		overlaps = new_addr - old_addr < len;

Does the code really work properly if old_addr < new_addr and overlaps ==
true ? If not, then we should add a warning here in the else IMHO:

	if (old_addr >= new_addr) {
		overlaps = old_addr - new_addr < len;
	} else {
		overlaps = new_addr - old_addr < len;
		WARN_ON(overlaps);
	}

(More so, since you have added code that detects overlaps for such a case).

thanks,

 - Joel

>  
>  	old_end = old_addr + len;
>  	flush_cache_range(vma, old_addr, old_end);
> @@ -282,7 +294,7 @@ unsigned long move_page_tables(struct vm_area_struct *vma,
>  			split_huge_pmd(vma, old_pmd, old_addr);
>  			if (pmd_trans_unstable(old_pmd))
>  				continue;
> -		} else if (extent == PMD_SIZE) {
> +		} else if (!overlaps && extent == PMD_SIZE) {
>  #ifdef CONFIG_HAVE_MOVE_PMD
>  			/*
>  			 * If the extent is PMD-sized, try to speed the move by
> -- 
> 2.26.2
> 
>
Kirill A. Shutemov July 15, 2020, 1:42 p.m. UTC | #2
On Wed, Jul 15, 2020 at 09:14:51AM -0400, Joel Fernandes wrote:
> On Wed, Jul 15, 2020 at 03:35:13PM +0300, Kirill A. Shutemov wrote:
> > mremap(2) does not allow source and destination regions to overlap, but
> > shift_arg_pages() calls move_page_tables() directly and in this case the
> > source and destination overlap often. It confuses move_normal_pmd():
> > 
> >   WARNING: CPU: 3 PID: 27091 at mm/mremap.c:211 move_page_tables+0x6ef/0x720
> > 
> > move_normal_pmd() expects the destination PMD to be empty, but when
> > ranges overlap nobody removes PTE page tables on source side.
> > move_ptes() only removes PTE entries, leaving tables behind.
> > When the source PMD becomes destination and alignment/size is right we
> > step onto the warning.
> > 
> > The warning is harmless: kernel correctly fallbacks to handle entries on
> > per-entry basis.
> 
> A link to the debugging effort could be added to the change log:
> https://lore.kernel.org/lkml/20200713025354.GB3644504@google.com/
> 
> > The fix is to avoid move_normal_pmd() if we see that source and
> > destination ranges overlap.
> 
> Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>

Thanks.

> And one thing that bothers me:
> 
> >  mm/mremap.c | 14 +++++++++++++-
> >  1 file changed, 13 insertions(+), 1 deletion(-)
> > 
> > diff --git a/mm/mremap.c b/mm/mremap.c
> > index 5dd572d57ca9..e33fcee541fe 100644
> > --- a/mm/mremap.c
> > +++ b/mm/mremap.c
> > @@ -245,6 +245,18 @@ unsigned long move_page_tables(struct vm_area_struct *vma,
> >  	unsigned long extent, next, old_end;
> >  	struct mmu_notifier_range range;
> >  	pmd_t *old_pmd, *new_pmd;
> > +	bool overlaps;
> > +
> > +	/*
> > +	 * shift_arg_pages() can call move_page_tables() on overlapping ranges.
> > +	 * In this case we cannot use move_normal_pmd() because destination pmd
> > +	 * might be established page table: move_ptes() doesn't free page
> > +	 * table.
> > +	 */
> > +	if (old_addr > new_addr)
> > +		overlaps = old_addr - new_addr < len;
> > +	else
> > +		overlaps = new_addr - old_addr < len;
> 
> Does the code really work properly if old_addr < new_addr and overlaps ==
> true ? If not, then we should add a warning here in the else IMHO:
> 
> 	if (old_addr >= new_addr) {
> 		overlaps = old_addr - new_addr < len;
> 	} else {
> 		overlaps = new_addr - old_addr < len;
> 		WARN_ON(overlaps);
> 	}
> 
> (More so, since you have added code that detects overlaps for such a case).

Right, I'll update. But we should warn for old_addr == new_addr too.
Nobody should do this.

> 
> thanks,
> 
>  - Joel
> 
> >  
> >  	old_end = old_addr + len;
> >  	flush_cache_range(vma, old_addr, old_end);
> > @@ -282,7 +294,7 @@ unsigned long move_page_tables(struct vm_area_struct *vma,
> >  			split_huge_pmd(vma, old_pmd, old_addr);
> >  			if (pmd_trans_unstable(old_pmd))
> >  				continue;
> > -		} else if (extent == PMD_SIZE) {
> > +		} else if (!overlaps && extent == PMD_SIZE) {
> >  #ifdef CONFIG_HAVE_MOVE_PMD
> >  			/*
> >  			 * If the extent is PMD-sized, try to speed the move by
> > -- 
> > 2.26.2
> > 
> > 
>

Patch
diff mbox series

diff --git a/mm/mremap.c b/mm/mremap.c
index 5dd572d57ca9..e33fcee541fe 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -245,6 +245,18 @@  unsigned long move_page_tables(struct vm_area_struct *vma,
 	unsigned long extent, next, old_end;
 	struct mmu_notifier_range range;
 	pmd_t *old_pmd, *new_pmd;
+	bool overlaps;
+
+	/*
+	 * shift_arg_pages() can call move_page_tables() on overlapping ranges.
+	 * In this case we cannot use move_normal_pmd() because destination pmd
+	 * might be established page table: move_ptes() doesn't free page
+	 * table.
+	 */
+	if (old_addr > new_addr)
+		overlaps = old_addr - new_addr < len;
+	else
+		overlaps = new_addr - old_addr < len;
 
 	old_end = old_addr + len;
 	flush_cache_range(vma, old_addr, old_end);
@@ -282,7 +294,7 @@  unsigned long move_page_tables(struct vm_area_struct *vma,
 			split_huge_pmd(vma, old_pmd, old_addr);
 			if (pmd_trans_unstable(old_pmd))
 				continue;
-		} else if (extent == PMD_SIZE) {
+		} else if (!overlaps && extent == PMD_SIZE) {
 #ifdef CONFIG_HAVE_MOVE_PMD
 			/*
 			 * If the extent is PMD-sized, try to speed the move by