| Message ID | 20200715123513.42240-1-kirill.shutemov@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | mm: Fix warning in move_normal_pmd() | expand |
On Wed, Jul 15, 2020 at 03:35:13PM +0300, Kirill A. Shutemov wrote: > mremap(2) does not allow source and destination regions to overlap, but > shift_arg_pages() calls move_page_tables() directly and in this case the > source and destination overlap often. It confuses move_normal_pmd(): > > WARNING: CPU: 3 PID: 27091 at mm/mremap.c:211 move_page_tables+0x6ef/0x720 > > move_normal_pmd() expects the destination PMD to be empty, but when > ranges overlap nobody removes PTE page tables on source side. > move_ptes() only removes PTE entries, leaving tables behind. > When the source PMD becomes destination and alignment/size is right we > step onto the warning. > > The warning is harmless: kernel correctly fallbacks to handle entries on > per-entry basis. A link to the debugging effort could be added to the change log: https://lore.kernel.org/lkml/20200713025354.GB3644504@google.com/ > The fix is to avoid move_normal_pmd() if we see that source and > destination ranges overlap. Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org> And one thing that bothers me: > mm/mremap.c | 14 +++++++++++++- > 1 file changed, 13 insertions(+), 1 deletion(-) > > diff --git a/mm/mremap.c b/mm/mremap.c > index 5dd572d57ca9..e33fcee541fe 100644 > --- a/mm/mremap.c > +++ b/mm/mremap.c > @@ -245,6 +245,18 @@ unsigned long move_page_tables(struct vm_area_struct *vma, > unsigned long extent, next, old_end; > struct mmu_notifier_range range; > pmd_t *old_pmd, *new_pmd; > + bool overlaps; > + > + /* > + * shift_arg_pages() can call move_page_tables() on overlapping ranges. > + * In this case we cannot use move_normal_pmd() because destination pmd > + * might be established page table: move_ptes() doesn't free page > + * table. > + */ > + if (old_addr > new_addr) > + overlaps = old_addr - new_addr < len; > + else > + overlaps = new_addr - old_addr < len; Does the code really work properly if old_addr < new_addr and overlaps == true ? If not, then we should add a warning here in the else IMHO: if (old_addr >= new_addr) { overlaps = old_addr - new_addr < len; } else { overlaps = new_addr - old_addr < len; WARN_ON(overlaps); } (More so, since you have added code that detects overlaps for such a case). thanks, - Joel > > old_end = old_addr + len; > flush_cache_range(vma, old_addr, old_end); > @@ -282,7 +294,7 @@ unsigned long move_page_tables(struct vm_area_struct *vma, > split_huge_pmd(vma, old_pmd, old_addr); > if (pmd_trans_unstable(old_pmd)) > continue; > - } else if (extent == PMD_SIZE) { > + } else if (!overlaps && extent == PMD_SIZE) { > #ifdef CONFIG_HAVE_MOVE_PMD > /* > * If the extent is PMD-sized, try to speed the move by > -- > 2.26.2 > >
On Wed, Jul 15, 2020 at 09:14:51AM -0400, Joel Fernandes wrote: > On Wed, Jul 15, 2020 at 03:35:13PM +0300, Kirill A. Shutemov wrote: > > mremap(2) does not allow source and destination regions to overlap, but > > shift_arg_pages() calls move_page_tables() directly and in this case the > > source and destination overlap often. It confuses move_normal_pmd(): > > > > WARNING: CPU: 3 PID: 27091 at mm/mremap.c:211 move_page_tables+0x6ef/0x720 > > > > move_normal_pmd() expects the destination PMD to be empty, but when > > ranges overlap nobody removes PTE page tables on source side. > > move_ptes() only removes PTE entries, leaving tables behind. > > When the source PMD becomes destination and alignment/size is right we > > step onto the warning. > > > > The warning is harmless: kernel correctly fallbacks to handle entries on > > per-entry basis. > > A link to the debugging effort could be added to the change log: > https://lore.kernel.org/lkml/20200713025354.GB3644504@google.com/ > > > The fix is to avoid move_normal_pmd() if we see that source and > > destination ranges overlap. > > Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org> Thanks. > And one thing that bothers me: > > > mm/mremap.c | 14 +++++++++++++- > > 1 file changed, 13 insertions(+), 1 deletion(-) > > > > diff --git a/mm/mremap.c b/mm/mremap.c > > index 5dd572d57ca9..e33fcee541fe 100644 > > --- a/mm/mremap.c > > +++ b/mm/mremap.c > > @@ -245,6 +245,18 @@ unsigned long move_page_tables(struct vm_area_struct *vma, > > unsigned long extent, next, old_end; > > struct mmu_notifier_range range; > > pmd_t *old_pmd, *new_pmd; > > + bool overlaps; > > + > > + /* > > + * shift_arg_pages() can call move_page_tables() on overlapping ranges. > > + * In this case we cannot use move_normal_pmd() because destination pmd > > + * might be established page table: move_ptes() doesn't free page > > + * table. > > + */ > > + if (old_addr > new_addr) > > + overlaps = old_addr - new_addr < len; > > + else > > + overlaps = new_addr - old_addr < len; > > Does the code really work properly if old_addr < new_addr and overlaps == > true ? If not, then we should add a warning here in the else IMHO: > > if (old_addr >= new_addr) { > overlaps = old_addr - new_addr < len; > } else { > overlaps = new_addr - old_addr < len; > WARN_ON(overlaps); > } > > (More so, since you have added code that detects overlaps for such a case). Right, I'll update. But we should warn for old_addr == new_addr too. Nobody should do this. > > thanks, > > - Joel > > > > > old_end = old_addr + len; > > flush_cache_range(vma, old_addr, old_end); > > @@ -282,7 +294,7 @@ unsigned long move_page_tables(struct vm_area_struct *vma, > > split_huge_pmd(vma, old_pmd, old_addr); > > if (pmd_trans_unstable(old_pmd)) > > continue; > > - } else if (extent == PMD_SIZE) { > > + } else if (!overlaps && extent == PMD_SIZE) { > > #ifdef CONFIG_HAVE_MOVE_PMD > > /* > > * If the extent is PMD-sized, try to speed the move by > > -- > > 2.26.2 > > > > >
diff --git a/mm/mremap.c b/mm/mremap.c index 5dd572d57ca9..e33fcee541fe 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -245,6 +245,18 @@ unsigned long move_page_tables(struct vm_area_struct *vma, unsigned long extent, next, old_end; struct mmu_notifier_range range; pmd_t *old_pmd, *new_pmd; + bool overlaps; + + /* + * shift_arg_pages() can call move_page_tables() on overlapping ranges. + * In this case we cannot use move_normal_pmd() because destination pmd + * might be established page table: move_ptes() doesn't free page + * table. + */ + if (old_addr > new_addr) + overlaps = old_addr - new_addr < len; + else + overlaps = new_addr - old_addr < len; old_end = old_addr + len; flush_cache_range(vma, old_addr, old_end); @@ -282,7 +294,7 @@ unsigned long move_page_tables(struct vm_area_struct *vma, split_huge_pmd(vma, old_pmd, old_addr); if (pmd_trans_unstable(old_pmd)) continue; - } else if (extent == PMD_SIZE) { + } else if (!overlaps && extent == PMD_SIZE) { #ifdef CONFIG_HAVE_MOVE_PMD /* * If the extent is PMD-sized, try to speed the move by
mremap(2) does not allow source and destination regions to overlap, but shift_arg_pages() calls move_page_tables() directly and in this case the source and destination overlap often. It confuses move_normal_pmd(): WARNING: CPU: 3 PID: 27091 at mm/mremap.c:211 move_page_tables+0x6ef/0x720 move_normal_pmd() expects the destination PMD to be empty, but when ranges overlap nobody removes PTE page tables on source side. move_ptes() only removes PTE entries, leaving tables behind. When the source PMD becomes destination and alignment/size is right we step onto the warning. The warning is harmless: kernel correctly fallbacks to handle entries on per-entry basis. The fix is to avoid move_normal_pmd() if we see that source and destination ranges overlap. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Reported-and-tested-by: Naresh Kamboju <naresh.kamboju@linaro.org> Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions") Cc: Joel Fernandes <joel@joelfernandes.org> Cc: William Kucharski <william.kucharski@oracle.com> --- mm/mremap.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)