diff mbox series

[2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd()

Message ID 20230731074829.79309-3-wangkefeng.wang@huawei.com (mailing list archive)
State New, archived
Headers show
Series mm: mremap: fix move page tables | expand

Commit Message

Kefeng Wang July 31, 2023, 7:48 a.m. UTC 
Archs may need to do special things when flushing thp tlb,
so use the more applicable flush_pud_tlb_range() instead of
flush_tlb_range().

Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
 mm/mremap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Catalin Marinas July 31, 2023, 11:05 a.m. UTC | #1 
On Mon, Jul 31, 2023 at 03:48:27PM +0800, Kefeng Wang wrote:
> Archs may need to do special things when flushing thp tlb,
> so use the more applicable flush_pud_tlb_range() instead of
> flush_tlb_range().
> 
> Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
> ---
>  mm/mremap.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/mremap.c b/mm/mremap.c
> index 11e06e4ab33b..1883205fa22b 100644
> --- a/mm/mremap.c
> +++ b/mm/mremap.c
> @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,
>  	VM_BUG_ON(!pmd_none(*new_pmd));
>  
>  	pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
> -	flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
> +	flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);

I don't think that's correct for arm64. The assumption in the
flush_p*d_tlb_range() was that they are called only for block mappings
at that p*d level (and we use FEAT_TTL on arm64 indicating that the leaf
level is level 2 for pmd, 1 for pud). IIUC move_normal_pmd() is only
called for table pmds which would have a leaf level of 3 (the pte).

Same for the next patch doing the equivalent for the pud.
Kefeng Wang July 31, 2023, 11:20 a.m. UTC | #2 
On 2023/7/31 19:05, Catalin Marinas wrote:
> On Mon, Jul 31, 2023 at 03:48:27PM +0800, Kefeng Wang wrote:
>> Archs may need to do special things when flushing thp tlb,
>> so use the more applicable flush_pud_tlb_range() instead of
>> flush_tlb_range().
>>
>> Fixes: 2c91bd4a4e2e ("mm: speed up mremap by 20x on large regions")
>> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
>> ---
>>   mm/mremap.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/mm/mremap.c b/mm/mremap.c
>> index 11e06e4ab33b..1883205fa22b 100644
>> --- a/mm/mremap.c
>> +++ b/mm/mremap.c
>> @@ -284,7 +284,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,
>>   	VM_BUG_ON(!pmd_none(*new_pmd));
>>   
>>   	pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
>> -	flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
>> +	flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
> 
> I don't think that's correct for arm64. The assumption in the
> flush_p*d_tlb_range() was that they are called only for block mappings
> at that p*d level (and we use FEAT_TTL on arm64 indicating that the leaf
> level is level 2 for pmd, 1 for pud). IIUC move_normal_pmd() is only
> called for table pmds which would have a leaf level of 3 (the pte).

oops, yes, this is for NORMAL_PMD case, not HPAGE_PMD, please ignore 
patch 2/3.

> 
> Same for the next patch doing the equivalent for the pud.
>
kernel test robot July 31, 2023, 1:58 p.m. UTC | #3 
Hi Kefeng,

kernel test robot noticed the following build errors:

[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on arm-perf/for-next/perf linus/master v6.5-rc4 next-20230731]
[cannot apply to akpm-mm/mm-everything]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Kefeng-Wang/mm-hugetlb-use-flush_hugetlb_tlb_range-in-move_hugetlb_page_tables/20230731-154016
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link:    https://lore.kernel.org/r/20230731074829.79309-3-wangkefeng.wang%40huawei.com
patch subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd()
config: x86_64-defconfig (https://download.01.org/0day-ci/archive/20230731/202307312137.ormxuS5g-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230731/202307312137.ormxuS5g-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202307312137.ormxuS5g-lkp@intel.com/

All errors (new ones prefixed by >>):

   In file included from <command-line>:
   In function 'move_normal_pmd',
       inlined from 'move_pgt_entry' at mm/mremap.c:463:11,
       inlined from 'move_page_tables' at mm/mremap.c:565:8:
>> include/linux/compiler_types.h:397:45: error: call to '__compiletime_assert_338' declared with attribute error: BUILD_BUG failed
     397 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |                                             ^
   include/linux/compiler_types.h:378:25: note: in definition of macro '__compiletime_assert'
     378 |                         prefix ## suffix();                             \
         |                         ^~~~~~
   include/linux/compiler_types.h:397:9: note: in expansion of macro '_compiletime_assert'
     397 |         _compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
         |         ^~~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
      39 | #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
         |                                     ^~~~~~~~~~~~~~~~~~
   include/linux/build_bug.h:59:21: note: in expansion of macro 'BUILD_BUG_ON_MSG'
      59 | #define BUILD_BUG() BUILD_BUG_ON_MSG(1, "BUILD_BUG failed")
         |                     ^~~~~~~~~~~~~~~~
   include/linux/pgtable.h:1415:49: note: in expansion of macro 'BUILD_BUG'
    1415 | #define flush_pmd_tlb_range(vma, addr, end)     BUILD_BUG()
         |                                                 ^~~~~~~~~
   mm/mremap.c:287:9: note: in expansion of macro 'flush_pmd_tlb_range'
     287 |         flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
         |         ^~~~~~~~~~~~~~~~~~~


vim +/__compiletime_assert_338 +397 include/linux/compiler_types.h

eb5c2d4b45e3d2 Will Deacon 2020-07-21  383  
eb5c2d4b45e3d2 Will Deacon 2020-07-21  384  #define _compiletime_assert(condition, msg, prefix, suffix) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21  385  	__compiletime_assert(condition, msg, prefix, suffix)
eb5c2d4b45e3d2 Will Deacon 2020-07-21  386  
eb5c2d4b45e3d2 Will Deacon 2020-07-21  387  /**
eb5c2d4b45e3d2 Will Deacon 2020-07-21  388   * compiletime_assert - break build and emit msg if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21  389   * @condition: a compile-time constant condition to check
eb5c2d4b45e3d2 Will Deacon 2020-07-21  390   * @msg:       a message to emit if condition is false
eb5c2d4b45e3d2 Will Deacon 2020-07-21  391   *
eb5c2d4b45e3d2 Will Deacon 2020-07-21  392   * In tradition of POSIX assert, this macro will break the build if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21  393   * supplied condition is *false*, emitting the supplied error message if the
eb5c2d4b45e3d2 Will Deacon 2020-07-21  394   * compiler has support to do so.
eb5c2d4b45e3d2 Will Deacon 2020-07-21  395   */
eb5c2d4b45e3d2 Will Deacon 2020-07-21  396  #define compiletime_assert(condition, msg) \
eb5c2d4b45e3d2 Will Deacon 2020-07-21 @397  	_compiletime_assert(condition, msg, __compiletime_assert_, __COUNTER__)
eb5c2d4b45e3d2 Will Deacon 2020-07-21  398
kernel test robot July 31, 2023, 9:43 p.m. UTC | #4 
Hi Kefeng,

kernel test robot noticed the following build errors:

[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on arm-perf/for-next/perf linus/master v6.5-rc4]
[cannot apply to akpm-mm/mm-everything]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Kefeng-Wang/mm-hugetlb-use-flush_hugetlb_tlb_range-in-move_hugetlb_page_tables/20230731-154016
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link:    https://lore.kernel.org/r/20230731074829.79309-3-wangkefeng.wang%40huawei.com
patch subject: [PATCH 2/4] mm: mremap: use flush_pmd_tlb_range() in move_normal_pmd()
config: x86_64-randconfig-x003-20230731 (https://download.01.org/0day-ci/archive/20230801/202308010553.KxefZFdO-lkp@intel.com/config)
compiler: clang version 16.0.4 (https://github.com/llvm/llvm-project.git ae42196bc493ffe877a7e3dff8be32035dea4d07)
reproduce: (https://download.01.org/0day-ci/archive/20230801/202308010553.KxefZFdO-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308010553.KxefZFdO-lkp@intel.com/

All errors (new ones prefixed by >>):

>> ld.lld: error: call to __compiletime_assert_860 marked "dontcall-error": BUILD_BUG failed
diff mbox series

Patch

diff --git a/mm/mremap.c b/mm/mremap.c
index 11e06e4ab33b..1883205fa22b 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -284,7 +284,7 @@  static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,
 	VM_BUG_ON(!pmd_none(*new_pmd));
 
 	pmd_populate(mm, new_pmd, pmd_pgtable(pmd));
-	flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
+	flush_pmd_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
 	if (new_ptl != old_ptl)
 		spin_unlock(new_ptl);
 	spin_unlock(old_ptl);