| Message ID | 20230105101844.1893104-14-jthoughton@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Based on latest mm-unstable (85b44c25cd1e). | expand |
Hi James,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on next-20230105]
[cannot apply to kvm/queue shuah-kselftest/next shuah-kselftest/fixes arnd-asm-generic/master linus/master kvm/linux-next v6.2-rc2 v6.2-rc1 v6.1 v6.2-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/James-Houghton/hugetlb-don-t-set-PageUptodate-for-UFFDIO_CONTINUE/20230105-182428
patch link: https://lore.kernel.org/r/20230105101844.1893104-14-jthoughton%40google.com
patch subject: [PATCH 13/46] hugetlb: add hugetlb_hgm_walk and hugetlb_walk_step
config: m68k-allmodconfig
compiler: m68k-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/5395f068d45f39d202240799d3a8146226387f5c
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review James-Houghton/hugetlb-don-t-set-PageUptodate-for-UFFDIO_CONTINUE/20230105-182428
git checkout 5395f068d45f39d202240799d3a8146226387f5c
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k SHELL=/bin/bash fs/proc/ mm/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from include/linux/migrate.h:8,
from mm/zsmalloc.c:60:
>> include/linux/hugetlb.h:1275:34: error: return type is an incomplete type
1275 | static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
| ^~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: In function 'hpage_size_to_level':
>> include/linux/hugetlb.h:1277:16: error: 'HUGETLB_LEVEL_PTE' undeclared (first use in this function)
1277 | return HUGETLB_LEVEL_PTE;
| ^~~~~~~~~~~~~~~~~
include/linux/hugetlb.h:1277:16: note: each undeclared identifier is reported only once for each function it appears in
include/linux/hugetlb.h:1277:16: error: 'return' with a value, in function returning void [-Werror=return-type]
include/linux/hugetlb.h:1275:34: note: declared here
1275 | static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
| ^~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1306:40: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1306 | spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
| ^~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_lockptr':
include/linux/hugetlb.h:1308:20: error: invalid use of undefined type 'struct hugetlb_pte'
1308 | return hpte->ptl;
| ^~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1312:37: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1312 | spinlock_t *hugetlb_pte_lock(struct hugetlb_pte *hpte)
| ^~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_lock':
include/linux/hugetlb.h:1314:47: error: passing argument 1 of 'hugetlb_pte_lockptr' from incompatible pointer type [-Werror=incompatible-pointer-types]
1314 | spinlock_t *ptl = hugetlb_pte_lockptr(hpte);
| ^~~~
| |
| struct hugetlb_pte *
include/linux/hugetlb.h:1306:53: note: expected 'struct hugetlb_pte *' but argument is of type 'struct hugetlb_pte *'
1306 | spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
| ~~~~~~~~~~~~~~~~~~~~^~~~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1321:56: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1321 | void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
| ^~~~~~~~~~~
include/linux/hugetlb.h:1323:46: error: parameter 5 ('level') has incomplete type
1323 | enum hugetlb_level level)
| ~~~~~~~~~~~~~~~~~~~^~~~~
include/linux/hugetlb.h:1321:6: error: function declaration isn't a prototype [-Werror=strict-prototypes]
1321 | void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
| ^~~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_populate':
include/linux/hugetlb.h:1325:9: error: implicit declaration of function '__hugetlb_pte_populate'; did you mean 'hugetlb_pte_populate'? [-Werror=implicit-function-declaration]
1325 | __hugetlb_pte_populate(hpte, ptep, shift, level,
| ^~~~~~~~~~~~~~~~~~~~~~
| hugetlb_pte_populate
cc1: some warnings being treated as errors
--
In file included from mm/madvise.c:16:
>> include/linux/hugetlb.h:1275:34: error: return type is an incomplete type
1275 | static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
| ^~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: In function 'hpage_size_to_level':
>> include/linux/hugetlb.h:1277:16: error: 'HUGETLB_LEVEL_PTE' undeclared (first use in this function)
1277 | return HUGETLB_LEVEL_PTE;
| ^~~~~~~~~~~~~~~~~
include/linux/hugetlb.h:1277:16: note: each undeclared identifier is reported only once for each function it appears in
include/linux/hugetlb.h:1277:16: error: 'return' with a value, in function returning void [-Werror=return-type]
include/linux/hugetlb.h:1275:34: note: declared here
1275 | static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
| ^~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1306:40: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1306 | spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
| ^~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_lockptr':
include/linux/hugetlb.h:1308:20: error: invalid use of undefined type 'struct hugetlb_pte'
1308 | return hpte->ptl;
| ^~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1312:37: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1312 | spinlock_t *hugetlb_pte_lock(struct hugetlb_pte *hpte)
| ^~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_lock':
include/linux/hugetlb.h:1314:47: error: passing argument 1 of 'hugetlb_pte_lockptr' from incompatible pointer type [-Werror=incompatible-pointer-types]
1314 | spinlock_t *ptl = hugetlb_pte_lockptr(hpte);
| ^~~~
| |
| struct hugetlb_pte *
include/linux/hugetlb.h:1306:53: note: expected 'struct hugetlb_pte *' but argument is of type 'struct hugetlb_pte *'
1306 | spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
| ~~~~~~~~~~~~~~~~~~~~^~~~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1321:56: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1321 | void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
| ^~~~~~~~~~~
include/linux/hugetlb.h:1323:46: error: parameter 5 ('level') has incomplete type
1323 | enum hugetlb_level level)
| ~~~~~~~~~~~~~~~~~~~^~~~~
include/linux/hugetlb.h:1321:6: error: function declaration isn't a prototype [-Werror=strict-prototypes]
1321 | void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
| ^~~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_populate':
include/linux/hugetlb.h:1325:9: error: implicit declaration of function '__hugetlb_pte_populate'; did you mean 'hugetlb_pte_populate'? [-Werror=implicit-function-declaration]
1325 | __hugetlb_pte_populate(hpte, ptep, shift, level,
| ^~~~~~~~~~~~~~~~~~~~~~
| hugetlb_pte_populate
mm/madvise.c: In function 'madvise_split':
mm/madvise.c:1023:9: error: implicit declaration of function 'hugetlb_vma_lock_alloc'; did you mean 'hugetlb_vma_lock_write'? [-Werror=implicit-function-declaration]
1023 | hugetlb_vma_lock_alloc(vma);
| ^~~~~~~~~~~~~~~~~~~~~~
| hugetlb_vma_lock_write
cc1: some warnings being treated as errors
--
In file included from fs/proc/meminfo.c:6:
>> include/linux/hugetlb.h:1275:34: error: return type is an incomplete type
1275 | static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
| ^~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: In function 'hpage_size_to_level':
>> include/linux/hugetlb.h:1277:16: error: 'HUGETLB_LEVEL_PTE' undeclared (first use in this function)
1277 | return HUGETLB_LEVEL_PTE;
| ^~~~~~~~~~~~~~~~~
include/linux/hugetlb.h:1277:16: note: each undeclared identifier is reported only once for each function it appears in
include/linux/hugetlb.h:1277:16: error: 'return' with a value, in function returning void [-Werror=return-type]
include/linux/hugetlb.h:1275:34: note: declared here
1275 | static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
| ^~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1306:40: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1306 | spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
| ^~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_lockptr':
include/linux/hugetlb.h:1308:20: error: invalid use of undefined type 'struct hugetlb_pte'
1308 | return hpte->ptl;
| ^~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1312:37: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1312 | spinlock_t *hugetlb_pte_lock(struct hugetlb_pte *hpte)
| ^~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_lock':
include/linux/hugetlb.h:1314:47: error: passing argument 1 of 'hugetlb_pte_lockptr' from incompatible pointer type [-Werror=incompatible-pointer-types]
1314 | spinlock_t *ptl = hugetlb_pte_lockptr(hpte);
| ^~~~
| |
| struct hugetlb_pte *
include/linux/hugetlb.h:1306:53: note: expected 'struct hugetlb_pte *' but argument is of type 'struct hugetlb_pte *'
1306 | spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
| ~~~~~~~~~~~~~~~~~~~~^~~~
include/linux/hugetlb.h: At top level:
include/linux/hugetlb.h:1321:56: warning: 'struct hugetlb_pte' declared inside parameter list will not be visible outside of this definition or declaration
1321 | void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
| ^~~~~~~~~~~
include/linux/hugetlb.h:1323:46: error: parameter 5 ('level') has incomplete type
1323 | enum hugetlb_level level)
| ~~~~~~~~~~~~~~~~~~~^~~~~
include/linux/hugetlb.h:1321:6: error: function declaration isn't a prototype [-Werror=strict-prototypes]
1321 | void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
| ^~~~~~~~~~~~~~~~~~~~
include/linux/hugetlb.h: In function 'hugetlb_pte_populate':
include/linux/hugetlb.h:1325:9: error: implicit declaration of function '__hugetlb_pte_populate'; did you mean 'hugetlb_pte_populate'? [-Werror=implicit-function-declaration]
1325 | __hugetlb_pte_populate(hpte, ptep, shift, level,
| ^~~~~~~~~~~~~~~~~~~~~~
| hugetlb_pte_populate
fs/proc/meminfo.c: At top level:
fs/proc/meminfo.c:22:28: warning: no previous prototype for 'arch_report_meminfo' [-Wmissing-prototypes]
22 | void __attribute__((weak)) arch_report_meminfo(struct seq_file *m)
| ^~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +1275 include/linux/hugetlb.h
1274
> 1275 static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
1276 {
> 1277 return HUGETLB_LEVEL_PTE;
1278 }
1279 #endif /* CONFIG_HUGETLB_PAGE */
1280
Hi James,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on next-20230105]
[cannot apply to kvm/queue shuah-kselftest/next shuah-kselftest/fixes arnd-asm-generic/master linus/master kvm/linux-next v6.2-rc2 v6.2-rc1 v6.1 v6.2-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/James-Houghton/hugetlb-don-t-set-PageUptodate-for-UFFDIO_CONTINUE/20230105-182428
patch link: https://lore.kernel.org/r/20230105101844.1893104-14-jthoughton%40google.com
patch subject: [PATCH 13/46] hugetlb: add hugetlb_hgm_walk and hugetlb_walk_step
config: powerpc-randconfig-r005-20230105
compiler: clang version 16.0.0 (https://github.com/llvm/llvm-project 8d9828ef5aa9688500657d36cd2aefbe12bbd162)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install powerpc cross compiling tool for clang build
# apt-get install binutils-powerpc-linux-gnu
# https://github.com/intel-lab-lkp/linux/commit/5395f068d45f39d202240799d3a8146226387f5c
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review James-Houghton/hugetlb-don-t-set-PageUptodate-for-UFFDIO_CONTINUE/20230105-182428
git checkout 5395f068d45f39d202240799d3a8146226387f5c
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=powerpc olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=powerpc SHELL=/bin/bash fs/proc/ mm/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from mm/filemap.c:36:
>> include/linux/hugetlb.h:1275:34: error: incomplete result type 'enum hugetlb_level' in function definition
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
include/linux/hugetlb.h:1275:20: note: forward declaration of 'enum hugetlb_level'
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
>> include/linux/hugetlb.h:1277:9: error: use of undeclared identifier 'HUGETLB_LEVEL_PTE'
return HUGETLB_LEVEL_PTE;
^
include/linux/hugetlb.h:1306:40: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1308:13: error: incomplete definition of type 'struct hugetlb_pte'
return hpte->ptl;
~~~~^
include/linux/hugetlb.h:1306:40: note: forward declaration of 'struct hugetlb_pte'
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1312:37: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
spinlock_t *hugetlb_pte_lock(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1314:40: error: incompatible pointer types passing 'struct hugetlb_pte *' to parameter of type 'struct hugetlb_pte *' [-Werror,-Wincompatible-pointer-types]
spinlock_t *ptl = hugetlb_pte_lockptr(hpte);
^~~~
include/linux/hugetlb.h:1306:53: note: passing argument to parameter 'hpte' here
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1321:56: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
^
include/linux/hugetlb.h:1323:25: error: variable has incomplete type 'enum hugetlb_level'
enum hugetlb_level level)
^
include/linux/hugetlb.h:1275:20: note: forward declaration of 'enum hugetlb_level'
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
include/linux/hugetlb.h:1325:2: error: call to undeclared function '__hugetlb_pte_populate'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
__hugetlb_pte_populate(hpte, ptep, shift, level,
^
include/linux/hugetlb.h:1325:2: note: did you mean 'hugetlb_pte_populate'?
include/linux/hugetlb.h:1321:6: note: 'hugetlb_pte_populate' declared here
void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
^
3 warnings and 6 errors generated.
--
In file included from mm/shmem.c:38:
>> include/linux/hugetlb.h:1275:34: error: incomplete result type 'enum hugetlb_level' in function definition
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
include/linux/hugetlb.h:1275:20: note: forward declaration of 'enum hugetlb_level'
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
>> include/linux/hugetlb.h:1277:9: error: use of undeclared identifier 'HUGETLB_LEVEL_PTE'
return HUGETLB_LEVEL_PTE;
^
include/linux/hugetlb.h:1306:40: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1308:13: error: incomplete definition of type 'struct hugetlb_pte'
return hpte->ptl;
~~~~^
include/linux/hugetlb.h:1306:40: note: forward declaration of 'struct hugetlb_pte'
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1312:37: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
spinlock_t *hugetlb_pte_lock(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1314:40: error: incompatible pointer types passing 'struct hugetlb_pte *' to parameter of type 'struct hugetlb_pte *' [-Werror,-Wincompatible-pointer-types]
spinlock_t *ptl = hugetlb_pte_lockptr(hpte);
^~~~
include/linux/hugetlb.h:1306:53: note: passing argument to parameter 'hpte' here
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1321:56: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
^
include/linux/hugetlb.h:1323:25: error: variable has incomplete type 'enum hugetlb_level'
enum hugetlb_level level)
^
include/linux/hugetlb.h:1275:20: note: forward declaration of 'enum hugetlb_level'
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
include/linux/hugetlb.h:1325:2: error: call to undeclared function '__hugetlb_pte_populate'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
__hugetlb_pte_populate(hpte, ptep, shift, level,
^
include/linux/hugetlb.h:1325:2: note: did you mean 'hugetlb_pte_populate'?
include/linux/hugetlb.h:1321:6: note: 'hugetlb_pte_populate' declared here
void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
^
In file included from mm/shmem.c:57:
include/linux/mman.h:153:9: warning: division by zero is undefined [-Wdivision-by-zero]
_calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED ) |
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mman.h:132:21: note: expanded from macro '_calc_vm_trans'
: ((x) & (bit1)) / ((bit1) / (bit2))))
^ ~~~~~~~~~~~~~~~~~
include/linux/mman.h:154:9: warning: division by zero is undefined [-Wdivision-by-zero]
_calc_vm_trans(flags, MAP_SYNC, VM_SYNC ) |
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mman.h:132:21: note: expanded from macro '_calc_vm_trans'
: ((x) & (bit1)) / ((bit1) / (bit2))))
^ ~~~~~~~~~~~~~~~~~
5 warnings and 6 errors generated.
--
In file included from fs/proc/meminfo.c:6:
>> include/linux/hugetlb.h:1275:34: error: incomplete result type 'enum hugetlb_level' in function definition
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
include/linux/hugetlb.h:1275:20: note: forward declaration of 'enum hugetlb_level'
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
>> include/linux/hugetlb.h:1277:9: error: use of undeclared identifier 'HUGETLB_LEVEL_PTE'
return HUGETLB_LEVEL_PTE;
^
include/linux/hugetlb.h:1306:40: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1308:13: error: incomplete definition of type 'struct hugetlb_pte'
return hpte->ptl;
~~~~^
include/linux/hugetlb.h:1306:40: note: forward declaration of 'struct hugetlb_pte'
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1312:37: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
spinlock_t *hugetlb_pte_lock(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1314:40: error: incompatible pointer types passing 'struct hugetlb_pte *' to parameter of type 'struct hugetlb_pte *' [-Werror,-Wincompatible-pointer-types]
spinlock_t *ptl = hugetlb_pte_lockptr(hpte);
^~~~
include/linux/hugetlb.h:1306:53: note: passing argument to parameter 'hpte' here
spinlock_t *hugetlb_pte_lockptr(struct hugetlb_pte *hpte)
^
include/linux/hugetlb.h:1321:56: warning: declaration of 'struct hugetlb_pte' will not be visible outside of this function [-Wvisibility]
void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
^
include/linux/hugetlb.h:1323:25: error: variable has incomplete type 'enum hugetlb_level'
enum hugetlb_level level)
^
include/linux/hugetlb.h:1275:20: note: forward declaration of 'enum hugetlb_level'
static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
^
include/linux/hugetlb.h:1325:2: error: call to undeclared function '__hugetlb_pte_populate'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
__hugetlb_pte_populate(hpte, ptep, shift, level,
^
include/linux/hugetlb.h:1325:2: note: did you mean 'hugetlb_pte_populate'?
include/linux/hugetlb.h:1321:6: note: 'hugetlb_pte_populate' declared here
void hugetlb_pte_populate(struct mm_struct *mm, struct hugetlb_pte *hpte,
^
In file included from fs/proc/meminfo.c:7:
include/linux/mman.h:153:9: warning: division by zero is undefined [-Wdivision-by-zero]
_calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED ) |
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mman.h:132:21: note: expanded from macro '_calc_vm_trans'
: ((x) & (bit1)) / ((bit1) / (bit2))))
^ ~~~~~~~~~~~~~~~~~
include/linux/mman.h:154:9: warning: division by zero is undefined [-Wdivision-by-zero]
_calc_vm_trans(flags, MAP_SYNC, VM_SYNC ) |
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/mman.h:132:21: note: expanded from macro '_calc_vm_trans'
: ((x) & (bit1)) / ((bit1) / (bit2))))
^ ~~~~~~~~~~~~~~~~~
fs/proc/meminfo.c:22:28: warning: no previous prototype for function 'arch_report_meminfo' [-Wmissing-prototypes]
void __attribute__((weak)) arch_report_meminfo(struct seq_file *m)
^
fs/proc/meminfo.c:22:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
void __attribute__((weak)) arch_report_meminfo(struct seq_file *m)
^
static
6 warnings and 6 errors generated.
vim +1275 include/linux/hugetlb.h
1274
> 1275 static inline enum hugetlb_level hpage_size_to_level(unsigned long sz)
1276 {
> 1277 return HUGETLB_LEVEL_PTE;
1278 }
1279 #endif /* CONFIG_HUGETLB_PAGE */
1280
On Thu, Jan 05, 2023 at 10:18:11AM +0000, James Houghton wrote: [...] > +static int hugetlb_hgm_walk_uninit(struct hugetlb_pte *hpte, Nitpick on the name: the "uninit" can be misread into pairing with some other "init()" calls.. How about just call it hugetlb_hgm_walk (since it's the higher level API comparing to the existing one)? Then the existing hugetlb_hgm_walk can be called hugetlb_hgm_do_walk/__hugetlb_hgm_walk since it's one level down. > + pte_t *ptep, > + struct vm_area_struct *vma, > + unsigned long addr, > + unsigned long target_sz, > + bool alloc) > +{ > + struct hstate *h = hstate_vma(vma); > + > + hugetlb_pte_populate(vma->vm_mm, hpte, ptep, huge_page_shift(h), > + hpage_size_to_level(huge_page_size(h))); Another nitpick on name: I remembered we used to reach a consensus of using hugetlb_pte_init before? Can we still avoid the word "populate" (if "init" is not suitable since it can be updated during stepping, how about "setup")? [...] > +int hugetlb_walk_step(struct mm_struct *mm, struct hugetlb_pte *hpte, > + unsigned long addr, unsigned long sz) > +{ > + pte_t *ptep; > + spinlock_t *ptl; > + > + switch (hpte->level) { > + case HUGETLB_LEVEL_PUD: > + ptep = (pte_t *)hugetlb_alloc_pmd(mm, hpte, addr); > + if (IS_ERR(ptep)) > + return PTR_ERR(ptep); > + hugetlb_pte_populate(mm, hpte, ptep, PMD_SHIFT, > + HUGETLB_LEVEL_PMD); > + break; > + case HUGETLB_LEVEL_PMD: > + ptep = hugetlb_alloc_pte(mm, hpte, addr); > + if (IS_ERR(ptep)) > + return PTR_ERR(ptep); > + ptl = pte_lockptr(mm, (pmd_t *)hpte->ptep); > + __hugetlb_pte_populate(hpte, ptep, PAGE_SHIFT, > + HUGETLB_LEVEL_PTE, ptl); > + hpte->ptl = ptl; This line seems to be superfluous (even if benign). > + break; > + default: > + WARN_ONCE(1, "%s: got invalid level: %d (shift: %d)\n", > + __func__, hpte->level, hpte->shift); > + return -EINVAL; > + } > + return 0; > +} > + > /* > * Return a mask that can be used to update an address to the last huge > * page in a page table page mapping size. Used to skip non-present > -- > 2.39.0.314.g84b9a713c41-goog >
On Wed, Jan 11, 2023 at 4:51 PM Peter Xu <peterx@redhat.com> wrote: > > On Thu, Jan 05, 2023 at 10:18:11AM +0000, James Houghton wrote: > > [...] > > > +static int hugetlb_hgm_walk_uninit(struct hugetlb_pte *hpte, > > Nitpick on the name: the "uninit" can be misread into pairing with some > other "init()" calls.. > > How about just call it hugetlb_hgm_walk (since it's the higher level API > comparing to the existing one)? Then the existing hugetlb_hgm_walk can be > called hugetlb_hgm_do_walk/__hugetlb_hgm_walk since it's one level down. > > > + pte_t *ptep, > > + struct vm_area_struct *vma, > > + unsigned long addr, > > + unsigned long target_sz, > > + bool alloc) > > +{ > > + struct hstate *h = hstate_vma(vma); > > + > > + hugetlb_pte_populate(vma->vm_mm, hpte, ptep, huge_page_shift(h), > > + hpage_size_to_level(huge_page_size(h))); > > Another nitpick on name: I remembered we used to reach a consensus of using > hugetlb_pte_init before? Can we still avoid the word "populate" (if "init" > is not suitable since it can be updated during stepping, how about "setup")? Right, we did talk about this, sorry. Ok I'll go ahead with this name change. - hugetlb_hgm_walk => __hugetlb_hgm_walk - hugetlb_hgm_walk_uninit => hugetlb_hgm_walk - [__,]hugetlb_pte_populate => [__,]hugetlb_pte_init > > [...] > > > +int hugetlb_walk_step(struct mm_struct *mm, struct hugetlb_pte *hpte, > > + unsigned long addr, unsigned long sz) > > +{ > > + pte_t *ptep; > > + spinlock_t *ptl; > > + > > + switch (hpte->level) { > > + case HUGETLB_LEVEL_PUD: > > + ptep = (pte_t *)hugetlb_alloc_pmd(mm, hpte, addr); > > + if (IS_ERR(ptep)) > > + return PTR_ERR(ptep); > > + hugetlb_pte_populate(mm, hpte, ptep, PMD_SHIFT, > > + HUGETLB_LEVEL_PMD); > > + break; > > + case HUGETLB_LEVEL_PMD: > > + ptep = hugetlb_alloc_pte(mm, hpte, addr); > > + if (IS_ERR(ptep)) > > + return PTR_ERR(ptep); > > + ptl = pte_lockptr(mm, (pmd_t *)hpte->ptep); > > + __hugetlb_pte_populate(hpte, ptep, PAGE_SHIFT, > > + HUGETLB_LEVEL_PTE, ptl); > > + hpte->ptl = ptl; > > This line seems to be superfluous (even if benign). Nice catch! It shouldn't be there; I accidentally left it in when I changed how `ptl` was handled. Thanks Peter!
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ad9d19f0d1b9..2fcd8f313628 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -239,6 +239,14 @@ u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx); pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pud_t *pud); +int hugetlb_full_walk(struct hugetlb_pte *hpte, struct vm_area_struct *vma, + unsigned long addr); +void hugetlb_full_walk_continue(struct hugetlb_pte *hpte, + struct vm_area_struct *vma, unsigned long addr); +int hugetlb_full_walk_alloc(struct hugetlb_pte *hpte, + struct vm_area_struct *vma, unsigned long addr, + unsigned long target_sz); + struct address_space *hugetlb_page_mapping_lock_write(struct page *hpage); extern int sysctl_hugetlb_shm_group; @@ -288,6 +296,8 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long sz); unsigned long hugetlb_mask_last_page(struct hstate *h); +int hugetlb_walk_step(struct mm_struct *mm, struct hugetlb_pte *hpte, + unsigned long addr, unsigned long sz); int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pte_t *ptep); void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, @@ -1067,6 +1077,8 @@ void hugetlb_register_node(struct node *node); void hugetlb_unregister_node(struct node *node); #endif +enum hugetlb_level hpage_size_to_level(unsigned long sz); + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; @@ -1259,6 +1271,11 @@ static inline void hugetlb_register_node(struct node *node) static inline void hugetlb_unregister_node(struct node *node) { } + +static inline enum hugetlb_level hpage_size_to_level(unsigned long sz) +{ + return HUGETLB_LEVEL_PTE; +} #endif /* CONFIG_HUGETLB_PAGE */ #ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING @@ -1333,12 +1350,8 @@ __vma_has_hugetlb_vma_lock(struct vm_area_struct *vma) return (vma->vm_flags & VM_MAYSHARE) && vma->vm_private_data; } -/* - * Safe version of huge_pte_offset() to check the locks. See comments - * above huge_pte_offset(). - */ -static inline pte_t * -hugetlb_walk(struct vm_area_struct *vma, unsigned long addr, unsigned long sz) +static inline void +hugetlb_walk_lock_check(struct vm_area_struct *vma) { #if defined(CONFIG_HUGETLB_PAGE) && \ defined(CONFIG_ARCH_WANT_HUGE_PMD_SHARE) && defined(CONFIG_LOCKDEP) @@ -1360,6 +1373,16 @@ hugetlb_walk(struct vm_area_struct *vma, unsigned long addr, unsigned long sz) !lockdep_is_held( &vma->vm_file->f_mapping->i_mmap_rwsem)); #endif +} + +/* + * Safe version of huge_pte_offset() to check the locks. See comments + * above huge_pte_offset(). + */ +static inline pte_t * +hugetlb_walk(struct vm_area_struct *vma, unsigned long addr, unsigned long sz) +{ + hugetlb_walk_lock_check(vma); return huge_pte_offset(vma->vm_mm, addr, sz); } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 2160cbaf3311..aa8e59cbca69 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -94,6 +94,29 @@ static int hugetlb_acct_memory(struct hstate *h, long delta); static void hugetlb_vma_lock_free(struct vm_area_struct *vma); static void __hugetlb_vma_unlock_write_free(struct vm_area_struct *vma); +/* + * hpage_size_to_level() - convert @sz to the corresponding page table level + * + * @sz must be less than or equal to a valid hugepage size. + */ +enum hugetlb_level hpage_size_to_level(unsigned long sz) +{ + /* + * We order the conditionals from smallest to largest to pick the + * smallest level when multiple levels have the same size (i.e., + * when levels are folded). + */ + if (sz < PMD_SIZE) + return HUGETLB_LEVEL_PTE; + if (sz < PUD_SIZE) + return HUGETLB_LEVEL_PMD; + if (sz < P4D_SIZE) + return HUGETLB_LEVEL_PUD; + if (sz < PGDIR_SIZE) + return HUGETLB_LEVEL_P4D; + return HUGETLB_LEVEL_PGD; +} + static inline bool subpool_is_free(struct hugepage_subpool *spool) { if (spool->count) @@ -7276,6 +7299,153 @@ bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) } #endif /* CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ +/* hugetlb_hgm_walk - walks a high-granularity HugeTLB page table to resolve + * the page table entry for @addr. We might allocate new PTEs. + * + * @hpte must always be pointing at an hstate-level PTE or deeper. + * + * This function will never walk further if it encounters a PTE of a size + * less than or equal to @sz. + * + * @alloc determines what we do when we encounter an empty PTE. If false, + * we stop walking. If true and @sz is less than the current PTE's size, + * we make that PTE point to the next level down, going until @sz is the same + * as our current PTE. + * + * If @alloc is false and @sz is PAGE_SIZE, this function will always + * succeed, but that does not guarantee that hugetlb_pte_size(hpte) is @sz. + * + * Return: + * -ENOMEM if we couldn't allocate new PTEs. + * -EEXIST if the caller wanted to walk further than a migration PTE, + * poison PTE, or a PTE marker. The caller needs to manually deal + * with this scenario. + * -EINVAL if called with invalid arguments (@sz invalid, @hpte not + * initialized). + * 0 otherwise. + * + * Even if this function fails, @hpte is guaranteed to always remain + * valid. + */ +static int hugetlb_hgm_walk(struct mm_struct *mm, struct vm_area_struct *vma, + struct hugetlb_pte *hpte, unsigned long addr, + unsigned long sz, bool alloc) +{ + int ret = 0; + pte_t pte; + + if (WARN_ON_ONCE(sz < PAGE_SIZE)) + return -EINVAL; + + if (WARN_ON_ONCE(!hpte->ptep)) + return -EINVAL; + + /* We have the same synchronization requirements as hugetlb_walk. */ + hugetlb_walk_lock_check(vma); + + while (hugetlb_pte_size(hpte) > sz && !ret) { + pte = huge_ptep_get(hpte->ptep); + if (!pte_present(pte)) { + if (!alloc) + return 0; + if (unlikely(!huge_pte_none(pte))) + return -EEXIST; + } else if (hugetlb_pte_present_leaf(hpte, pte)) + return 0; + ret = hugetlb_walk_step(mm, hpte, addr, sz); + } + + return ret; +} + +static int hugetlb_hgm_walk_uninit(struct hugetlb_pte *hpte, + pte_t *ptep, + struct vm_area_struct *vma, + unsigned long addr, + unsigned long target_sz, + bool alloc) +{ + struct hstate *h = hstate_vma(vma); + + hugetlb_pte_populate(vma->vm_mm, hpte, ptep, huge_page_shift(h), + hpage_size_to_level(huge_page_size(h))); + return hugetlb_hgm_walk(vma->vm_mm, vma, hpte, addr, target_sz, + alloc); +} + +/* + * hugetlb_full_walk_continue - continue a high-granularity page-table walk. + * + * If a user has a valid @hpte but knows that @hpte is not a leaf, they can + * attempt to continue walking by calling this function. + * + * This function may never fail, but @hpte might not change. + * + * If @hpte is not valid, then this function is a no-op. + */ +void hugetlb_full_walk_continue(struct hugetlb_pte *hpte, + struct vm_area_struct *vma, + unsigned long addr) +{ + /* hugetlb_hgm_walk will never fail with these arguments. */ + WARN_ON_ONCE(hugetlb_hgm_walk(vma->vm_mm, vma, hpte, addr, + PAGE_SIZE, false)); +} + +/* + * hugetlb_full_walk - do a high-granularity page-table walk; never allocate. + * + * This function can only fail if we find that the hstate-level PTE is not + * allocated. Callers can take advantage of this fact to skip address regions + * that cannot be mapped in that case. + * + * If this function succeeds, @hpte is guaranteed to be valid. + */ +int hugetlb_full_walk(struct hugetlb_pte *hpte, + struct vm_area_struct *vma, + unsigned long addr) +{ + struct hstate *h = hstate_vma(vma); + unsigned long sz = huge_page_size(h); + /* + * We must mask the address appropriately so that we pick up the first + * PTE in a contiguous group. + */ + pte_t *ptep = hugetlb_walk(vma, addr & huge_page_mask(h), sz); + + if (!ptep) + return -ENOMEM; + + /* hugetlb_hgm_walk_uninit will never fail with these arguments. */ + WARN_ON_ONCE(hugetlb_hgm_walk_uninit(hpte, ptep, vma, addr, + PAGE_SIZE, false)); + return 0; +} + +/* + * hugetlb_full_walk_alloc - do a high-granularity walk, potentially allocate + * new PTEs. + */ +int hugetlb_full_walk_alloc(struct hugetlb_pte *hpte, + struct vm_area_struct *vma, + unsigned long addr, + unsigned long target_sz) +{ + struct hstate *h = hstate_vma(vma); + unsigned long sz = huge_page_size(h); + /* + * We must mask the address appropriately so that we pick up the first + * PTE in a contiguous group. + */ + pte_t *ptep = huge_pte_alloc(vma->vm_mm, vma, addr & huge_page_mask(h), + sz); + + if (!ptep) + return -ENOMEM; + + return hugetlb_hgm_walk_uninit(hpte, ptep, vma, addr, target_sz, true); +} + #ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz) @@ -7343,6 +7513,49 @@ pte_t *huge_pte_offset(struct mm_struct *mm, return (pte_t *)pmd; } +/* + * hugetlb_walk_step() - Walk the page table one step to resolve the page + * (hugepage or subpage) entry at address @addr. + * + * @sz always points at the final target PTE size (e.g. PAGE_SIZE for the + * lowest level PTE). + * + * @hpte will always remain valid, even if this function fails. + * + * Architectures that implement this function must ensure that if @hpte does + * not change levels, then its PTL must also stay the same. + */ +int hugetlb_walk_step(struct mm_struct *mm, struct hugetlb_pte *hpte, + unsigned long addr, unsigned long sz) +{ + pte_t *ptep; + spinlock_t *ptl; + + switch (hpte->level) { + case HUGETLB_LEVEL_PUD: + ptep = (pte_t *)hugetlb_alloc_pmd(mm, hpte, addr); + if (IS_ERR(ptep)) + return PTR_ERR(ptep); + hugetlb_pte_populate(mm, hpte, ptep, PMD_SHIFT, + HUGETLB_LEVEL_PMD); + break; + case HUGETLB_LEVEL_PMD: + ptep = hugetlb_alloc_pte(mm, hpte, addr); + if (IS_ERR(ptep)) + return PTR_ERR(ptep); + ptl = pte_lockptr(mm, (pmd_t *)hpte->ptep); + __hugetlb_pte_populate(hpte, ptep, PAGE_SHIFT, + HUGETLB_LEVEL_PTE, ptl); + hpte->ptl = ptl; + break; + default: + WARN_ONCE(1, "%s: got invalid level: %d (shift: %d)\n", + __func__, hpte->level, hpte->shift); + return -EINVAL; + } + return 0; +} + /* * Return a mask that can be used to update an address to the last huge * page in a page table page mapping size. Used to skip non-present
hugetlb_hgm_walk implements high-granularity page table walks for HugeTLB. It is safe to call on non-HGM enabled VMAs; it will return immediately. hugetlb_walk_step implements how we step forwards in the walk. For architectures that don't use GENERAL_HUGETLB, they will need to provide their own implementation. Signed-off-by: James Houghton <jthoughton@google.com> --- include/linux/hugetlb.h | 35 +++++-- mm/hugetlb.c | 213 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 242 insertions(+), 6 deletions(-)