Message ID | 20210719184001.1750630-3-willy@infradead.org (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | Folio support in block + iomap layers | expand |
On Mon, Jul 19, 2021 at 07:39:46PM +0100, Matthew Wilcox (Oracle) wrote: > Allow callers to iterate over each folio instead of each page. The > bio need not have been constructed using folios originally. > > Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Whoops, I never did remember to circle back and ack this patch. Reviewed-by: Darrick J. Wong <djwong@kernel.org> --D > --- > Documentation/core-api/kernel-api.rst | 1 + > include/linux/bio.h | 53 ++++++++++++++++++++++++++- > 2 files changed, 53 insertions(+), 1 deletion(-) > > diff --git a/Documentation/core-api/kernel-api.rst b/Documentation/core-api/kernel-api.rst > index 2a7444e3a4c2..b804605f705e 100644 > --- a/Documentation/core-api/kernel-api.rst > +++ b/Documentation/core-api/kernel-api.rst > @@ -279,6 +279,7 @@ Accounting Framework > Block Devices > ============= > > +.. kernel-doc:: include/linux/bio.h > .. kernel-doc:: block/blk-core.c > :export: > > diff --git a/include/linux/bio.h b/include/linux/bio.h > index ade93e2de6a1..a90a79ad7bd1 100644 > --- a/include/linux/bio.h > +++ b/include/linux/bio.h > @@ -189,7 +189,7 @@ static inline void bio_advance_iter_single(const struct bio *bio, > */ > #define bio_for_each_bvec_all(bvl, bio, i) \ > for (i = 0, bvl = bio_first_bvec_all(bio); \ > - i < (bio)->bi_vcnt; i++, bvl++) \ > + i < (bio)->bi_vcnt; i++, bvl++) > > #define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len) > > @@ -314,6 +314,57 @@ static inline struct bio_vec *bio_last_bvec_all(struct bio *bio) > return &bio->bi_io_vec[bio->bi_vcnt - 1]; > } > > +/** > + * struct folio_iter - State for iterating all folios in a bio. > + * @folio: The current folio we're iterating. NULL after the last folio. > + * @offset: The byte offset within the current folio. > + * @length: The number of bytes in this iteration (will not cross folio > + * boundary). > + */ > +struct folio_iter { > + struct folio *folio; > + size_t offset; > + size_t length; > + /* private: for use by the iterator */ > + size_t _seg_count; > + int _i; > +}; > + > +static inline > +void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) > +{ > + struct bio_vec *bvec = bio_first_bvec_all(bio) + i; > + > + fi->folio = page_folio(bvec->bv_page); > + fi->offset = bvec->bv_offset + > + PAGE_SIZE * (bvec->bv_page - &fi->folio->page); > + fi->_seg_count = bvec->bv_len; > + fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); > + fi->_i = i; > +} > + > +static inline void bio_next_folio(struct folio_iter *fi, struct bio *bio) > +{ > + fi->_seg_count -= fi->length; > + if (fi->_seg_count) { > + fi->folio = folio_next(fi->folio); > + fi->offset = 0; > + fi->length = min(folio_size(fi->folio), fi->_seg_count); > + } else if (fi->_i + 1 < bio->bi_vcnt) { > + bio_first_folio(fi, bio, fi->_i + 1); > + } else { > + fi->folio = NULL; > + } > +} > + > +/** > + * bio_for_each_folio_all - Iterate over each folio in a bio. > + * @fi: struct folio_iter which is updated for each folio. > + * @bio: struct bio to iterate over. > + */ > +#define bio_for_each_folio_all(fi, bio) \ > + for (bio_first_folio(&fi, bio, 0); fi.folio; bio_next_folio(&fi, bio)) > + > enum bip_flags { > BIP_BLOCK_INTEGRITY = 1 << 0, /* block layer owns integrity data */ > BIP_MAPPED_INTEGRITY = 1 << 1, /* ref tag has been remapped */ > -- > 2.30.2 >
On Mon, Jul 19, 2021 at 06:29:45PM -0700, Darrick J. Wong wrote: > On Mon, Jul 19, 2021 at 07:39:46PM +0100, Matthew Wilcox (Oracle) wrote: > > Allow callers to iterate over each folio instead of each page. The > > bio need not have been constructed using folios originally. > > > > Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> > > Whoops, I never did remember to circle back and ack this patch. > Reviewed-by: Darrick J. Wong <djwong@kernel.org> That's all good; it made me ask "What more can I do?" and "Add documentation" as always a good answer ...
On Mon, Jul 19, 2021 at 07:39:46PM +0100, Matthew Wilcox (Oracle) wrote: > #define bio_for_each_bvec_all(bvl, bio, i) \ > for (i = 0, bvl = bio_first_bvec_all(bio); \ > - i < (bio)->bi_vcnt; i++, bvl++) \ > + i < (bio)->bi_vcnt; i++, bvl++) Pleae split out this unrelated fixup. > +static inline > +void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) Please fix the strange formatting. > +{ > + struct bio_vec *bvec = bio_first_bvec_all(bio) + i; > + > + fi->folio = page_folio(bvec->bv_page); > + fi->offset = bvec->bv_offset + > + PAGE_SIZE * (bvec->bv_page - &fi->folio->page); Can we have a little helper for the offset in folio calculation, like: static inline size_t offset_of_page_in_folio(struct page *page) { return (bvec->bv_page - &page_folio(page)->page) * PAGE; } as that makes the callers a lot easier to read.
Hi "Matthew, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on linus/master] [also build test WARNING on v5.14-rc2 next-20210720] [cannot apply to xfs-linux/for-next block/for-next] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-support-in-block-iomap-layers/20210720-152323 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2734d6c1b1a089fb593ef6a23d4b70903526fe0c config: x86_64-randconfig-r016-20210720 (attached as .config) compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 87039c048c0cbc3d8cbba86187269b006bf2f373) reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # install x86_64 cross compiling tool for clang build # apt-get install binutils-x86-64-linux-gnu # https://github.com/0day-ci/linux/commit/d611eec3f37483e1acbcc299bf47e929766afc8d git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-support-in-block-iomap-layers/20210720-152323 git checkout d611eec3f37483e1acbcc299bf47e929766afc8d # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64 If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All warnings (new ones prefixed by >>): In file included from arch/x86/kernel/asm-offsets.c:13: In file included from include/linux/suspend.h:5: In file included from include/linux/swap.h:9: In file included from include/linux/memcontrol.h:22: In file included from include/linux/writeback.h:14: In file included from include/linux/blk-cgroup.h:23: In file included from include/linux/blkdev.h:18: include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration] fi->folio = page_folio(bvec->bv_page); ^ >> include/linux/bio.h:338:12: warning: incompatible integer to pointer conversion assigning to 'struct folio *' from 'int' [-Wint-conversion] fi->folio = page_folio(bvec->bv_page); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/bio.h:340:43: error: incomplete definition of type 'struct folio' PAGE_SIZE * (bvec->bv_page - &fi->folio->page); ~~~~~~~~~^ include/linux/bio.h:325:9: note: forward declaration of 'struct folio' struct folio *folio; ^ include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror,-Wimplicit-function-declaration] fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); ^ include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror,-Wimplicit-function-declaration] fi->folio = folio_next(fi->folio); ^ include/linux/bio.h:350:13: warning: incompatible integer to pointer conversion assigning to 'struct folio *' from 'int' [-Wint-conversion] fi->folio = folio_next(fi->folio); ^ ~~~~~~~~~~~~~~~~~~~~~ include/linux/bio.h:352:20: error: implicit declaration of function 'folio_size' [-Werror,-Wimplicit-function-declaration] fi->length = min(folio_size(fi->folio), fi->_seg_count); ^ >> include/linux/bio.h:352:16: warning: comparison of distinct pointer types ('typeof (folio_size(fi->folio)) *' (aka 'int *') and 'typeof (fi->_seg_count) *' (aka 'unsigned long *')) [-Wcompare-distinct-pointer-types] fi->length = min(folio_size(fi->folio), fi->_seg_count); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/minmax.h:45:19: note: expanded from macro 'min' #define min(x, y) __careful_cmp(x, y, <) ^~~~~~~~~~~~~~~~~~~~~~ include/linux/minmax.h:36:24: note: expanded from macro '__careful_cmp' __builtin_choose_expr(__safe_cmp(x, y), \ ^~~~~~~~~~~~~~~~ include/linux/minmax.h:26:4: note: expanded from macro '__safe_cmp' (__typecheck(x, y) && __no_side_effects(x, y)) ^~~~~~~~~~~~~~~~~ include/linux/minmax.h:20:28: note: expanded from macro '__typecheck' (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~ 3 warnings and 5 errors generated. -- In file included from arch/x86/kernel/asm-offsets.c:13: In file included from include/linux/suspend.h:5: In file included from include/linux/swap.h:9: In file included from include/linux/memcontrol.h:22: In file included from include/linux/writeback.h:14: In file included from include/linux/blk-cgroup.h:23: In file included from include/linux/blkdev.h:18: include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration] fi->folio = page_folio(bvec->bv_page); ^ >> include/linux/bio.h:338:12: warning: incompatible integer to pointer conversion assigning to 'struct folio *' from 'int' [-Wint-conversion] fi->folio = page_folio(bvec->bv_page); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/bio.h:340:43: error: incomplete definition of type 'struct folio' PAGE_SIZE * (bvec->bv_page - &fi->folio->page); ~~~~~~~~~^ include/linux/bio.h:325:9: note: forward declaration of 'struct folio' struct folio *folio; ^ include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror,-Wimplicit-function-declaration] fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); ^ include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror,-Wimplicit-function-declaration] fi->folio = folio_next(fi->folio); ^ include/linux/bio.h:350:13: warning: incompatible integer to pointer conversion assigning to 'struct folio *' from 'int' [-Wint-conversion] fi->folio = folio_next(fi->folio); ^ ~~~~~~~~~~~~~~~~~~~~~ include/linux/bio.h:352:20: error: implicit declaration of function 'folio_size' [-Werror,-Wimplicit-function-declaration] fi->length = min(folio_size(fi->folio), fi->_seg_count); ^ >> include/linux/bio.h:352:16: warning: comparison of distinct pointer types ('typeof (folio_size(fi->folio)) *' (aka 'int *') and 'typeof (fi->_seg_count) *' (aka 'unsigned long *')) [-Wcompare-distinct-pointer-types] fi->length = min(folio_size(fi->folio), fi->_seg_count); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/minmax.h:45:19: note: expanded from macro 'min' #define min(x, y) __careful_cmp(x, y, <) ^~~~~~~~~~~~~~~~~~~~~~ include/linux/minmax.h:36:24: note: expanded from macro '__careful_cmp' __builtin_choose_expr(__safe_cmp(x, y), \ ^~~~~~~~~~~~~~~~ include/linux/minmax.h:26:4: note: expanded from macro '__safe_cmp' (__typecheck(x, y) && __no_side_effects(x, y)) ^~~~~~~~~~~~~~~~~ include/linux/minmax.h:20:28: note: expanded from macro '__typecheck' (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~ 3 warnings and 5 errors generated. make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1 make[2]: Target '__build' not remade because of errors. make[1]: *** [Makefile:1213: prepare0] Error 2 make[1]: Target 'modules_prepare' not remade because of errors. make: *** [Makefile:220: __sub-make] Error 2 make: Target 'modules_prepare' not remade because of errors. -- In file included from arch/x86/kernel/asm-offsets.c:13: In file included from include/linux/suspend.h:5: In file included from include/linux/swap.h:9: In file included from include/linux/memcontrol.h:22: In file included from include/linux/writeback.h:14: In file included from include/linux/blk-cgroup.h:23: In file included from include/linux/blkdev.h:18: include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration] fi->folio = page_folio(bvec->bv_page); ^ >> include/linux/bio.h:338:12: warning: incompatible integer to pointer conversion assigning to 'struct folio *' from 'int' [-Wint-conversion] fi->folio = page_folio(bvec->bv_page); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/bio.h:340:43: error: incomplete definition of type 'struct folio' PAGE_SIZE * (bvec->bv_page - &fi->folio->page); ~~~~~~~~~^ include/linux/bio.h:325:9: note: forward declaration of 'struct folio' struct folio *folio; ^ include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror,-Wimplicit-function-declaration] fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); ^ include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror,-Wimplicit-function-declaration] fi->folio = folio_next(fi->folio); ^ include/linux/bio.h:350:13: warning: incompatible integer to pointer conversion assigning to 'struct folio *' from 'int' [-Wint-conversion] fi->folio = folio_next(fi->folio); ^ ~~~~~~~~~~~~~~~~~~~~~ include/linux/bio.h:352:20: error: implicit declaration of function 'folio_size' [-Werror,-Wimplicit-function-declaration] fi->length = min(folio_size(fi->folio), fi->_seg_count); ^ >> include/linux/bio.h:352:16: warning: comparison of distinct pointer types ('typeof (folio_size(fi->folio)) *' (aka 'int *') and 'typeof (fi->_seg_count) *' (aka 'unsigned long *')) [-Wcompare-distinct-pointer-types] fi->length = min(folio_size(fi->folio), fi->_seg_count); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/minmax.h:45:19: note: expanded from macro 'min' #define min(x, y) __careful_cmp(x, y, <) ^~~~~~~~~~~~~~~~~~~~~~ include/linux/minmax.h:36:24: note: expanded from macro '__careful_cmp' __builtin_choose_expr(__safe_cmp(x, y), \ ^~~~~~~~~~~~~~~~ include/linux/minmax.h:26:4: note: expanded from macro '__safe_cmp' (__typecheck(x, y) && __no_side_effects(x, y)) ^~~~~~~~~~~~~~~~~ include/linux/minmax.h:20:28: note: expanded from macro '__typecheck' (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~ 3 warnings and 5 errors generated. make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1 make[2]: Target '__build' not remade because of errors. make[1]: *** [Makefile:1213: prepare0] Error 2 make[1]: Target 'prepare' not remade because of errors. make: *** [Makefile:220: __sub-make] Error 2 make: Target 'prepare' not remade because of errors. vim +338 include/linux/bio.h 332 333 static inline 334 void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) 335 { 336 struct bio_vec *bvec = bio_first_bvec_all(bio) + i; 337 > 338 fi->folio = page_folio(bvec->bv_page); 339 fi->offset = bvec->bv_offset + 340 PAGE_SIZE * (bvec->bv_page - &fi->folio->page); 341 fi->_seg_count = bvec->bv_len; 342 fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); 343 fi->_i = i; 344 } 345 346 static inline void bio_next_folio(struct folio_iter *fi, struct bio *bio) 347 { 348 fi->_seg_count -= fi->length; 349 if (fi->_seg_count) { 350 fi->folio = folio_next(fi->folio); 351 fi->offset = 0; > 352 fi->length = min(folio_size(fi->folio), fi->_seg_count); 353 } else if (fi->_i + 1 < bio->bi_vcnt) { 354 bio_first_folio(fi, bio, fi->_i + 1); 355 } else { 356 fi->folio = NULL; 357 } 358 } 359 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Hi "Matthew, Thank you for the patch! Yet something to improve: [auto build test ERROR on linus/master] [also build test ERROR on v5.14-rc2 next-20210720] [cannot apply to xfs-linux/for-next block/for-next] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-support-in-block-iomap-layers/20210720-152323 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2734d6c1b1a089fb593ef6a23d4b70903526fe0c config: sparc-randconfig-r015-20210720 (attached as .config) compiler: sparc64-linux-gcc (GCC) 10.3.0 reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # https://github.com/0day-ci/linux/commit/d611eec3f37483e1acbcc299bf47e929766afc8d git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-support-in-block-iomap-layers/20210720-152323 git checkout d611eec3f37483e1acbcc299bf47e929766afc8d # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-10.3.0 make.cross ARCH=sparc If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All errors (new ones prefixed by >>): In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/tracehook.h:50, from arch/sparc/kernel/signal_64.c:18: include/linux/bio.h: In function 'bio_first_folio': include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration] 338 | fi->folio = page_folio(bvec->bv_page); | ^~~~~~~~~~ >> include/linux/bio.h:338:12: error: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion] 338 | fi->folio = page_folio(bvec->bv_page); | ^ include/linux/bio.h:340:43: error: invalid use of undefined type 'struct folio' 340 | PAGE_SIZE * (bvec->bv_page - &fi->folio->page); | ^~ In file included from include/linux/kernel.h:16, from include/linux/list.h:9, from include/linux/rculist.h:10, from include/linux/pid.h:5, from include/linux/sched.h:14, from arch/sparc/kernel/signal_64.c:12: include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror=implicit-function-declaration] 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~~~~~~~~ include/linux/minmax.h:20:21: note: in definition of macro '__typecheck' 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:342:15: note: in expansion of macro 'min' 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~ In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/tracehook.h:50, from arch/sparc/kernel/signal_64.c:18: include/linux/bio.h: In function 'bio_next_folio': include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror=implicit-function-declaration] 350 | fi->folio = folio_next(fi->folio); | ^~~~~~~~~~ include/linux/bio.h:350:13: error: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion] 350 | fi->folio = folio_next(fi->folio); | ^ In file included from include/linux/kernel.h:16, from include/linux/list.h:9, from include/linux/rculist.h:10, from include/linux/pid.h:5, from include/linux/sched.h:14, from arch/sparc/kernel/signal_64.c:12: include/linux/minmax.h:20:28: error: comparison of distinct pointer types lacks a cast [-Werror] 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ include/linux/minmax.h:26:4: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:352:16: note: in expansion of macro 'min' 352 | fi->length = min(folio_size(fi->folio), fi->_seg_count); | ^~~ cc1: all warnings being treated as errors -- In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/swap.h:9, from arch/sparc/include/asm/tlb_64.h:5, from arch/sparc/include/asm/tlb.h:5, from arch/sparc/kernel/smp_64.c:49: include/linux/bio.h: In function 'bio_first_folio': include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio'; did you mean 'page_endio'? [-Werror=implicit-function-declaration] 338 | fi->folio = page_folio(bvec->bv_page); | ^~~~~~~~~~ | page_endio >> include/linux/bio.h:338:12: error: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion] 338 | fi->folio = page_folio(bvec->bv_page); | ^ include/linux/bio.h:340:43: error: invalid use of undefined type 'struct folio' 340 | PAGE_SIZE * (bvec->bv_page - &fi->folio->page); | ^~ In file included from include/linux/kernel.h:16, from arch/sparc/kernel/smp_64.c:8: include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror=implicit-function-declaration] 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~~~~~~~~ include/linux/minmax.h:20:21: note: in definition of macro '__typecheck' 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:342:15: note: in expansion of macro 'min' 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~ In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/swap.h:9, from arch/sparc/include/asm/tlb_64.h:5, from arch/sparc/include/asm/tlb.h:5, from arch/sparc/kernel/smp_64.c:49: include/linux/bio.h: In function 'bio_next_folio': include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror=implicit-function-declaration] 350 | fi->folio = folio_next(fi->folio); | ^~~~~~~~~~ include/linux/bio.h:350:13: error: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion] 350 | fi->folio = folio_next(fi->folio); | ^ In file included from include/linux/kernel.h:16, from arch/sparc/kernel/smp_64.c:8: include/linux/minmax.h:20:28: error: comparison of distinct pointer types lacks a cast [-Werror] 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ include/linux/minmax.h:26:4: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:352:16: note: in expansion of macro 'min' 352 | fi->length = min(folio_size(fi->folio), fi->_seg_count); | ^~~ cc1: all warnings being treated as errors -- In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/net/sock.h:53, from include/linux/tcp.h:19, from include/linux/ipv6.h:88, from arch/sparc/kernel/sys_sparc32.c:33: include/linux/bio.h: In function 'bio_first_folio': include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration] 338 | fi->folio = page_folio(bvec->bv_page); | ^~~~~~~~~~ >> include/linux/bio.h:338:12: error: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion] 338 | fi->folio = page_folio(bvec->bv_page); | ^ include/linux/bio.h:340:43: error: invalid use of undefined type 'struct folio' 340 | PAGE_SIZE * (bvec->bv_page - &fi->folio->page); | ^~ In file included from include/linux/kernel.h:16, from arch/sparc/kernel/sys_sparc32.c:11: include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror=implicit-function-declaration] 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~~~~~~~~ include/linux/minmax.h:20:21: note: in definition of macro '__typecheck' 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:342:15: note: in expansion of macro 'min' 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~ In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/net/sock.h:53, from include/linux/tcp.h:19, from include/linux/ipv6.h:88, from arch/sparc/kernel/sys_sparc32.c:33: include/linux/bio.h: In function 'bio_next_folio': include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next'; did you mean 'nla_next'? [-Werror=implicit-function-declaration] 350 | fi->folio = folio_next(fi->folio); | ^~~~~~~~~~ | nla_next include/linux/bio.h:350:13: error: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion] 350 | fi->folio = folio_next(fi->folio); | ^ In file included from include/linux/kernel.h:16, from arch/sparc/kernel/sys_sparc32.c:11: include/linux/minmax.h:20:28: error: comparison of distinct pointer types lacks a cast [-Werror] 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ include/linux/minmax.h:26:4: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:352:16: note: in expansion of macro 'min' 352 | fi->length = min(folio_size(fi->folio), fi->_seg_count); | ^~~ cc1: all warnings being treated as errors vim +338 include/linux/bio.h 332 333 static inline 334 void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) 335 { 336 struct bio_vec *bvec = bio_first_bvec_all(bio) + i; 337 > 338 fi->folio = page_folio(bvec->bv_page); 339 fi->offset = bvec->bv_offset + 340 PAGE_SIZE * (bvec->bv_page - &fi->folio->page); 341 fi->_seg_count = bvec->bv_len; 342 fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); 343 fi->_i = i; 344 } 345 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Hi "Matthew, Thank you for the patch! Yet something to improve: [auto build test ERROR on linus/master] [also build test ERROR on v5.14-rc2 next-20210720] [cannot apply to xfs-linux/for-next block/for-next] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-support-in-block-iomap-layers/20210720-152323 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 2734d6c1b1a089fb593ef6a23d4b70903526fe0c config: i386-tinyconfig (attached as .config) compiler: gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0 reproduce (this is a W=1 build): # https://github.com/0day-ci/linux/commit/d611eec3f37483e1acbcc299bf47e929766afc8d git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-support-in-block-iomap-layers/20210720-152323 git checkout d611eec3f37483e1acbcc299bf47e929766afc8d # save the attached .config to linux build tree make W=1 ARCH=i386 If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All error/warnings (new ones prefixed by >>): In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/swap.h:9, from include/linux/suspend.h:5, from arch/x86/kernel/asm-offsets.c:13: include/linux/bio.h: In function 'bio_first_folio': >> include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration] 338 | fi->folio = page_folio(bvec->bv_page); | ^~~~~~~~~~ >> include/linux/bio.h:338:12: warning: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Wint-conversion] 338 | fi->folio = page_folio(bvec->bv_page); | ^ >> include/linux/bio.h:340:43: error: invalid use of undefined type 'struct folio' 340 | PAGE_SIZE * (bvec->bv_page - &fi->folio->page); | ^~ In file included from include/linux/kernel.h:16, from include/linux/crypto.h:16, from arch/x86/kernel/asm-offsets.c:9: >> include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror=implicit-function-declaration] 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~~~~~~~~ include/linux/minmax.h:20:21: note: in definition of macro '__typecheck' 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:342:15: note: in expansion of macro 'min' 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~ In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/swap.h:9, from include/linux/suspend.h:5, from arch/x86/kernel/asm-offsets.c:13: include/linux/bio.h: In function 'bio_next_folio': >> include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror=implicit-function-declaration] 350 | fi->folio = folio_next(fi->folio); | ^~~~~~~~~~ include/linux/bio.h:350:13: warning: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Wint-conversion] 350 | fi->folio = folio_next(fi->folio); | ^ In file included from include/linux/kernel.h:16, from include/linux/crypto.h:16, from arch/x86/kernel/asm-offsets.c:9: include/linux/minmax.h:20:28: warning: comparison of distinct pointer types lacks a cast 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ include/linux/minmax.h:26:4: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:352:16: note: in expansion of macro 'min' 352 | fi->length = min(folio_size(fi->folio), fi->_seg_count); | ^~~ cc1: some warnings being treated as errors -- In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/swap.h:9, from include/linux/suspend.h:5, from arch/x86/kernel/asm-offsets.c:13: include/linux/bio.h: In function 'bio_first_folio': >> include/linux/bio.h:338:14: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration] 338 | fi->folio = page_folio(bvec->bv_page); | ^~~~~~~~~~ >> include/linux/bio.h:338:12: warning: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Wint-conversion] 338 | fi->folio = page_folio(bvec->bv_page); | ^ >> include/linux/bio.h:340:43: error: invalid use of undefined type 'struct folio' 340 | PAGE_SIZE * (bvec->bv_page - &fi->folio->page); | ^~ In file included from include/linux/kernel.h:16, from include/linux/crypto.h:16, from arch/x86/kernel/asm-offsets.c:9: >> include/linux/bio.h:342:19: error: implicit declaration of function 'folio_size' [-Werror=implicit-function-declaration] 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~~~~~~~~ include/linux/minmax.h:20:21: note: in definition of macro '__typecheck' 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:342:15: note: in expansion of macro 'min' 342 | fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); | ^~~ In file included from include/linux/blkdev.h:18, from include/linux/blk-cgroup.h:23, from include/linux/writeback.h:14, from include/linux/memcontrol.h:22, from include/linux/swap.h:9, from include/linux/suspend.h:5, from arch/x86/kernel/asm-offsets.c:13: include/linux/bio.h: In function 'bio_next_folio': >> include/linux/bio.h:350:15: error: implicit declaration of function 'folio_next' [-Werror=implicit-function-declaration] 350 | fi->folio = folio_next(fi->folio); | ^~~~~~~~~~ include/linux/bio.h:350:13: warning: assignment to 'struct folio *' from 'int' makes pointer from integer without a cast [-Wint-conversion] 350 | fi->folio = folio_next(fi->folio); | ^ In file included from include/linux/kernel.h:16, from include/linux/crypto.h:16, from arch/x86/kernel/asm-offsets.c:9: include/linux/minmax.h:20:28: warning: comparison of distinct pointer types lacks a cast 20 | (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) | ^~ include/linux/minmax.h:26:4: note: in expansion of macro '__typecheck' 26 | (__typecheck(x, y) && __no_side_effects(x, y)) | ^~~~~~~~~~~ include/linux/minmax.h:36:24: note: in expansion of macro '__safe_cmp' 36 | __builtin_choose_expr(__safe_cmp(x, y), \ | ^~~~~~~~~~ include/linux/minmax.h:45:19: note: in expansion of macro '__careful_cmp' 45 | #define min(x, y) __careful_cmp(x, y, <) | ^~~~~~~~~~~~~ include/linux/bio.h:352:16: note: in expansion of macro 'min' 352 | fi->length = min(folio_size(fi->folio), fi->_seg_count); | ^~~ cc1: some warnings being treated as errors make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1 make[2]: Target '__build' not remade because of errors. make[1]: *** [Makefile:1213: prepare0] Error 2 make[1]: Target 'prepare' not remade because of errors. make: *** [Makefile:220: __sub-make] Error 2 make: Target 'prepare' not remade because of errors. vim +/page_folio +338 include/linux/bio.h 332 333 static inline 334 void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) 335 { 336 struct bio_vec *bvec = bio_first_bvec_all(bio) + i; 337 > 338 fi->folio = page_folio(bvec->bv_page); 339 fi->offset = bvec->bv_offset + > 340 PAGE_SIZE * (bvec->bv_page - &fi->folio->page); 341 fi->_seg_count = bvec->bv_len; > 342 fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); 343 fi->_i = i; 344 } 345 346 static inline void bio_next_folio(struct folio_iter *fi, struct bio *bio) 347 { 348 fi->_seg_count -= fi->length; 349 if (fi->_seg_count) { > 350 fi->folio = folio_next(fi->folio); 351 fi->offset = 0; 352 fi->length = min(folio_size(fi->folio), fi->_seg_count); 353 } else if (fi->_i + 1 < bio->bi_vcnt) { 354 bio_first_folio(fi, bio, fi->_i + 1); 355 } else { 356 fi->folio = NULL; 357 } 358 } 359 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
On Tue, Jul 20, 2021 at 08:48:07AM +0200, Christoph Hellwig wrote: > On Mon, Jul 19, 2021 at 07:39:46PM +0100, Matthew Wilcox (Oracle) wrote: > > #define bio_for_each_bvec_all(bvl, bio, i) \ > > for (i = 0, bvl = bio_first_bvec_all(bio); \ > > - i < (bio)->bi_vcnt; i++, bvl++) \ > > + i < (bio)->bi_vcnt; i++, bvl++) > > Pleae split out this unrelated fixup. > > > +static inline > > +void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) > > Please fix the strange formatting. static inline void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) > > +{ > > + struct bio_vec *bvec = bio_first_bvec_all(bio) + i; > > + > > + fi->folio = page_folio(bvec->bv_page); > > + fi->offset = bvec->bv_offset + > > + PAGE_SIZE * (bvec->bv_page - &fi->folio->page); > > Can we have a little helper for the offset in folio calculation, like: > > static inline size_t offset_of_page_in_folio(struct page *page) > { > return (bvec->bv_page - &page_folio(page)->page) * PAGE; > } > > as that makes the callers a lot easier to read. I've spent most of today thinking about this one. I actually don't want to make this easy to read. This is code that, in an ideal world, would not exist. A bio_vec should not contain a struct page; it should probably be: struct bio_vec { phys_addr_t bv_start; unsigned int bv_len; }; and then the helper to get from a bio_vec to a folio_iter looks like: fi->folio = pfn_folio(bvec->bv_start >> PAGE_SHIFT); fi->offset = offset_in_folio(fi->folio, bvec->bv_start); If instead we decide to keep bvecs the way they are, we can at least turn the bv_page into bv_folio, and then we won't need this code either.
diff --git a/Documentation/core-api/kernel-api.rst b/Documentation/core-api/kernel-api.rst index 2a7444e3a4c2..b804605f705e 100644 --- a/Documentation/core-api/kernel-api.rst +++ b/Documentation/core-api/kernel-api.rst @@ -279,6 +279,7 @@ Accounting Framework Block Devices ============= +.. kernel-doc:: include/linux/bio.h .. kernel-doc:: block/blk-core.c :export: diff --git a/include/linux/bio.h b/include/linux/bio.h index ade93e2de6a1..a90a79ad7bd1 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -189,7 +189,7 @@ static inline void bio_advance_iter_single(const struct bio *bio, */ #define bio_for_each_bvec_all(bvl, bio, i) \ for (i = 0, bvl = bio_first_bvec_all(bio); \ - i < (bio)->bi_vcnt; i++, bvl++) \ + i < (bio)->bi_vcnt; i++, bvl++) #define bio_iter_last(bvec, iter) ((iter).bi_size == (bvec).bv_len) @@ -314,6 +314,57 @@ static inline struct bio_vec *bio_last_bvec_all(struct bio *bio) return &bio->bi_io_vec[bio->bi_vcnt - 1]; } +/** + * struct folio_iter - State for iterating all folios in a bio. + * @folio: The current folio we're iterating. NULL after the last folio. + * @offset: The byte offset within the current folio. + * @length: The number of bytes in this iteration (will not cross folio + * boundary). + */ +struct folio_iter { + struct folio *folio; + size_t offset; + size_t length; + /* private: for use by the iterator */ + size_t _seg_count; + int _i; +}; + +static inline +void bio_first_folio(struct folio_iter *fi, struct bio *bio, int i) +{ + struct bio_vec *bvec = bio_first_bvec_all(bio) + i; + + fi->folio = page_folio(bvec->bv_page); + fi->offset = bvec->bv_offset + + PAGE_SIZE * (bvec->bv_page - &fi->folio->page); + fi->_seg_count = bvec->bv_len; + fi->length = min(folio_size(fi->folio) - fi->offset, fi->_seg_count); + fi->_i = i; +} + +static inline void bio_next_folio(struct folio_iter *fi, struct bio *bio) +{ + fi->_seg_count -= fi->length; + if (fi->_seg_count) { + fi->folio = folio_next(fi->folio); + fi->offset = 0; + fi->length = min(folio_size(fi->folio), fi->_seg_count); + } else if (fi->_i + 1 < bio->bi_vcnt) { + bio_first_folio(fi, bio, fi->_i + 1); + } else { + fi->folio = NULL; + } +} + +/** + * bio_for_each_folio_all - Iterate over each folio in a bio. + * @fi: struct folio_iter which is updated for each folio. + * @bio: struct bio to iterate over. + */ +#define bio_for_each_folio_all(fi, bio) \ + for (bio_first_folio(&fi, bio, 0); fi.folio; bio_next_folio(&fi, bio)) + enum bip_flags { BIP_BLOCK_INTEGRITY = 1 << 0, /* block layer owns integrity data */ BIP_MAPPED_INTEGRITY = 1 << 1, /* ref tag has been remapped */
Allow callers to iterate over each folio instead of each page. The bio need not have been constructed using folios originally. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> --- Documentation/core-api/kernel-api.rst | 1 + include/linux/bio.h | 53 ++++++++++++++++++++++++++- 2 files changed, 53 insertions(+), 1 deletion(-)