Message ID | 20200825054808.16241-5-wqu@suse.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | btrfs: basic refactor of btrfs_buffered_write() | expand |
Hi Qu, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on kdave/for-next] [also build test WARNING on next-20200824] [cannot apply to v5.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/btrfs-basic-refactor-of-btrfs_buffered_write/20200825-135114 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next config: s390-randconfig-r012-20200825 (attached as .config) compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 77e5a195f818b9ace91f7b12ab948b21d7918238) reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # install s390 cross compiling tool for clang build # apt-get install binutils-s390x-linux-gnu # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=s390 If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All warnings (new ones prefixed by >>): include/uapi/linux/swab.h:19:12: note: expanded from macro '___constant_swab32' (((__u32)(x) & (__u32)0x000000ffUL) << 24) | \ ^ In file included from fs/btrfs/file.c:11: In file included from include/linux/backing-dev.h:15: In file included from include/linux/blkdev.h:25: In file included from include/linux/scatterlist.h:9: In file included from arch/s390/include/asm/io.h:72: include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr)); ~~~~~~~~~~ ^ include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu' #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x)) ^ include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32' ___constant_swab32(x) : \ ^ include/uapi/linux/swab.h:20:12: note: expanded from macro '___constant_swab32' (((__u32)(x) & (__u32)0x0000ff00UL) << 8) | \ ^ In file included from fs/btrfs/file.c:11: In file included from include/linux/backing-dev.h:15: In file included from include/linux/blkdev.h:25: In file included from include/linux/scatterlist.h:9: In file included from arch/s390/include/asm/io.h:72: include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr)); ~~~~~~~~~~ ^ include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu' #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x)) ^ include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32' ___constant_swab32(x) : \ ^ include/uapi/linux/swab.h:21:12: note: expanded from macro '___constant_swab32' (((__u32)(x) & (__u32)0x00ff0000UL) >> 8) | \ ^ In file included from fs/btrfs/file.c:11: In file included from include/linux/backing-dev.h:15: In file included from include/linux/blkdev.h:25: In file included from include/linux/scatterlist.h:9: In file included from arch/s390/include/asm/io.h:72: include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr)); ~~~~~~~~~~ ^ include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu' #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x)) ^ include/uapi/linux/swab.h:119:21: note: expanded from macro '__swab32' ___constant_swab32(x) : \ ^ include/uapi/linux/swab.h:22:12: note: expanded from macro '___constant_swab32' (((__u32)(x) & (__u32)0xff000000UL) >> 24))) ^ In file included from fs/btrfs/file.c:11: In file included from include/linux/backing-dev.h:15: In file included from include/linux/blkdev.h:25: In file included from include/linux/scatterlist.h:9: In file included from arch/s390/include/asm/io.h:72: include/asm-generic/io.h:490:61: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] val = __le32_to_cpu((__le32 __force)__raw_readl(PCI_IOBASE + addr)); ~~~~~~~~~~ ^ include/uapi/linux/byteorder/big_endian.h:34:59: note: expanded from macro '__le32_to_cpu' #define __le32_to_cpu(x) __swab32((__force __u32)(__le32)(x)) ^ include/uapi/linux/swab.h:120:12: note: expanded from macro '__swab32' __fswab32(x)) ^ In file included from fs/btrfs/file.c:11: In file included from include/linux/backing-dev.h:15: In file included from include/linux/blkdev.h:25: In file included from include/linux/scatterlist.h:9: In file included from arch/s390/include/asm/io.h:72: include/asm-generic/io.h:501:33: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] __raw_writeb(value, PCI_IOBASE + addr); ~~~~~~~~~~ ^ include/asm-generic/io.h:511:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] __raw_writew((u16 __force)cpu_to_le16(value), PCI_IOBASE + addr); ~~~~~~~~~~ ^ include/asm-generic/io.h:521:59: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] __raw_writel((u32 __force)cpu_to_le32(value), PCI_IOBASE + addr); ~~~~~~~~~~ ^ include/asm-generic/io.h:609:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] readsb(PCI_IOBASE + addr, buffer, count); ~~~~~~~~~~ ^ include/asm-generic/io.h:617:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] readsw(PCI_IOBASE + addr, buffer, count); ~~~~~~~~~~ ^ include/asm-generic/io.h:625:20: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] readsl(PCI_IOBASE + addr, buffer, count); ~~~~~~~~~~ ^ include/asm-generic/io.h:634:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] writesb(PCI_IOBASE + addr, buffer, count); ~~~~~~~~~~ ^ include/asm-generic/io.h:643:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] writesw(PCI_IOBASE + addr, buffer, count); ~~~~~~~~~~ ^ include/asm-generic/io.h:652:21: warning: performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic] writesl(PCI_IOBASE + addr, buffer, count); ~~~~~~~~~~ ^ >> fs/btrfs/file.c:1571:13: warning: comparison of distinct pointer types ('typeof (nr_pages) *' (aka 'int *') and 'typeof (65536 / ((1UL) << 12)) *' (aka 'unsigned long *')) [-Wcompare-distinct-pointer-types] nr_pages = min(nr_pages, SZ_64K / PAGE_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/kernel.h:883:19: note: expanded from macro 'min' #define min(x, y) __careful_cmp(x, y, <) ^~~~~~~~~~~~~~~~~~~~~~ include/linux/kernel.h:874:24: note: expanded from macro '__careful_cmp' __builtin_choose_expr(__safe_cmp(x, y), \ ^~~~~~~~~~~~~~~~ include/linux/kernel.h:864:4: note: expanded from macro '__safe_cmp' (__typecheck(x, y) && __no_side_effects(x, y)) ^~~~~~~~~~~~~~~~~ include/linux/kernel.h:850:29: note: expanded from macro '__typecheck' (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~ 21 warnings generated. # https://github.com/0day-ci/linux/commit/a73ab37ebab960522a0b353a6f20c8094ab911c5 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Qu-Wenruo/btrfs-basic-refactor-of-btrfs_buffered_write/20200825-135114 git checkout a73ab37ebab960522a0b353a6f20c8094ab911c5 vim +1571 fs/btrfs/file.c 1548 1549 /* Helper to get how many pages we should alloc for the batch */ 1550 static int calc_nr_pages(loff_t pos, struct iov_iter *iov) 1551 { 1552 int nr_pages; 1553 1554 /* 1555 * Try to cover the full iov range, as btrfs metadata/data reserve 1556 * and release can be pretty slow, thus the more pages we process in 1557 * one batch the better. 1558 */ 1559 nr_pages = (round_up(pos + iov_iter_count(iov), PAGE_SIZE) - 1560 round_down(pos, PAGE_SIZE)) / PAGE_SIZE; 1561 1562 nr_pages = min(nr_pages, current->nr_dirtied_pause - 1563 current->nr_dirtied); 1564 1565 /* 1566 * Limit the batch to 64K, too large batch may lead to higher memory 1567 * pressure and increase the possibility of short-copy. 1568 * With more and more short-copy, the benefit of batch copy would be 1569 * hugely reduced, as we will fall back to page-by-page copy. 1570 */ > 1571 nr_pages = min(nr_pages, SZ_64K / PAGE_SIZE); 1572 return nr_pages; 1573 } 1574 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Hi Qu, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on kdave/for-next] [also build test WARNING on next-20200824] [cannot apply to v5.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/btrfs-basic-refactor-of-btrfs_buffered_write/20200825-135114 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next config: x86_64-randconfig-s022-20200825 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0 reproduce: # apt-get install sparse # sparse version: v0.6.2-191-g10164920-dirty # save the attached .config to linux build tree make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=x86_64 If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> sparse warnings: (new ones prefixed by >>) >> fs/btrfs/file.c:1571:20: sparse: sparse: incompatible types in comparison expression (different type sizes): >> fs/btrfs/file.c:1571:20: sparse: int * >> fs/btrfs/file.c:1571:20: sparse: unsigned long * # https://github.com/0day-ci/linux/commit/a73ab37ebab960522a0b353a6f20c8094ab911c5 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Qu-Wenruo/btrfs-basic-refactor-of-btrfs_buffered_write/20200825-135114 git checkout a73ab37ebab960522a0b353a6f20c8094ab911c5 vim +1571 fs/btrfs/file.c 1548 1549 /* Helper to get how many pages we should alloc for the batch */ 1550 static int calc_nr_pages(loff_t pos, struct iov_iter *iov) 1551 { 1552 int nr_pages; 1553 1554 /* 1555 * Try to cover the full iov range, as btrfs metadata/data reserve 1556 * and release can be pretty slow, thus the more pages we process in 1557 * one batch the better. 1558 */ 1559 nr_pages = (round_up(pos + iov_iter_count(iov), PAGE_SIZE) - 1560 round_down(pos, PAGE_SIZE)) / PAGE_SIZE; 1561 1562 nr_pages = min(nr_pages, current->nr_dirtied_pause - 1563 current->nr_dirtied); 1564 1565 /* 1566 * Limit the batch to 64K, too large batch may lead to higher memory 1567 * pressure and increase the possibility of short-copy. 1568 * With more and more short-copy, the benefit of batch copy would be 1569 * hugely reduced, as we will fall back to page-by-page copy. 1570 */ > 1571 nr_pages = min(nr_pages, SZ_64K / PAGE_SIZE); 1572 return nr_pages; 1573 } 1574 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
Hi Qu, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on kdave/for-next] [also build test WARNING on next-20200826] [cannot apply to btrfs/next v5.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/btrfs-basic-refactor-of-btrfs_buffered_write/20200825-135114 base: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next config: riscv-randconfig-r033-20200826 (attached as .config) compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 7cfcecece0e0430937cf529ce74d3a071a4dedc6) reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # install riscv cross compiling tool for clang build # apt-get install binutils-riscv64-linux-gnu # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=riscv If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All warnings (new ones prefixed by >>): >> fs/btrfs/file.c:1571:13: warning: comparison of distinct pointer types ('typeof (nr_pages) *' (aka 'int *') and 'typeof (65536 / ((1UL) << (12))) *' (aka 'unsigned long *')) [-Wcompare-distinct-pointer-types] nr_pages = min(nr_pages, SZ_64K / PAGE_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ include/linux/kernel.h:883:19: note: expanded from macro 'min' #define min(x, y) __careful_cmp(x, y, <) ^~~~~~~~~~~~~~~~~~~~~~ include/linux/kernel.h:874:24: note: expanded from macro '__careful_cmp' __builtin_choose_expr(__safe_cmp(x, y), \ ^~~~~~~~~~~~~~~~ include/linux/kernel.h:864:4: note: expanded from macro '__safe_cmp' (__typecheck(x, y) && __no_side_effects(x, y)) ^~~~~~~~~~~~~~~~~ include/linux/kernel.h:850:29: note: expanded from macro '__typecheck' (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~ 1 warning generated. # https://github.com/0day-ci/linux/commit/a73ab37ebab960522a0b353a6f20c8094ab911c5 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Qu-Wenruo/btrfs-basic-refactor-of-btrfs_buffered_write/20200825-135114 git checkout a73ab37ebab960522a0b353a6f20c8094ab911c5 vim +1571 fs/btrfs/file.c 1548 1549 /* Helper to get how many pages we should alloc for the batch */ 1550 static int calc_nr_pages(loff_t pos, struct iov_iter *iov) 1551 { 1552 int nr_pages; 1553 1554 /* 1555 * Try to cover the full iov range, as btrfs metadata/data reserve 1556 * and release can be pretty slow, thus the more pages we process in 1557 * one batch the better. 1558 */ 1559 nr_pages = (round_up(pos + iov_iter_count(iov), PAGE_SIZE) - 1560 round_down(pos, PAGE_SIZE)) / PAGE_SIZE; 1561 1562 nr_pages = min(nr_pages, current->nr_dirtied_pause - 1563 current->nr_dirtied); 1564 1565 /* 1566 * Limit the batch to 64K, too large batch may lead to higher memory 1567 * pressure and increase the possibility of short-copy. 1568 * With more and more short-copy, the benefit of batch copy would be 1569 * hugely reduced, as we will fall back to page-by-page copy. 1570 */ > 1571 nr_pages = min(nr_pages, SZ_64K / PAGE_SIZE); 1572 return nr_pages; 1573 } 1574 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 67d2368a8fa6..de6d1c313042 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1561,7 +1561,14 @@ static int calc_nr_pages(loff_t pos, struct iov_iter *iov) nr_pages = min(nr_pages, current->nr_dirtied_pause - current->nr_dirtied); - nr_pages = max(nr_pages, 8); + + /* + * Limit the batch to 64K, too large batch may lead to higher memory + * pressure and increase the possibility of short-copy. + * With more and more short-copy, the benefit of batch copy would be + * hugely reduced, as we will fall back to page-by-page copy. + */ + nr_pages = min(nr_pages, SZ_64K / PAGE_SIZE); return nr_pages; }
Commit 142349f541d0 ("btrfs: lower the dirty balance poll interval") introduced one limit which is definitely suspicious: - ensure we always have 8 pages allocated The 8 lower limit looks pretty strange, this means even we're just writing 4K, we will allocate page pointers for 8 pages no matter what. To me, this 8 pages look more like a upper limit. This 8 pages upper limit looks so incorrect that my eyes alawys correct it into "min(, 8)" other than "max(, 8)". This patch will use a fixed size (SZ_64K) other than page numbers to setup the upper limit. Also, with comment added to show why we need a upper limit. Signed-off-by: Qu Wenruo <wqu@suse.com> --- fs/btrfs/file.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)