Message ID | cover.1725334811.git.bo.wu@vivo.com (mailing list archive) |
---|---|
Headers | show |
Series | f2fs: introduce inline tail | expand |
On Tue, Sep 03, 2024 at 02:54:44AM -0600, Wu Bo via Linux-f2fs-devel wrote: > The inode in F2FS occupies an entire 4k block. For many small files, this means > they consume much more space than their actual size. Therefore, there is > significant potential to better utilize the inode block space. > > Currently, F2FS has two features to make use of the inode block space: inline > data and inline xattr. > > Inline data stores file which size is smaller then 3.5k in inode block. However, > for slightly larger small files, there still have much waste. > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > more than twice the size of the file itself! > > Additionally, the end of a file often does not occupy an entire block. If we can > store the end of the file data within the inode block, we can save an entire > block for the file. This is particularly important for small files. > > In fact, the current inline data is a special case of inline tail, and > inline tail is an extension of inline data. > > To make it simple, inline tail only on small files(<64k). And for larger files, > inline tails don't provide any significant benefits. > > The layout of an inline tail inode block is following: > > | inode block | 4096 | inline tail enable | > | --------------- | ---- | --------------------------| > | inode info | 360 | | > | --------------- | ---- | --------------------------| > | | | extra info | 0~36 | > | | | **compact_addr[16] | 64 | > | addr table[923] | 3692 | reserved | 4 | > | | | **tail data | | > | | | inline_xattr | 200 | > | --------------- | ---- | --------------------------| > | nid table[5] | 20 | > | node footer | 24 | > > F2fs-tools to support inline tail: > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > copy time also reduced by around 10%. > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > -g quick' both with and without the patch; no regressions were observed. > The test result is: > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > generic/735 > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > Wu Bo (13): > f2fs: add inline tail mount option > f2fs: add inline tail disk layout definition > f2fs: implement inline tail write & truncate > f2fs: implement inline tail read & fiemap > f2fs: set inline tail flag when create inode > f2fs: fix address info has been truncated > f2fs: support seek for inline tail > f2fs: convert inline tail when inode expand > f2fs: fix data loss during inline tail writing > f2fs: avoid inlining quota files > f2fs: fix inline tail data lost > f2fs: convert inline tails to avoid potential issues > f2fs: implement inline tail forward recovery > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > fs/f2fs/f2fs.h | 46 ++++++++++++- > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > fs/f2fs/inode.c | 6 ++ > fs/f2fs/namei.c | 3 + > fs/f2fs/node.c | 6 +- > fs/f2fs/recovery.c | 9 ++- > fs/f2fs/super.c | 25 +++++++ > fs/f2fs/verity.c | 4 ++ > 10 files changed, 409 insertions(+), 27 deletions(-) This is disabled on encrypted files, right? - Eric
On Tue, Sep 03, 2024 at 09:29:30AM -0700, Eric Biggers via Linux-f2fs-devel wrote: > On Tue, Sep 03, 2024 at 02:54:44AM -0600, Wu Bo via Linux-f2fs-devel wrote: > > The inode in F2FS occupies an entire 4k block. For many small files, this means > > they consume much more space than their actual size. Therefore, there is > > significant potential to better utilize the inode block space. > > > > Currently, F2FS has two features to make use of the inode block space: inline > > data and inline xattr. > > > > Inline data stores file which size is smaller then 3.5k in inode block. However, > > for slightly larger small files, there still have much waste. > > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > > more than twice the size of the file itself! > > > > Additionally, the end of a file often does not occupy an entire block. If we can > > store the end of the file data within the inode block, we can save an entire > > block for the file. This is particularly important for small files. > > > > In fact, the current inline data is a special case of inline tail, and > > inline tail is an extension of inline data. > > > > To make it simple, inline tail only on small files(<64k). And for larger files, > > inline tails don't provide any significant benefits. > > > > The layout of an inline tail inode block is following: > > > > | inode block | 4096 | inline tail enable | > > | --------------- | ---- | --------------------------| > > | inode info | 360 | | > > | --------------- | ---- | --------------------------| > > | | | extra info | 0~36 | > > | | | **compact_addr[16] | 64 | > > | addr table[923] | 3692 | reserved | 4 | > > | | | **tail data | | > > | | | inline_xattr | 200 | > > | --------------- | ---- | --------------------------| > > | nid table[5] | 20 | > > | node footer | 24 | > > > > F2fs-tools to support inline tail: > > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > > copy time also reduced by around 10%. > > > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > > -g quick' both with and without the patch; no regressions were observed. > > The test result is: > > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > > generic/735 > > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > > > Wu Bo (13): > > f2fs: add inline tail mount option > > f2fs: add inline tail disk layout definition > > f2fs: implement inline tail write & truncate > > f2fs: implement inline tail read & fiemap > > f2fs: set inline tail flag when create inode > > f2fs: fix address info has been truncated > > f2fs: support seek for inline tail > > f2fs: convert inline tail when inode expand > > f2fs: fix data loss during inline tail writing > > f2fs: avoid inlining quota files > > f2fs: fix inline tail data lost > > f2fs: convert inline tails to avoid potential issues > > f2fs: implement inline tail forward recovery > > > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > > fs/f2fs/f2fs.h | 46 ++++++++++++- > > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > > fs/f2fs/inode.c | 6 ++ > > fs/f2fs/namei.c | 3 + > > fs/f2fs/node.c | 6 +- > > fs/f2fs/recovery.c | 9 ++- > > fs/f2fs/super.c | 25 +++++++ > > fs/f2fs/verity.c | 4 ++ > > 10 files changed, 409 insertions(+), 27 deletions(-) > > This is disabled on encrypted files, right? Yes, conditions for enabling inline tail are the same as for inline data now. Thanks > > - Eric > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Hi Wu, Interesting patch-set. A quick test here on my daily-driver phone's data (785558 inodes) with both compression and encryption disabled, copied via rsync to 2 fresh f2fs partitions with and without inline tail: Before: 170064928KiB After: 169249780KiB So about 0.48% saved. Let me know if this is an unexpected result. Thanks, On Tue, Sep 3, 2024 at 5:42 PM Wu Bo via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net> wrote: > > The inode in F2FS occupies an entire 4k block. For many small files, this means > they consume much more space than their actual size. Therefore, there is > significant potential to better utilize the inode block space. > > Currently, F2FS has two features to make use of the inode block space: inline > data and inline xattr. > > Inline data stores file which size is smaller then 3.5k in inode block. However, > for slightly larger small files, there still have much waste. > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > more than twice the size of the file itself! > > Additionally, the end of a file often does not occupy an entire block. If we can > store the end of the file data within the inode block, we can save an entire > block for the file. This is particularly important for small files. > > In fact, the current inline data is a special case of inline tail, and > inline tail is an extension of inline data. > > To make it simple, inline tail only on small files(<64k). And for larger files, > inline tails don't provide any significant benefits. > > The layout of an inline tail inode block is following: > > | inode block | 4096 | inline tail enable | > | --------------- | ---- | --------------------------| > | inode info | 360 | | > | --------------- | ---- | --------------------------| > | | | extra info | 0~36 | > | | | **compact_addr[16] | 64 | > | addr table[923] | 3692 | reserved | 4 | > | | | **tail data | | > | | | inline_xattr | 200 | > | --------------- | ---- | --------------------------| > | nid table[5] | 20 | > | node footer | 24 | > > F2fs-tools to support inline tail: > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > copy time also reduced by around 10%. > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > -g quick' both with and without the patch; no regressions were observed. > The test result is: > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > generic/735 > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > Wu Bo (13): > f2fs: add inline tail mount option > f2fs: add inline tail disk layout definition > f2fs: implement inline tail write & truncate > f2fs: implement inline tail read & fiemap > f2fs: set inline tail flag when create inode > f2fs: fix address info has been truncated > f2fs: support seek for inline tail > f2fs: convert inline tail when inode expand > f2fs: fix data loss during inline tail writing > f2fs: avoid inlining quota files > f2fs: fix inline tail data lost > f2fs: convert inline tails to avoid potential issues > f2fs: implement inline tail forward recovery > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > fs/f2fs/f2fs.h | 46 ++++++++++++- > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > fs/f2fs/inode.c | 6 ++ > fs/f2fs/namei.c | 3 + > fs/f2fs/node.c | 6 +- > fs/f2fs/recovery.c | 9 ++- > fs/f2fs/super.c | 25 +++++++ > fs/f2fs/verity.c | 4 ++ > 10 files changed, 409 insertions(+), 27 deletions(-) > > -- > 2.35.3 > > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
On Fri, Sep 06, 2024 at 06:02:42PM +0900, Juhyung Park wrote: > Hi Wu, > > Interesting patch-set. > > A quick test here on my daily-driver phone's data (785558 inodes) with > both compression and encryption disabled, copied via rsync to 2 fresh > f2fs partitions with and without inline tail: > Before: 170064928KiB > After: 169249780KiB > > So about 0.48% saved. > Hi Juhyung, Thanks for your test. If it's not too much trouble, please help check the f2fs statistics: cat /sys/kernel/debug/f2fs/status | grep Utilization However, it’s more likely that the benefits of inline tail are indeed limited on mobile devices. I previously evaluated the gains on my own phone, and they were less than 1% too. From the data on your phone, the average size is 170064928KiB/785558≈200KiB. Storage space on phones is primarily consumed by multimedia files, so the proportion of files smaller than 64KB is quite small. Therefore, compared to phone storage, scenarios like storing Linux source code, which involve many small files, are more likely to yield noticeable benefits. However, don't be too disappointed with it, as it can still double the benefits based on the existing gains from inline data. Inline data: https://lore.kernel.org/linux-f2fs-devel/1384096401-25169-1-git-send-email-huajun.li.lee@gmail.com/T/#u Thanks, Wu Bo > Let me know if this is an unexpected result. > > Thanks, > > On Tue, Sep 3, 2024 at 5:42 PM Wu Bo via Linux-f2fs-devel > <linux-f2fs-devel@lists.sourceforge.net> wrote: > > > > The inode in F2FS occupies an entire 4k block. For many small files, this means > > they consume much more space than their actual size. Therefore, there is > > significant potential to better utilize the inode block space. > > > > Currently, F2FS has two features to make use of the inode block space: inline > > data and inline xattr. > > > > Inline data stores file which size is smaller then 3.5k in inode block. However, > > for slightly larger small files, there still have much waste. > > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > > more than twice the size of the file itself! > > > > Additionally, the end of a file often does not occupy an entire block. If we can > > store the end of the file data within the inode block, we can save an entire > > block for the file. This is particularly important for small files. > > > > In fact, the current inline data is a special case of inline tail, and > > inline tail is an extension of inline data. > > > > To make it simple, inline tail only on small files(<64k). And for larger files, > > inline tails don't provide any significant benefits. > > > > The layout of an inline tail inode block is following: > > > > | inode block | 4096 | inline tail enable | > > | --------------- | ---- | --------------------------| > > | inode info | 360 | | > > | --------------- | ---- | --------------------------| > > | | | extra info | 0~36 | > > | | | **compact_addr[16] | 64 | > > | addr table[923] | 3692 | reserved | 4 | > > | | | **tail data | | > > | | | inline_xattr | 200 | > > | --------------- | ---- | --------------------------| > > | nid table[5] | 20 | > > | node footer | 24 | > > > > F2fs-tools to support inline tail: > > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > > copy time also reduced by around 10%. > > > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > > -g quick' both with and without the patch; no regressions were observed. > > The test result is: > > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > > generic/735 > > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > > > Wu Bo (13): > > f2fs: add inline tail mount option > > f2fs: add inline tail disk layout definition > > f2fs: implement inline tail write & truncate > > f2fs: implement inline tail read & fiemap > > f2fs: set inline tail flag when create inode > > f2fs: fix address info has been truncated > > f2fs: support seek for inline tail > > f2fs: convert inline tail when inode expand > > f2fs: fix data loss during inline tail writing > > f2fs: avoid inlining quota files > > f2fs: fix inline tail data lost > > f2fs: convert inline tails to avoid potential issues > > f2fs: implement inline tail forward recovery > > > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > > fs/f2fs/f2fs.h | 46 ++++++++++++- > > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > > fs/f2fs/inode.c | 6 ++ > > fs/f2fs/namei.c | 3 + > > fs/f2fs/node.c | 6 +- > > fs/f2fs/recovery.c | 9 ++- > > fs/f2fs/super.c | 25 +++++++ > > fs/f2fs/verity.c | 4 ++ > > 10 files changed, 409 insertions(+), 27 deletions(-) > > > > -- > > 2.35.3 > > > > > > > > _______________________________________________ > > Linux-f2fs-devel mailing list > > Linux-f2fs-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Hi Wu, On Fri, Sep 6, 2024 at 7:42 PM Wu Bo <bo.wu@vivo.com> wrote: > > On Fri, Sep 06, 2024 at 06:02:42PM +0900, Juhyung Park wrote: > > Hi Wu, > > > > Interesting patch-set. > > > > A quick test here on my daily-driver phone's data (785558 inodes) with > > both compression and encryption disabled, copied via rsync to 2 fresh > > f2fs partitions with and without inline tail: > > Before: 170064928KiB > > After: 169249780KiB > > > > So about 0.48% saved. > > > Hi Juhyung, > > Thanks for your test. If it's not too much trouble, please help check the f2fs > statistics: > cat /sys/kernel/debug/f2fs/status | grep Utilization # w/o inline tail Utilization: 70% (42153224 valid blocks, 0 discard blocks) # w/ inline tail Utilization: 69% (41949437 valid blocks, 0 discard blocks) > > However, it’s more likely that the benefits of inline tail are indeed limited on > mobile devices. I previously evaluated the gains on my own phone, and they were > less than 1% too. > > From the data on your phone, the average size is 170064928KiB/785558≈200KiB. > Storage space on phones is primarily consumed by multimedia files, so the > proportion of files smaller than 64KB is quite small. > > Therefore, compared to phone storage, scenarios like storing Linux source code, > which involve many small files, are more likely to yield noticeable benefits. Mmm. I do have my own Linux setups using f2fs as well with extended node bitmap: # Workstation, 2.5T/3.5T, 55602404 inodes Utilization: 70% (657421720 valid blocks, 902 discard blocks) # Laptop, 736G/1.9T, 12229380 inodes Utilization: 39% (190980868 valid blocks, 2887 discard blocks) I do have a lot of small files here, it'll be interesting to test this out but I can't afford to run an unstable kernel for those atm. (Not to mention finding a new SSD for migration.) > > However, don't be too disappointed with it, as it can still double the benefits > based on the existing gains from inline data. > Inline data: > https://lore.kernel.org/linux-f2fs-devel/1384096401-25169-1-git-send-email-huajun.li.lee@gmail.com/T/#u > > Thanks, > Wu Bo > > > Let me know if this is an unexpected result. > > > > Thanks, > > > > On Tue, Sep 3, 2024 at 5:42 PM Wu Bo via Linux-f2fs-devel > > <linux-f2fs-devel@lists.sourceforge.net> wrote: > > > > > > The inode in F2FS occupies an entire 4k block. For many small files, this means > > > they consume much more space than their actual size. Therefore, there is > > > significant potential to better utilize the inode block space. > > > > > > Currently, F2FS has two features to make use of the inode block space: inline > > > data and inline xattr. > > > > > > Inline data stores file which size is smaller then 3.5k in inode block. However, > > > for slightly larger small files, there still have much waste. > > > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > > > more than twice the size of the file itself! > > > > > > Additionally, the end of a file often does not occupy an entire block. If we can > > > store the end of the file data within the inode block, we can save an entire > > > block for the file. This is particularly important for small files. > > > > > > In fact, the current inline data is a special case of inline tail, and > > > inline tail is an extension of inline data. > > > > > > To make it simple, inline tail only on small files(<64k). And for larger files, > > > inline tails don't provide any significant benefits. > > > > > > The layout of an inline tail inode block is following: > > > > > > | inode block | 4096 | inline tail enable | > > > | --------------- | ---- | --------------------------| > > > | inode info | 360 | | > > > | --------------- | ---- | --------------------------| > > > | | | extra info | 0~36 | > > > | | | **compact_addr[16] | 64 | > > > | addr table[923] | 3692 | reserved | 4 | > > > | | | **tail data | | > > > | | | inline_xattr | 200 | > > > | --------------- | ---- | --------------------------| > > > | nid table[5] | 20 | > > > | node footer | 24 | > > > > > > F2fs-tools to support inline tail: > > > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > > > > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > > > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > > > copy time also reduced by around 10%. > > > > > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > > > -g quick' both with and without the patch; no regressions were observed. > > > The test result is: > > > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > > > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > > > generic/735 > > > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > > > > > Wu Bo (13): > > > f2fs: add inline tail mount option > > > f2fs: add inline tail disk layout definition > > > f2fs: implement inline tail write & truncate > > > f2fs: implement inline tail read & fiemap > > > f2fs: set inline tail flag when create inode > > > f2fs: fix address info has been truncated > > > f2fs: support seek for inline tail > > > f2fs: convert inline tail when inode expand > > > f2fs: fix data loss during inline tail writing > > > f2fs: avoid inlining quota files > > > f2fs: fix inline tail data lost > > > f2fs: convert inline tails to avoid potential issues > > > f2fs: implement inline tail forward recovery > > > > > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > > > fs/f2fs/f2fs.h | 46 ++++++++++++- > > > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > > > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > > > fs/f2fs/inode.c | 6 ++ > > > fs/f2fs/namei.c | 3 + > > > fs/f2fs/node.c | 6 +- > > > fs/f2fs/recovery.c | 9 ++- > > > fs/f2fs/super.c | 25 +++++++ > > > fs/f2fs/verity.c | 4 ++ > > > 10 files changed, 409 insertions(+), 27 deletions(-) > > > > > > -- > > > 2.35.3 > > > > > > > > > > > > _______________________________________________ > > > Linux-f2fs-devel mailing list > > > Linux-f2fs-devel@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > > > > > > _______________________________________________ > > Linux-f2fs-devel mailing list > > Linux-f2fs-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel