mbox series

[00/12] btrfs: support read-write for subpage metadata

Message ID 20210222063357.92930-1-wqu@suse.com (mailing list archive)
Headers show
Series btrfs: support read-write for subpage metadata | expand

Message

Qu Wenruo Feb. 22, 2021, 6:33 a.m. UTC
This patchset can be fetched from the following github repo, along with
the full subpage RW support:
https://github.com/adam900710/linux/tree/subpage

This patchset is for metadata read write support.

[TEST]
Since the data write path is not included in this patchset, we can't
really test it, but during the lunar new year vocation, I have tested
the full RW patchset with "fstresss -n 10000 -p2" on my Aarch64 board.

And the full RW patchset survives without any crash for a full week.

There is only one remaining bug exposed during the test, that we have
random data checksum mismatch, which is still under investigation.

But the metadata part should be OK for submission.

[DIFFERENCE AGAINST REGULAR SECTORSIZE]
The metadata part in fact has more new code than data part, as it has
some different behaviors compared to the regular sector size handling:

- No more page locking
  Now metadata read/write relies on extent io tree locking, other than
  page locking.
  This is to allow behaviors like read lock one eb while also try to
  read lock another eb in the same page.
  We can't rely on page lock as now we have multiple extent buffers in
  the same page.

- Page status update
  Now we use subpage wrappers to handle page status update.

- How to submit dirty extent buffers
  Instead of just grabbing extent buffer from page::private, we need to
  iterate all dirty extent buffers in the page and submit them.

Qu Wenruo (12):
  btrfs: subpage: introduce helper for subpage dirty status
  btrfs: subpage: introduce helper for subpage writeback status
  btrfs: disk-io: allow btree_set_page_dirty() to do more sanity check
    on subpage metadata
  btrfs: disk-io: support subpage metadata csum calculation at write
    time
  btrfs: extent_io: make alloc_extent_buffer() check subpage dirty
    bitmap
  btrfs: extent_io: make the page uptodate assert check to handle
    subpage
  btrfs: extent_io: make set/clear_extent_buffer_dirty() to support
    subpage sized metadata
  btrfs: extent_io: make set_btree_ioerr() accept extent buffer and
    handle subpage metadata
  btrfs: extent_io: introduce end_bio_subpage_eb_writepage() function
  btrfs: extent_io: introduce write_one_subpage_eb() function
  btrfs: extent_io: make lock_extent_buffer_for_io() to support subpage
    metadata
  btrfs: extent_io: introduce submit_eb_subpage() to submit a subpage
    metadata page

 fs/btrfs/disk-io.c   | 143 +++++++++++----
 fs/btrfs/extent_io.c | 420 ++++++++++++++++++++++++++++++++++++-------
 fs/btrfs/subpage.c   |  72 ++++++++
 fs/btrfs/subpage.h   |  17 ++
 4 files changed, 559 insertions(+), 93 deletions(-)

Comments

David Sterba March 1, 2021, 4:22 p.m. UTC | #1
On Mon, Feb 22, 2021 at 02:33:45PM +0800, Qu Wenruo wrote:
> This patchset can be fetched from the following github repo, along with
> the full subpage RW support:
> https://github.com/adam900710/linux/tree/subpage
> 
> This patchset is for metadata read write support.

I've skimmed the patches, it's adding further helpers and special cases
for subpage (but I could have missed something). From that I think it's
ok to add it to for-next for some test coverage, but rather to make sure
the subpage changes do not bleed to the regular case.

> [TEST]
> Since the data write path is not included in this patchset, we can't
> really test it, but during the lunar new year vocation, I have tested
> the full RW patchset with "fstresss -n 10000 -p2" on my Aarch64 board.
> 
> And the full RW patchset survives without any crash for a full week.
> 
> There is only one remaining bug exposed during the test, that we have
> random data checksum mismatch, which is still under investigation.
> 
> But the metadata part should be OK for submission.
> 
> [DIFFERENCE AGAINST REGULAR SECTORSIZE]
> The metadata part in fact has more new code than data part, as it has
> some different behaviors compared to the regular sector size handling:
> 
> - No more page locking
>   Now metadata read/write relies on extent io tree locking, other than
>   page locking.
>   This is to allow behaviors like read lock one eb while also try to
>   read lock another eb in the same page.
>   We can't rely on page lock as now we have multiple extent buffers in
>   the same page.
> 
> - Page status update
>   Now we use subpage wrappers to handle page status update.
> 
> - How to submit dirty extent buffers
>   Instead of just grabbing extent buffer from page::private, we need to
>   iterate all dirty extent buffers in the page and submit them.

I'm not sure if all this information is also preserved in some comments,
if not it definitely should.
David Sterba March 1, 2021, 4:30 p.m. UTC | #2
On Mon, Feb 22, 2021 at 02:33:45PM +0800, Qu Wenruo wrote:
> This patchset can be fetched from the following github repo, along with
> the full subpage RW support:
> https://github.com/adam900710/linux/tree/subpage
> 
> This patchset is for metadata read write support.
> 
> [TEST]
> Since the data write path is not included in this patchset, we can't
> really test it, but during the lunar new year vocation, I have tested
> the full RW patchset with "fstresss -n 10000 -p2" on my Aarch64 board.
> 
> And the full RW patchset survives without any crash for a full week.
> 
> There is only one remaining bug exposed during the test, that we have
> random data checksum mismatch, which is still under investigation.
> 
> But the metadata part should be OK for submission.
> 
> [DIFFERENCE AGAINST REGULAR SECTORSIZE]
> The metadata part in fact has more new code than data part, as it has
> some different behaviors compared to the regular sector size handling:
> 
> - No more page locking
>   Now metadata read/write relies on extent io tree locking, other than
>   page locking.
>   This is to allow behaviors like read lock one eb while also try to
>   read lock another eb in the same page.
>   We can't rely on page lock as now we have multiple extent buffers in
>   the same page.
> 
> - Page status update
>   Now we use subpage wrappers to handle page status update.
> 
> - How to submit dirty extent buffers
>   Instead of just grabbing extent buffer from page::private, we need to
>   iterate all dirty extent buffers in the page and submit them.
> 
> Qu Wenruo (12):
>   btrfs: subpage: introduce helper for subpage dirty status
>   btrfs: subpage: introduce helper for subpage writeback status
>   btrfs: disk-io: allow btree_set_page_dirty() to do more sanity check
>     on subpage metadata
>   btrfs: disk-io: support subpage metadata csum calculation at write
>     time
>   btrfs: extent_io: make alloc_extent_buffer() check subpage dirty
>     bitmap
>   btrfs: extent_io: make the page uptodate assert check to handle
>     subpage
>   btrfs: extent_io: make set/clear_extent_buffer_dirty() to support
>     subpage sized metadata
>   btrfs: extent_io: make set_btree_ioerr() accept extent buffer and
>     handle subpage metadata
>   btrfs: extent_io: introduce end_bio_subpage_eb_writepage() function
>   btrfs: extent_io: introduce write_one_subpage_eb() function
>   btrfs: extent_io: make lock_extent_buffer_for_io() to support subpage
>     metadata
>   btrfs: extent_io: introduce submit_eb_subpage() to submit a subpage
>     metadata page

Please don't use "extent_io" nor "disk-io" in subjects.
Qu Wenruo March 2, 2021, 12:18 a.m. UTC | #3
On 2021/3/2 上午12:30, David Sterba wrote:
> On Mon, Feb 22, 2021 at 02:33:45PM +0800, Qu Wenruo wrote:
>> This patchset can be fetched from the following github repo, along with
>> the full subpage RW support:
>> https://github.com/adam900710/linux/tree/subpage
>>
>> This patchset is for metadata read write support.
>>
>> [TEST]
>> Since the data write path is not included in this patchset, we can't
>> really test it, but during the lunar new year vocation, I have tested
>> the full RW patchset with "fstresss -n 10000 -p2" on my Aarch64 board.
>>
>> And the full RW patchset survives without any crash for a full week.
>>
>> There is only one remaining bug exposed during the test, that we have
>> random data checksum mismatch, which is still under investigation.
>>
>> But the metadata part should be OK for submission.
>>
>> [DIFFERENCE AGAINST REGULAR SECTORSIZE]
>> The metadata part in fact has more new code than data part, as it has
>> some different behaviors compared to the regular sector size handling:
>>
>> - No more page locking
>>    Now metadata read/write relies on extent io tree locking, other than
>>    page locking.
>>    This is to allow behaviors like read lock one eb while also try to
>>    read lock another eb in the same page.
>>    We can't rely on page lock as now we have multiple extent buffers in
>>    the same page.
>>
>> - Page status update
>>    Now we use subpage wrappers to handle page status update.
>>
>> - How to submit dirty extent buffers
>>    Instead of just grabbing extent buffer from page::private, we need to
>>    iterate all dirty extent buffers in the page and submit them.
>>
>> Qu Wenruo (12):
>>    btrfs: subpage: introduce helper for subpage dirty status
>>    btrfs: subpage: introduce helper for subpage writeback status
>>    btrfs: disk-io: allow btree_set_page_dirty() to do more sanity check
>>      on subpage metadata
>>    btrfs: disk-io: support subpage metadata csum calculation at write
>>      time
>>    btrfs: extent_io: make alloc_extent_buffer() check subpage dirty
>>      bitmap
>>    btrfs: extent_io: make the page uptodate assert check to handle
>>      subpage
>>    btrfs: extent_io: make set/clear_extent_buffer_dirty() to support
>>      subpage sized metadata
>>    btrfs: extent_io: make set_btree_ioerr() accept extent buffer and
>>      handle subpage metadata
>>    btrfs: extent_io: introduce end_bio_subpage_eb_writepage() function
>>    btrfs: extent_io: introduce write_one_subpage_eb() function
>>    btrfs: extent_io: make lock_extent_buffer_for_io() to support subpage
>>      metadata
>>    btrfs: extent_io: introduce submit_eb_subpage() to submit a subpage
>>      metadata page
> 
> Please don't use "extent_io" nor "disk-io" in subjects.
> 
Oh, those patches are too old before the naming schema change.

I should recheck all the checklist on them.

Thanks,
Qu