[RFC,-next,00/26] md/md-bitmap: introduce bitmap_operations

Message

Yu Kuai Aug. 10, 2024, 2:08 a.m. UTC

From: Yu Kuai <[e-mail]>

The background is that currently bitmap is using a global spin_lock,
cauing lock contention and huge IO performance degration for all raid
levels.

However, it's impossible to implement a new lock free bitmap with
current situation that md-bitmap exposes the internal implementation
with lots of exported apis. Hence bitmap_operations is invented, to
describe bitmap core implementation, and a new bitmap can be introduced
with a new bitmap_operations, we only need to switch to the new one
during initialization.

And with this we can build bitmap as kernel module, but that's not
our concern for now.

Noted I just compile this patchset, not tested yet.

Yu Kuai (26):
  md/md-bitmap: introduce struct bitmap_operations
  md/md-bitmap: merge md_bitmap_create() into bitmap_operations
  md/md-bitmap: merge md_bitmap_load() into bitmap_operations
  md/md-bitmap: merge md_bitmap_destroy() into bitmap_operations
  md/md-bitmap: merge md_bitmap_flush() into bitmap_operations
  md/md-bitmap: don't expose md_bitmap_print_sb()
  md/md-bitmap: merge md_bitmap_update_sb() into bitmap_operations
  md/md-bitmap: merge md_bitmap_status() into bitmap_operations
  md/md-bitmap: remove md_bitmap_setallbits()
  md/md-bitmap: merge bitmap_write_all() into bitmap_operations
  md/md-bitmap: merge md_bitmap_dirty_bits() into bitmap_operations
  md/md-bitmap: merge md_bitmap_startwrite() into bitmap_operations
  md/md-bitmap: merge md_bitmap_endwrite() into bitmap_operations
  md/md-bitmap: merge md_bitmap_start_sync() into bitmap_operations
  md/md-bitmap: merge md_bitmap_end_sync() into bitmap_operations
  md/md-bitmap: merge md_bitmap_close_sync() into bitmap_operations
  md/md-bitmap: mrege md_bitmap_cond_end_sync() into bitmap_operations
  md/md-bitmap: merge bitmap_sync_with_cluster() into bitmap_operations
  md/md-bitmap: merge md_bitmap_resize() into bitmap_operations
  md/md-bitmap: merge get_bitmap_from_slot() into bitmap_operations
  md/md-bitmap: merge md_bitmap_copy_from_slot() into bitmap_operations
  md/md-bitmap: merge md_bitmap_free() into bitmap_operations
  md/md-bitmap: merge md_bitmap_wait_behind_writes() into
    bitmap_operations
  md/md-bitmap: merge md_bitmap_daemon_work() into bitmap_operations
  md/md-bitmap: merge md_bitmap_unplug() and md_bitmap_unplug_async()
  md/md-bitmap: merge bitmap_unplug() into bitmap_operations

 drivers/md/dm-raid.c     |   2 +-
 drivers/md/md-bitmap.c   | 216 +++++++++++++++++---------------
 drivers/md/md-bitmap.h   | 259 ++++++++++++++++++++++++++++++++++-----
 drivers/md/md-cluster.c  |  21 ++--
 drivers/md/md.c          |  13 +-
 drivers/md/md.h          |   1 +
 drivers/md/raid1-10.c    |   7 +-
 drivers/md/raid1.c       |  22 ++--
 drivers/md/raid10.c      |  32 +++--
 drivers/md/raid5-cache.c |   2 +-
 drivers/md/raid5.c       |  25 ++--
 11 files changed, 401 insertions(+), 199 deletions(-)

Comments

Christoph Hellwig Aug. 13, 2024, 7:21 a.m. UTC | #1

On Sat, Aug 10, 2024 at 10:08:28AM +0800, Yu Kuai wrote:
> From: Yu Kuai <[e-mail]>
> 
> The background is that currently bitmap is using a global spin_lock,
> cauing lock contention and huge IO performance degration for all raid
> levels.
> 
> However, it's impossible to implement a new lock free bitmap with
> current situation that md-bitmap exposes the internal implementation
> with lots of exported apis. Hence bitmap_operations is invented, to
> describe bitmap core implementation, and a new bitmap can be introduced
> with a new bitmap_operations, we only need to switch to the new one
> during initialization.
> 
> And with this we can build bitmap as kernel module, but that's not
> our concern for now.
> 
> Noted I just compile this patchset, not tested yet.

Refactoring the bitmap code to be modular seems like a good idea.

But I'd just turn this into plain function calls and maybe a hidden
data structure if you feel really fancy.  No need to introduce expensive
indirect calls and a separate module.

Yu Kuai Aug. 13, 2024, 7:36 a.m. UTC | #2

在 2024/08/13 15:21, Christoph Hellwig 写道:
> On Sat, Aug 10, 2024 at 10:08:28AM +0800, Yu Kuai wrote:
>> From: Yu Kuai <[e-mail]>
>>
>> The background is that currently bitmap is using a global spin_lock,
>> cauing lock contention and huge IO performance degration for all raid
>> levels.
>>
>> However, it's impossible to implement a new lock free bitmap with
>> current situation that md-bitmap exposes the internal implementation
>> with lots of exported apis. Hence bitmap_operations is invented, to
>> describe bitmap core implementation, and a new bitmap can be introduced
>> with a new bitmap_operations, we only need to switch to the new one
>> during initialization.
>>
>> And with this we can build bitmap as kernel module, but that's not
>> our concern for now.
>>
>> Noted I just compile this patchset, not tested yet.
> 
> Refactoring the bitmap code to be modular seems like a good idea.
> 
> But I'd just turn this into plain function calls and maybe a hidden
> data structure if you feel really fancy.  No need to introduce expensive
> indirect calls and a separate module.

Got it, and will make the struct bitmap a hidden structure in the next
version. This is exactly what I'm doing now locally, get rid of bitmap
direct dereference :).

Thanks for the review!
Kuai
> 
> .
>