[V3,0/6] blk-mq: support batching dispatch from scheduler

Message

Ming Lei May 28, 2020, 8 a.m. UTC

Hi Jens,

More and more drivers want to get batching requests queued from
block layer, such as mmc[1], and tcp based storage drivers[2]. Also
current in-tree users have virtio-scsi, virtio-blk and nvme.

For none, we already support batching dispatch.

But for io scheduler, every time we just take one request from scheduler
and pass the single request to blk_mq_dispatch_rq_list(). This way makes
batching dispatch not possible when io scheduler is applied. One reason
is that we don't want to hurt sequential IO performance, becasue IO
merge chance is reduced if more requests are dequeued from scheduler
queue.

Tries to start the support by dequeuing more requests from scheduler
if budget is enough and device isn't busy.

Simple fio test over virtio-scsi shows IO can get improved by 5~10%.

Baolin has tested V1 and found performance on MMC can be improved.

Patches can be found from the following tree too:

	https://github.com/ming1/linux/commits/v5.7-rc-blk-mq-batching-submission

Patch 1 ~ 4 are improvement and cleanup, which can't applied without
supporting batching dispatch.

Patch 5 ~ 6 starts to support batching dispatch from scheduler.


[1] https://lore.kernel.org/linux-block/20200512075501.GF1531898@T590/#r
[2] https://lore.kernel.org/linux-block/[e-mail]/

V3:
	- add reviewed-by tag
	- fix one typo
	- fix one budget leak issue in case that .queue_rq returned *_RESOURCE in 5/6

V2:
	- remove 'got_budget' from blk_mq_dispatch_rq_list
	- drop patch for getting driver tag & handling partial dispatch

Ming Lei (6):
  blk-mq: pass request queue into get/put budget callback
  blk-mq: pass hctx to blk_mq_dispatch_rq_list
  blk-mq: move getting driver tag and budget into one helper
  blk-mq: remove dead check from blk_mq_dispatch_rq_list
  blk-mq: pass obtained budget count to blk_mq_dispatch_rq_list
  blk-mq: support batching dispatch in case of io scheduler

 block/blk-mq-sched.c    |  95 ++++++++++++++++++++++++++++-----
 block/blk-mq.c          | 115 ++++++++++++++++++++++++++--------------
 block/blk-mq.h          |  15 +++---
 drivers/scsi/scsi_lib.c |   8 ++-
 include/linux/blk-mq.h  |   4 +-
 5 files changed, 168 insertions(+), 69 deletions(-)

Cc: Sagi Grimberg <[e-mail]>
Cc: Baolin Wang <[e-mail]>
Cc: Christoph Hellwig <[e-mail]>

Comments

Baolin Wang May 31, 2020, 4:09 a.m. UTC | #1

On Thu, May 28, 2020 at 4:01 PM Ming Lei <[e-mail]> wrote:
>
> Hi Jens,
>
> More and more drivers want to get batching requests queued from
> block layer, such as mmc[1], and tcp based storage drivers[2]. Also
> current in-tree users have virtio-scsi, virtio-blk and nvme.
>
> For none, we already support batching dispatch.
>
> But for io scheduler, every time we just take one request from scheduler
> and pass the single request to blk_mq_dispatch_rq_list(). This way makes
> batching dispatch not possible when io scheduler is applied. One reason
> is that we don't want to hurt sequential IO performance, becasue IO
> merge chance is reduced if more requests are dequeued from scheduler
> queue.
>
> Tries to start the support by dequeuing more requests from scheduler
> if budget is enough and device isn't busy.
>
> Simple fio test over virtio-scsi shows IO can get improved by 5~10%.
>
> Baolin has tested V1 and found performance on MMC can be improved.
>
> Patches can be found from the following tree too:
>
>         https://github.com/ming1/linux/commits/v5.7-rc-blk-mq-batching-submission
>
> Patch 1 ~ 4 are improvement and cleanup, which can't applied without
> supporting batching dispatch.
>
> Patch 5 ~ 6 starts to support batching dispatch from scheduler.
>
>
> [1] https://lore.kernel.org/linux-block/20200512075501.GF1531898@T590/#r
> [2] https://lore.kernel.org/linux-block/[e-mail]/
>
> V3:
>         - add reviewed-by tag
>         - fix one typo
>         - fix one budget leak issue in case that .queue_rq returned *_RESOURCE in 5/6
>
> V2:
>         - remove 'got_budget' from blk_mq_dispatch_rq_list
>         - drop patch for getting driver tag & handling partial dispatch
>
> Ming Lei (6):
>   blk-mq: pass request queue into get/put budget callback
>   blk-mq: pass hctx to blk_mq_dispatch_rq_list
>   blk-mq: move getting driver tag and budget into one helper
>   blk-mq: remove dead check from blk_mq_dispatch_rq_list
>   blk-mq: pass obtained budget count to blk_mq_dispatch_rq_list
>   blk-mq: support batching dispatch in case of io scheduler
>
>  block/blk-mq-sched.c    |  95 ++++++++++++++++++++++++++++-----
>  block/blk-mq.c          | 115 ++++++++++++++++++++++++++--------------
>  block/blk-mq.h          |  15 +++---
>  drivers/scsi/scsi_lib.c |   8 ++-
>  include/linux/blk-mq.h  |   4 +-
>  5 files changed, 168 insertions(+), 69 deletions(-)
>
> Cc: Sagi Grimberg <[e-mail]>
> Cc: Baolin Wang <[e-mail]>
> Cc: Christoph Hellwig <[e-mail]>

Works well for me again. So for the whole patch set:
Tested-by: Baolin Wang <[e-mail]>