[V3,0/10] blk-mq: cleanup map queues & fix blk_mq_alloc_request_hctx

Message

Ming Lei July 9, 2021, 8:09 a.m. UTC

Hi,

blk_mq_alloc_request_hctx() is used by NVMe fc/rdma/tcp/loop to connect
io queue. Also the sw ctx is chosen as the 1st online cpu in hctx->cpumask.
However, all cpus in hctx->cpumask may be offline.

This usage model isn't well supported by blk-mq which supposes allocator is
always done on one online CPU in hctx->cpumask. This assumption is
related with managed irq, which also requires blk-mq to drain inflight
request in this hctx when the last cpu in hctx->cpumask is going to
offline.

However, NVMe fc/rdma/tcp/loop don't use managed irq, so we should allow
them to ask for request allocation when the specified hctx is inactive
(all cpus in hctx->cpumask are offline). Fix blk_mq_alloc_request_hctx() by
allowing to allocate request when all CPUs of this hctx are offline.

Also cleans up map queues helpers, replace current pci/virtio/rdma
helpers with blk_mq_dev_map_queues(), and deal with the device
difference by passing one callback from driver, and the actual only
difference is that how to retrieve queue affinity. Finally the single helper
can meet all driver's requirement.


V3:
	- cleanup map queues helpers, and remove pci/virtio/rdma queue
	  helpers
	- store use managed irq info into qmap


V2:
	- use flag of BLK_MQ_F_MANAGED_IRQ
	- pass BLK_MQ_F_MANAGED_IRQ from driver explicitly
	- kill BLK_MQ_F_STACKING


Ming Lei (10):
  blk-mq: rename blk-mq-cpumap.c as blk-mq-map.c
  blk-mq: Introduce blk_mq_dev_map_queues
  blk-mq: pass use managed irq info to blk_mq_dev_map_queues
  scsi: replace blk_mq_pci_map_queues with blk_mq_dev_map_queues
  nvme: replace blk_mq_pci_map_queues with blk_mq_dev_map_queues
  virito: add APIs for retrieving vq affinity
  virtio: blk/scsi: replace blk_mq_virtio_map_queues with
    blk_mq_dev_map_queues
  nvme: rdma: replace blk_mq_rdma_map_queues with blk_mq_dev_map_queues
  blk-mq: remove map queue helpers for pci, rdma and virtio
  blk-mq: don't deactivate hctx if managed irq isn't used

 block/Makefile                            |  5 +-
 block/{blk-mq-cpumap.c => blk-mq-map.c}   | 57 +++++++++++++++++++++++
 block/blk-mq-pci.c                        | 48 -------------------
 block/blk-mq-rdma.c                       | 44 -----------------
 block/blk-mq-virtio.c                     | 46 ------------------
 block/blk-mq.c                            | 27 +++++++----
 block/blk-mq.h                            |  5 ++
 drivers/block/virtio_blk.c                | 12 ++++-
 drivers/nvme/host/pci.c                   | 12 ++++-
 drivers/nvme/host/rdma.c                  | 18 +++++--
 drivers/scsi/hisi_sas/hisi_sas_v2_hw.c    | 21 ++++-----
 drivers/scsi/hisi_sas/hisi_sas_v3_hw.c    |  5 +-
 drivers/scsi/megaraid/megaraid_sas_base.c |  4 +-
 drivers/scsi/mpi3mr/mpi3mr_os.c           |  9 ++--
 drivers/scsi/mpt3sas/mpt3sas_scsih.c      |  6 ++-
 drivers/scsi/qla2xxx/qla_os.c             |  4 +-
 drivers/scsi/scsi_priv.h                  |  9 ++++
 drivers/scsi/smartpqi/smartpqi_init.c     |  7 ++-
 drivers/scsi/virtio_scsi.c                | 11 ++++-
 drivers/virtio/virtio.c                   | 10 ++++
 include/linux/blk-mq.h                    |  8 +++-
 include/linux/virtio.h                    |  2 +
 22 files changed, 186 insertions(+), 184 deletions(-)
 rename block/{blk-mq-cpumap.c => blk-mq-map.c} (58%)
 delete mode 100644 block/blk-mq-pci.c
 delete mode 100644 block/blk-mq-rdma.c
 delete mode 100644 block/blk-mq-virtio.c