| Message ID | 20250311132616.1049687-1-stefanha@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | virtio-scsi: add iothread-vq-mapping parameter | expand |
On Tue, Mar 11, 2025 at 21:26:03 +0800, Stefan Hajnoczi wrote: > v4: > - Squash fixup commit properly this time >_< [Peter] > v3: > - Use vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED] as the AioContext for the Block > Backend [Kevin] > v2: > - Only expose cmd vqs via iothread-vq-mapping [Kevin, Peter] > > Implement --device virtio-scsi-pci,iothread-vq-mapping= support so that > virtqueues can be assigned to different IOThreads. This improves SMP guest > scalability where I/O-intensive applications can become bottlenecked on a > single IOThread. > > The following benchmark results show the effect of iothread-vq-mapping. fio > randread 4k iodepth=64 results from a 4 vCPU guest with an Intel P4800X SSD: > iothreads IOPS > ------------------------------ > 1 189576 > 2 312698 > 4 346744 I've modified my libvirt patches to take only the command queues into account same as with virtio-blk. After learning how 'fio' works I was able to set multiple iothreads on fire based on the mapping. Tested-by: Peter Krempa <pkrempa@redhat.com>
Am 11.03.2025 um 14:26 hat Stefan Hajnoczi geschrieben: > v4: > - Squash fixup commit properly this time >_< [Peter] > v3: > - Use vq_aio_context[VIRTIO_SCSI_VQ_NUM_FIXED] as the AioContext for the Block > Backend [Kevin] > v2: > - Only expose cmd vqs via iothread-vq-mapping [Kevin, Peter] > > Implement --device virtio-scsi-pci,iothread-vq-mapping= support so that > virtqueues can be assigned to different IOThreads. This improves SMP guest > scalability where I/O-intensive applications can become bottlenecked on a > single IOThread. > > The following benchmark results show the effect of iothread-vq-mapping. fio > randread 4k iodepth=64 results from a 4 vCPU guest with an Intel P4800X SSD: > iothreads IOPS > ------------------------------ > 1 189576 > 2 312698 > 4 346744 > > The virtio-scsi device model and core SCSI emulation currently assume that > requests are processed in a single AioContext. This patch series goes about > modifying this as follows: > > scsi-disk: drop unused SCSIDiskState->bh field > dma: use current AioContext for dma_blk_io() > > Make dma-helpers.c support the QEMU multi-queue block layer by using > qemu_get_current_aio_context(). > > scsi: track per-SCSIRequest AioContext > scsi: introduce requests_lock > > Make the core SCSI emulation code support processing requests in multiple > AioContexts by protecting the per-SCSIDevice requests list. > > virtio-scsi: introduce event and ctrl virtqueue locks > virtio-scsi: protect events_dropped field > virtio-scsi: perform TMFs in appropriate AioContexts > > Make the virtio-scsi emulation code support processing requests in multiple > AioContexts. The event and ctrl virtqueues can interact with multiple > AioContexts. Especially the SCSI Task Management Functions (TMFs) handled by > the ctrl virtqueue need to be made thread-safe. > > virtio-blk: extract cleanup_iothread_vq_mapping() function > virtio-blk: tidy up iothread_vq_mapping functions > virtio: extract iothread-vq-mapping.h API > virtio-scsi: add iothread-vq-mapping parameter > > Port over the iothread-vq-mapping qdev property from virtio-blk to virtio-scsi. > > virtio-scsi: handle ctrl virtqueue in main loop > > Simplify TMF handling now that there is no longer a single AioContext where all > requests are processed. > > virtio-scsi: only expose cmd vqs via iothread-vq-mapping > > Make the user interface more intuitive. > > Stefan Hajnoczi (13): > scsi-disk: drop unused SCSIDiskState->bh field > dma: use current AioContext for dma_blk_io() > scsi: track per-SCSIRequest AioContext > scsi: introduce requests_lock > virtio-scsi: introduce event and ctrl virtqueue locks > virtio-scsi: protect events_dropped field > virtio-scsi: perform TMFs in appropriate AioContexts > virtio-blk: extract cleanup_iothread_vq_mapping() function > virtio-blk: tidy up iothread_vq_mapping functions > virtio: extract iothread-vq-mapping.h API > virtio-scsi: add iothread-vq-mapping parameter > virtio-scsi: handle ctrl virtqueue in main loop > virtio-scsi: only expose cmd vqs via iothread-vq-mapping > > include/hw/scsi/scsi.h | 8 +- > include/hw/virtio/iothread-vq-mapping.h | 45 ++ > include/hw/virtio/virtio-scsi.h | 15 +- > include/system/dma.h | 3 +- > hw/block/virtio-blk.c | 132 +----- > hw/ide/core.c | 3 +- > hw/ide/macio.c | 3 +- > hw/scsi/scsi-bus.c | 121 ++++-- > hw/scsi/scsi-disk.c | 24 +- > hw/scsi/virtio-scsi-dataplane.c | 103 +++-- > hw/scsi/virtio-scsi.c | 542 ++++++++++++++---------- > hw/virtio/iothread-vq-mapping.c | 131 ++++++ > system/dma-helpers.c | 8 +- > hw/virtio/meson.build | 1 + > 14 files changed, 683 insertions(+), 456 deletions(-) > create mode 100644 include/hw/virtio/iothread-vq-mapping.h > create mode 100644 hw/virtio/iothread-vq-mapping.c Thanks, applied to the block branch. Kevin