From patchwork Fri Jan 24 10:01:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 11350041 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 919A8138C for ; Fri, 24 Jan 2020 10:03:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 68FCB214DB for ; Fri, 24 Jan 2020 10:03:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="U6OcHUiT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 68FCB214DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:39682 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvo9-0008Nh-Ff for patchwork-qemu-devel@patchwork.kernel.org; Fri, 24 Jan 2020 05:03:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38115) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvn4-0006BD-2f for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iuvn2-0006OT-UQ for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:14 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:37798 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iuvn2-0006OB-Qp for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579860132; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QeMtCxkFpvpr8tuwH+DUSIg6/oyH0JkxWISM4qWiQEM=; b=U6OcHUiT1xOmw9qEO5YEmHEcHOm/MQVCQAwzEk8OJgeslaXyCJAsPaFeacKx6NHJLm0sKl U4iOC7KVjxkb81EJ7+yqPSwJBYUffdTXNTL3g4yWCkLiWhHenRnNJAI/QNv5dBwrhB2Dc1 ZwGZ8sVUS9Sy8Kj2NCRhsIQFz8TWbCE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-337-3_ygj4oqOW2v14stt-vKEg-1; Fri, 24 Jan 2020 05:02:09 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F1DC08017CC; Fri, 24 Jan 2020 10:02:07 +0000 (UTC) Received: from localhost (ovpn-117-162.ams2.redhat.com [10.36.117.162]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8596C19756; Fri, 24 Jan 2020 10:02:07 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v2 1/4] virtio-scsi: introduce a constant for fixed virtqueues Date: Fri, 24 Jan 2020 10:01:56 +0000 Message-Id: <20200124100159.736209-2-stefanha@redhat.com> In-Reply-To: <20200124100159.736209-1-stefanha@redhat.com> References: <20200124100159.736209-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: 3_ygj4oqOW2v14stt-vKEg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Stefan Hajnoczi , Paolo Bonzini , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The event and control virtqueues are always present, regardless of the multi-queue configuration. Define a constant so that virtqueue number calculations are easier to read. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- hw/scsi/vhost-user-scsi.c | 2 +- hw/scsi/virtio-scsi.c | 7 ++++--- include/hw/virtio/virtio-scsi.h | 3 +++ 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 23f972df59..eb37733bd0 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -115,7 +115,7 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) goto free_virtio; } - vsc->dev.nvqs = 2 + vs->conf.num_queues; + vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; vsc->dev.vqs = g_new0(struct vhost_virtqueue, vsc->dev.nvqs); vsc->dev.vq_index = 0; vsc->dev.backend_features = 0; diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index d3af42ef92..224a290498 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -191,7 +191,7 @@ static void virtio_scsi_save_request(QEMUFile *f, SCSIRequest *sreq) VirtIOSCSIReq *req = sreq->hba_private; VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(req->dev); VirtIODevice *vdev = VIRTIO_DEVICE(req->dev); - uint32_t n = virtio_get_queue_index(req->vq) - 2; + uint32_t n = virtio_get_queue_index(req->vq) - VIRTIO_SCSI_VQ_NUM_FIXED; assert(n < vs->conf.num_queues); qemu_put_be32s(f, &n); @@ -892,10 +892,11 @@ void virtio_scsi_common_realize(DeviceState *dev, sizeof(VirtIOSCSIConfig)); if (s->conf.num_queues == 0 || - s->conf.num_queues > VIRTIO_QUEUE_MAX - 2) { + s->conf.num_queues > VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIXED) { error_setg(errp, "Invalid number of queues (= %" PRIu32 "), " "must be a positive integer less than %d.", - s->conf.num_queues, VIRTIO_QUEUE_MAX - 2); + s->conf.num_queues, + VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIXED); virtio_cleanup(vdev); return; } diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h index 24e768909d..9f293bcb80 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -36,6 +36,9 @@ #define VIRTIO_SCSI_MAX_TARGET 255 #define VIRTIO_SCSI_MAX_LUN 16383 +/* Number of virtqueues that are always present */ +#define VIRTIO_SCSI_VQ_NUM_FIXED 2 + typedef struct virtio_scsi_cmd_req VirtIOSCSICmdReq; typedef struct virtio_scsi_cmd_resp VirtIOSCSICmdResp; typedef struct virtio_scsi_ctrl_tmf_req VirtIOSCSICtrlTMFReq; From patchwork Fri Jan 24 10:01:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 11350037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1AF91139A for ; Fri, 24 Jan 2020 10:03:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E1B1921556 for ; Fri, 24 Jan 2020 10:03:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bnFKJZ1j" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E1B1921556 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:39680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvo4-0008B8-T7 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 24 Jan 2020 05:03:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38143) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvn8-0006GQ-Oq for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iuvn6-0006Rs-Pt for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:18 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:51147 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iuvn6-0006Pr-Ib for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579860134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MQxVhFjVOtXTswwkqeSbsQS/+HjZ0EegRuzS2Ji1B60=; b=bnFKJZ1joN3tt/xVdlJYSlW9lfVwg0gNkRDBHFkdvRGVusfadqjhNeF3CrRpyykBLr7/Xy 0enuD0BrHQA5zPGngOPQ/AoM65PM8w5mA+6eGi7H+RpmV8mkF9rKBrkPcNg75lMyFiv47A jgwkqA1KGPPzdTEQ0yOMlUql8AXwUSA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-100-qBO7CEEPPvakh24HXfb3kw-1; Fri, 24 Jan 2020 05:02:12 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2943C107ACC5; Fri, 24 Jan 2020 10:02:11 +0000 (UTC) Received: from localhost (ovpn-117-162.ams2.redhat.com [10.36.117.162]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE1C119C69; Fri, 24 Jan 2020 10:02:10 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v2 2/4] virtio-scsi: default num_queues to -smp N Date: Fri, 24 Jan 2020 10:01:57 +0000 Message-Id: <20200124100159.736209-3-stefanha@redhat.com> In-Reply-To: <20200124100159.736209-1-stefanha@redhat.com> References: <20200124100159.736209-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: qBO7CEEPPvakh24HXfb3kw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Stefan Hajnoczi , Paolo Bonzini , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Automatically size the number of request virtqueues to match the number of vCPUs. This ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. Signed-off-by: Stefan Hajnoczi --- hw/core/machine.c | 3 +++ hw/scsi/vhost-scsi.c | 3 ++- hw/scsi/vhost-user-scsi.c | 3 ++- hw/scsi/virtio-scsi.c | 6 +++++- hw/virtio/vhost-scsi-pci.c | 10 ++++++++-- hw/virtio/vhost-user-scsi-pci.c | 10 ++++++++-- hw/virtio/virtio-scsi-pci.c | 10 ++++++++-- include/hw/virtio/virtio-scsi.h | 2 ++ 8 files changed, 38 insertions(+), 9 deletions(-) diff --git a/hw/core/machine.c b/hw/core/machine.c index 3e288bfceb..d6e2370c77 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -30,8 +30,11 @@ GlobalProperty hw_compat_4_2[] = { { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" }, { "virtio-blk-device", "seg-max-adjust", "off"}, + { "virtio-scsi-device", "num_queues", "1"}, { "virtio-scsi-device", "seg_max_adjust", "off"}, { "vhost-blk-device", "seg_max_adjust", "off"}, + { "vhost-scsi", "num_queues", "1"}, + { "vhost-user-scsi", "num_queues", "1"}, { "usb-host", "suppress-remote-wake", "off" }, { "usb-redir", "suppress-remote-wake", "off" }, }; diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 26f710d3ec..80fe5d999a 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -272,7 +272,8 @@ static Property vhost_scsi_properties[] = { DEFINE_PROP_STRING("vhostfd", VirtIOSCSICommon, conf.vhostfd), DEFINE_PROP_STRING("wwpn", VirtIOSCSICommon, conf.wwpn), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), - DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_size, 128), DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSICommon, conf.seg_max_adjust, diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index eb37733bd0..655d300875 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -163,7 +163,8 @@ static void vhost_user_scsi_unrealize(DeviceState *dev, Error **errp) static Property vhost_user_scsi_properties[] = { DEFINE_PROP_CHR("chardev", VirtIOSCSICommon, conf.chardev), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), - DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_size, 128), DEFINE_PROP_UINT32("max_sectors", VirtIOSCSICommon, conf.max_sectors, diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 224a290498..c9342004ef 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -891,6 +891,9 @@ void virtio_scsi_common_realize(DeviceState *dev, virtio_init(vdev, "virtio-scsi", VIRTIO_ID_SCSI, sizeof(VirtIOSCSIConfig)); + if (s->conf.num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + s->conf.num_queues = 1; + } if (s->conf.num_queues == 0 || s->conf.num_queues > VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIXED) { error_setg(errp, "Invalid number of queues (= %" PRIu32 "), " @@ -964,7 +967,8 @@ static void virtio_scsi_device_unrealize(DeviceState *dev, Error **errp) } static Property virtio_scsi_properties[] = { - DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI, parent_obj.conf.virtqueue_size, 128), DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI, diff --git a/hw/virtio/vhost-scsi-pci.c b/hw/virtio/vhost-scsi-pci.c index e8dfbfc60f..38a8f0c3ef 100644 --- a/hw/virtio/vhost-scsi-pci.c +++ b/hw/virtio/vhost-scsi-pci.c @@ -17,6 +17,7 @@ #include "qemu/osdep.h" #include "standard-headers/linux/virtio_pci.h" +#include "hw/boards.h" #include "hw/qdev-properties.h" #include "hw/virtio/vhost-scsi.h" #include "qapi/error.h" @@ -47,10 +48,15 @@ static void vhost_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VHostSCSIPCI *dev = VHOST_SCSI_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); - VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); + VirtIOSCSIConf *conf = &dev->vdev.parent_obj.parent_obj.conf; + + /* 1:1 vq to vcpu mapping is ideal because it avoids IPIs */ + if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues = current_machine->smp.cpus; + } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = vs->conf.num_queues + 3; + vpci_dev->nvectors = conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus)); diff --git a/hw/virtio/vhost-user-scsi-pci.c b/hw/virtio/vhost-user-scsi-pci.c index ff13af7030..0cad29eb67 100644 --- a/hw/virtio/vhost-user-scsi-pci.c +++ b/hw/virtio/vhost-user-scsi-pci.c @@ -18,6 +18,7 @@ #include "qemu/osdep.h" #include "standard-headers/linux/virtio_pci.h" +#include "hw/boards.h" #include "hw/virtio/vhost-user-scsi.h" #include "hw/virtio/virtio.h" #include "hw/virtio/virtio-scsi.h" @@ -53,10 +54,15 @@ static void vhost_user_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VHostUserSCSIPCI *dev = VHOST_USER_SCSI_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); - VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); + VirtIOSCSIConf *conf = &dev->vdev.parent_obj.parent_obj.conf; + + /* 1:1 vq to vcpu mapping is ideal because it avoids IPIs */ + if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues = current_machine->smp.cpus; + } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = vs->conf.num_queues + 3; + vpci_dev->nvectors = conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus)); diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c index 3c55dc19a1..b22c8b79e2 100644 --- a/hw/virtio/virtio-scsi-pci.c +++ b/hw/virtio/virtio-scsi-pci.c @@ -15,6 +15,7 @@ #include "qemu/osdep.h" +#include "hw/boards.h" #include "hw/qdev-properties.h" #include "hw/virtio/virtio-scsi.h" #include "qemu/module.h" @@ -46,12 +47,17 @@ static void virtio_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VirtIOSCSIPCI *dev = VIRTIO_SCSI_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); - VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(vdev); DeviceState *proxy = DEVICE(vpci_dev); + VirtIOSCSIConf *conf = &dev->vdev.parent_obj.conf; char *bus_name; + /* 1:1 vq to vcpu mapping is ideal because it avoids IPIs */ + if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues = current_machine->smp.cpus; + } + if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = vs->conf.num_queues + 3; + vpci_dev->nvectors = conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } /* diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h index 9f293bcb80..c0b8e4dd7e 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -39,6 +39,8 @@ /* Number of virtqueues that are always present */ #define VIRTIO_SCSI_VQ_NUM_FIXED 2 +#define VIRTIO_SCSI_AUTO_NUM_QUEUES UINT32_MAX + typedef struct virtio_scsi_cmd_req VirtIOSCSICmdReq; typedef struct virtio_scsi_cmd_resp VirtIOSCSICmdResp; typedef struct virtio_scsi_ctrl_tmf_req VirtIOSCSICtrlTMFReq; From patchwork Fri Jan 24 10:01:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 11350043 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3334F139A for ; Fri, 24 Jan 2020 10:04:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A011222C2 for ; Fri, 24 Jan 2020 10:04:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Nc9FEWhw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A011222C2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:39706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvpB-0001xf-7h for patchwork-qemu-devel@patchwork.kernel.org; Fri, 24 Jan 2020 05:04:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38206) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvnE-0006Lb-GA for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iuvnD-0006Yr-65 for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:24 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:26488 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iuvnC-0006YI-P1 for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579860142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jvQ5GA+mBgNZIB4n151iIGtmYdm0/MUwPyFgTl16W54=; b=Nc9FEWhwBW0LEdywtgf85re5InHw8nSTUJ+SS4nlGUxr/bK8kmqaJMOEC+MmmUpD+ZVjXD sUc70XX0GWwckYNgiqRd0Ps+uZ9ShuDNNwjfJFcQ+TcwWcP6K9kigXmTBaE9Qy6FXH+xj1 +DTQ+DscU5TCgdv6iXj1Y0XdktmXrE0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-396-Qup9UDWPNqCqCKtvIJ_NxA-1; Fri, 24 Jan 2020 05:02:18 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 971DB1005510; Fri, 24 Jan 2020 10:02:17 +0000 (UTC) Received: from localhost (ovpn-117-162.ams2.redhat.com [10.36.117.162]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7322985784; Fri, 24 Jan 2020 10:02:12 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v2 3/4] virtio-blk: default num_queues to -smp N Date: Fri, 24 Jan 2020 10:01:58 +0000 Message-Id: <20200124100159.736209-4-stefanha@redhat.com> In-Reply-To: <20200124100159.736209-1-stefanha@redhat.com> References: <20200124100159.736209-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: Qup9UDWPNqCqCKtvIJ_NxA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Stefan Hajnoczi , Paolo Bonzini , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Automatically size the number of request virtqueues to match the number of vCPUs. This ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- hw/block/virtio-blk.c | 6 +++++- hw/core/machine.c | 1 + hw/virtio/virtio-blk-pci.c | 9 ++++++++- include/hw/virtio/virtio-blk.h | 2 ++ 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 9bee514c4e..d3ffaffc93 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1135,6 +1135,9 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) error_setg(errp, "Device needs media, but drive is empty"); return; } + if (conf->num_queues == VIRTIO_BLK_AUTO_NUM_QUEUES) { + conf->num_queues = 1; + } if (!conf->num_queues) { error_setg(errp, "num-queues property must be larger than 0"); return; @@ -1271,7 +1274,8 @@ static Property virtio_blk_properties[] = { #endif DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, 0, true), - DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1), + DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, + VIRTIO_BLK_AUTO_NUM_QUEUES), DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 128), DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, true), DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD, diff --git a/hw/core/machine.c b/hw/core/machine.c index d6e2370c77..de6ceaa97f 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -28,6 +28,7 @@ #include "hw/mem/nvdimm.h" GlobalProperty hw_compat_4_2[] = { + { "virtio-blk-device", "num-queues", "1"}, { "virtio-blk-device", "x-enable-wce-if-config-wce", "off" }, { "virtio-blk-device", "seg-max-adjust", "off"}, { "virtio-scsi-device", "num_queues", "1"}, diff --git a/hw/virtio/virtio-blk-pci.c b/hw/virtio/virtio-blk-pci.c index d9b69a5af3..7e6d863963 100644 --- a/hw/virtio/virtio-blk-pci.c +++ b/hw/virtio/virtio-blk-pci.c @@ -17,6 +17,7 @@ #include "qemu/osdep.h" +#include "hw/boards.h" #include "hw/qdev-properties.h" #include "hw/virtio/virtio-blk.h" #include "virtio-pci.h" @@ -50,9 +51,15 @@ static void virtio_blk_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { VirtIOBlkPCI *dev = VIRTIO_BLK_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); + VirtIOBlkConf *conf = &dev->vdev.conf; + + /* 1:1 vq to vcpu mapping is ideal because it avoids IPIs */ + if (conf->num_queues == VIRTIO_BLK_AUTO_NUM_QUEUES) { + conf->num_queues = current_machine->smp.cpus; + } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors = dev->vdev.conf.num_queues + 1; + vpci_dev->nvectors = conf->num_queues + 1; } qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus)); diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index 1e62f869b2..4e5e903f4a 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -30,6 +30,8 @@ struct virtio_blk_inhdr unsigned char status; }; +#define VIRTIO_BLK_AUTO_NUM_QUEUES UINT16_MAX + struct VirtIOBlkConf { BlockConf conf; From patchwork Fri Jan 24 10:01:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 11350047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FBE2139A for ; Fri, 24 Jan 2020 10:05:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 76B1F20704 for ; Fri, 24 Jan 2020 10:05:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="K7+nafv0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76B1F20704 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:39762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvqZ-0004IF-LL for patchwork-qemu-devel@patchwork.kernel.org; Fri, 24 Jan 2020 05:05:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:38235) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iuvnI-0006UN-S6 for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iuvnH-0006cK-I4 for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:28 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:37666 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iuvnH-0006c3-Dk for qemu-devel@nongnu.org; Fri, 24 Jan 2020 05:02:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1579860147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zapCsf3yIAWy9YVSk9S/kiNcaPecTeVpB4Pj1Kno3O8=; b=K7+nafv0Q7dQ3XDPiUbrC25XC2/2+83ubNEjvkhyl7SYf6k2H3GqmXoito3d30F0HrkhjM 6+HuiVMdGr2zz9ZoBBrE8KN2dPjL2O1M4ZNZP8/2teYaEABmUxGwPLKu2jH/g2HAKupKva wrdzRBnexcytHBIP3LB8uj4yuEEr/Jo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-181-raErzDmaMa-J_zpET0Ha8A-1; Fri, 24 Jan 2020 05:02:25 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5BF71477; Fri, 24 Jan 2020 10:02:24 +0000 (UTC) Received: from localhost (ovpn-117-162.ams2.redhat.com [10.36.117.162]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0456219C69; Fri, 24 Jan 2020 10:02:18 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v2 4/4] vhost-user-blk: default num_queues to -smp N Date: Fri, 24 Jan 2020 10:01:59 +0000 Message-Id: <20200124100159.736209-5-stefanha@redhat.com> In-Reply-To: <20200124100159.736209-1-stefanha@redhat.com> References: <20200124100159.736209-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: raErzDmaMa-J_zpET0Ha8A-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Stefan Hajnoczi , Paolo Bonzini , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Automatically size the number of request virtqueues to match the number of vCPUs. This ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- hw/block/vhost-user-blk.c | 6 +++++- hw/core/machine.c | 1 + hw/virtio/vhost-user-blk-pci.c | 6 ++++++ include/hw/virtio/vhost-user-blk.h | 2 ++ 4 files changed, 14 insertions(+), 1 deletion(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 98b383f90e..2ee26a434c 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -403,6 +403,9 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) return; } + if (s->num_queues == VHOST_USER_BLK_AUTO_NUM_QUEUES) { + s->num_queues = 1; + } if (!s->num_queues || s->num_queues > VIRTIO_QUEUE_MAX) { error_setg(errp, "vhost-user-blk: invalid number of IO queues"); return; @@ -500,7 +503,8 @@ static const VMStateDescription vmstate_vhost_user_blk = { static Property vhost_user_blk_properties[] = { DEFINE_PROP_CHR("chardev", VHostUserBlk, chardev), - DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues, 1), + DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues, + VHOST_USER_BLK_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("queue-size", VHostUserBlk, queue_size, 128), DEFINE_PROP_BIT("config-wce", VHostUserBlk, config_wce, 0, true), DEFINE_PROP_END_OF_LIST(), diff --git a/hw/core/machine.c b/hw/core/machine.c index de6ceaa97f..d4c67f4d6e 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -35,6 +35,7 @@ GlobalProperty hw_compat_4_2[] = { { "virtio-scsi-device", "seg_max_adjust", "off"}, { "vhost-blk-device", "seg_max_adjust", "off"}, { "vhost-scsi", "num_queues", "1"}, + { "vhost-user-blk", "num-queues", "1"}, { "vhost-user-scsi", "num_queues", "1"}, { "usb-host", "suppress-remote-wake", "off" }, { "usb-redir", "suppress-remote-wake", "off" }, diff --git a/hw/virtio/vhost-user-blk-pci.c b/hw/virtio/vhost-user-blk-pci.c index 1dc834a3ff..cf72b21c16 100644 --- a/hw/virtio/vhost-user-blk-pci.c +++ b/hw/virtio/vhost-user-blk-pci.c @@ -19,6 +19,7 @@ #include "qemu/osdep.h" #include "standard-headers/linux/virtio_pci.h" +#include "hw/boards.h" #include "hw/virtio/virtio.h" #include "hw/virtio/vhost-user-blk.h" #include "hw/pci/pci.h" @@ -54,6 +55,11 @@ static void vhost_user_blk_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) VHostUserBlkPCI *dev = VHOST_USER_BLK_PCI(vpci_dev); DeviceState *vdev = DEVICE(&dev->vdev); + /* 1:1 vq to vcpu mapping is ideal because it avoids IPIs */ + if (dev->vdev.num_queues == VHOST_USER_BLK_AUTO_NUM_QUEUES) { + dev->vdev.num_queues = current_machine->smp.cpus; + } + if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { vpci_dev->nvectors = dev->vdev.num_queues + 1; } diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h index 108bfadeeb..5a353dc1c6 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -25,6 +25,8 @@ #define VHOST_USER_BLK(obj) \ OBJECT_CHECK(VHostUserBlk, (obj), TYPE_VHOST_USER_BLK) +#define VHOST_USER_BLK_AUTO_NUM_QUEUES UINT16_MAX + typedef struct VHostUserBlk { VirtIODevice parent_obj; CharBackend chardev;