From patchwork Wed May 27 10:29:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 11572555 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AE7D14B7 for ; Wed, 27 May 2020 10:30:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CD50D208B8 for ; Wed, 27 May 2020 10:30:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="K3Mp8yFI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD50D208B8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:36758 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdtKM-0008Fb-Fm for patchwork-qemu-devel@patchwork.kernel.org; Wed, 27 May 2020 06:30:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55814) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdtJc-0006m5-TU for qemu-devel@nongnu.org; Wed, 27 May 2020 06:29:40 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:40821 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jdtJb-0000zw-AQ for qemu-devel@nongnu.org; Wed, 27 May 2020 06:29:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590575377; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=CEYSJmVzTNIYluSOttOVKcndsZi4ILRg3ku5sStIvc0=; b=K3Mp8yFImUj4a0THSCG4835abg1ut9FqenqJiTm9gt28Qmeq5LRXvdeMUlnsmbiTzlTK8t MH0CnhqltVQP5E2xOIDFae+VyyWLL6VT04OEuy1eG8Zd1iu3r4t5JukBwVi8wBUzzeiDHm sIt+TXDvZVSt/boqBtD6lrrOcxi4ozg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-286-EkIdHvXsM4-0usgXR7RhvA-1; Wed, 27 May 2020 06:29:33 -0400 X-MC-Unique: EkIdHvXsM4-0usgXR7RhvA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 68EFA8005AA; Wed, 27 May 2020 10:29:32 +0000 (UTC) Received: from localhost (ovpn-114-250.ams2.redhat.com [10.36.114.250]) by smtp.corp.redhat.com (Postfix) with ESMTP id 574DB5C1B0; Wed, 27 May 2020 10:29:26 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v4 0/5] virtio-pci: enable blk and scsi multi-queue by default Date: Wed, 27 May 2020 11:29:20 +0100 Message-Id: <20200527102925.128013-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/27 05:46:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Pankaj Gupta , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Max Reitz , Stefan Hajnoczi , Paolo Bonzini , Fam Zheng , Raphael Norwitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" v3: * Introduce virtio_pci_optimal_num_queues() helper to enforce VIRTIO_QUEUE_MAX in one place * Use VIRTIO_SCSI_VQ_NUM_FIXED constant in all cases [Cornelia] * Update hw/core/machine.c compat properties for QEMU 5.0 [Michael] v3: * Add new performance results that demonstrate the scalability * Mention that this is PCI-specific [Cornelia] v2: * Let the virtio-DEVICE-pci device select num-queues because the optimal multi-queue configuration may differ between virtio-pci, virtio-mmio, and virtio-ccw [Cornelia] Enabling multi-queue on virtio-pci storage devices improves performance on SMP guests because the completion interrupt is handled on the vCPU that submitted the I/O request. This avoids IPIs inside the guest. Note that performance is unchanged in these cases: 1. Uniprocessor guests. They don't have IPIs. 2. Application threads might be scheduled on the sole vCPU that handles completion interrupts purely by chance. (This is one reason why benchmark results can vary noticably between runs.) 3. Users may bind the application to the vCPU that handles completion interrupts. Set the number of queues to the number of vCPUs by default on virtio-blk and virtio-scsi PCI devices. Older machine types continue to default to 1 queue for live migration compatibility. Random read performance: IOPS q=1 78k q=32 104k +33% Boot time: Duration q=1 51s q=32 1m41s +98% Guest configuration: 32 vCPUs, 101 virtio-blk-pci disks Previously measured results on a 4 vCPU guest were also positive but showed a smaller 1-4% performance improvement. They are no longer valid because significant event loop optimizations have been merged. Stefan Hajnoczi (5): virtio-pci: add virtio_pci_optimal_num_queues() helper virtio-scsi: introduce a constant for fixed virtqueues virtio-scsi: default num_queues to -smp N virtio-blk: default num_queues to -smp N vhost-user-blk: default num_queues to -smp N hw/virtio/virtio-pci.h | 9 +++++++++ include/hw/virtio/vhost-user-blk.h | 2 ++ include/hw/virtio/virtio-blk.h | 2 ++ include/hw/virtio/virtio-scsi.h | 5 +++++ hw/block/vhost-user-blk.c | 6 +++++- hw/block/virtio-blk.c | 6 +++++- hw/core/machine.c | 8 +++++++- hw/scsi/vhost-scsi.c | 3 ++- hw/scsi/vhost-user-scsi.c | 5 +++-- hw/scsi/virtio-scsi.c | 13 +++++++++---- hw/virtio/vhost-scsi-pci.c | 9 +++++++-- hw/virtio/vhost-user-blk-pci.c | 4 ++++ hw/virtio/vhost-user-scsi-pci.c | 9 +++++++-- hw/virtio/virtio-blk-pci.c | 7 ++++++- hw/virtio/virtio-pci.c | 7 +++++++ hw/virtio/virtio-scsi-pci.c | 9 +++++++-- 16 files changed, 87 insertions(+), 17 deletions(-)