From patchwork Thu Aug 11 09:47:22 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 1056242 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter2.kernel.org (8.14.4/8.14.4) with ESMTP id p7B9mdK4022560 for ; Thu, 11 Aug 2011 09:48:40 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754936Ab1HKJsh (ORCPT ); Thu, 11 Aug 2011 05:48:37 -0400 Received: from mail-fx0-f46.google.com ([209.85.161.46]:64154 "EHLO mail-fx0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754818Ab1HKJsg (ORCPT ); Thu, 11 Aug 2011 05:48:36 -0400 Received: by fxh19 with SMTP id 19so1542924fxh.19 for ; Thu, 11 Aug 2011 02:48:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:date:message-id:x-mailer; bh=xdhAgkit5iU0b/Z94NfVS5i6k8W245rp85XLUeE5k44=; b=aSAQngOPLJ+cyYFagJoc8/vhZl53wVhWawIa92p9OSLhChxc9IwSnOqsdklK4bnHU/ dGf+QCWJPrgNI3C7pw04Sw6B/JsmyeT5S9Lck0mACUjs3j3Is+MZNoe0NM2/XsOKEX39 hDNYkjP7BQSDRr5X83+AB94Kd87Fd77a4d7Nk= Received: by 10.223.57.140 with SMTP id c12mr25330fah.126.1313056115690; Thu, 11 Aug 2011 02:48:35 -0700 (PDT) Received: from localhost.localdomain ([77.127.203.32]) by mx.google.com with ESMTPS id p3sm1426250faa.33.2011.08.11.02.48.33 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 11 Aug 2011 02:48:35 -0700 (PDT) From: Sasha Levin To: penberg@kernel.org Cc: kvm@vger.kernel.org, mingo@elte.hu, asias.hejun@gmail.com, gorcunov@gmail.com, Sasha Levin Subject: [PATCH] kvm tools: Add MSI-X support to virtio-net Date: Thu, 11 Aug 2011 12:47:22 +0300 Message-Id: <1313056043-17405-1-git-send-email-levinsasha928@gmail.com> X-Mailer: git-send-email 1.7.6 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter2.kernel.org [140.211.167.43]); Thu, 11 Aug 2011 09:48:40 +0000 (UTC) The device uses the virtio preferred method of working with MSI-X by creating one vector for configuration and one vector for each vq in the device. Signed-off-by: Sasha Levin --- tools/kvm/virtio/net.c | 56 ++++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 52 insertions(+), 4 deletions(-) kvm__irq_line(kvm, pci_header.irq_line, VIRTIO_IRQ_LOW); ndev.isr = VIRTIO_IRQ_LOW; break; + case VIRTIO_MSI_CONFIG_VECTOR: + ioport__write16(data, ndev.config_vector); + break; + case VIRTIO_MSI_QUEUE_VECTOR: + ioport__write16(data, ndev.vq_vector[ndev.queue_selector]); + break; default: ret = virtio_net_pci_io_device_specific_in(data, offset, size, count); }; @@ -285,10 +297,22 @@ static bool virtio_net_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po ndev.status = ioport__read8(data); break; case VIRTIO_MSI_CONFIG_VECTOR: - ndev.config_vector = VIRTIO_MSI_NO_VECTOR; + ndev.config_vector = ioport__read16(data); break; - case VIRTIO_MSI_QUEUE_VECTOR: + case VIRTIO_MSI_QUEUE_VECTOR: { + u32 gsi; + u32 vec; + + vec = ndev.vq_vector[ndev.queue_selector] = ioport__read16(data); + + gsi = irq__add_msix_route(kvm, + pci_header.msix.table[vec].low, + pci_header.msix.table[vec].high, + pci_header.msix.table[vec].data); + + ndev.gsis[ndev.queue_selector] = gsi; break; + } default: ret = false; }; @@ -308,6 +332,15 @@ static struct ioport_operations virtio_net_io_ops = { .io_out = virtio_net_pci_io_out, }; +static void callback_mmio(u64 addr, u8 *data, u32 len, u8 is_write, void *ptr) +{ + void *table = pci_header.msix.table; + if (is_write) + memcpy(table + addr - ndev.msix_io_block, data, len); + else + memcpy(data, table + addr - ndev.msix_io_block, len); +} + static bool virtio_net__tap_init(const struct virtio_net_parameters *params) { int sock = socket(AF_INET, SOCK_STREAM, 0); @@ -467,6 +500,21 @@ void virtio_net__init(const struct virtio_net_parameters *params) ndev.ops = &uip_ops; } + ndev.msix_io_block = pci_get_io_space_block(); + kvm__register_mmio(params->kvm, ndev.msix_io_block, 0x100, callback_mmio, NULL); + pci_header.bar[1] = ndev.msix_io_block | + PCI_BASE_ADDRESS_SPACE_MEMORY | + PCI_BASE_ADDRESS_MEM_TYPE_64; + /* bar[2] is the continuation of bar[1] for 64bit addressing */ + pci_header.bar[2] = 0; + pci_header.status = PCI_STATUS_CAP_LIST; + pci_header.capabilities = (void *)&pci_header.msix - (void *)&pci_header; + + pci_header.msix.cap = PCI_CAP_ID_MSIX; + pci_header.msix.next = 0; + pci_header.msix.table_size = (VIRTIO_NET_NUM_QUEUES + 1) | PCI_MSIX_FLAGS_ENABLE; + pci_header.msix.table_offset = 1; /* Use BAR 1 */ + virtio_net__io_thread_init(params->kvm); for (i = 0; i < VIRTIO_NET_NUM_QUEUES; i++) { diff --git a/tools/kvm/virtio/net.c b/tools/kvm/virtio/net.c index e865b7f..e9fe45d 100644 --- a/tools/kvm/virtio/net.c +++ b/tools/kvm/virtio/net.c @@ -60,6 +60,9 @@ struct net_dev { u8 isr; u16 queue_selector; u16 base_addr; + u32 vq_vector[VIRTIO_NET_NUM_QUEUES]; + u32 gsis[VIRTIO_NET_NUM_QUEUES]; + u32 msix_io_block; pthread_t io_rx_thread; pthread_mutex_t io_rx_lock; @@ -125,7 +128,8 @@ static void *virtio_net_rx_thread(void *p) virt_queue__set_used_elem(vq, head, len); /* We should interrupt guest right now, otherwise latency is huge. */ - virt_queue__trigger_irq(vq, pci_header.irq_line, &ndev.isr, kvm); + kvm__irq_line(kvm, ndev.gsis[VIRTIO_NET_RX_QUEUE], VIRTIO_IRQ_HIGH); + kvm__irq_line(kvm, ndev.gsis[VIRTIO_NET_RX_QUEUE], VIRTIO_IRQ_LOW); } } @@ -162,7 +166,9 @@ static void *virtio_net_tx_thread(void *p) virt_queue__set_used_elem(vq, head, len); } - virt_queue__trigger_irq(vq, pci_header.irq_line, &ndev.isr, kvm); + kvm__irq_line(kvm, ndev.gsis[VIRTIO_NET_TX_QUEUE], VIRTIO_IRQ_HIGH); + kvm__irq_line(kvm, ndev.gsis[VIRTIO_NET_TX_QUEUE], VIRTIO_IRQ_LOW); } @@ -219,6 +225,12 @@ static bool virtio_net_pci_io_in(struct ioport *ioport, struct kvm *kvm, u16 por