From patchwork Thu May 26 10:30:07 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 820342 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id p4QAUlAf008872 for ; Thu, 26 May 2011 10:30:48 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757428Ab1EZKan (ORCPT ); Thu, 26 May 2011 06:30:43 -0400 Received: from mail-wy0-f174.google.com ([74.125.82.174]:45357 "EHLO mail-wy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754488Ab1EZKal (ORCPT ); Thu, 26 May 2011 06:30:41 -0400 Received: by wya21 with SMTP id 21so381453wya.19 for ; Thu, 26 May 2011 03:30:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:cc:subject:date:message-id:x-mailer :in-reply-to:references; bh=LJX2SzxNeZiDobqGvLrdj4ON3g1cDx6uexKDY+42isQ=; b=XAdy01BDdFU1TGMZOsPtvSZoRKrDNRMb5LZWXtRQsASUtQDgSJFAWHIs5vuY7LPt2d V3UKXlbad/qA+6T0i4V4oYanzuMlAfoX1GwKl1NjTXE0+twWyoTcJbSU6lYvOgtswNrx c1ugW4a3X2rCWI7FpWEui31Qn+TLTTgM9yPzs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references; b=Ve49TgjCgl5rOC5vzUy8bA1HqWtVq3Wra/70fVnGykc7vpYVQaYUGGgzUDT3cisMXd v/AWalpb3UmF2cn6dxetdN+1K2kTJUN2Ye0ycuCxRFBm9g/7WWEnFCQWOI5vBdgSWFuR wMz+ESh6OCXCFhMpbQhr47vCV26t+TFV9xlp8= Received: by 10.227.162.193 with SMTP id w1mr688139wbx.66.1306405840315; Thu, 26 May 2011 03:30:40 -0700 (PDT) Received: from localhost.localdomain ([109.66.201.145]) by mx.google.com with ESMTPS id et5sm365858wbb.33.2011.05.26.03.30.38 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 26 May 2011 03:30:40 -0700 (PDT) From: Sasha Levin To: penberg@kernel.org Cc: john@jfloren.net, kvm@vger.kernel.org, mingo@elte.hu, asias.hejun@gmail.com, gorcunov@gmail.com, prasadjoshi124@gmail.com, Sasha Levin Subject: [PATCH v3 4/8] kvm tools: Add support for multiple virtio-rng devices Date: Thu, 26 May 2011 13:30:07 +0300 Message-Id: <1306405811-4482-4-git-send-email-levinsasha928@gmail.com> X-Mailer: git-send-email 1.7.5.rc3 In-Reply-To: <1306405811-4482-1-git-send-email-levinsasha928@gmail.com> References: <1306405811-4482-1-git-send-email-levinsasha928@gmail.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter1.kernel.org [140.211.167.41]); Thu, 26 May 2011 10:30:48 +0000 (UTC) Since multiple hardware rng devices of the same type are currently unsupported by the kernel, this serves more as an example of a basic virtio driver under kvm tools and can be used to debug the PCI layer. Signed-off-by: Sasha Levin --- tools/kvm/include/kvm/ioport.h | 2 - tools/kvm/include/kvm/parse-options.h | 9 +++ tools/kvm/include/kvm/virtio-rng.h | 1 + tools/kvm/kvm-run.c | 8 ++- tools/kvm/virtio/rng.c | 118 ++++++++++++++++++++++---------- 5 files changed, 96 insertions(+), 42 deletions(-) diff --git a/tools/kvm/include/kvm/ioport.h b/tools/kvm/include/kvm/ioport.h index e53c03c..55d53e0 100644 --- a/tools/kvm/include/kvm/ioport.h +++ b/tools/kvm/include/kvm/ioport.h @@ -20,8 +20,6 @@ #define IOPORT_VIRTIO_CONSOLE_SIZE 256 #define IOPORT_VIRTIO_NET 0xe200 /* Virtio network device */ #define IOPORT_VIRTIO_NET_SIZE 256 -#define IOPORT_VIRTIO_RNG 0xf200 /* Virtio network device */ -#define IOPORT_VIRTIO_RNG_SIZE 256 #define IOPORT_EMPTY USHRT_MAX diff --git a/tools/kvm/include/kvm/parse-options.h b/tools/kvm/include/kvm/parse-options.h index 2d5c99e..6bf9a1d 100644 --- a/tools/kvm/include/kvm/parse-options.h +++ b/tools/kvm/include/kvm/parse-options.h @@ -132,6 +132,15 @@ intptr_t defval; .help = (h) \ } +#define OPT_INCR(s, l, v, h) \ +{ \ + .type = OPTION_INCR, \ + .short_name = (s), \ + .long_name = (l), \ + .value = check_vtype(v, int *), \ + .help = (h) \ +} + #define OPT_GROUP(h) \ { \ .type = OPTION_GROUP, \ diff --git a/tools/kvm/include/kvm/virtio-rng.h b/tools/kvm/include/kvm/virtio-rng.h index 7015c1f..c0a413b 100644 --- a/tools/kvm/include/kvm/virtio-rng.h +++ b/tools/kvm/include/kvm/virtio-rng.h @@ -4,5 +4,6 @@ struct kvm; void virtio_rng__init(struct kvm *kvm); +void virtio_rng__delete_all(struct kvm *kvm); #endif /* KVM__RNG_VIRTIO_H */ diff --git a/tools/kvm/kvm-run.c b/tools/kvm/kvm-run.c index adbb25b..76b5782 100644 --- a/tools/kvm/kvm-run.c +++ b/tools/kvm/kvm-run.c @@ -52,6 +52,7 @@ static __thread struct kvm_cpu *current_kvm_cpu; static u64 ram_size; static u8 image_count; +static int virtio_rng; static const char *kernel_cmdline; static const char *kernel_filename; static const char *vmlinux_filename; @@ -66,7 +67,6 @@ static const char *script; static const char *virtio_9p_dir; static bool single_step; static bool readonly_image[MAX_DISK_IMAGES]; -static bool virtio_rng; static bool vnc; extern bool ioport_debug; extern int active_console; @@ -107,7 +107,7 @@ static const struct option options[] = { OPT_CALLBACK('d', "disk", NULL, "image", "Disk image", img_name_parser), OPT_STRING('\0', "console", &console, "serial or virtio", "Console to use"), - OPT_BOOLEAN('\0', "rng", &virtio_rng, + OPT_INCR('\0', "rng", &virtio_rng, "Enable virtio Random Number Generator"), OPT_STRING('\0', "kvm-dev", &kvm_dev, "kvm-dev", "KVM device file"), OPT_STRING('\0', "virtio-9p", &virtio_9p_dir, "root dir", @@ -570,7 +570,8 @@ int kvm_cmd_run(int argc, const char **argv, const char *prefix) virtio_console__init(kvm); if (virtio_rng) - virtio_rng__init(kvm); + while (virtio_rng--) + virtio_rng__init(kvm); if (!network) network = DEFAULT_NETWORK; @@ -631,6 +632,7 @@ int kvm_cmd_run(int argc, const char **argv, const char *prefix) } virtio_blk__delete_all(kvm); + virtio_rng__delete_all(kvm); disk_image__close_all(kvm->disks, image_count); kvm__delete(kvm); diff --git a/tools/kvm/virtio/rng.c b/tools/kvm/virtio/rng.c index a553f6b..1a3bda3 100644 --- a/tools/kvm/virtio/rng.c +++ b/tools/kvm/virtio/rng.c @@ -5,7 +5,6 @@ #include "kvm/disk-image.h" #include "kvm/virtio.h" #include "kvm/ioport.h" -#include "kvm/mutex.h" #include "kvm/util.h" #include "kvm/kvm.h" #include "kvm/pci.h" @@ -15,6 +14,7 @@ #include #include +#include #include #include #include @@ -23,18 +23,17 @@ #define NUM_VIRT_QUEUES 1 #define VIRTIO_RNG_QUEUE_SIZE 128 -static struct pci_device_header virtio_rng_pci_device = { - .vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET, - .device_id = PCI_DEVICE_ID_VIRTIO_RNG, - .header_type = PCI_HEADER_TYPE_NORMAL, - .revision_id = 0, - .class = 0x010000, - .subsys_vendor_id = PCI_SUBSYSTEM_VENDOR_ID_REDHAT_QUMRANET, - .subsys_id = VIRTIO_ID_RNG, - .bar[0] = IOPORT_VIRTIO_RNG | PCI_BASE_ADDRESS_SPACE_IO, +struct rng_dev_job { + struct virt_queue *vq; + struct rng_dev *rdev; + void *job_id; }; struct rng_dev { + struct pci_device_header pci_hdr; + struct list_head list; + + u16 base_addr; u8 status; u8 isr; u16 config_vector; @@ -43,17 +42,19 @@ struct rng_dev { /* virtio queue */ u16 queue_selector; struct virt_queue vqs[NUM_VIRT_QUEUES]; - void *jobs[NUM_VIRT_QUEUES]; + struct rng_dev_job jobs[NUM_VIRT_QUEUES]; }; -static struct rng_dev rdev; +static LIST_HEAD(rdevs); static bool virtio_rng_pci_io_in(struct ioport *ioport, struct kvm *kvm, u16 port, void *data, int size, u32 count) { unsigned long offset; bool ret = true; + struct rng_dev *rdev; - offset = port - IOPORT_VIRTIO_RNG; + rdev = ioport->priv; + offset = port - rdev->base_addr; switch (offset) { case VIRTIO_PCI_HOST_FEATURES: @@ -63,21 +64,21 @@ static bool virtio_rng_pci_io_in(struct ioport *ioport, struct kvm *kvm, u16 por ret = false; break; case VIRTIO_PCI_QUEUE_PFN: - ioport__write32(data, rdev.vqs[rdev.queue_selector].pfn); + ioport__write32(data, rdev->vqs[rdev->queue_selector].pfn); break; case VIRTIO_PCI_QUEUE_NUM: ioport__write16(data, VIRTIO_RNG_QUEUE_SIZE); break; case VIRTIO_PCI_STATUS: - ioport__write8(data, rdev.status); + ioport__write8(data, rdev->status); break; case VIRTIO_PCI_ISR: - ioport__write8(data, rdev.isr); - kvm__irq_line(kvm, virtio_rng_pci_device.irq_line, VIRTIO_IRQ_LOW); - rdev.isr = VIRTIO_IRQ_LOW; + ioport__write8(data, rdev->isr); + kvm__irq_line(kvm, rdev->pci_hdr.irq_line, VIRTIO_IRQ_LOW); + rdev->isr = VIRTIO_IRQ_LOW; break; case VIRTIO_MSI_CONFIG_VECTOR: - ioport__write16(data, rdev.config_vector); + ioport__write16(data, rdev->config_vector); break; default: ret = false; @@ -87,14 +88,14 @@ static bool virtio_rng_pci_io_in(struct ioport *ioport, struct kvm *kvm, u16 por return ret; } -static bool virtio_rng_do_io_request(struct kvm *kvm, struct virt_queue *queue) +static bool virtio_rng_do_io_request(struct kvm *kvm, struct rng_dev *rdev, struct virt_queue *queue) { struct iovec iov[VIRTIO_RNG_QUEUE_SIZE]; unsigned int len = 0; u16 out, in, head; head = virt_queue__get_iov(queue, iov, &out, &in, kvm); - len = readv(rdev.fd, iov, in); + len = readv(rdev->fd, iov, in); virt_queue__set_used_elem(queue, head, len); @@ -103,11 +104,13 @@ static bool virtio_rng_do_io_request(struct kvm *kvm, struct virt_queue *queue) static void virtio_rng_do_io(struct kvm *kvm, void *param) { - struct virt_queue *vq = param; + struct rng_dev_job *job = param; + struct virt_queue *vq = job->vq; + struct rng_dev *rdev = job->rdev; while (virt_queue__available(vq)) { - virtio_rng_do_io_request(kvm, vq); - virt_queue__trigger_irq(vq, virtio_rng_pci_device.irq_line, &rdev.isr, kvm); + virtio_rng_do_io_request(kvm, rdev, vq); + virt_queue__trigger_irq(vq, rdev->pci_hdr.irq_line, &rdev->isr, kvm); } } @@ -115,8 +118,10 @@ static bool virtio_rng_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po { unsigned long offset; bool ret = true; + struct rng_dev *rdev; - offset = port - IOPORT_VIRTIO_RNG; + rdev = ioport->priv; + offset = port - rdev->base_addr; switch (offset) { case VIRTIO_MSI_QUEUE_VECTOR: @@ -124,32 +129,40 @@ static bool virtio_rng_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po break; case VIRTIO_PCI_QUEUE_PFN: { struct virt_queue *queue; + struct rng_dev_job *job; void *p; - queue = &rdev.vqs[rdev.queue_selector]; + queue = &rdev->vqs[rdev->queue_selector]; queue->pfn = ioport__read32(data); p = guest_pfn_to_host(kvm, queue->pfn); + job = &rdev->jobs[rdev->queue_selector]; + vring_init(&queue->vring, VIRTIO_RNG_QUEUE_SIZE, p, VIRTIO_PCI_VRING_ALIGN); - rdev.jobs[rdev.queue_selector] = thread_pool__add_job(kvm, virtio_rng_do_io, queue); + *job = (struct rng_dev_job) { + .vq = queue, + .rdev = rdev, + }; + + job->job_id = thread_pool__add_job(kvm, virtio_rng_do_io, job); break; } case VIRTIO_PCI_QUEUE_SEL: - rdev.queue_selector = ioport__read16(data); + rdev->queue_selector = ioport__read16(data); break; case VIRTIO_PCI_QUEUE_NOTIFY: { u16 queue_index; queue_index = ioport__read16(data); - thread_pool__do_job(rdev.jobs[queue_index]); + thread_pool__do_job(rdev->jobs[queue_index].job_id); break; } case VIRTIO_PCI_STATUS: - rdev.status = ioport__read8(data); + rdev->status = ioport__read8(data); break; case VIRTIO_MSI_CONFIG_VECTOR: - rdev.config_vector = VIRTIO_MSI_NO_VECTOR; + rdev->config_vector = VIRTIO_MSI_NO_VECTOR; break; default: ret = false; @@ -167,17 +180,48 @@ static struct ioport_operations virtio_rng_io_ops = { void virtio_rng__init(struct kvm *kvm) { u8 pin, line, dev; + u16 rdev_base_addr; + struct rng_dev *rdev; + + rdev = malloc(sizeof(*rdev)); + if (rdev == NULL) + return; + + rdev_base_addr = ioport__register(IOPORT_EMPTY, &virtio_rng_io_ops, IOPORT_SIZE, rdev); + + rdev->pci_hdr = (struct pci_device_header) { + .vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET, + .device_id = PCI_DEVICE_ID_VIRTIO_RNG, + .header_type = PCI_HEADER_TYPE_NORMAL, + .revision_id = 0, + .class = 0x010000, + .subsys_vendor_id = PCI_SUBSYSTEM_VENDOR_ID_REDHAT_QUMRANET, + .subsys_id = VIRTIO_ID_RNG, + .bar[0] = rdev_base_addr | PCI_BASE_ADDRESS_SPACE_IO, + }; - rdev.fd = open("/dev/urandom", O_RDONLY); - if (rdev.fd < 0) + rdev->base_addr = rdev_base_addr; + rdev->fd = open("/dev/urandom", O_RDONLY); + if (rdev->fd < 0) die("Failed initializing RNG"); if (irq__register_device(VIRTIO_ID_RNG, &dev, &pin, &line) < 0) return; - virtio_rng_pci_device.irq_pin = pin; - virtio_rng_pci_device.irq_line = line; - pci__register(&virtio_rng_pci_device, dev); + rdev->pci_hdr.irq_pin = pin; + rdev->pci_hdr.irq_line = line; + pci__register(&rdev->pci_hdr, dev); + + list_add_tail(&rdev->list, &rdevs); +} + +void virtio_rng__delete_all(struct kvm *kvm) +{ + while (!list_empty(&rdevs)) { + struct rng_dev *rdev; - ioport__register(IOPORT_VIRTIO_RNG, &virtio_rng_io_ops, IOPORT_VIRTIO_RNG_SIZE, NULL); + rdev = list_first_entry(&rdevs, struct rng_dev, list); + list_del(&rdev->list); + free(rdev); + } }