From patchwork Wed Jun 29 18:02:10 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sasha Levin X-Patchwork-Id: 929862 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter2.kernel.org (8.14.4/8.14.4) with ESMTP id p5TI4W3b005339 for ; Wed, 29 Jun 2011 18:04:33 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754860Ab1F2SE3 (ORCPT ); Wed, 29 Jun 2011 14:04:29 -0400 Received: from mail-qy0-f181.google.com ([209.85.216.181]:52561 "EHLO mail-qy0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754092Ab1F2SE2 (ORCPT ); Wed, 29 Jun 2011 14:04:28 -0400 Received: by qyk9 with SMTP id 9so856813qyk.19 for ; Wed, 29 Jun 2011 11:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:date:message-id:x-mailer; bh=LMqrEey8BbAEwIk4mr/rQ1yBDDOipwA5qqsMpHFzUCs=; b=H9g5CMUEAeV8FxeEJcnWVCjGxMsdEtLXQ23Uy6dy7M3yQym1Q0USYJp9gyd9xzqFlR 7p9gpqGHQ53gYLplYA8nILFPTaKDf3EC8b1U1yzszkLf1y3XybNQE6OlabB/st+SyFM7 4owHYIBw6um2Bu5WZVZIK9dUo4x3pDHNVp76E= Received: by 10.224.38.197 with SMTP id c5mr819555qae.173.1309370667734; Wed, 29 Jun 2011 11:04:27 -0700 (PDT) Received: from localhost.localdomain (c-71-232-157-243.hsd1.ma.comcast.net [71.232.157.243]) by mx.google.com with ESMTPS id e18sm1067677qcs.29.2011.06.29.11.04.25 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 29 Jun 2011 11:04:26 -0700 (PDT) From: Sasha Levin To: penberg@kernel.org Cc: kvm@vger.kernel.org, mingo@elte.hu, asias.hejun@gmail.com, gorcunov@gmail.com, prasadjoshi124@gmail.com, Sasha Levin Subject: [PATCH 1/9] kvm tools: Don't dynamically allocate threadpool jobs Date: Wed, 29 Jun 2011 14:02:10 -0400 Message-Id: <1309370538-7947-1-git-send-email-levinsasha928@gmail.com> X-Mailer: git-send-email 1.7.6 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter2.kernel.org [140.211.167.43]); Wed, 29 Jun 2011 18:04:33 +0000 (UTC) To allow efficient use of shorter-term threadpool jobs, don't allocate them dynamically upon creation. Instead, store them within 'job' structures. This will prevent some overhead creating/destroying jobs which live for a short time. Signed-off-by: Sasha Levin --- tools/kvm/include/kvm/threadpool.h | 29 ++++++++++++++++++++++++++--- tools/kvm/threadpool.c | 30 ++---------------------------- tools/kvm/virtio/9p.c | 12 ++++++------ tools/kvm/virtio/blk.c | 8 ++++---- tools/kvm/virtio/console.c | 10 +++++----- tools/kvm/virtio/rng.c | 16 ++++++++-------- 6 files changed, 51 insertions(+), 54 deletions(-) diff --git a/tools/kvm/include/kvm/threadpool.h b/tools/kvm/include/kvm/threadpool.h index 62826a6..768239f 100644 --- a/tools/kvm/include/kvm/threadpool.h +++ b/tools/kvm/include/kvm/threadpool.h @@ -1,14 +1,37 @@ #ifndef KVM__THREADPOOL_H #define KVM__THREADPOOL_H +#include "kvm/mutex.h" + +#include + struct kvm; typedef void (*kvm_thread_callback_fn_t)(struct kvm *kvm, void *data); -int thread_pool__init(unsigned long thread_count); +struct thread_pool__job { + kvm_thread_callback_fn_t callback; + struct kvm *kvm; + void *data; + + int signalcount; + pthread_mutex_t mutex; -void *thread_pool__add_job(struct kvm *kvm, kvm_thread_callback_fn_t callback, void *data); + struct list_head queue; +}; + +static inline void thread_pool__init_job(struct thread_pool__job *job, struct kvm *kvm, kvm_thread_callback_fn_t callback, void *data) +{ + *job = (struct thread_pool__job) { + .kvm = kvm, + .callback = callback, + .data = data, + .mutex = PTHREAD_MUTEX_INITIALIZER, + }; +} + +int thread_pool__init(unsigned long thread_count); -void thread_pool__do_job(void *job); +void thread_pool__do_job(struct thread_pool__job *job); #endif diff --git a/tools/kvm/threadpool.c b/tools/kvm/threadpool.c index 2db02184..fdc5fa7 100644 --- a/tools/kvm/threadpool.c +++ b/tools/kvm/threadpool.c @@ -6,17 +6,6 @@ #include #include -struct thread_pool__job { - kvm_thread_callback_fn_t callback; - struct kvm *kvm; - void *data; - - int signalcount; - pthread_mutex_t mutex; - - struct list_head queue; -}; - static pthread_mutex_t job_mutex = PTHREAD_MUTEX_INITIALIZER; static pthread_mutex_t thread_mutex = PTHREAD_MUTEX_INITIALIZER; static pthread_cond_t job_cond = PTHREAD_COND_INITIALIZER; @@ -139,26 +128,11 @@ int thread_pool__init(unsigned long thread_count) return i; } -void *thread_pool__add_job(struct kvm *kvm, - kvm_thread_callback_fn_t callback, void *data) -{ - struct thread_pool__job *job = calloc(1, sizeof(*job)); - - *job = (struct thread_pool__job) { - .kvm = kvm, - .data = data, - .callback = callback, - .mutex = PTHREAD_MUTEX_INITIALIZER - }; - - return job; -} - -void thread_pool__do_job(void *job) +void thread_pool__do_job(struct thread_pool__job *job) { struct thread_pool__job *jobinfo = job; - if (jobinfo == NULL) + if (jobinfo == NULL || jobinfo->callback == NULL) return; mutex_lock(&jobinfo->mutex); diff --git a/tools/kvm/virtio/9p.c b/tools/kvm/virtio/9p.c index d2d738d..b1a8c01 100644 --- a/tools/kvm/virtio/9p.c +++ b/tools/kvm/virtio/9p.c @@ -46,9 +46,9 @@ struct p9_fid { }; struct p9_dev_job { - struct virt_queue *vq; - struct p9_dev *p9dev; - void *job_id; + struct virt_queue *vq; + struct p9_dev *p9dev; + struct thread_pool__job job_id; }; struct p9_dev { @@ -696,7 +696,7 @@ static void ioevent_callback(struct kvm *kvm, void *param) { struct p9_dev_job *job = param; - thread_pool__do_job(job->job_id); + thread_pool__do_job(&job->job_id); } static bool virtio_p9_pci_io_out(struct ioport *ioport, struct kvm *kvm, @@ -731,7 +731,7 @@ static bool virtio_p9_pci_io_out(struct ioport *ioport, struct kvm *kvm, .vq = queue, .p9dev = p9dev, }; - job->job_id = thread_pool__add_job(kvm, virtio_p9_do_io, job); + thread_pool__init_job(&job->job_id, kvm, virtio_p9_do_io, job); ioevent = (struct ioevent) { .io_addr = p9dev->base_addr + VIRTIO_PCI_QUEUE_NOTIFY, @@ -754,7 +754,7 @@ static bool virtio_p9_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 queue_index; queue_index = ioport__read16(data); - thread_pool__do_job(p9dev->jobs[queue_index].job_id); + thread_pool__do_job(&p9dev->jobs[queue_index].job_id); break; } case VIRTIO_PCI_STATUS: diff --git a/tools/kvm/virtio/blk.c b/tools/kvm/virtio/blk.c index 8d54f5a..1fdfc1e 100644 --- a/tools/kvm/virtio/blk.c +++ b/tools/kvm/virtio/blk.c @@ -31,7 +31,7 @@ struct blk_dev_job { struct virt_queue *vq; struct blk_dev *bdev; - void *job_id; + struct thread_pool__job job_id; }; struct blk_dev { @@ -206,7 +206,7 @@ static bool virtio_blk_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po .bdev = bdev, }; - job->job_id = thread_pool__add_job(kvm, virtio_blk_do_io, job); + thread_pool__init_job(&job->job_id, kvm, virtio_blk_do_io, job); break; } @@ -217,7 +217,7 @@ static bool virtio_blk_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po u16 queue_index; queue_index = ioport__read16(data); - thread_pool__do_job(bdev->jobs[queue_index].job_id); + thread_pool__do_job(&bdev->jobs[queue_index].job_id); break; } @@ -248,7 +248,7 @@ static void ioevent_callback(struct kvm *kvm, void *param) { struct blk_dev_job *job = param; - thread_pool__do_job(job->job_id); + thread_pool__do_job(&job->job_id); } void virtio_blk__init(struct kvm *kvm, struct disk_image *disk) diff --git a/tools/kvm/virtio/console.c b/tools/kvm/virtio/console.c index 038e53f..e5d59c0 100644 --- a/tools/kvm/virtio/console.c +++ b/tools/kvm/virtio/console.c @@ -51,7 +51,7 @@ struct con_dev { u16 queue_selector; u16 base_addr; - void *jobs[VIRTIO_CONSOLE_NUM_QUEUES]; + struct thread_pool__job jobs[VIRTIO_CONSOLE_NUM_QUEUES]; }; static struct con_dev cdev = { @@ -93,7 +93,7 @@ static void virtio_console__inject_interrupt_callback(struct kvm *kvm, void *par void virtio_console__inject_interrupt(struct kvm *kvm) { - thread_pool__do_job(cdev.jobs[VIRTIO_CONSOLE_RX_QUEUE]); + thread_pool__do_job(&cdev.jobs[VIRTIO_CONSOLE_RX_QUEUE]); } static bool virtio_console_pci_io_device_specific_in(void *data, unsigned long offset, int size, u32 count) @@ -203,9 +203,9 @@ static bool virtio_console_pci_io_out(struct ioport *ioport, struct kvm *kvm, u1 vring_init(&queue->vring, VIRTIO_CONSOLE_QUEUE_SIZE, p, VIRTIO_PCI_VRING_ALIGN); if (cdev.queue_selector == VIRTIO_CONSOLE_TX_QUEUE) - cdev.jobs[cdev.queue_selector] = thread_pool__add_job(kvm, virtio_console_handle_callback, queue); + thread_pool__init_job(&cdev.jobs[cdev.queue_selector], kvm, virtio_console_handle_callback, queue); else if (cdev.queue_selector == VIRTIO_CONSOLE_RX_QUEUE) - cdev.jobs[cdev.queue_selector] = thread_pool__add_job(kvm, virtio_console__inject_interrupt_callback, queue); + thread_pool__init_job(&cdev.jobs[cdev.queue_selector], kvm, virtio_console__inject_interrupt_callback, queue); break; } @@ -214,7 +214,7 @@ static bool virtio_console_pci_io_out(struct ioport *ioport, struct kvm *kvm, u1 break; case VIRTIO_PCI_QUEUE_NOTIFY: { u16 queue_index = ioport__read16(data); - thread_pool__do_job(cdev.jobs[queue_index]); + thread_pool__do_job(&cdev.jobs[queue_index]); break; } case VIRTIO_PCI_STATUS: diff --git a/tools/kvm/virtio/rng.c b/tools/kvm/virtio/rng.c index ede0e2b..1a7f569 100644 --- a/tools/kvm/virtio/rng.c +++ b/tools/kvm/virtio/rng.c @@ -21,13 +21,13 @@ #include #include -#define NUM_VIRT_QUEUES 1 -#define VIRTIO_RNG_QUEUE_SIZE 128 +#define NUM_VIRT_QUEUES 1 +#define VIRTIO_RNG_QUEUE_SIZE 128 struct rng_dev_job { - struct virt_queue *vq; - struct rng_dev *rdev; - void *job_id; + struct virt_queue *vq; + struct rng_dev *rdev; + struct thread_pool__job job_id; }; struct rng_dev { @@ -146,7 +146,7 @@ static bool virtio_rng_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po .rdev = rdev, }; - job->job_id = thread_pool__add_job(kvm, virtio_rng_do_io, job); + thread_pool__init_job(&job->job_id, kvm, virtio_rng_do_io, job); break; } @@ -156,7 +156,7 @@ static bool virtio_rng_pci_io_out(struct ioport *ioport, struct kvm *kvm, u16 po case VIRTIO_PCI_QUEUE_NOTIFY: { u16 queue_index; queue_index = ioport__read16(data); - thread_pool__do_job(rdev->jobs[queue_index].job_id); + thread_pool__do_job(&rdev->jobs[queue_index].job_id); break; } case VIRTIO_PCI_STATUS: @@ -182,7 +182,7 @@ static void ioevent_callback(struct kvm *kvm, void *param) { struct rng_dev_job *job = param; - thread_pool__do_job(job->job_id); + thread_pool__do_job(&job->job_id); } void virtio_rng__init(struct kvm *kvm)