From patchwork Wed Jul 18 15:07:44 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 1211621 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 45D5540071 for ; Wed, 18 Jul 2012 15:09:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754828Ab2GRPIz (ORCPT ); Wed, 18 Jul 2012 11:08:55 -0400 Received: from e06smtp17.uk.ibm.com ([195.75.94.113]:38327 "EHLO e06smtp17.uk.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754794Ab2GRPIo (ORCPT ); Wed, 18 Jul 2012 11:08:44 -0400 Received: from /spool/local by e06smtp17.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 18 Jul 2012 16:08:43 +0100 Received: from d06nrmr1707.portsmouth.uk.ibm.com (9.149.39.225) by e06smtp17.uk.ibm.com (192.168.101.147) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 18 Jul 2012 16:08:40 +0100 Received: from d06av07.portsmouth.uk.ibm.com (d06av07.portsmouth.uk.ibm.com [9.149.37.248]) by d06nrmr1707.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q6IF8dLD2359382 for ; Wed, 18 Jul 2012 16:08:39 +0100 Received: from d06av07.portsmouth.uk.ibm.com (d06av07.portsmouth.uk.ibm.com [127.0.0.1]) by d06av07.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q6IEnE15003645 for ; Wed, 18 Jul 2012 10:49:14 -0400 Received: from localhost (sig-9-145-185-169.de.ibm.com [9.145.185.169]) by d06av07.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q6IEnDxW003633; Wed, 18 Jul 2012 10:49:14 -0400 From: Stefan Hajnoczi To: Cc: , Anthony Liguori , Kevin Wolf , Paolo Bonzini , "Michael S. Tsirkin" , Asias He , Khoa Huynh , Stefan Hajnoczi Subject: [RFC v9 17/27] virtio-blk: Use guest notifier to raise interrupts Date: Wed, 18 Jul 2012 16:07:44 +0100 Message-Id: <1342624074-24650-18-git-send-email-stefanha@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1342624074-24650-1-git-send-email-stefanha@linux.vnet.ibm.com> References: <1342624074-24650-1-git-send-email-stefanha@linux.vnet.ibm.com> x-cbid: 12071815-0542-0000-0000-00000280014F Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The data plane thread isn't allowed to call virtio_irq() directly because that function is not thread-safe. Use the guest notifier just like virtio-net to handle IRQs. When MSI-X is in use and the vector is unmasked, the guest notifier directly sets the IRQ inside the host kernel. If the vector is masked, then QEMU's iothread needs to take note of the IRQ. If MSI-X is not in use, then QEMU's iothread handles the IRQ and this will be slower than synchronously calling notify_irq() from the data plane thread. --- hw/virtio-blk.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/hw/virtio-blk.c b/hw/virtio-blk.c index d75c187..bdff68a 100644 --- a/hw/virtio-blk.c +++ b/hw/virtio-blk.c @@ -73,6 +73,18 @@ static int get_raw_posix_fd_hack(VirtIOBlock *s) return *(int*)s->bs->file->opaque; } +/* Raise an interrupt to signal guest, if necessary */ +static void virtio_blk_notify_guest(VirtIOBlock *s) +{ + /* Always notify when queue is empty (when feature acknowledge) */ + if ((s->vring.vr.avail->flags & VRING_AVAIL_F_NO_INTERRUPT) && + (s->vring.vr.avail->idx != s->vring.last_avail_idx || + !(s->vdev.guest_features & (1 << VIRTIO_F_NOTIFY_ON_EMPTY)))) + return; + + event_notifier_set(virtio_queue_get_guest_notifier(s->vq)); +} + static void complete_request(struct iocb *iocb, ssize_t ret, void *opaque) { VirtIOBlock *s = opaque; @@ -154,7 +166,7 @@ static void process_request(IOQueue *ioq, struct iovec iov[], unsigned int out_n fdatasync(get_raw_posix_fd_hack(s)); inhdr->status = VIRTIO_BLK_S_OK; vring_push(&s->vring, head, sizeof *inhdr); - virtio_irq(s->vq); + virtio_blk_notify_guest(s); } return; @@ -222,8 +234,7 @@ static bool handle_io(EventHandler *handler) VirtIOBlock *s = container_of(handler, VirtIOBlock, io_handler); if (ioq_run_completion(&s->ioqueue, complete_request, s) > 0) { - /* TODO is this thread-safe and can it be done faster? */ - virtio_irq(s->vq); + virtio_blk_notify_guest(s); } /* If there were more requests than iovecs, the vring will not be empty yet @@ -251,11 +262,17 @@ static void data_plane_start(VirtIOBlock *s) vring_setup(&s->vring, &s->vdev, 0); + /* Set up guest notifier (irq) */ + if (s->vdev.binding->set_guest_notifier(s->vdev.binding_opaque, 0, true) != 0) { + fprintf(stderr, "virtio-blk failed to set guest notifier, ensure -enable-kvm is set\n"); + exit(1); + } + event_poll_init(&s->event_poll); /* Set up virtqueue notify */ if (s->vdev.binding->set_host_notifier(s->vdev.binding_opaque, 0, true) != 0) { - fprintf(stderr, "virtio-blk failed to set host notifier, ensure -enable-kvm is set\n"); + fprintf(stderr, "virtio-blk failed to set host notifier\n"); exit(1); } event_poll_add(&s->event_poll, &s->notify_handler, @@ -296,6 +313,9 @@ static void data_plane_stop(VirtIOBlock *s) s->vdev.binding->set_host_notifier(s->vdev.binding_opaque, 0, false); event_poll_cleanup(&s->event_poll); + + /* Clean up guest notifier (irq) */ + s->vdev.binding->set_guest_notifier(s->vdev.binding_opaque, 0, false); } static void virtio_blk_set_status(VirtIODevice *vdev, uint8_t val)