From patchwork Mon May 23 13:22:07 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gerd Hoffmann X-Patchwork-Id: 9131827 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 600E3607D3 for ; Mon, 23 May 2016 13:23:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58D5B281F0 for ; Mon, 23 May 2016 13:23:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4974728239; Mon, 23 May 2016 13:23:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.9 required=2.0 tests=BAYES_00,FSL_HELO_HOME, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 77A92281F0 for ; Mon, 23 May 2016 13:23:24 +0000 (UTC) Received: from localhost ([::1]:48104 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4ppL-0000tk-At for patchwork-qemu-devel@patchwork.kernel.org; Mon, 23 May 2016 09:23:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39487) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4poK-0000Ia-RH for qemu-devel@nongnu.org; Mon, 23 May 2016 09:22:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b4poD-0003S6-2u for qemu-devel@nongnu.org; Mon, 23 May 2016 09:22:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51286) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b4poC-0003Rz-QJ for qemu-devel@nongnu.org; Mon, 23 May 2016 09:22:13 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C146C04B302 for ; Mon, 23 May 2016 13:22:12 +0000 (UTC) Received: from nilsson.home.kraxel.org (ovpn-116-17.ams2.redhat.com [10.36.116.17]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u4NDMAeU023770; Mon, 23 May 2016 09:22:11 -0400 Received: by nilsson.home.kraxel.org (Postfix, from userid 500) id 273E482905; Mon, 23 May 2016 15:22:10 +0200 (CEST) From: Gerd Hoffmann To: qemu-devel@nongnu.org Date: Mon, 23 May 2016 15:22:07 +0200 Message-Id: <1464009727-7753-1-git-send-email-kraxel@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 23 May 2016 13:22:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH] virtio-gpu: add live migration support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Gerd Hoffmann , "Michael S. Tsirkin" Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Store some additional state for cursor and resource backing storage, so we can write out and reload things. Implement vmsave+vmload for 2d mode. Continue blocking live migration in 3d/virgl mode. Signed-off-by: Gerd Hoffmann --- hw/display/virtio-gpu-3d.c | 2 +- hw/display/virtio-gpu.c | 188 +++++++++++++++++++++++++++++++++++++++-- hw/display/virtio-vga.c | 12 +++ include/hw/virtio/virtio-gpu.h | 4 +- 4 files changed, 198 insertions(+), 8 deletions(-) diff --git a/hw/display/virtio-gpu-3d.c b/hw/display/virtio-gpu-3d.c index fa19294..4627046 100644 --- a/hw/display/virtio-gpu-3d.c +++ b/hw/display/virtio-gpu-3d.c @@ -283,7 +283,7 @@ static void virgl_resource_attach_backing(VirtIOGPU *g, VIRTIO_GPU_FILL_CMD(att_rb); trace_virtio_gpu_cmd_res_back_attach(att_rb.resource_id); - ret = virtio_gpu_create_mapping_iov(&att_rb, cmd, &res_iovs); + ret = virtio_gpu_create_mapping_iov(&att_rb, cmd, NULL, &res_iovs); if (ret != 0) { cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC; return; diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c index 91345bd..537a405 100644 --- a/hw/display/virtio-gpu.c +++ b/hw/display/virtio-gpu.c @@ -21,6 +21,8 @@ #include "hw/virtio/virtio-bus.h" #include "qemu/log.h" +#define VIRTIO_GPU_VM_VERSION 1 + static struct virtio_gpu_simple_resource* virtio_gpu_find_resource(VirtIOGPU *g, uint32_t resource_id); @@ -93,7 +95,7 @@ static void update_cursor_data_virgl(VirtIOGPU *g, static void update_cursor(VirtIOGPU *g, struct virtio_gpu_update_cursor *cursor) { struct virtio_gpu_scanout *s; - bool move = cursor->hdr.type != VIRTIO_GPU_CMD_MOVE_CURSOR; + bool move = cursor->hdr.type == VIRTIO_GPU_CMD_MOVE_CURSOR; if (cursor->pos.scanout_id >= g->conf.max_outputs) { return; @@ -106,7 +108,7 @@ static void update_cursor(VirtIOGPU *g, struct virtio_gpu_update_cursor *cursor) move ? "move" : "update", cursor->resource_id); - if (move) { + if (!move) { if (!s->current_cursor) { s->current_cursor = cursor_alloc(64, 64); } @@ -119,6 +121,11 @@ static void update_cursor(VirtIOGPU *g, struct virtio_gpu_update_cursor *cursor) g, s, cursor->resource_id); } dpy_cursor_define(s->con, s->current_cursor); + + s->cursor = *cursor; + } else { + s->cursor.pos.x = cursor->pos.x; + s->cursor.pos.y = cursor->pos.y; } dpy_mouse_set(s->con, cursor->pos.x, cursor->pos.y, cursor->resource_id ? 1 : 0); @@ -591,7 +598,7 @@ static void virtio_gpu_set_scanout(VirtIOGPU *g, int virtio_gpu_create_mapping_iov(struct virtio_gpu_resource_attach_backing *ab, struct virtio_gpu_ctrl_command *cmd, - struct iovec **iov) + uint64_t **addr, struct iovec **iov) { struct virtio_gpu_mem_entry *ents; size_t esize, s; @@ -617,10 +624,16 @@ int virtio_gpu_create_mapping_iov(struct virtio_gpu_resource_attach_backing *ab, } *iov = g_malloc0(sizeof(struct iovec) * ab->nr_entries); + if (addr) { + *addr = g_malloc0(sizeof(uint64_t) * ab->nr_entries); + } for (i = 0; i < ab->nr_entries; i++) { hwaddr len = ents[i].length; (*iov)[i].iov_len = ents[i].length; (*iov)[i].iov_base = cpu_physical_memory_map(ents[i].addr, &len, 1); + if (addr) { + (*addr)[i] = ents[i].addr; + } if (!(*iov)[i].iov_base || len != ents[i].length) { qemu_log_mask(LOG_GUEST_ERROR, "%s: failed to map MMIO memory for" " resource %d element %d\n", @@ -628,6 +641,10 @@ int virtio_gpu_create_mapping_iov(struct virtio_gpu_resource_attach_backing *ab, virtio_gpu_cleanup_mapping_iov(*iov, i); g_free(ents); *iov = NULL; + if (addr) { + g_free(*addr); + *addr = NULL; + } return -1; } } @@ -651,6 +668,8 @@ static void virtio_gpu_cleanup_mapping(struct virtio_gpu_simple_resource *res) virtio_gpu_cleanup_mapping_iov(res->iov, res->iov_cnt); res->iov = NULL; res->iov_cnt = 0; + g_free(res->addrs); + res->addrs = NULL; } static void @@ -672,7 +691,7 @@ virtio_gpu_resource_attach_backing(VirtIOGPU *g, return; } - ret = virtio_gpu_create_mapping_iov(&ab, cmd, &res->iov); + ret = virtio_gpu_create_mapping_iov(&ab, cmd, &res->addrs, &res->iov); if (ret != 0) { cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC; return; @@ -918,11 +937,163 @@ const GraphicHwOps virtio_gpu_ops = { .gl_block = virtio_gpu_gl_block, }; +static const VMStateDescription vmstate_virtio_gpu_scanout = { + .name = "virtio-gpu-one-scanout", + .version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT32(resource_id, struct virtio_gpu_scanout), + VMSTATE_UINT32(width, struct virtio_gpu_scanout), + VMSTATE_UINT32(height, struct virtio_gpu_scanout), + VMSTATE_INT32(x, struct virtio_gpu_scanout), + VMSTATE_INT32(y, struct virtio_gpu_scanout), + VMSTATE_UINT32(cursor.resource_id, struct virtio_gpu_scanout), + VMSTATE_UINT32(cursor.hot_x, struct virtio_gpu_scanout), + VMSTATE_UINT32(cursor.hot_y, struct virtio_gpu_scanout), + VMSTATE_UINT32(cursor.pos.x, struct virtio_gpu_scanout), + VMSTATE_UINT32(cursor.pos.y, struct virtio_gpu_scanout), + VMSTATE_END_OF_LIST() + }, +}; + +static const VMStateDescription vmstate_virtio_gpu_scanouts = { + .name = "virtio-gpu-scanouts", + .version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_INT32(enable, struct VirtIOGPU), + VMSTATE_UINT32_EQUAL(conf.max_outputs, struct VirtIOGPU), + VMSTATE_STRUCT_VARRAY_UINT32(scanout, struct VirtIOGPU, + conf.max_outputs, 1, + vmstate_virtio_gpu_scanout, + struct virtio_gpu_scanout), + VMSTATE_END_OF_LIST() + }, +}; + static const VMStateDescription vmstate_virtio_gpu_unmigratable = { - .name = "virtio-gpu", + .name = "virtio-gpu-with-virgl", .unmigratable = 1, }; +static void virtio_gpu_save(QEMUFile *f, void *opaque) +{ + VirtIOGPU *g = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(g); + struct virtio_gpu_simple_resource *res; + int i; + + virtio_save(vdev, f); + + /* in 2d mode we should never find unprocessed commands here */ + assert(QTAILQ_EMPTY(&g->cmdq)); + + QTAILQ_FOREACH(res, &g->reslist, next) { + qemu_put_be32(f, res->resource_id); + qemu_put_be32(f, res->width); + qemu_put_be32(f, res->height); + qemu_put_be32(f, res->format); + qemu_put_be32(f, res->iov_cnt); + for (i = 0; i < res->iov_cnt; i++) { + qemu_put_be64(f, res->addrs[i]); + qemu_put_be32(f, res->iov[i].iov_len); + } + qemu_put_buffer(f, (void *)pixman_image_get_data(res->image), + pixman_image_get_stride(res->image) * res->height); + } + qemu_put_be32(f, 0); /* end of list */ + + vmstate_save_state(f, &vmstate_virtio_gpu_scanouts, g, NULL); +} + +static int virtio_gpu_load(QEMUFile *f, void *opaque, int version_id) +{ + VirtIOGPU *g = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(g); + struct virtio_gpu_simple_resource *res; + struct virtio_gpu_scanout *scanout; + uint32_t resource_id, pformat; + int i, ret; + + if (version_id != VIRTIO_GPU_VM_VERSION) { + return -EINVAL; + } + + ret = virtio_load(vdev, f, version_id); + if (ret) { + return ret; + } + + resource_id = qemu_get_be32(f); + while (resource_id != 0) { + res = g_new0(struct virtio_gpu_simple_resource, 1); + res->resource_id = resource_id; + res->width = qemu_get_be32(f); + res->height = qemu_get_be32(f); + res->format = qemu_get_be32(f); + res->iov_cnt = qemu_get_be32(f); + + /* allocate */ + pformat = get_pixman_format(res->format); + if (!pformat) { + return -EINVAL; + } + res->image = pixman_image_create_bits(pformat, + res->width, res->height, + NULL, 0); + if (!res->image) { + return -EINVAL; + } + + res->addrs = g_new(uint64_t, res->iov_cnt); + res->iov = g_new(struct iovec, res->iov_cnt); + + /* read data */ + for (i = 0; i < res->iov_cnt; i++) { + res->addrs[i] = qemu_get_be64(f); + res->iov[i].iov_len = qemu_get_be32(f); + } + qemu_get_buffer(f, (void *)pixman_image_get_data(res->image), + pixman_image_get_stride(res->image) * res->height); + + /* restore mapping */ + for (i = 0; i < res->iov_cnt; i++) { + hwaddr len = res->iov[i].iov_len; + res->iov[i].iov_base = + cpu_physical_memory_map(res->addrs[i], &len, 1); + if (!res->iov[i].iov_base || len != res->iov[i].iov_len) { + return -EINVAL; + } + } + + QTAILQ_INSERT_HEAD(&g->reslist, res, next); + + resource_id = qemu_get_be32(f); + } + + /* load & apply scanout state */ + vmstate_load_state(f, &vmstate_virtio_gpu_scanouts, g, 1); + for (i = 0; i < g->conf.max_outputs; i++) { + scanout = &g->scanout[i]; + if (!scanout->resource_id) { + continue; + } + res = virtio_gpu_find_resource(g, scanout->resource_id); + if (!res) { + return -EINVAL; + } + scanout->ds = qemu_create_displaysurface_pixman(res->image); + if (!scanout->ds) { + return -EINVAL; + } + + dpy_gfx_replace_surface(scanout->con, scanout->ds); + dpy_gfx_update(scanout->con, 0, 0, scanout->width, scanout->height); + update_cursor(g, &scanout->cursor); + res->scanout_bitmask |= (1 << i); + } + + return 0; +} + static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp) { VirtIODevice *vdev = VIRTIO_DEVICE(qdev); @@ -975,7 +1146,12 @@ static void virtio_gpu_device_realize(DeviceState *qdev, Error **errp) } } - vmstate_register(qdev, -1, &vmstate_virtio_gpu_unmigratable, g); + if (virtio_gpu_virgl_enabled(g->conf)) { + vmstate_register(qdev, -1, &vmstate_virtio_gpu_unmigratable, g); + } else { + register_savevm(qdev, "virtio-gpu", -1, VIRTIO_GPU_VM_VERSION, + virtio_gpu_save, virtio_gpu_load, g); + } } static void virtio_gpu_instance_init(Object *obj) diff --git a/hw/display/virtio-vga.c b/hw/display/virtio-vga.c index e58b165..4933b08 100644 --- a/hw/display/virtio-vga.c +++ b/hw/display/virtio-vga.c @@ -83,6 +83,17 @@ static const GraphicHwOps virtio_vga_ops = { .gl_block = virtio_vga_gl_block, }; +static const VMStateDescription vmstate_virtio_vga = { + .name = "virtio-vga", + .version_id = 2, + .minimum_version_id = 2, + .fields = (VMStateField[]) { + /* no pci stuff here, saving the virtio device will handle that */ + VMSTATE_STRUCT(vga, VirtIOVGA, 0, vmstate_vga_common, VGACommonState), + VMSTATE_END_OF_LIST() + } +}; + /* VGA device wrapper around PCI device around virtio GPU */ static void virtio_vga_realize(VirtIOPCIProxy *vpci_dev, Error **errp) { @@ -162,6 +173,7 @@ static void virtio_vga_class_init(ObjectClass *klass, void *data) set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories); dc->props = virtio_vga_properties; dc->reset = virtio_vga_reset; + dc->vmsd = &vmstate_virtio_vga; dc->hotpluggable = false; k->realize = virtio_vga_realize; diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h index 13b0ab0..6d7422c 100644 --- a/include/hw/virtio/virtio-gpu.h +++ b/include/hw/virtio/virtio-gpu.h @@ -34,6 +34,7 @@ struct virtio_gpu_simple_resource { uint32_t width; uint32_t height; uint32_t format; + uint64_t *addrs; struct iovec *iov; unsigned int iov_cnt; uint32_t scanout_bitmask; @@ -48,6 +49,7 @@ struct virtio_gpu_scanout { int x, y; int invalidate; uint32_t resource_id; + struct virtio_gpu_update_cursor cursor; QEMUCursor *current_cursor; }; @@ -152,7 +154,7 @@ void virtio_gpu_get_display_info(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd); int virtio_gpu_create_mapping_iov(struct virtio_gpu_resource_attach_backing *ab, struct virtio_gpu_ctrl_command *cmd, - struct iovec **iov); + uint64_t **addr, struct iovec **iov); void virtio_gpu_cleanup_mapping_iov(struct iovec *iov, uint32_t count); void virtio_gpu_process_cmdq(VirtIOGPU *g);