From patchwork Wed Aug 24 13:42:33 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 9297863 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3411C607F0 for ; Wed, 24 Aug 2016 13:46:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 214B328FAF for ; Wed, 24 Aug 2016 13:46:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 15E2228FDD; Wed, 24 Aug 2016 13:46:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.9 required=2.0 tests=BAYES_00,HK_NAME_DR, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 31B9A28FAF for ; Wed, 24 Aug 2016 13:46:48 +0000 (UTC) Received: from localhost ([::1]:51517 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bcYVr-0007KP-RF for patchwork-qemu-devel@patchwork.kernel.org; Wed, 24 Aug 2016 09:46:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54567) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bcYS9-00045P-Cn for qemu-devel@nongnu.org; Wed, 24 Aug 2016 09:42:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bcYS6-000387-K4 for qemu-devel@nongnu.org; Wed, 24 Aug 2016 09:42:48 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45560) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bcYS6-00037v-B4 for qemu-devel@nongnu.org; Wed, 24 Aug 2016 09:42:46 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C96118124A; Wed, 24 Aug 2016 13:42:45 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-116-84.ams2.redhat.com [10.36.116.84]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u7ODgX25027186; Wed, 24 Aug 2016 09:42:44 -0400 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, mst@redhat.com, amit.shah@redhat.com, quintela@redhat.com Date: Wed, 24 Aug 2016 14:42:33 +0100 Message-Id: <1472046153-22123-7-git-send-email-dgilbert@redhat.com> In-Reply-To: <1472046153-22123-1-git-send-email-dgilbert@redhat.com> References: <1472046153-22123-1-git-send-email-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Wed, 24 Aug 2016 13:42:45 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC 6/6] virtio/migration: Migrate virtio-net to VMState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cornelia.huck@de.ibm.com, duanj@linux.vnet.ibm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: "Dr. David Alan Gilbert" Only lightly smoke-tested so far Signed-off-by: Dr. David Alan Gilbert --- hw/net/virtio-net.c | 256 ++++++++++++++++++++++++----------------- include/hw/virtio/virtio-net.h | 10 +- 2 files changed, 157 insertions(+), 109 deletions(-) diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 01f1351..a06d07e 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -1503,127 +1503,44 @@ static void virtio_net_save(QEMUFile *f, void *opaque, size_t size) virtio_save(vdev, f); } -static void virtio_net_save_device(VirtIODevice *vdev, QEMUFile *f) -{ - VirtIONet *n = VIRTIO_NET(vdev); - int i; - - qemu_put_buffer(f, n->mac, ETH_ALEN); - qemu_put_be32(f, n->vqs[0].tx_waiting); - qemu_put_be32(f, n->mergeable_rx_bufs); - qemu_put_be16(f, n->status); - qemu_put_byte(f, n->promisc); - qemu_put_byte(f, n->allmulti); - qemu_put_be32(f, n->mac_table.in_use); - qemu_put_buffer(f, n->mac_table.macs, n->mac_table.in_use * ETH_ALEN); - qemu_put_buffer(f, (uint8_t *)n->vlans, MAX_VLAN >> 3); - qemu_put_be32(f, n->has_vnet_hdr); - qemu_put_byte(f, n->mac_table.multi_overflow); - qemu_put_byte(f, n->mac_table.uni_overflow); - qemu_put_byte(f, n->alluni); - qemu_put_byte(f, n->nomulti); - qemu_put_byte(f, n->nouni); - qemu_put_byte(f, n->nobcast); - qemu_put_byte(f, n->has_ufo); - if (n->max_queues > 1) { - qemu_put_be16(f, n->max_queues); - qemu_put_be16(f, n->curr_queues); - for (i = 1; i < n->curr_queues; i++) { - qemu_put_be32(f, n->vqs[i].tx_waiting); - } - } - - if (virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)) { - qemu_put_be64(f, n->curr_guest_offloads); - } -} - -static int virtio_net_load(QEMUFile *f, void *opaque, size_t size) +static int virtio_net_post_load_device(void *opaque, int version_id) { VirtIONet *n = opaque; VirtIODevice *vdev = VIRTIO_DEVICE(n); - - return virtio_load(vdev, f, VIRTIO_NET_VM_VERSION); -} - -static int virtio_net_load_device(VirtIODevice *vdev, QEMUFile *f, - int version_id) -{ - VirtIONet *n = VIRTIO_NET(vdev); int i, link_down; + bool ufo_on_wire; - qemu_get_buffer(f, n->mac, ETH_ALEN); - n->vqs[0].tx_waiting = qemu_get_be32(f); - - virtio_net_set_mrg_rx_bufs(n, qemu_get_be32(f), - virtio_vdev_has_feature(vdev, - VIRTIO_F_VERSION_1)); - - n->status = qemu_get_be16(f); - - n->promisc = qemu_get_byte(f); - n->allmulti = qemu_get_byte(f); - - n->mac_table.in_use = qemu_get_be32(f); - /* MAC_TABLE_ENTRIES may be different from the saved image */ - if (n->mac_table.in_use <= MAC_TABLE_ENTRIES) { - qemu_get_buffer(f, n->mac_table.macs, - n->mac_table.in_use * ETH_ALEN); - } else { - int64_t i; - - /* Overflow detected - can happen if source has a larger MAC table. - * We simply set overflow flag so there's no need to maintain the - * table of addresses, discard them all. - * Note: 64 bit math to avoid integer overflow. - */ - for (i = 0; i < (int64_t)n->mac_table.in_use * ETH_ALEN; ++i) { - qemu_get_byte(f); - } - n->mac_table.multi_overflow = n->mac_table.uni_overflow = 1; - n->mac_table.in_use = 0; - } - - qemu_get_buffer(f, (uint8_t *)n->vlans, MAX_VLAN >> 3); - - if (qemu_get_be32(f) && !peer_has_vnet_hdr(n)) { + /* The received has_vnet_hdr is only compared against what the configured + * state, we always end up using the configured state we stashed. + */ + if (n->has_vnet_hdr && !n->mig_has_vnet_hdr) { error_report("virtio-net: saved image requires vnet_hdr=on"); return -1; } + n->has_vnet_hdr = n->mig_has_vnet_hdr; - n->mac_table.multi_overflow = qemu_get_byte(f); - n->mac_table.uni_overflow = qemu_get_byte(f); - - n->alluni = qemu_get_byte(f); - n->nomulti = qemu_get_byte(f); - n->nouni = qemu_get_byte(f); - n->nobcast = qemu_get_byte(f); - - if (qemu_get_byte(f) && !peer_has_ufo(n)) { + /* The read has_ufo value is only tested; the used value is either + * the value prior to migration (which preload stashed in mig_has_ufo) + * OR a value that peer_has_ufo sets during the following test. + */ + ufo_on_wire = n->has_ufo; + n->has_ufo = n->mig_has_ufo; + if (ufo_on_wire && !peer_has_ufo(n)) { error_report("virtio-net: saved image requires TUN_F_UFO support"); return -1; } - if (n->max_queues > 1) { - if (n->max_queues != qemu_get_be16(f)) { - error_report("virtio-net: different max_queues "); - return -1; - } + virtio_net_set_mrg_rx_bufs(n, n->mergeable_rx_bufs, + virtio_vdev_has_feature(vdev, + VIRTIO_F_VERSION_1)); - n->curr_queues = qemu_get_be16(f); - if (n->curr_queues > n->max_queues) { - error_report("virtio-net: curr_queues %x > max_queues %x", - n->curr_queues, n->max_queues); - return -1; - } - for (i = 1; i < n->curr_queues; i++) { - n->vqs[i].tx_waiting = qemu_get_be32(f); - } + /* MAC_TABLE_ENTRIES may be different from the saved image */ + if (n->mac_table.in_use > MAC_TABLE_ENTRIES) { + /* TODO: Should we or in multi_overflow/uni_overflow ? */ + n->mac_table.in_use = 0; } - if (virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)) { - n->curr_guest_offloads = qemu_get_be64(f); - } else { + if (!virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)) { n->curr_guest_offloads = virtio_net_supported_guest_offloads(n); } @@ -1657,6 +1574,132 @@ static int virtio_net_load_device(VirtIODevice *vdev, QEMUFile *f, return 0; } +static int virtio_net_pre_load_device(void *opaque) +{ + VirtIONet *n = opaque; + n->mig_has_vnet_hdr = peer_has_vnet_hdr(n); + n->mig_has_ufo = n->has_ufo; + + return 0; +} + +/* tx_waiting field of a VirtIONetQueue */ +static const VMStateDescription vmstate_virtio_net_queue_tx_waiting = { + .name = "virtio-net-queue-tx_waiting", + .fields = (VMStateField[]) { + VMSTATE_UINT32(tx_waiting, VirtIONetQueue), + VMSTATE_END_OF_LIST() + }, +}; + +static bool max_queues_gt_1(void *opaque, int version_id) +{ + return VIRTIO_NET(opaque)->max_queues > 1; +} + +static bool has_ctrl_guest_offloads(void *opaque, int version_id) +{ + return virtio_vdev_has_feature(VIRTIO_DEVICE(opaque), + VIRTIO_NET_F_CTRL_GUEST_OFFLOADS); +} + +static bool mac_table_fits(void *opaque, int version_id) +{ + return VIRTIO_NET(opaque)->mac_table.in_use <= MAC_TABLE_ENTRIES; +} + +static bool mac_table_doesnt_fit(void *opaque, int version_id) +{ + return !mac_table_fits(opaque, version_id); +} + +/* NOTE! Has side effect - it updates mig_curr_queues_1 for the + * varray macro after it. + */ +static bool validate_curr_queues(void *opaque, int version_id) +{ + VirtIONet *n = opaque; + + /* This pair make it easy to migrate all-but-the-first element + * of vqs. + */ + n->mig_vqs_1 = n->vqs + 1; + n->mig_curr_queues_1 = 0; + if (n->curr_queues > 1) { + if (n->curr_queues > n->max_queues) { + error_report("virtio-net: curr_queues %x > max_queues %x", + n->curr_queues, n->max_queues); + return false; + } + + n->mig_curr_queues_1 = n->curr_queues - 1; + } + + return true; /* All good */ +} + +static const VMStateDescription vmstate_virtio_net_device = { + .name = "virtio-net-device", + .version_id = VIRTIO_NET_VM_VERSION, + .minimum_version_id = VIRTIO_NET_VM_VERSION, + .pre_load = virtio_net_pre_load_device, + .post_load = virtio_net_post_load_device, + .fields = (VMStateField[]) { + VMSTATE_UINT8_ARRAY(mac, VirtIONet, ETH_ALEN), + VMSTATE_STRUCT_POINTER(vqs, VirtIONet, + vmstate_virtio_net_queue_tx_waiting, + VirtIONetQueue), + VMSTATE_UINT32(mergeable_rx_bufs, VirtIONet), + VMSTATE_UINT16(status, VirtIONet), + VMSTATE_UINT8(promisc, VirtIONet), + VMSTATE_UINT8(allmulti, VirtIONet), + VMSTATE_UINT32(mac_table.in_use, VirtIONet), + + /* Guarded pair: If it fits we load it, else we throw it away + * - can happen if source has a larger MAC table.; post-load + * sets flags in this case. + */ + VMSTATE_VBUFFER_MULTIPLY(mac_table.macs, VirtIONet, + 0, mac_table_fits, 0, mac_table.in_use, + ETH_ALEN), + VMSTATE_UNUSED_VARRAY_UINT32(VirtIONet, mac_table_doesnt_fit, 0, + mac_table.in_use, ETH_ALEN), + + /* Note: This is an array of uint32's that's always been saved as a + * buffer; hold onto your endiannesses; it's actually used as a bitmap + * but based on the uint. + */ + VMSTATE_BUFFER_POINTER_UNSAFE(vlans, VirtIONet, 0, MAX_VLAN >> 3), + VMSTATE_UINT32(has_vnet_hdr, VirtIONet), + VMSTATE_UINT8(mac_table.multi_overflow, VirtIONet), + VMSTATE_UINT8(mac_table.uni_overflow, VirtIONet), + VMSTATE_UINT8(alluni, VirtIONet), + VMSTATE_UINT8(nomulti, VirtIONet), + VMSTATE_UINT8(nouni, VirtIONet), + VMSTATE_UINT8(nobcast, VirtIONet), + VMSTATE_UINT8(has_ufo, VirtIONet), + VMSTATE_SINGLE_TEST(max_queues, VirtIONet, max_queues_gt_1, 0, + vmstate_info_uint16_equal, uint16_t), + VMSTATE_UINT16_TEST(curr_queues, VirtIONet, max_queues_gt_1), + VMSTATE_VALIDATE("curr_queues <= max_queues", validate_curr_queues), + VMSTATE_STRUCT_VARRAY_POINTER_UINT16(mig_vqs_1, VirtIONet, + mig_curr_queues_1, + vmstate_virtio_net_queue_tx_waiting, + VirtIONetQueue), + VMSTATE_UINT64_TEST(curr_guest_offloads, VirtIONet, + has_ctrl_guest_offloads), + VMSTATE_END_OF_LIST() + }, +}; + +static int virtio_net_load(QEMUFile *f, void *opaque, size_t size) +{ + VirtIONet *n = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(n); + + return virtio_load(vdev, f, VIRTIO_NET_VM_VERSION); +} + static NetClientInfo net_virtio_info = { .type = NET_CLIENT_DRIVER_NIC, .size = sizeof(NICState), @@ -1902,8 +1945,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->set_status = virtio_net_set_status; vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending; - vdc->load = virtio_net_load_device; - vdc->save = virtio_net_save_device; + vdc->vmsd = &vmstate_virtio_net_device; } static const TypeInfo virtio_net_info = { diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index 91ed97c..d86ee19 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -45,7 +45,7 @@ typedef struct VirtIONetQueue { VirtQueue *tx_vq; QEMUTimer *tx_timer; QEMUBH *tx_bh; - int tx_waiting; + uint32_t tx_waiting; struct { VirtQueueElement *elem; } async_tx; @@ -66,7 +66,7 @@ typedef struct VirtIONet { size_t guest_hdr_len; uint32_t host_features; uint8_t has_ufo; - int mergeable_rx_bufs; + uint32_t mergeable_rx_bufs; uint8_t promisc; uint8_t allmulti; uint8_t alluni; @@ -95,6 +95,12 @@ typedef struct VirtIONet { QEMUTimer *announce_timer; int announce_counter; bool needs_vnet_hdr_swap; + + /* Helper variables just for migration - see validate_curr_queues */ + VirtIONetQueue *mig_vqs_1; /* vqs+1 */ + uint16_t mig_curr_queues_1; /* vqs-1 */ + bool mig_has_vnet_hdr; /* Stashed copy during load */ + bool mig_has_ufo; /* Stashed copy during load */ } VirtIONet; void virtio_net_set_netclient_name(VirtIONet *n, const char *name,