From patchwork Fri Oct 11 13:58:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikhail Sennikovsky X-Patchwork-Id: 11185575 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F054118B7 for ; Fri, 11 Oct 2019 14:04:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C6CC320640 for ; Fri, 11 Oct 2019 14:04:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cloud.ionos.com header.i=@cloud.ionos.com header.b="HHH0aoB9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C6CC320640 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=cloud.ionos.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:50884 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iIvWX-0006Fn-Ce for patchwork-qemu-devel@patchwork.kernel.org; Fri, 11 Oct 2019 10:04:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40487) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iIvRF-0008Ei-8t for qemu-devel@nongnu.org; Fri, 11 Oct 2019 09:58:38 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iIvRD-0000XU-VZ for qemu-devel@nongnu.org; Fri, 11 Oct 2019 09:58:36 -0400 Received: from mail-ed1-x542.google.com ([2a00:1450:4864:20::542]:36124) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iIvRD-0000X7-MG for qemu-devel@nongnu.org; Fri, 11 Oct 2019 09:58:35 -0400 Received: by mail-ed1-x542.google.com with SMTP id h2so8777158edn.3 for ; Fri, 11 Oct 2019 06:58:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id; bh=pn/7UX82hnkyCmlRyxAFvkcLsirvCKVjBf2U+cH5riY=; b=HHH0aoB9fZ4oIf004cr1sVdGFpcUN4W9o7Qr98qduFW+7rNulcbj3yos2RginL6nGD VGLNYvfiAYzxQr15Rk6XxFmNvmkPLLXY9dyTnh1IandyNLztlF86i704GD7Rsmq1wHtH cFgq7rVeLtkQ3SK94Jp4LXj7+mS33PxUHyrkgrec/MVHTVTVKvNTJLn9iEmiR8EVb2Lg uUNiOsdY0K/nWZDZSEjoj0smNK1Xlan5lruC7psMU8gg8N83rrfd99emZOMPgFOAdMOV nzvHeOzK/Hbcme2ZdSkepLWYEgOxCRf5KlYO4ApXZ3R30e/vcavAiu+nIY32UjYA8Ftd vaaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=pn/7UX82hnkyCmlRyxAFvkcLsirvCKVjBf2U+cH5riY=; b=hCUTUg2cleUHHRVY3FljdIxhOw9qXVXI1DlAjYjb0+cqKaZzBBAsvEe/8WERoTWJYJ tAb+o3AH0do57Sm3oeYwMgfXLQLZaRxT0fNu1ICFeesfPSd6DIxHmoq/rAbh1R9oQe0a PqQIunxXYezH8Qfjs900BhTG3gNRWHCruXb9y4jlT9yCQpftsDO0UttmUoY3EWteQ6kw 5qf8qWNast4Ycluc/+cg3CnZJf36e3MQEdN9mnj/RliJ68gVEh3F+AHEQSkAAHqXLiAf XT9VeO85hb2/N7aqTtJIYwPKKzilHgLf8CplqAluXEf79wb6XDitukX9nQkDJsugyrdc xvVg== X-Gm-Message-State: APjAAAUghEaZhFVQGYRyarR3SuZ44RkWGeh+/Sm/mRRUijjx/AxtGCb/ LRRoH0Hj3/IzsoKUY+UfMtx9xg== X-Google-Smtp-Source: APXvYqzAKc3TFtE/+d23x/KqY0f0ZAEYcf9bLjwHoRODgLkDesGD5phMs5HZnecgoHgn37LKO48WSA== X-Received: by 2002:a17:906:4d85:: with SMTP id s5mr14316718eju.192.1570802314444; Fri, 11 Oct 2019 06:58:34 -0700 (PDT) Received: from msennikovskii2.pb.local ([2001:1438:4010:2558:9cc:2ca5:45db:2acf]) by smtp.googlemail.com with ESMTPSA id 36sm1547982edz.92.2019.10.11.06.58.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 11 Oct 2019 06:58:33 -0700 (PDT) From: Mikhail Sennikovsky To: "Michael S . Tsirkin" , qemu-devel@nongnu.org, stefanha@redhat.com, dgilbert@redhat.com, Jason Wang Subject: [PATCH v4 1/2] virtio: new post_load hook Date: Fri, 11 Oct 2019 15:58:03 +0200 Message-Id: <1570802284-3064-1-git-send-email-mikhail.sennikovskii@cloud.ionos.com> X-Mailer: git-send-email 2.7.4 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::542 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhail Sennikovsky Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: "Michael S. Tsirkin" Post load hook in virtio vmsd is called early while device is processed, and when VirtIODevice core isn't fully initialized. Most device specific code isn't ready to deal with a device in such state, and behaves weirdly. Add a new post_load hook in a device class instead. Devices should use this unless they specifically want to verify the migration stream as it's processed, e.g. for bounds checking. Suggested-by: "Dr. David Alan Gilbert" Cc: Mikhail Sennikovsky Signed-off-by: Michael S. Tsirkin --- hw/virtio/virtio.c | 7 +++++++ include/hw/virtio/virtio.h | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index a94ea18..7c3822c 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2287,6 +2287,13 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id) } rcu_read_unlock(); + if (vdc->post_load) { + ret = vdc->post_load(vdev); + if (ret) { + return ret; + } + } + return 0; } diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index b189788..f9f6237 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -158,6 +158,12 @@ typedef struct VirtioDeviceClass { */ void (*save)(VirtIODevice *vdev, QEMUFile *f); int (*load)(VirtIODevice *vdev, QEMUFile *f, int version_id); + /* Post load hook in vmsd is called early while device is processed, and + * when VirtIODevice isn't fully initialized. Devices should use this instead, + * unless they specifically want to verify the migration stream as it's + * processed, e.g. for bounds checking. + */ + int (*post_load)(VirtIODevice *vdev); const VMStateDescription *vmsd; } VirtioDeviceClass; From patchwork Fri Oct 11 13:58:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mikhail Sennikovsky X-Patchwork-Id: 11185629 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 372281668 for ; Fri, 11 Oct 2019 14:12:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0CF02214E0 for ; Fri, 11 Oct 2019 14:12:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cloud.ionos.com header.i=@cloud.ionos.com header.b="brjyj0pQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CF02214E0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=cloud.ionos.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:51100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iIved-0000YR-LL for patchwork-qemu-devel@patchwork.kernel.org; Fri, 11 Oct 2019 10:12:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40500) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iIvRG-0008Gr-Ef for qemu-devel@nongnu.org; Fri, 11 Oct 2019 09:58:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iIvRF-0000Xv-0l for qemu-devel@nongnu.org; Fri, 11 Oct 2019 09:58:38 -0400 Received: from mail-ed1-x542.google.com ([2a00:1450:4864:20::542]:39270) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iIvRE-0000Xb-Om for qemu-devel@nongnu.org; Fri, 11 Oct 2019 09:58:36 -0400 Received: by mail-ed1-x542.google.com with SMTP id a15so8767190edt.6 for ; Fri, 11 Oct 2019 06:58:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nNPD57i+zdP3HSzedsX84NREYpBQ/HXdziN+13fyCCQ=; b=brjyj0pQCvV4N1+yy5EwxCMUOSFqyqdMeQnXYvYdMnTiEQsca4UC2rGkVK2n8VGTCn /PLYWLHyVI5yrrPpcgHHjWm1suxNwoZn0r897Ta2OKFRmPLHngH7TkuVNdzAcq+2MskF uzGsJ8I2pcmlI4uksh/y14TcGUgITiDZEczZDswWiTmnM3txR4L+B+ICT7lGxJwxLwn5 WsHfy4oYq5g44N6WASeoUJGW+2FNqm9BrHUytnf6oovl3kuuFh/0XSNsx24ptpOvHtkg FXE7xwAFP3Km70R4cuu1DfnwZGjlxWRwgVMt1tHMrJR3HTpOLENNPHFJLmn/HKGmtdk1 +Bhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nNPD57i+zdP3HSzedsX84NREYpBQ/HXdziN+13fyCCQ=; b=nIcgHz8wxLfr4ZolGGS9CJYe2nCZeq24+nr/Mn02oqxY2m0A4CCaObAASYRJqPs+vy vUeOtIqDvHs3CPZQSxKyvRBFntCnPYLVrvA6scypLtMKuFuhzqeWtIKuruxDUOntqCCR 0w9wS/9KGFd0cmJ1sGrPscBsPZSiebENB6s/GtJ/qnT/AXXlwuSZMH+WyXbvszcjZaes fRC6cig+fOnY5eZHdT9bOQFOaIT4Di6w7P/o2ajApzrisvEvCILdDz1DLhe1vl0m/z37 Lw0CIf5yHBDgd8lIGMtPqz/8N7MeZJ5UOuhdTuR9+Wok8eR2nVUwfEN/CS2s/SEqgV8L d65Q== X-Gm-Message-State: APjAAAXmc3PzdwKKinbKyC0AB5ZNN0d/6DbfpubD5aPRRrA4BWwNBMKh YqxM6nat3scjCVTTit0KYEqntQ== X-Google-Smtp-Source: APXvYqwpj+rnC0j9t1hvN8td1G/OYp4PoDR0Sw0ZDRUAOhw9Ox3IxA0sAeKwMIdXMHoSx35eKR0Pnw== X-Received: by 2002:a17:906:3385:: with SMTP id v5mr13914224eja.154.1570802315775; Fri, 11 Oct 2019 06:58:35 -0700 (PDT) Received: from msennikovskii2.pb.local ([2001:1438:4010:2558:9cc:2ca5:45db:2acf]) by smtp.googlemail.com with ESMTPSA id 36sm1547982edz.92.2019.10.11.06.58.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 11 Oct 2019 06:58:34 -0700 (PDT) From: Mikhail Sennikovsky To: "Michael S . Tsirkin" , qemu-devel@nongnu.org, stefanha@redhat.com, dgilbert@redhat.com, Jason Wang Subject: [PATCH v4 2/2] virtio-net: prevent offloads reset on migration Date: Fri, 11 Oct 2019 15:58:04 +0200 Message-Id: <1570802284-3064-2-git-send-email-mikhail.sennikovskii@cloud.ionos.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1570802284-3064-1-git-send-email-mikhail.sennikovskii@cloud.ionos.com> References: <1570802284-3064-1-git-send-email-mikhail.sennikovskii@cloud.ionos.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::542 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mikhail Sennikovsky Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Currently offloads disabled by guest via the VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET command are not preserved on VM migration. Instead all offloads reported by guest features (via VIRTIO_PCI_GUEST_FEATURES) get enabled. What happens is: first the VirtIONet::curr_guest_offloads gets restored and offloads are getting set correctly: #0 qemu_set_offload (nc=0x555556a11400, csum=1, tso4=0, tso6=0, ecn=0, ufo=0) at net/net.c:474 #1 virtio_net_apply_guest_offloads (n=0x555557701ca0) at hw/net/virtio-net.c:720 #2 virtio_net_post_load_device (opaque=0x555557701ca0, version_id=11) at hw/net/virtio-net.c:2334 #3 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577c80 , opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:168 #4 virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at hw/virtio/virtio.c:2197 #5 virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036 #6 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 , opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:143 #7 vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at migration/savevm.c:829 #8 qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2211 #9 qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2395 #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467 #11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449 However later on the features are getting restored, and offloads get reset to everything supported by features: #0 qemu_set_offload (nc=0x555556a11400, csum=1, tso4=1, tso6=1, ecn=0, ufo=0) at net/net.c:474 #1 virtio_net_apply_guest_offloads (n=0x555557701ca0) at hw/net/virtio-net.c:720 #2 virtio_net_set_features (vdev=0x555557701ca0, features=5104441767) at hw/net/virtio-net.c:773 #3 virtio_set_features_nocheck (vdev=0x555557701ca0, val=5104441767) at hw/virtio/virtio.c:2052 #4 virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at hw/virtio/virtio.c:2220 #5 virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036 #6 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 , opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:143 #7 vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at migration/savevm.c:829 #8 qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2211 #9 qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2395 #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467 #11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449 Fix this by preserving the state in saved_guest_offloads field and pushing out offload initialization to the new post load hook. Signed-off-by: Mikhail Sennikovsky Reviewed-by: Michael S. Tsirkin --- hw/net/virtio-net.c | 27 ++++++++++++++++++++++++--- include/hw/virtio/virtio-net.h | 2 ++ 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index b9e1cd7..6adb0fe 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -2330,9 +2330,13 @@ static int virtio_net_post_load_device(void *opaque, int version_id) n->curr_guest_offloads = virtio_net_supported_guest_offloads(n); } - if (peer_has_vnet_hdr(n)) { - virtio_net_apply_guest_offloads(n); - } + /* + * curr_guest_offloads will be later overwritten by the + * virtio_set_features_nocheck call done from the virtio_load. + * Here we make sure it is preserved and restored accordingly + * in the virtio_net_post_load_virtio callback. + */ + n->saved_guest_offloads = n->curr_guest_offloads; virtio_net_set_queues(n); @@ -2367,6 +2371,22 @@ static int virtio_net_post_load_device(void *opaque, int version_id) return 0; } +static int virtio_net_post_load_virtio(VirtIODevice *vdev) +{ + VirtIONet *n = VIRTIO_NET(vdev); + /* + * The actual needed state is now in saved_guest_offloads, + * see virtio_net_post_load_device for detail. + * Restore it back and apply the desired offloads. + */ + n->curr_guest_offloads = n->saved_guest_offloads; + if (peer_has_vnet_hdr(n)) { + virtio_net_apply_guest_offloads(n); + } + + return 0; +} + /* tx_waiting field of a VirtIONetQueue */ static const VMStateDescription vmstate_virtio_net_queue_tx_waiting = { .name = "virtio-net-queue-tx_waiting", @@ -2909,6 +2929,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data) vdc->guest_notifier_mask = virtio_net_guest_notifier_mask; vdc->guest_notifier_pending = virtio_net_guest_notifier_pending; vdc->legacy_features |= (0x1 << VIRTIO_NET_F_GSO); + vdc->post_load = virtio_net_post_load_virtio; vdc->vmsd = &vmstate_virtio_net_device; } diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index b96f0c6..07a9319 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -182,6 +182,8 @@ struct VirtIONet { char *netclient_name; char *netclient_type; uint64_t curr_guest_offloads; + /* used on saved state restore phase to preserve the curr_guest_offloads */ + uint64_t saved_guest_offloads; AnnounceTimer announce_timer; bool needs_vnet_hdr_swap; bool mtu_bypass_backend;