From patchwork Thu Jan 12 17:24:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13098497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 794AAC54EBE for ; Thu, 12 Jan 2023 17:26:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pG1Jp-0006R7-S7; Thu, 12 Jan 2023 12:24:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pG1Jn-0006QX-8R for qemu-devel@nongnu.org; Thu, 12 Jan 2023 12:24:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pG1Jl-0008EO-FW for qemu-devel@nongnu.org; Thu, 12 Jan 2023 12:24:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673544284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2eVgQnytJqbJEosfGhHUCnXQYlQAfjp7LLW5DrztsQQ=; b=DQemzaaquLvi4nwRddKvqJlLCkagz+ZHiCAo7U7SwiII4TitJ6miW8nOaTVGgwTvsvbhtY zDAPVnn9kpd6vw4hieWdliA/VgSXrgJjqbpQHHmtLKD/h0m1AOLeiqPt6PCH/+rNQSc6f9 f6hcqZHkIbRMnfTRhUXmQ85RLMwETfE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-417-jEXm1EPBNhCTVT3T-Jzp2A-1; Thu, 12 Jan 2023 12:24:40 -0500 X-MC-Unique: jEXm1EPBNhCTVT3T-Jzp2A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 01E813806721; Thu, 12 Jan 2023 17:24:40 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.72]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCA514078903; Thu, 12 Jan 2023 17:24:36 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: si-wei.liu@oracle.com, Liuxiangdong , Zhu Lingshan , "Gonglei (Arei)" , alvaro.karsz@solid-run.com, Shannon Nelson , Laurent Vivier , Harpreet Singh Anand , Gautam Dawar , Stefano Garzarella , Cornelia Huck , Cindy Lu , Eli Cohen , Paolo Bonzini , "Michael S. Tsirkin" , Jason Wang , Stefan Hajnoczi , Parav Pandit Subject: [RFC v2 00/13] Dinamycally switch to vhost shadow virtqueues at vdpa net migration Date: Thu, 12 Jan 2023 18:24:21 +0100 Message-Id: <20230112172434.760850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org It's possible to migrate vdpa net devices if they are shadowed from the start. But to always shadow the dataplane is effectively break its host passthrough, so its not convenient in vDPA scenarios. This series enables dynamically switching to shadow mode only at migration time. This allow full data virtqueues passthrough all the time qemu is not migrating. Successfully tested with vdpa_sim_net (but it needs some patches, I will send them soon) and qemu emulated device with vp_vdpa with some restrictions: * No CVQ. * VIRTIO_RING_F_STATE patches. * Expose _F_SUSPEND, but ignore it and suspend on ring state fetch like DPDK. Comments are welcome, especially in the patcheswith RFC in the message. v2: - Use a migration listener instead of a memory listener to know when the migration starts. - Add stuff not picked with ASID patches, like enable rings after driver_ok - Add rewinding on the migration src, not in dst - v1 at https://lists.gnu.org/archive/html/qemu-devel/2022-08/msg01664.html Eugenio Pérez (13): vdpa: fix VHOST_BACKEND_F_IOTLB_ASID flag check vdpa net: move iova tree creation from init to start vdpa: copy cvq shadow_data from data vqs, not from x-svq vdpa: rewind at get_base, not set_base vdpa net: add migration blocker if cannot migrate cvq vhost: delay set_vring_ready after DRIVER_OK vdpa: delay set_vring_ready after DRIVER_OK vdpa: Negotiate _F_SUSPEND feature vdpa: add feature_log parameter to vhost_vdpa vdpa net: allow VHOST_F_LOG_ALL vdpa: add vdpa net migration state notifier vdpa: preemptive kick at enable vdpa: Conditionally expose _F_LOG in vhost_net devices include/hw/virtio/vhost-backend.h | 4 + include/hw/virtio/vhost-vdpa.h | 1 + hw/net/vhost_net.c | 25 ++- hw/virtio/vhost-vdpa.c | 64 +++++--- hw/virtio/vhost.c | 3 + net/vhost-vdpa.c | 247 +++++++++++++++++++++++++----- 6 files changed, 278 insertions(+), 66 deletions(-)