From patchwork Thu Dec 15 11:31:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13074080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6851C3DA6E for ; Thu, 15 Dec 2022 11:33:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5mT6-0002WV-DV; Thu, 15 Dec 2022 06:32:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5mT4-0002W1-BQ for qemu-devel@nongnu.org; Thu, 15 Dec 2022 06:32:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5mT2-0006i7-6g for qemu-devel@nongnu.org; Thu, 15 Dec 2022 06:32:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1671103918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+FgMluJl7FAJ334+76uBKCfgm7N2VpXkx+EvMvxNZ0w=; b=WyZY9t2XVsn2c1XDXX+t2dG2yGTaF4e8ZuS8oaTG1HIg+mqwwXWmp7uGt/HB8ecwCzFHbb Q2lqdPHz6WdAAgfvHTiVkrayhEskv1OMLWVRwpv2m9EUeN14NEl5hEs6pjUyemFWtdDgq/ ErJCgS6ONW+36FEPJDaYA2pV9sg3rsI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-172-nea6zczNOLOi9qWz2ao1Mw-1; Thu, 15 Dec 2022 06:31:53 -0500 X-MC-Unique: nea6zczNOLOi9qWz2ao1Mw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7424A382C969; Thu, 15 Dec 2022 11:31:52 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0160F2166B26; Thu, 15 Dec 2022 11:31:48 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefano Garzarella , Zhu Lingshan , Si-Wei Liu , Laurent Vivier , "Gonglei (Arei)" , Stefan Hajnoczi , Jason Wang , "Michael S. Tsirkin" , Cindy Lu , Gautam Dawar , Eli Cohen , Cornelia Huck , Paolo Bonzini , Longpeng , Harpreet Singh Anand , Parav Pandit , kvm@vger.kernel.org, virtualization@lists.linux-foundation.org Subject: [PATCH v9 00/12] ASID support in vhost-vdpa net Date: Thu, 15 Dec 2022 12:31:32 +0100 Message-Id: <20221215113144.322011-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Control VQ is the way net devices use to send changes to the device state, like the number of active queues or its mac address. QEMU needs to intercept this queue so it can track these changes and is able to migrate the device. It can do it from 1576dbb5bbc4 ("vdpa: Add x-svq to NetdevVhostVDPAOptions"). However, to enable x-svq implies to shadow all VirtIO device's virtqueues, which will damage performance. This series adds address space isolation, so the device and the guest communicate directly with them (passthrough) and CVQ communication is split in two: The guest communicates with qemu and qemu forwards the commands to the device. Comments are welcome. Thanks! v9: - Reuse iova_range fetched from the device at initialization, instead of fetch it again at vhost_vdpa_net_cvq_start. - Add comment about how migration is blocked in case ASID does not met our expectations. - Delete warning about CVQ group not being independent. v8: - Do not allocate iova_tree on net_init_vhost_vdpa if only CVQ is shadowed. Move the iova_tree allocation to vhost_vdpa_net_cvq_start and vhost_vdpa_net_cvq_stop in this case. v7: - Never ask for number of address spaces, just react if isolation is not possible. - Return ASID ioctl errors instead of masking them as if the device has no asid. - Rename listener_shadow_vq to shadow_data - Move comment on zero initailization of vhost_vdpa_dma_map above the functions. - Add VHOST_VDPA_GUEST_PA_ASID macro. v6: - Do not allocate SVQ resources like file descriptors if SVQ cannot be used. - Disable shadow CVQ if the device does not support it because of net features. v5: - Move vring state in vhost_vdpa_get_vring_group instead of using a parameter. - Rename VHOST_VDPA_NET_CVQ_PASSTHROUGH to VHOST_VDPA_NET_DATA_ASID v4: - Rebased on last CVQ start series, that allocated CVQ cmd bufs at load - Squash vhost_vdpa_cvq_group_is_independent. - Do not check for cvq index on vhost_vdpa_net_prepare, we only have one that callback registered in that NetClientInfo. - Add comment specifying behavior if device does not support _F_ASID - Update headers to a later Linux commit to not to remove SETUP_RNG_SEED v3: - Do not return an error but just print a warning if vdpa device initialization returns failure while getting AS num of VQ groups - Delete extra newline v2: - Much as commented on series [1], handle vhost_net backend through NetClientInfo callbacks instead of directly. - Fix not freeing SVQ properly when device does not support CVQ - Add BIT_ULL missed checking device's backend feature for _F_ASID. Eugenio Pérez (12): vdpa: use v->shadow_vqs_enabled in vhost_vdpa_svqs_start & stop vhost: set SVQ device call handler at SVQ start vhost: allocate SVQ device file descriptors at device start vhost: move iova_tree set to vhost_svq_start vdpa: add vhost_vdpa_net_valid_svq_features vdpa: request iova_range only once vdpa: move SVQ vring features check to net/ vdpa: allocate SVQ array unconditionally vdpa: add asid parameter to vhost_vdpa_dma_map/unmap vdpa: store x-svq parameter in VhostVDPAState vdpa: add shadow_data to vhost_vdpa vdpa: always start CVQ in SVQ mode if possible hw/virtio/vhost-shadow-virtqueue.h | 5 +- include/hw/virtio/vhost-vdpa.h | 16 ++- hw/virtio/vhost-shadow-virtqueue.c | 44 ++------ hw/virtio/vhost-vdpa.c | 140 +++++++++++------------ net/vhost-vdpa.c | 174 ++++++++++++++++++++++++----- hw/virtio/trace-events | 4 +- 6 files changed, 237 insertions(+), 146 deletions(-)