From patchwork Mon Dec 5 17:04:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 169C1C4332F for ; Mon, 5 Dec 2022 17:06:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Etm-0000TM-KX; Mon, 05 Dec 2022 12:04:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etj-0000SD-7M for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:04:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Eth-0002eb-Qw for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:04:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259893; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=COOJJLORpxTJlPJ0nnDaaaQkM6ejKdslCpuafsEQe1c=; b=U5l5Mys+re7ryKBUjsyisblM/H3XGIO6acCU5IBNNClkCW7hCqZt1I2eWGawhuWsCNa5+I PkZ9Z6zQJ1xMZsMMISHqbk/EFmFFz1HN4rkrgtnW2B8QdZ9WarhdP+eFfKUwLtiTC9vhOe U82mCoYd08irXsQc1dG+ZjNOYo3JQ8Y= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-646-976d8h7KNfWMkqYttIvSVw-1; Mon, 05 Dec 2022 12:04:46 -0500 X-MC-Unique: 976d8h7KNfWMkqYttIvSVw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 476538828C4; Mon, 5 Dec 2022 17:04:46 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id 856B1C2C7D9; Mon, 5 Dec 2022 17:04:43 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 01/13] vhost: add available descriptor list in SVQ Date: Mon, 5 Dec 2022 18:04:24 +0100 Message-Id: <20221205170436.2977336-2-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This helps to track the inflight buffers, make easier to transverse all of them, and return them in order. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 6 ++++++ hw/virtio/vhost-shadow-virtqueue.c | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index d04c34a589..a01a7d4a18 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -23,6 +23,9 @@ typedef struct SVQDescState { * guest's */ unsigned int ndescs; + + /* List to save or free inflight descriptors */ + QTAILQ_ENTRY(SVQDescState) entry; } SVQDescState; typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; @@ -81,6 +84,9 @@ typedef struct VhostShadowVirtqueue { /* SVQ vring descriptors state */ SVQDescState *desc_state; + /* Linked list to follow avail descriptors */ + QTAILQ_HEAD(, SVQDescState) desc_state_avail; + /* Next VirtQueue element that guest made available */ VirtQueueElement *next_guest_avail_elem; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 5bd14cad96..0da72cb0ec 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -265,6 +265,8 @@ int vhost_svq_add(VhostShadowVirtqueue *svq, const struct iovec *out_sg, svq->desc_state[qemu_head].elem = elem; svq->desc_state[qemu_head].ndescs = ndescs; + QTAILQ_INSERT_TAIL(&svq->desc_state_avail, &svq->desc_state[qemu_head], + entry); vhost_svq_kick(svq); return 0; } @@ -451,6 +453,8 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq, svq->free_head = used_elem.id; *len = used_elem.len; + QTAILQ_REMOVE(&svq->desc_state_avail, &svq->desc_state[used_elem.id], + entry); return g_steal_pointer(&svq->desc_state[used_elem.id].elem); } @@ -665,6 +669,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, svq->vring.used = qemu_memalign(qemu_real_host_page_size(), device_size); memset(svq->vring.used, 0, device_size); svq->desc_state = g_new0(SVQDescState, svq->vring.num); + QTAILQ_INIT(&svq->desc_state_avail); svq->desc_next = g_new0(uint16_t, svq->vring.num); for (unsigned i = 0; i < svq->vring.num - 1; i++) { svq->desc_next[i] = cpu_to_le16(i + 1); From patchwork Mon Dec 5 17:04:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064824 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9DBD4C4321E for ; Mon, 5 Dec 2022 17:06:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Etm-0000TO-K2; Mon, 05 Dec 2022 12:04:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etl-0000Sc-5Y for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:04:57 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etj-0002eg-2V for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:04:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259894; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pM2/rSL7g84lHihy8JL+xsaUEf/Ol5aaFNVkigp8LR0=; b=Ufv82K3aUU4k//CQLic8eALkRF1izU38fLE0O1BHi0qfUUWCDTSqTvGliXj9Dhk1FOg4p7 EU7nf+zkFFbRPor5KdBQPiY218p39rj4bfFWPQQioAke6PI8N+QOoQ2MMzJvJ9A1ObSNNJ qiRhwQ7fgHnMzKBEOsCfxCI/UAG0X5U= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-657-2p4itpD7Oxyk6xTWQwCXEw-1; Mon, 05 Dec 2022 12:04:50 -0500 X-MC-Unique: 2p4itpD7Oxyk6xTWQwCXEw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 52D983C1068D; Mon, 5 Dec 2022 17:04:49 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E4B6C16922; Mon, 5 Dec 2022 17:04:46 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 02/13] vhost: iterate only available descriptors at SVQ stop Date: Mon, 5 Dec 2022 18:04:25 +0100 Message-Id: <20221205170436.2977336-3-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org While we're at it, simplify the free path making just transverse the list instead of all of them. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 0da72cb0ec..1bda8ca4bf 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -692,12 +692,13 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) /* Send all pending used descriptors to guest */ vhost_svq_flush(svq, false); - for (unsigned i = 0; i < svq->vring.num; ++i) { + while (!QTAILQ_EMPTY(&svq->desc_state_avail)) { + SVQDescState *s = QTAILQ_FIRST(&svq->desc_state_avail); g_autofree VirtQueueElement *elem = NULL; - elem = g_steal_pointer(&svq->desc_state[i].elem); - if (elem) { - virtqueue_detach_element(svq->vq, elem, 0); - } + + elem = g_steal_pointer(&s->elem); + virtqueue_detach_element(svq->vq, elem, 0); + QTAILQ_REMOVE(&svq->desc_state_avail, s, entry); } next_avail_elem = g_steal_pointer(&svq->next_guest_avail_elem); From patchwork Mon Dec 5 17:04:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 056DBC4332F for ; Mon, 5 Dec 2022 17:06:03 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Eto-0000Ta-LN; Mon, 05 Dec 2022 12:05:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etn-0000TR-4y for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:04:59 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etl-0002ey-KW for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:04:58 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zC19UHkvMu8PnACVlAc3vwHKzXaHdSok8GEtSaD3nNc=; b=X2CeihXJobbE6dlUhP03DavRonkhGT7Uv9SZVe4/CVi6ZrVj4vwHAn56XbmkGVNR96HtTd XUwV59YwM/dr6ccJlhX2Qgke2irra1BpZY9qx5w/kalt8mdcAxkIth4GzKt4grCBHz1R8p HcCATH12h5VPw/OOXzHjs47lut1tFUs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-389-4ENSNoYeN0C_tN6Nlg6jnQ-1; Mon, 05 Dec 2022 12:04:53 -0500 X-MC-Unique: 4ENSNoYeN0C_tN6Nlg6jnQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5CFA038164C3; Mon, 5 Dec 2022 17:04:52 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9AA2DC2C8C5; Mon, 5 Dec 2022 17:04:49 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 03/13] vhost: merge avail list and next avail descriptors detach Date: Mon, 5 Dec 2022 18:04:26 +0100 Message-Id: <20221205170436.2977336-4-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We need to perform the same actions for the two. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 1bda8ca4bf..e50bfba6dc 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -692,18 +692,18 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) /* Send all pending used descriptors to guest */ vhost_svq_flush(svq, false); - while (!QTAILQ_EMPTY(&svq->desc_state_avail)) { + while (!QTAILQ_EMPTY(&svq->desc_state_avail) + || svq->next_guest_avail_elem) { SVQDescState *s = QTAILQ_FIRST(&svq->desc_state_avail); g_autofree VirtQueueElement *elem = NULL; - elem = g_steal_pointer(&s->elem); - virtqueue_detach_element(svq->vq, elem, 0); - QTAILQ_REMOVE(&svq->desc_state_avail, s, entry); - } - - next_avail_elem = g_steal_pointer(&svq->next_guest_avail_elem); - if (next_avail_elem) { - virtqueue_detach_element(svq->vq, next_avail_elem, 0); + elem = g_steal_pointer(s ? &s->elem : &svq->next_guest_avail_elem); + if (elem) { + virtqueue_detach_element(svq->vq, elem, 0); + } + if (s) { + QTAILQ_REMOVE(&svq->desc_state_avail, s, entry); + } } svq->vq = NULL; g_free(svq->desc_next); From patchwork Mon Dec 5 17:04:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C134C4332F for ; Mon, 5 Dec 2022 17:07:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Ets-0000U9-Cz; Mon, 05 Dec 2022 12:05:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etq-0000U0-Go for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Eto-0002fD-VX for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PZJN+lQaiR6aayLRZJJMIFQUTsms4Mb/PeILWZOWtuY=; b=Luxu4sc4lKSDCAiwyScGAkIb4Up1qUEoY3oLaWzhil1agvn6sKvZDriru+2uOOsEUcYGNM eDv6sJxciHf26emPwZZI9A2+28DCKLkmlAyUJ6T7X8LJboOX3dREQ08RnBW2KW7i46cbzC 1awoYHWBTq7ydEBQT5SY6YOVRo6V0c4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-628-8VQ_liRANpKIKSszfRXboA-1; Mon, 05 Dec 2022 12:04:56 -0500 X-MC-Unique: 8VQ_liRANpKIKSszfRXboA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6AB86185A794; Mon, 5 Dec 2022 17:04:55 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id A4816C16922; Mon, 5 Dec 2022 17:04:52 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 04/13] vhost: add vhost_svq_save_inflight Date: Mon, 5 Dec 2022 18:04:27 +0100 Message-Id: <20221205170436.2977336-5-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This allows SVQ to return all the inflight descriptors in order. TODO: Rewind descriptors that can be fetched from avail ring again. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index a01a7d4a18..e82da4b55c 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -133,6 +133,8 @@ size_t vhost_svq_device_area_size(const VhostShadowVirtqueue *svq); void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, VirtQueue *vq); +VirtQueueElement **vhost_svq_save_inflight(VhostShadowVirtqueue *svq, + uint32_t *num); void vhost_svq_stop(VhostShadowVirtqueue *svq); VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index e50bfba6dc..029ccee957 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -676,6 +676,26 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev, } } +VirtQueueElement **vhost_svq_save_inflight(VhostShadowVirtqueue *svq, + uint32_t *num) +{ + GPtrArray *aret = g_ptr_array_new(); + SVQDescState *s, *tmp; + + QTAILQ_FOREACH_SAFE(s, &svq->desc_state_avail, entry, tmp) { + VirtQueueElement *elem = s->elem; + + virtqueue_detach_element(svq->vq, elem, 0); + g_ptr_array_add(aret, elem); + QTAILQ_REMOVE(&svq->desc_state_avail, s, entry); + } + + *num = aret->len; + + /* TODO: return g_ptr_array_steal(aret) when min glib >= 2.64 */ + return (void *)g_ptr_array_free(aret, false); +} + /** * Stop the shadow virtqueue operation. * @svq: Shadow Virtqueue From patchwork Mon Dec 5 17:04:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99D90C4332F for ; Mon, 5 Dec 2022 17:06:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Eud-0000WH-1l; Mon, 05 Dec 2022 12:05:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etx-0000VT-Pc for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Ets-0002nw-1j for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MNbGMGhn4osZDau4Cy6mZ/L5GKVeEbNHeE8/WkHLsBc=; b=KaGCB9deP13ZOBuMJJcSgfS6uEQFSeYZpXWlu5yKG6yYkPpm9sLA1cb9TWPR2hh4omhdHE 3VCPw6eDRdAo2+EqS4Rr8rkr5Xis0hXk6cK7WzlggXo30JHmai24bOeAahjM4LCSNuECaC 8LFOFLSXCPnCOnelhKmjNCNc+2MaSjU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-275-0-OCdQ-SPFK8tWVLumWFmA-1; Mon, 05 Dec 2022 12:04:59 -0500 X-MC-Unique: 0-OCdQ-SPFK8tWVLumWFmA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 75849857F8F; Mon, 5 Dec 2022 17:04:58 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id B1298C1912A; Mon, 5 Dec 2022 17:04:55 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 05/13] virtio: Specify uint32_t as VirtQueueElementOld members type Date: Mon, 5 Dec 2022 18:04:28 +0100 Message-Id: <20221205170436.2977336-6-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This struct will be the one used as a base to serialize and deserialize the virtqueue elements within this RFC. The VMState fields are way more simpler using this struct. TODO: Use VirtQueueElement to serialize and deserialize inflight elements. Signed-off-by: Eugenio Pérez --- hw/virtio/virtio.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index eb6347ab5d..6efff3d441 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2288,9 +2288,9 @@ unsigned int virtqueue_drop_all(VirtQueue *vq) * used before the change. */ typedef struct VirtQueueElementOld { - unsigned int index; - unsigned int out_num; - unsigned int in_num; + uint32_t index; + uint32_t out_num; + uint32_t in_num; hwaddr in_addr[VIRTQUEUE_MAX_SIZE]; hwaddr out_addr[VIRTQUEUE_MAX_SIZE]; struct iovec in_sg[VIRTQUEUE_MAX_SIZE]; From patchwork Mon Dec 5 17:04:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2F3B1C4321E for ; Mon, 5 Dec 2022 17:06:10 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Euk-0000l6-Bj; Mon, 05 Dec 2022 12:06:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Ety-0000Vb-7B for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etv-0002qt-Mk for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259907; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2RFw7L+2BXaLuOkr7eQfNV4/dHHQbEBLPykQdO9g9zg=; b=f3/A1rSb6o7BLdKpeYxHRO8qWYTuMWRqPNY2ByP5/YgelbWlH7bpzx8QQl7F5O5ghTgmyi Eer0/pbg6z3svZyf74ZXfwaGgVLW6Gbje19nZcSJZybiDPezOh0VvQeKEIfH8eb3/zrMlq FSwOf4orF7ODnLfqb8b1XvlscxLLe1M= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-444-16wV_l61OiKReFzbYO32FA-1; Mon, 05 Dec 2022 12:05:02 -0500 X-MC-Unique: 16wV_l61OiKReFzbYO32FA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8080D894E89; Mon, 5 Dec 2022 17:05:01 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id BD0DAC15BA8; Mon, 5 Dec 2022 17:04:58 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 06/13] virtio: refactor qemu_get_virtqueue_element Date: Mon, 5 Dec 2022 18:04:29 +0100 Message-Id: <20221205170436.2977336-7-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The core of the function is useful to transform from VirtQueueElementOld to VirtQueueElement. Extract from qemu_get_virtqueue_element, and leave there the handling of QEMUFile. No functional change intended. Signed-off-by: Eugenio Pérez --- hw/virtio/virtio.c | 68 ++++++++++++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 26 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 6efff3d441..b0245ce4e8 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2297,50 +2297,66 @@ typedef struct VirtQueueElementOld { struct iovec out_sg[VIRTQUEUE_MAX_SIZE]; } VirtQueueElementOld; -void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz) +/* Convert VirtQueueElementOld to VirtQueueElement */ +static void *qemu_get_virtqueue_element_from_old(VirtIODevice *vdev, + const VirtQueueElementOld *data, + size_t sz) { - VirtQueueElement *elem; - VirtQueueElementOld data; - int i; - - qemu_get_buffer(f, (uint8_t *)&data, sizeof(VirtQueueElementOld)); - - /* TODO: teach all callers that this can fail, and return failure instead - * of asserting here. - * This is just one thing (there are probably more) that must be - * fixed before we can allow NDEBUG compilation. - */ - assert(ARRAY_SIZE(data.in_addr) >= data.in_num); - assert(ARRAY_SIZE(data.out_addr) >= data.out_num); - - elem = virtqueue_alloc_element(sz, data.out_num, data.in_num); - elem->index = data.index; + VirtQueueElement *elem = virtqueue_alloc_element(sz, data->out_num, + data->in_num); + elem->index = data->index; - for (i = 0; i < elem->in_num; i++) { - elem->in_addr[i] = data.in_addr[i]; + for (uint16_t i = 0; i < elem->in_num; i++) { + elem->in_addr[i] = data->in_addr[i]; } - for (i = 0; i < elem->out_num; i++) { - elem->out_addr[i] = data.out_addr[i]; + for (uint16_t i = 0; i < elem->out_num; i++) { + elem->out_addr[i] = data->out_addr[i]; } - for (i = 0; i < elem->in_num; i++) { + for (uint16_t i = 0; i < elem->in_num; i++) { /* Base is overwritten by virtqueue_map. */ elem->in_sg[i].iov_base = 0; - elem->in_sg[i].iov_len = data.in_sg[i].iov_len; + elem->in_sg[i].iov_len = data->in_sg[i].iov_len; } - for (i = 0; i < elem->out_num; i++) { + for (uint16_t i = 0; i < elem->out_num; i++) { /* Base is overwritten by virtqueue_map. */ elem->out_sg[i].iov_base = 0; - elem->out_sg[i].iov_len = data.out_sg[i].iov_len; + elem->out_sg[i].iov_len = data->out_sg[i].iov_len; } + virtqueue_map(vdev, elem); + return elem; +} + +static bool vq_element_in_range(void *opaque, int version_id) +{ + VirtQueueElementOld *data = opaque; + + return ARRAY_SIZE(data->in_addr) >= data->in_num && + ARRAY_SIZE(data->out_addr) >= data->out_num; +} + +void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz) +{ + VirtQueueElement *elem; + VirtQueueElementOld data; + + qemu_get_buffer(f, (uint8_t *)&data, sizeof(VirtQueueElementOld)); + + /* TODO: teach all callers that this can fail, and return failure instead + * of asserting here. + * This is just one thing (there are probably more) that must be + * fixed before we can allow NDEBUG compilation. + */ + assert(vq_element_in_range(&data, 0)); + + elem = qemu_get_virtqueue_element_from_old(vdev, &data, sz); if (virtio_host_has_feature(vdev, VIRTIO_F_RING_PACKED)) { qemu_get_be32s(f, &elem->ndescs); } - virtqueue_map(vdev, elem); return elem; } From patchwork Mon Dec 5 17:04:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D0881C4321E for ; Mon, 5 Dec 2022 17:06:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Eu1-0000Vg-QV; Mon, 05 Dec 2022 12:05:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etw-0000VE-Nx for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:08 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Etv-0002qn-7x for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259906; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4kPovz/7WAZ3ZL6rq70rmWA7JywhPmZS0LtaB3TU55g=; b=IvPKTMHLwepAVh+jNY8U9YhhA6ZXJyMYpUwyvorFLCFUz3rvxUAYVz0xqYSiQDJIxvCuhn 6e0cdEfzWKPqSQhfx0C5pFf+FTpj8zh+pWy0A0a76I94xiLkQc0Wy5Bk9U8JcJdZLR+GHY GvDcB0mXsV8knkf4V0T84zHp/E7MqrI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-519-jE-hW_RZNOmD9WvzUR02Cg-1; Mon, 05 Dec 2022 12:05:05 -0500 X-MC-Unique: jE-hW_RZNOmD9WvzUR02Cg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8C67F101AA45; Mon, 5 Dec 2022 17:05:04 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id C81CCC15BA8; Mon, 5 Dec 2022 17:05:01 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 07/13] virtio: refactor qemu_put_virtqueue_element Date: Mon, 5 Dec 2022 18:04:30 +0100 Message-Id: <20221205170436.2977336-8-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The core of the function is useful to transform from VirtQueueElement to VirtQueueElementOld. Extract these from qemu_put_virtqueue_element, and leave there the handling of QEMUFile. No functional change intended. Signed-off-by: Eugenio Pérez --- hw/virtio/virtio.c | 45 ++++++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index b0245ce4e8..bc3b474065 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2360,36 +2360,43 @@ void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz) return elem; } -void qemu_put_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, - VirtQueueElement *elem) +/* Convert VirtQueueElement to VirtQueueElementOld */ +static void qemu_put_virtqueue_element_old(const VirtQueueElement *elem, + VirtQueueElementOld *data) { - VirtQueueElementOld data; - int i; + memset(data, 0, sizeof(*data)); + data->index = elem->index; + data->in_num = elem->in_num; + data->out_num = elem->out_num; - memset(&data, 0, sizeof(data)); - data.index = elem->index; - data.in_num = elem->in_num; - data.out_num = elem->out_num; - - for (i = 0; i < elem->in_num; i++) { - data.in_addr[i] = elem->in_addr[i]; + for (int i = 0; i < elem->in_num; i++) { + data->in_addr[i] = elem->in_addr[i]; } - for (i = 0; i < elem->out_num; i++) { - data.out_addr[i] = elem->out_addr[i]; + for (int i = 0; i < elem->out_num; i++) { + data->out_addr[i] = elem->out_addr[i]; } - for (i = 0; i < elem->in_num; i++) { - /* Base is overwritten by virtqueue_map when loading. Do not - * save it, as it would leak the QEMU address space layout. */ - data.in_sg[i].iov_len = elem->in_sg[i].iov_len; + for (int i = 0; i < elem->in_num; i++) { + /* + * Base is overwritten by virtqueue_map when loading. Do not + * save it, as it would leak the QEMU address space layout. + */ + data->in_sg[i].iov_len = elem->in_sg[i].iov_len; } - for (i = 0; i < elem->out_num; i++) { + for (int i = 0; i < elem->out_num; i++) { /* Do not save iov_base as above. */ - data.out_sg[i].iov_len = elem->out_sg[i].iov_len; + data->out_sg[i].iov_len = elem->out_sg[i].iov_len; } +} + +void qemu_put_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, + VirtQueueElement *elem) +{ + VirtQueueElementOld data; + qemu_put_virtqueue_element_old(elem, &data); if (virtio_host_has_feature(vdev, VIRTIO_F_RING_PACKED)) { qemu_put_be32s(f, &elem->ndescs); } From patchwork Mon Dec 5 17:04:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A071C47089 for ; Mon, 5 Dec 2022 17:09:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Ev2-0001EC-EU; Mon, 05 Dec 2022 12:06:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Eu7-0000WI-HC for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Eu1-0002re-4j for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rclFwMvR8Pi/gGmSSgN0poZY/0G9+ys0W3dxJlqlY9Y=; b=Q4JRh/Ylpkh7Q6dnQ/4ZOMBhrT23PqdDRTwxThSp4+42OzIR2+8WtjyM+fi7kdFupGa3GE KiOmiVagncBEuyQcGzOFXVfqAbWPISXv2/2qk4KejRdYs199QfqoiaQorHln5sCbFq068l YA8H6flkfjKImf5Ndj1f0M6wdS6syk8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-333-mosG0CbGOGuzNrj44amX1Q-1; Mon, 05 Dec 2022 12:05:08 -0500 X-MC-Unique: mosG0CbGOGuzNrj44amX1Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9C6E2185A794; Mon, 5 Dec 2022 17:05:07 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id D7065C15BA8; Mon, 5 Dec 2022 17:05:04 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 08/13] virtio: expose VirtQueueElementOld Date: Mon, 5 Dec 2022 18:04:31 +0100 Message-Id: <20221205170436.2977336-9-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We will convert between VirtQueueElement and VirtQueueElementOld to migrate the inflight descriptors, so we need virtio-net to see these. This will not be exported in the final version, but working with VirtQueueElementOld is way more easier than with VirtQueueElement and VMState macros. Signed-off-by: Eugenio Pérez --- include/hw/virtio/virtio.h | 31 +++++++++++++++++++++++++++++++ hw/virtio/virtio.c | 27 +++++---------------------- 2 files changed, 36 insertions(+), 22 deletions(-) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index acfd4df125..b4c5163fb0 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -75,6 +75,37 @@ typedef struct VirtQueueElement struct iovec *out_sg; } VirtQueueElement; +/* + * Reading and writing a structure directly to QEMUFile is *awful*, but + * it is what QEMU has always done by mistake. We can change it sooner + * or later by bumping the version number of the affected vm states. + * In the meanwhile, since the in-memory layout of VirtQueueElement + * has changed, we need to marshal to and from the layout that was + * used before the change. + */ +typedef struct VirtQueueElementOld { + uint32_t index; + uint32_t out_num; + uint32_t in_num; + hwaddr in_addr[VIRTQUEUE_MAX_SIZE]; + hwaddr out_addr[VIRTQUEUE_MAX_SIZE]; + /* Unions help to serialize the descriptor using VMStateDescription */ + union { + struct iovec in_sg[VIRTQUEUE_MAX_SIZE]; + uint64_t in_sg_64[VIRTQUEUE_MAX_SIZE * 2]; + }; + union { + struct iovec out_sg[VIRTQUEUE_MAX_SIZE]; + uint64_t out_sg_64[VIRTQUEUE_MAX_SIZE * 2]; + }; +} VirtQueueElementOld; + +void *qemu_get_virtqueue_element_from_old(VirtIODevice *vdev, + const VirtQueueElementOld *data, + size_t sz); +void qemu_put_virtqueue_element_old(const VirtQueueElement *elem, + VirtQueueElementOld *data); + #define VIRTIO_QUEUE_MAX 1024 #define VIRTIO_NO_VECTOR 0xffff diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index bc3b474065..5ddc49610c 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2280,27 +2280,10 @@ unsigned int virtqueue_drop_all(VirtQueue *vq) } } -/* Reading and writing a structure directly to QEMUFile is *awful*, but - * it is what QEMU has always done by mistake. We can change it sooner - * or later by bumping the version number of the affected vm states. - * In the meanwhile, since the in-memory layout of VirtQueueElement - * has changed, we need to marshal to and from the layout that was - * used before the change. - */ -typedef struct VirtQueueElementOld { - uint32_t index; - uint32_t out_num; - uint32_t in_num; - hwaddr in_addr[VIRTQUEUE_MAX_SIZE]; - hwaddr out_addr[VIRTQUEUE_MAX_SIZE]; - struct iovec in_sg[VIRTQUEUE_MAX_SIZE]; - struct iovec out_sg[VIRTQUEUE_MAX_SIZE]; -} VirtQueueElementOld; - /* Convert VirtQueueElementOld to VirtQueueElement */ -static void *qemu_get_virtqueue_element_from_old(VirtIODevice *vdev, - const VirtQueueElementOld *data, - size_t sz) +void *qemu_get_virtqueue_element_from_old(VirtIODevice *vdev, + const VirtQueueElementOld *data, + size_t sz) { VirtQueueElement *elem = virtqueue_alloc_element(sz, data->out_num, data->in_num); @@ -2361,8 +2344,8 @@ void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz) } /* Convert VirtQueueElement to VirtQueueElementOld */ -static void qemu_put_virtqueue_element_old(const VirtQueueElement *elem, - VirtQueueElementOld *data) +void qemu_put_virtqueue_element_old(const VirtQueueElement *elem, + VirtQueueElementOld *data) { memset(data, 0, sizeof(*data)); data->index = elem->index; From patchwork Mon Dec 5 17:04:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F40E6C4332F for ; Mon, 5 Dec 2022 17:07:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Ev8-0001IM-6R; Mon, 05 Dec 2022 12:06:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuF-0000XC-4O for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Eu4-0002rp-ER for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259914; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w4JLQyJZReme83CRbMiFLssDIL5udQrhvg/dKOKdado=; b=WYQn83DfFmGT+XRtcuYq3pOu2KTLowT4/MbZoIoARxrs3V82NvGmNfTHkx82JNtSl79s/A ORHUEo9wKc1yD+p8T6RgsOFl+CCCgf66y7q7Amn6h7BstZnaw65td496JqaRPffswypzxt bRu2keEAKh8UrA4T2/wlXJzUdE75qvY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-292-LnN0iGOYOT-D26yHUFonsg-1; Mon, 05 Dec 2022 12:05:11 -0500 X-MC-Unique: LnN0iGOYOT-D26yHUFonsg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A964B894E80; Mon, 5 Dec 2022 17:05:10 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4126C15BA8; Mon, 5 Dec 2022 17:05:07 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 09/13] virtio: add vmstate_virtqueue_element_old Date: Mon, 5 Dec 2022 18:04:32 +0100 Message-Id: <20221205170436.2977336-10-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org VMStateDescription of an inflight descriptor represented in VirtQueueElementOld. TODO: Convert to VirtQueueElement. Signed-off-by: Eugenio Pérez --- include/hw/virtio/virtio.h | 1 + include/migration/vmstate.h | 11 +++++++++++ hw/virtio/virtio.c | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index b4c5163fb0..f3485e5748 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -105,6 +105,7 @@ void *qemu_get_virtqueue_element_from_old(VirtIODevice *vdev, size_t sz); void qemu_put_virtqueue_element_old(const VirtQueueElement *elem, VirtQueueElementOld *data); +extern const VMStateDescription vmstate_virtqueue_element_old; #define VIRTIO_QUEUE_MAX 1024 diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index ad24aa1934..9726d2d09e 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -448,6 +448,17 @@ extern const VMStateInfo vmstate_info_qlist; .offset = vmstate_offset_varray(_state, _field, _type), \ } +#define VMSTATE_VARRAY_UINT32_UNSAFE(_field, _state, _field_num, _version, \ + _info, _type) {\ + .name = (stringify(_field)), \ + .version_id = (_version), \ + .num_offset = vmstate_offset_value(_state, _field_num, uint32_t),\ + .info = &(_info), \ + .size = sizeof(_type), \ + .flags = VMS_VARRAY_UINT32, \ + .offset = vmstate_offset_varray(_state, _field, _type), \ +} + #define VMSTATE_VSTRUCT_TEST(_field, _state, _test, _version, _vmsd, _type, _struct_version) { \ .name = (stringify(_field)), \ .version_id = (_version), \ diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 5ddc49610c..7936fcfec2 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3165,6 +3165,38 @@ static bool virtio_disabled_needed(void *opaque) return vdev->disabled; } +const VMStateDescription vmstate_virtqueue_element_old = { + .name = "virtqueue_element", + .fields = (VMStateField[]) { + VMSTATE_UINT32(index, VirtQueueElementOld), + VMSTATE_UINT32(in_num, VirtQueueElementOld), + VMSTATE_UINT32(out_num, VirtQueueElementOld), + /* + * TODO: Needed for packed + * VMSTATE_UINT16(ndescs, VirtQueueElement), + */ + + VMSTATE_VALIDATE("fit", vq_element_in_range), + VMSTATE_VARRAY_UINT32_UNSAFE(in_addr, VirtQueueElementOld, in_num, 0, + vmstate_info_uint64, hwaddr), + VMSTATE_VARRAY_UINT32_UNSAFE(out_addr, VirtQueueElementOld, out_num, 0, + vmstate_info_uint64, hwaddr), + + /* + * Assume iovec[n] == uint64_t[n*2] + * TODO: Probably need to send each field individually because of + * endianess. + */ + VMSTATE_VARRAY_MULTIPLY(in_sg_64, VirtQueueElementOld, in_num, + sizeof(struct iovec) / sizeof(uint64_t), + vmstate_info_uint64, uint64_t), + VMSTATE_VARRAY_MULTIPLY(out_sg_64, VirtQueueElementOld, out_num, + sizeof(struct iovec) / sizeof(uint64_t), + vmstate_info_uint64, uint64_t), + VMSTATE_END_OF_LIST() + }, +}; + static const VMStateDescription vmstate_virtqueue = { .name = "virtqueue_state", .version_id = 1, From patchwork Mon Dec 5 17:04:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064822 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4ADA2C4332F for ; Mon, 5 Dec 2022 17:06:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Eur-000111-VP; Mon, 05 Dec 2022 12:06:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuI-0000fY-0z for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2Eu8-0002s4-Q8 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VCRp2XXSW3VVFVdJ/+N00VV0aDnfLIiZQIhbteXdvY4=; b=T9XQDtJjG1pLGkf0B5Y2bLUqr+oVjsHB6yJBw+hZXKSZqmbGFF/797oOV7lvwOR4KkLrTd hUrHKpR7FXZDBMJNZeO7M/qQ4Q8qk4arAGNJEVHDReRHb2BY9jMR+vGjc85Fr73OcDazye Uc6wYU8RxIz0oFj3q1Zt3wRfDQxbyMs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-541-VK-D7F7TN8ad9Z93IzL6Lw-1; Mon, 05 Dec 2022 12:05:15 -0500 X-MC-Unique: VK-D7F7TN8ad9Z93IzL6Lw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8255985A588; Mon, 5 Dec 2022 17:05:14 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id F2039C15BA8; Mon, 5 Dec 2022 17:05:10 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 10/13] virtio-net: Migrate vhost inflight descriptors Date: Mon, 5 Dec 2022 18:04:33 +0100 Message-Id: <20221205170436.2977336-11-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org There is currently no data to be migrated, since nothing populates or read the fields on virtio-net. The migration of in-flight descriptors is modelled after the migration of requests in virtio-blk. With some differences: * virtio-blk migrates queue number on each request. Here we only add a vq if it has descriptors to migrate, and then we make all descriptors in an array. * Use of QTAILQ since it works similar to signal the end of the inflight descriptors: 1 for more data, 0 if end. But do it for each vq instead of for each descriptor. * Usage of VMState macros. The fields of descriptors would be way more complicated if we use the VirtQueueElements directly, since there would be a few levels of indirections. Using VirtQueueElementOld for the moment, and migrate to VirtQueueElement for the final patch. TODO: Proper migration versioning TODO: Do not embed vhost-vdpa structs TODO: Migrate the VirtQueueElement, not VirtQueueElementOld. Signed-off-by: Eugenio Pérez --- include/hw/virtio/virtio-net.h | 2 + include/migration/vmstate.h | 11 +++ hw/net/virtio-net.c | 129 +++++++++++++++++++++++++++++++++ 3 files changed, 142 insertions(+) diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index ef234ffe7e..ae7c017ef0 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -151,9 +151,11 @@ typedef struct VirtIONetQueue { QEMUTimer *tx_timer; QEMUBH *tx_bh; uint32_t tx_waiting; + uint32_t tx_inflight_num, rx_inflight_num; struct { VirtQueueElement *elem; } async_tx; + VirtQueueElement **tx_inflight, **rx_inflight; struct VirtIONet *n; } VirtIONetQueue; diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 9726d2d09e..9e0dfef9ee 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -626,6 +626,17 @@ extern const VMStateInfo vmstate_info_qlist; .offset = vmstate_offset_varray(_state, _field, _type), \ } +#define VMSTATE_STRUCT_VARRAY_ALLOC_UINT16(_field, _state, _field_num, \ + _version, _vmsd, _type) { \ + .name = (stringify(_field)), \ + .version_id = (_version), \ + .vmsd = &(_vmsd), \ + .num_offset = vmstate_offset_value(_state, _field_num, uint16_t), \ + .size = sizeof(_type), \ + .flags = VMS_STRUCT | VMS_VARRAY_UINT16 | VMS_ALLOC | VMS_POINTER, \ + .offset = vmstate_offset_pointer(_state, _field, _type), \ +} + #define VMSTATE_STRUCT_VARRAY_ALLOC(_field, _state, _field_num, _version, _vmsd, _type) {\ .name = (stringify(_field)), \ .version_id = (_version), \ diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index aba12759d5..ffd7bf1fc7 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -3077,6 +3077,13 @@ static bool mac_table_doesnt_fit(void *opaque, int version_id) return !mac_table_fits(opaque, version_id); } +typedef struct VirtIONetInflightQueue { + uint16_t idx; + uint16_t num; + QTAILQ_ENTRY(VirtIONetInflightQueue) entry; + VirtQueueElementOld *elems; +} VirtIONetInflightQueue; + /* This temporary type is shared by all the WITH_TMP methods * although only some fields are used by each. */ @@ -3086,6 +3093,7 @@ struct VirtIONetMigTmp { uint16_t curr_queue_pairs_1; uint8_t has_ufo; uint32_t has_vnet_hdr; + QTAILQ_HEAD(, VirtIONetInflightQueue) queues_inflight; }; /* The 2nd and subsequent tx_waiting flags are loaded later than @@ -3231,6 +3239,124 @@ static const VMStateDescription vmstate_virtio_net_rss = { }, }; +static const VMStateDescription vmstate_virtio_net_inflight_queue = { + .name = "virtio-net-device/inflight/queue", + .fields = (VMStateField[]) { + VMSTATE_UINT16(idx, VirtIONetInflightQueue), + VMSTATE_UINT16(num, VirtIONetInflightQueue), + + VMSTATE_STRUCT_VARRAY_ALLOC_UINT16(elems, VirtIONetInflightQueue, num, + 0, vmstate_virtqueue_element_old, + VirtQueueElementOld), + VMSTATE_END_OF_LIST() + }, +}; + +static int virtio_net_inflight_init(void *opaque) +{ + struct VirtIONetMigTmp *tmp = opaque; + + QTAILQ_INIT(&tmp->queues_inflight); + return 0; +} + +static int virtio_net_inflight_pre_save(void *opaque) +{ + struct VirtIONetMigTmp *tmp = opaque; + VirtIONet *net = tmp->parent; + uint16_t curr_queue_pairs = net->multiqueue ? net->curr_queue_pairs : 1; + VirtIONetInflightQueue *qi = g_new0(VirtIONetInflightQueue, + curr_queue_pairs * 2); + + virtio_net_inflight_init(opaque); + for (uint16_t i = 0; i < curr_queue_pairs * 2; ++i) { + VirtIONetQueue *q = &net->vqs[vq2q(i)]; + size_t n = i % 2 ? q->tx_inflight_num : q->rx_inflight_num; + VirtQueueElement **inflight = i % 2 ? q->tx_inflight : q->rx_inflight; + + if (n == 0) { + continue; + } + + qi[i].idx = i; + qi[i].num = n; + qi[i].elems = g_new0(VirtQueueElementOld, n); + for (uint16_t j = 0; j < n; ++j) { + qemu_put_virtqueue_element_old(inflight[j], &qi[i].elems[j]); + } + QTAILQ_INSERT_TAIL(&tmp->queues_inflight, &qi[i], entry); + } + + return 0; +} + +static int virtio_net_inflight_post_save(void *opaque) +{ + struct VirtIONetMigTmp *tmp = opaque; + VirtIONetInflightQueue *qi; + + while ((qi = QTAILQ_FIRST(&tmp->queues_inflight))) { + QTAILQ_REMOVE(&tmp->queues_inflight, qi, entry); + g_free(qi->elems); + g_free(qi); + } + + return 0; +} + +static int virtio_net_inflight_post_load(void *opaque, int version_id) +{ + struct VirtIONetMigTmp *tmp = opaque; + VirtIONet *net = tmp->parent; + uint16_t curr_queue_pairs = net->multiqueue ? net->curr_queue_pairs : 1; + VirtIONetInflightQueue *qi; + + while ((qi = QTAILQ_FIRST(&tmp->queues_inflight))) { + VirtIONetQueue *q = &net->vqs[vq2q(qi->idx)]; + uint32_t *n = qi->idx % 2 ? &q->tx_inflight_num : &q->rx_inflight_num; + VirtQueueElement ***inflight = qi->idx % 2 ? + &q->tx_inflight : &q->rx_inflight; + if (unlikely(qi->num == 0)) { + /* TODO: error message */ + return -1; + } + + if (unlikely(qi->idx > curr_queue_pairs * 2)) { + /* TODO: error message */ + return -1; + } + + *n = qi->num; + *inflight = g_new(VirtQueueElement *, *n); + for (uint16_t j = 0; j < *n; ++j) { + (*inflight)[j] = qemu_get_virtqueue_element_from_old( + &net->parent_obj, &qi->elems[j], + sizeof(VirtQueueElement)); + } + + QTAILQ_REMOVE(&tmp->queues_inflight, qi, entry); + g_free(qi->elems); + g_free(qi); + } + + return 0; +} + +/* TODO: Allocate a temporal per queue / queue element, not all of them! */ +static const VMStateDescription vmstate_virtio_net_inflight = { + .name = "virtio-net-device/inflight", + .pre_save = virtio_net_inflight_pre_save, + .post_save = virtio_net_inflight_post_save, + .pre_load = virtio_net_inflight_init, + .post_load = virtio_net_inflight_post_load, + .fields = (VMStateField[]) { + VMSTATE_QTAILQ_V(queues_inflight, struct VirtIONetMigTmp, 0, + vmstate_virtio_net_inflight_queue, + VirtIONetInflightQueue, entry), + VMSTATE_END_OF_LIST() + }, +}; + static const VMStateDescription vmstate_virtio_net_device = { .name = "virtio-net-device", .version_id = VIRTIO_NET_VM_VERSION, @@ -3279,6 +3405,9 @@ static const VMStateDescription vmstate_virtio_net_device = { vmstate_virtio_net_tx_waiting), VMSTATE_UINT64_TEST(curr_guest_offloads, VirtIONet, has_ctrl_guest_offloads), + /* TODO: Move to subsection */ + VMSTATE_WITH_TMP(VirtIONet, struct VirtIONetMigTmp, + vmstate_virtio_net_inflight), VMSTATE_END_OF_LIST() }, .subsections = (const VMStateDescription * []) { From patchwork Mon Dec 5 17:04:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9B96C4332F for ; Mon, 5 Dec 2022 17:09:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Eut-00012S-A9; Mon, 05 Dec 2022 12:06:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuI-0000fZ-11 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:50 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuE-0002v7-Mj for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YJnDIFDpZAEXu6Ik7SJDneyObvRfod4mGLHDS8td1QM=; b=ItKFAnbMJbNizEy1R8i1CY8Q1ewG52LfpgKNR69D6N17T/3oXAL5eun/hFjDsG222PUnZs dTKJODDZZd7XFPfMDTCP6FhhAFLX5Fq8NlpUfdXc/h37IRpA9svlDxPSKDidIo70/sCY9D iGsCqgXozYhURVhFAWAVQrhqK0oDOZw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-152-qEAnpwi6NFyWCNdo4CfWYA-1; Mon, 05 Dec 2022 12:05:18 -0500 X-MC-Unique: qEAnpwi6NFyWCNdo4CfWYA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 93B23101A528; Mon, 5 Dec 2022 17:05:17 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id C89D1C2C7D9; Mon, 5 Dec 2022 17:05:14 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 11/13] virtio-net: save inflight descriptors at vhost shutdown Date: Mon, 5 Dec 2022 18:04:34 +0100 Message-Id: <20221205170436.2977336-12-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org So they can be migrated in virtio-net Signed-off-by: Eugenio Pérez --- hw/net/vhost_net.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 043058ff43..480f4ac0a1 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -21,6 +21,7 @@ #include "standard-headers/linux/vhost_types.h" #include "hw/virtio/virtio-net.h" +#include "hw/virtio/vhost-vdpa.h" /* TODO remove me */ #include "net/vhost_net.h" #include "qapi/error.h" #include "qemu/error-report.h" @@ -319,6 +320,7 @@ static void vhost_net_stop_one(struct vhost_net *net, VirtIODevice *dev) { struct vhost_vring_file file = { .fd = -1 }; + VirtIONet *n = VIRTIO_NET(dev); if (net->nc->info->type == NET_CLIENT_DRIVER_TAP) { for (file.index = 0; file.index < net->dev.nvqs; ++file.index) { @@ -329,6 +331,26 @@ static void vhost_net_stop_one(struct vhost_net *net, if (net->nc->info->poll) { net->nc->info->poll(net->nc, true); } + + for (size_t i = 0; i < net->dev.nvqs; ++i) { + struct vhost_vdpa *v = net->dev.opaque; + + if (net->dev.nvqs != 2) { + continue; + } + + if (!v->shadow_vqs_enabled) { + continue; + } + + n->vqs[i].rx_inflight = vhost_svq_save_inflight( + g_ptr_array_index(v->shadow_vqs, 0), + &n->vqs[i].rx_inflight_num); + n->vqs[i].tx_inflight = vhost_svq_save_inflight( + g_ptr_array_index(v->shadow_vqs, 1), + &n->vqs[i].tx_inflight_num); + } + vhost_dev_stop(&net->dev, dev, false); if (net->nc->info->stop) { net->nc->info->stop(net->nc); From patchwork Mon Dec 5 17:04:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8F5EBC4332F for ; Mon, 5 Dec 2022 17:09:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Eus-000125-U8; Mon, 05 Dec 2022 12:06:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuJ-0000g4-6w for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuF-0002wy-Oo for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259925; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a3m7wBYpBLUMr2K2HXblSpVto8qY0UVWMj56GggmWVc=; b=DpBFgzZCx8a+Fwno2/s8OqT6v8dLIhgQppMvuXIVF4f2F8+HYLyZMmXNM7nIrxPL9luzV8 aSaoThx5RiOgWL2Jf4t/7bW42nJBsIhV/BgTqr6sMlfB/hMC8TzL0+pd5y6T33HkmCvLwu zQfVrbcN7ngp5ANa7pZf2Ap2fKI4TkM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-604-QPi3xzu9O_Ku2Q05D6JXWA-1; Mon, 05 Dec 2022 12:05:21 -0500 X-MC-Unique: QPi3xzu9O_Ku2Q05D6JXWA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9E8868828C4; Mon, 5 Dec 2022 17:05:20 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id DA1A5C15BA8; Mon, 5 Dec 2022 17:05:17 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 12/13] vhost: expose vhost_svq_add_element Date: Mon, 5 Dec 2022 18:04:35 +0100 Message-Id: <20221205170436.2977336-13-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Needed to inject new inflight elements Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 12 +++++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index e82da4b55c..69910597dd 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -122,6 +122,7 @@ void vhost_svq_push_elem(VhostShadowVirtqueue *svq, int vhost_svq_add(VhostShadowVirtqueue *svq, const struct iovec *out_sg, size_t out_num, const struct iovec *in_sg, size_t in_num, VirtQueueElement *elem); +int vhost_svq_add_element(VhostShadowVirtqueue *svq, VirtQueueElement *elem); size_t vhost_svq_poll(VhostShadowVirtqueue *svq); void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 029ccee957..0668114897 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -271,9 +271,15 @@ int vhost_svq_add(VhostShadowVirtqueue *svq, const struct iovec *out_sg, return 0; } -/* Convenience wrapper to add a guest's element to SVQ */ -static int vhost_svq_add_element(VhostShadowVirtqueue *svq, - VirtQueueElement *elem) +/* + * Add a guest's element to SVQ. SVQ return the element When the device mark + * the descriptor as used, so make sure the guest will understand it. + * + * This function can be used to add the elements after a migration event or in + * the case the device is reset. To add elements that must not be seen by the + * guest use vhost_svq_add. + */ +int vhost_svq_add_element(VhostShadowVirtqueue *svq, VirtQueueElement *elem) { return vhost_svq_add(svq, elem->out_sg, elem->out_num, elem->in_sg, elem->in_num, elem); From patchwork Mon Dec 5 17:04:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 13064830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95B15C4332F for ; Mon, 5 Dec 2022 17:10:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2Euu-00013C-Ax; Mon, 05 Dec 2022 12:06:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuK-0000gE-Hc for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2EuJ-0002zH-0j for qemu-devel@nongnu.org; Mon, 05 Dec 2022 12:05:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670259930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r4aycp4adEJGmL77aJql7l+0FzaWMcJzSCjVB4Nz7k4=; b=dlN+xdNVHXu0QNDgj1dhrtSm+795/jZpEm2NeRlGxVDAAUGrF+EZlFZhkfSX3/PVP41wFS g5cZQQwG+oZyEKguF5af0YUo999taRmkfSm9St0kf5imn4LYEB2XLGgebwobuAQGllANOj zbc86ggi4Kn074wyx5UExNJkbcCZ/TE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-163--MmRbQAjN-G9YZ8ZNzlS8A-1; Mon, 05 Dec 2022 12:05:25 -0500 X-MC-Unique: -MmRbQAjN-G9YZ8ZNzlS8A-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A8C7838164C3; Mon, 5 Dec 2022 17:05:23 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.167]) by smtp.corp.redhat.com (Postfix) with ESMTP id E56BFC15BA8; Mon, 5 Dec 2022 17:05:20 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Liuxiangdong , Stefan Hajnoczi , Jason Wang , Harpreet Singh Anand , Gautam Dawar , Zhu Lingshan , Cindy Lu , Si-Wei Liu , "Michael S. Tsirkin" , "Dr. David Alan Gilbert" , Laurent Vivier , Eli Cohen , Stefano Garzarella , Juan Quintela , Parav Pandit Subject: [RFC PATCH for 8.0 13/13] vdpa: Recover inflight descriptors Date: Mon, 5 Dec 2022 18:04:36 +0100 Message-Id: <20221205170436.2977336-14-eperezma@redhat.com> In-Reply-To: <20221205170436.2977336-1-eperezma@redhat.com> References: <20221205170436.2977336-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Finally recover the inflight descriptors at vhost_net_start. TODO: Abstract it properly instead of using SVQ directly. Signed-off-by: Eugenio Pérez --- hw/net/vhost_net.c | 34 ++++++++++++++++++++++++++++++++++ hw/virtio/vhost-vdpa.c | 11 ----------- 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 480f4ac0a1..9a49046c53 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -428,6 +428,40 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, } } + for (i = 0; i < data_queue_pairs; i++) { + struct vhost_vdpa *v; + + peer = qemu_get_peer(ncs, i); + if (peer->info->type != NET_CLIENT_DRIVER_VHOST_VDPA) { + continue; + } + net = get_vhost_net(peer); + v = net->dev.opaque; + + if (!v->shadow_vqs_enabled) { + return 0; + } + + for (size_t i = 0; i < v->dev->nvqs; ++i) { + VirtIONetQueue *q = &n->vqs[(i + v->dev->vq_index) / 2]; + size_t num = i % 2 ? q->tx_inflight_num : q->rx_inflight_num; + g_autofree VirtQueueElement **inflight = NULL; + + assert(v->dev->vq_index % 2 == 0); + inflight = g_steal_pointer(i % 2 ? &q->tx_inflight + : &q->rx_inflight); + for (size_t j = 0; j < num; ++j) { + int r; + + r = vhost_svq_add_element(g_ptr_array_index(v->shadow_vqs, i), + inflight[j]); + + /* TODO: Proper error handling */ + assert(r == 0); + } + } + } + return 0; err_start: diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 7468e44b87..c54cb82cb5 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1167,18 +1167,7 @@ static int vhost_vdpa_set_vring_base(struct vhost_dev *dev, struct vhost_vring_state *ring) { struct vhost_vdpa *v = dev->opaque; - VirtQueue *vq = virtio_get_queue(dev->vdev, ring->index); - /* - * vhost-vdpa devices does not support in-flight requests. Set all of them - * as available. - * - * TODO: This is ok for networking, but other kinds of devices might - * have problems with these retransmissions. - */ - while (virtqueue_rewind(vq, 1)) { - continue; - } if (v->shadow_vqs_enabled) { /* * Device vring base was set at device start. SVQ base is handled by