From patchwork Wed Jul 19 07:53:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318504 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E6BFBEB64DA for ; Wed, 19 Jul 2023 07:54:42 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM20g-0004cA-Gv; Wed, 19 Jul 2023 03:54:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM20e-0004be-9n for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:08 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM20b-0003Kw-LK for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:07 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-67ef5af0ce8so6615135b3a.2 for ; Wed, 19 Jul 2023 00:54:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753244; x=1692345244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=44dnXMtDUhHeXA9lT48wQSwH+ZuRhBQiZncUCa72Y0E=; b=n3eyh3579R+nhcGBEqy/1xM7wVgV7OZihShX1jHCr+QrxMnR+a0iXefFXoKaNdaVTI C4hpDnjFaUKFZBgSYzXzKmUXJ2H6flGOtxuPANLEXiL2+W9en5iqn7huXRSsTrRfajPu ni+U58zYUmgAAr/h3vV+Jd8RQS5f0Dlg2qSUCuPx7fdIGHRgtJIZ7gGlr8wwTyDwJB/D n1CEf2LhWo3FZOqVOS6W6k5V6mkioiyWu1d3dJbGUlyZKx3Gs2peVUiQJGMjkXuY4938 QqtR6XBIDArpw+DmsNW/OVomsvoGfQoViyeWarb8w4rwKYAe1Kdy1yoIT64ICUbmorQc 1g0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753244; x=1692345244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=44dnXMtDUhHeXA9lT48wQSwH+ZuRhBQiZncUCa72Y0E=; b=W9xSVcL4vCKTaeeuL0flL7zSZplW8g7I/aV4OxRKR/nvVyuxi9GCItFqRw03zhHazA nXnZjdA3t7PVG4v0WS3S7/MVNGJEAuN0IV92IjsFyZd4rbW09gNRT/FmEiavBN1n4YwD 2PdhMmVT4JU0T4J0t8pahcIj6pu+ZnFXGhaRu0Ro9E6VIGb5uwIbXR0qqLyMRBT8KcOE ValnYcGN4QcC/frbuoD0YDL3Vsn1zgDU87KSry1+BS3ty6xHv7Ci1hblyo1Hcu2/3vI4 JR73KoE6Ro7DTXDgb08V5LZmnjg4//4fRGnv3uObt+VrFJDdPjYOZ1pbY+7YnLIthKcM J4Yw== X-Gm-Message-State: ABy/qLb8ei46D36RX7X2pxJ4B6XafxG/iw1XT8I0z2SPJjHrM5x9qYh7 ZB/WKZoCcPUxtGw6OiyNttI= X-Google-Smtp-Source: APBJJlEt5ehx3Qrpf4/NXGZJTp6IPM2m6KApIE+OctgmlMPBdEuLZPp3ChK/5TJxBw8P2+K3LzmdPw== X-Received: by 2002:a05:6a20:9684:b0:11f:c1a1:8c with SMTP id hp4-20020a056a20968400b0011fc1a1008cmr1430773pzc.54.1689753244055; Wed, 19 Jul 2023 00:54:04 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id k67-20020a632446000000b0056001f43726sm2865791pgk.92.2023.07.19.00.54.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:03 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 1/8] vhost: Add argument to vhost_svq_poll() Date: Wed, 19 Jul 2023 15:53:46 +0800 Message-Id: <77c1d8b358644b49992e6dbca55a5c9e62c941a8.1689748694.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=yin31149@gmail.com; helo=mail-pf1-x42d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Next patches in this series will no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. Instead, they will send CVQ state load commands in parallel by polling multiple pending buffers at once. To achieve this, this patch refactoring vhost_svq_poll() to accept a new argument `num`, which allows vhost_svq_poll() to wait for the device to use multiple elements, rather than polling for a single element. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.c | 36 ++++++++++++++++++------------ hw/virtio/vhost-shadow-virtqueue.h | 2 +- net/vhost-vdpa.c | 2 +- 3 files changed, 24 insertions(+), 16 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index 49e5aed931..e731b1d2ea 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -514,29 +514,37 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, } /** - * Poll the SVQ for one device used buffer. + * Poll the SVQ to wait for the device to use the specified number + * of elements and return the total length written by the device. * * This function race with main event loop SVQ polling, so extra * synchronization is needed. * - * Return the length written by the device. + * @svq: The svq + * @num: The number of elements that need to be used */ -size_t vhost_svq_poll(VhostShadowVirtqueue *svq) +size_t vhost_svq_poll(VhostShadowVirtqueue *svq, size_t num) { - int64_t start_us = g_get_monotonic_time(); - uint32_t len = 0; + size_t len = 0; + uint32_t r; - do { - if (vhost_svq_more_used(svq)) { - break; - } + while (num--) { + int64_t start_us = g_get_monotonic_time(); - if (unlikely(g_get_monotonic_time() - start_us > 10e6)) { - return 0; - } - } while (true); + do { + if (vhost_svq_more_used(svq)) { + break; + } + + if (unlikely(g_get_monotonic_time() - start_us > 10e6)) { + return len; + } + } while (true); + + vhost_svq_get_buf(svq, &r); + len += r; + } - vhost_svq_get_buf(svq, &len); return len; } diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index 6efe051a70..5bce67837b 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -119,7 +119,7 @@ void vhost_svq_push_elem(VhostShadowVirtqueue *svq, int vhost_svq_add(VhostShadowVirtqueue *svq, const struct iovec *out_sg, size_t out_num, const struct iovec *in_sg, size_t in_num, VirtQueueElement *elem); -size_t vhost_svq_poll(VhostShadowVirtqueue *svq); +size_t vhost_svq_poll(VhostShadowVirtqueue *svq, size_t num); void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); void vhost_svq_set_svq_call_fd(VhostShadowVirtqueue *svq, int call_fd); diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index dfd271c456..d1dd140bf6 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -625,7 +625,7 @@ static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, size_t out_len, * descriptor. Also, we need to take the answer before SVQ pulls by itself, * when BQL is released */ - return vhost_svq_poll(svq); + return vhost_svq_poll(svq, 1); } static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, From patchwork Wed Jul 19 07:53:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318506 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2F75DC001DC for ; Wed, 19 Jul 2023 07:54:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM20m-0004jl-DE; Wed, 19 Jul 2023 03:54:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM20i-0004eZ-2P for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:12 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM20g-0003MO-9y for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:11 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-666eef03ebdso4397565b3a.1 for ; Wed, 19 Jul 2023 00:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753248; x=1692345248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BQZe+3Ij+Tfx5OUTUv/4byPqpfQF2gQ9QsUZav2/M0A=; b=VdgUet2TCT08Gb793Mjmm7WaUcZqRnYEnoV2mFFOeDjZKjYAPJvQDWge/uvNzYU+Aa 2h6G2Wcm851dnjUI5B6i8d8xzqW2p3QY3s3z0bBBwvuGvmMFcxOaP3BiNivFE0RPz1TA GUaAP151jbkJPKkLMT4Agx4OtQjwJLPUSl5l/SF4S0LUoIamtvz0woTwRSXeQcS6PBCa Whotpn2QMGlGg2JXyqwtqWGSMxVk9fov55mZyrKKNng/E9aofbpJ0PAsFk33oydr/WT9 itSXc1Ck+Ry1shzwewHrNLsNKFGakPOy6YXmbpqbOva/UInT1IeSXk/FZtORFe9X9Sbg ZECg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753248; x=1692345248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BQZe+3Ij+Tfx5OUTUv/4byPqpfQF2gQ9QsUZav2/M0A=; b=gFXsQhUmAfYzxkgFJyKhs1ynwKuMTw03NiTlEH9VGt6pT8W4dKK1xI/WB0F9Pj0H7G 8oEV8HM85SJAgC6in/2v1bjnmEl9DvdFP8cDrxbZsiijz6RrJTFiNe1lRW3lmV+nxoT/ UTpcSaoz/fpx6h6MbrtR73d+g2y3JVzbpQNj/Ac+AtrAyCXU1E9gm571hXdjEu3g1ESO BRkcG0NjDlMAFE//Uhf7Aa2eZ7PhZLA59X/vYjM1AsOQ7xiEEcOSTc3coKIBm32kAs32 aGEx64Pgd3kOJ0ytspEuE7uii6Lnoexi0+bt2qG33aaaa9szRswB+pV1fos11bZdfP+e Is4g== X-Gm-Message-State: ABy/qLaJ0IIqGpCh/L6HwzYU9bBEUy0UpV8CXZwLMYcIT51pKuJyLy/I f1jNgOEK6zUtmetAF6BBpuW+anfhkuLoFw== X-Google-Smtp-Source: APBJJlGKPuyFs85ql0g7vMjtmV8ThfxA832UXY4EDJQ8N8vC5MGFKBV2o5zSs/HlFD8O0zBBUCVQkg== X-Received: by 2002:a05:6a20:12ca:b0:135:8a04:9045 with SMTP id v10-20020a056a2012ca00b001358a049045mr4362331pzg.1.1689753248328; Wed, 19 Jul 2023 00:54:08 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id j24-20020aa78d18000000b006732786b5f1sm2638962pfe.213.2023.07.19.00.54.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:08 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 2/8] vdpa: Use iovec for vhost_vdpa_net_cvq_add() Date: Wed, 19 Jul 2023 15:53:47 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=yin31149@gmail.com; helo=mail-pf1-x429.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Next patches in this series will no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. Consequently, there will be multiple pending buffers in the shadow VirtQueue, making it a must for every control command to have its own buffer. To achieve this, this patch refactor vhost_vdpa_net_cvq_add() to accept `struct iovec`, which eliminates the coupling of control commands to `s->cvq_cmd_out_buffer` and `s->status`, allowing them to use their own buffer. Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index d1dd140bf6..6b16c8ece0 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -596,22 +596,14 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) vhost_vdpa_net_client_stop(nc); } -static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, size_t out_len, - size_t in_len) +static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, + struct iovec *out_sg, size_t out_num, + struct iovec *in_sg, size_t in_num) { - /* Buffers for the device */ - const struct iovec out = { - .iov_base = s->cvq_cmd_out_buffer, - .iov_len = out_len, - }; - const struct iovec in = { - .iov_base = s->status, - .iov_len = sizeof(virtio_net_ctrl_ack), - }; VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); int r; - r = vhost_svq_add(svq, &out, 1, &in, 1, NULL); + r = vhost_svq_add(svq, out_sg, out_num, in_sg, in_num, NULL); if (unlikely(r != 0)) { if (unlikely(r == -ENOSPC)) { qemu_log_mask(LOG_GUEST_ERROR, "%s: No space on device queue\n", @@ -637,6 +629,15 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, .cmd = cmd, }; size_t data_size = iov_size(data_sg, data_num); + /* Buffers for the device */ + struct iovec out = { + .iov_base = s->cvq_cmd_out_buffer, + .iov_len = sizeof(ctrl) + data_size, + }; + struct iovec in = { + .iov_base = s->status, + .iov_len = sizeof(*s->status), + }; assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); @@ -647,8 +648,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, iov_to_buf(data_sg, data_num, 0, s->cvq_cmd_out_buffer + sizeof(ctrl), data_size); - return vhost_vdpa_net_cvq_add(s, data_size + sizeof(ctrl), - sizeof(virtio_net_ctrl_ack)); + return vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); } static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) @@ -1222,9 +1222,7 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, struct iovec out = { .iov_base = s->cvq_cmd_out_buffer, }; - /* in buffer used for device model */ - const struct iovec in = { - .iov_base = &status, + struct iovec in = { .iov_len = sizeof(status), }; ssize_t dev_written = -EINVAL; @@ -1232,6 +1230,8 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0, s->cvq_cmd_out_buffer, vhost_vdpa_net_cvq_cmd_page_len()); + /* In buffer used for the vdpa device */ + in.iov_base = s->status; ctrl = s->cvq_cmd_out_buffer; if (ctrl->class == VIRTIO_NET_CTRL_ANNOUNCE) { @@ -1260,7 +1260,7 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, goto out; } } else { - dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status)); + dev_written = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); if (unlikely(dev_written < 0)) { goto out; } @@ -1276,6 +1276,8 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, } status = VIRTIO_NET_ERR; + /* In buffer used for the device model */ + in.iov_base = &status; virtio_net_handle_ctrl_iov(svq->vdev, &in, 1, &out, 1); if (status != VIRTIO_NET_OK) { error_report("Bad CVQ processing in model"); From patchwork Wed Jul 19 07:53:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95174C001B0 for ; Wed, 19 Jul 2023 07:54:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM20r-00052Q-0R; Wed, 19 Jul 2023 03:54:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM20n-0004q8-7a for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:17 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM20l-0003Mz-Bj for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:16 -0400 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1b89bc52cd1so37227535ad.1 for ; Wed, 19 Jul 2023 00:54:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753254; x=1692345254; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NXg2eGLSXh1jDorZn4NnX3a2Wq1PPpMjkHiWexWxLa0=; b=LzdYdP57YMKCpRIhgFXpMVf41uhMofbDV1NpRTDYS7FbXk6ilBNzx1OaHPXUQzVKNC G77DzIi1ykx+cxX17cotO8OnPzRFyDEscuqAFH1it3BKTXGjtNWxOjqbUYLuUWlQVC7D db49LX/Q7iZqXppOOr6JMc435OwnYcdcNn6GvtD1O0BwF3CbCdNGDNIXUuNvBZARyueG 641cmCXCDFY16vJf+tXrO4Ep821zgd2HqfSAVrenb8XLMbV5zXZiBAeTJGKCLPV042bx gXWYgOIzvkO46BTtuj8vvPtojwDd3ff7r50UtYHwwQ/+/H0PEkYpeAQNv5asEFlroUVo xT7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753254; x=1692345254; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NXg2eGLSXh1jDorZn4NnX3a2Wq1PPpMjkHiWexWxLa0=; b=G1jBe12TanpZ7FHmeJ2EvIUL+t69ec+E7AZeFia2Kf2m9l7gTUnRfwu+4VPEG2Y1jE MDBSqJIZ0pGVRJnG2DdSW09lUPUxQF82Qf2sREpqe3mR3AQQHt72xeXE8UsZ+e+vQdFV S4r31QjM6v8nWhj0iGUTe16Na5Qv0T6rWyxspamPldvbbtXQHELsC44XVqidrucKDVPW 7ojQoYgiiFUoW1lvWCZlL23/4MFSOBMIzV1frqYyuH5WZV2boEt54eu3MlhMI6cc2a8J If3RxYUppBiTP/6jWpu7BRmInAtiFy+QbhBkHmrt7MJo6uXRTNExKYxwJKnwWP7CNIBH fnzw== X-Gm-Message-State: ABy/qLbowrCKS4KPQMERJV0c5gqeW9No2F1Q8RJAeSBBtejVAdeiHxU8 eWzpH8ISLrkXMVuXxD2pGIk= X-Google-Smtp-Source: APBJJlF+pCCcdzSY4TdhpPEyUt4CqdJnQAhlLE8hQyE/Q93dBPGfsP66CRcoAoMRaZ1RVjnO9pUtHQ== X-Received: by 2002:a17:902:b20b:b0:1b1:9272:55f3 with SMTP id t11-20020a170902b20b00b001b1927255f3mr13280691plr.66.1689753253724; Wed, 19 Jul 2023 00:54:13 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id iw10-20020a170903044a00b001b9ce2c3baesm3202094plb.143.2023.07.19.00.54.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:13 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 3/8] vhost: Expose vhost_svq_available_slots() Date: Wed, 19 Jul 2023 15:53:48 +0800 Message-Id: <110db78638438360da7e049343b06ea58af2a195.1689748694.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=yin31149@gmail.com; helo=mail-pl1-x62c.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Next patches in this series will delay the polling and checking of buffers until either the SVQ is full or control commands shadow buffers are full, no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. To achieve this, this patch exposes vhost_svq_available_slots() and introduces a helper function, allowing QEMU to know whether the SVQ is full. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.c | 2 +- hw/virtio/vhost-shadow-virtqueue.h | 1 + net/vhost-vdpa.c | 9 +++++++++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index e731b1d2ea..fc5f408f77 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -66,7 +66,7 @@ bool vhost_svq_valid_features(uint64_t features, Error **errp) * * @svq: The svq */ -static uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) +uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) { return svq->num_free; } diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index 5bce67837b..19c842a15b 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -114,6 +114,7 @@ typedef struct VhostShadowVirtqueue { bool vhost_svq_valid_features(uint64_t features, Error **errp); +uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq); void vhost_svq_push_elem(VhostShadowVirtqueue *svq, const VirtQueueElement *elem, uint32_t len); int vhost_svq_add(VhostShadowVirtqueue *svq, const struct iovec *out_sg, diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 6b16c8ece0..dd71008e08 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -620,6 +620,13 @@ static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, return vhost_svq_poll(svq, 1); } +/* Convenience wrapper to get number of available SVQ descriptors */ +static uint16_t vhost_vdpa_net_svq_available_slots(VhostVDPAState *s) +{ + VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); + return vhost_svq_available_slots(svq); +} + static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, uint8_t cmd, const struct iovec *data_sg, size_t data_num) @@ -640,6 +647,8 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, }; assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); + /* Each CVQ command has one out descriptor and one in descriptor */ + assert(vhost_vdpa_net_svq_available_slots(s) >= 2); /* pack the CVQ command header */ memcpy(s->cvq_cmd_out_buffer, &ctrl, sizeof(ctrl)); From patchwork Wed Jul 19 07:53:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EECA9C001DC for ; Wed, 19 Jul 2023 07:55:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM20s-0005AX-Sq; Wed, 19 Jul 2023 03:54:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM20r-00054P-CC for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:21 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM20p-0003NI-Q5 for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:21 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1b9c5e07c1bso52119775ad.2 for ; Wed, 19 Jul 2023 00:54:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753258; x=1692345258; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=x1AR3/MDpipxhoPzV6bjNpu4TYaX+Mc8H1OEub4GIXs=; b=WgRe7+E9mgqhAAbxkLAISMm5H+IzXLol7EgX50QnHYRAYUUMs+UY8wMyXOCOF4gyTr Bbj/19I017ynfcKJcAFSQ0WfBUINt/L+FB1Wbx4y1ZjxaRr3GFe9vliWlzxcQYswlfzW FvkI554ne06uNvL9Qx0mkRBYib29shkHMZfNk5hEooHTaiHk2jOxWWwe9IM99KZHvjJM OFMa7+cEI0PjA0HriJM0Zic7zZC+vth1sN3bM3kPG9uUnz3fr1PeVvhnhCUeaTFYU8fU vtOagSxIm9TBsfMjwvDxTeg183Nc/pVXeIZJCHgB8+bUznCiakfub0shwZojdLQo5DqO IKnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753258; x=1692345258; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x1AR3/MDpipxhoPzV6bjNpu4TYaX+Mc8H1OEub4GIXs=; b=i6FYBaSRTofxvSeJW4TM41uIQG8oys7EY1XNWA5hgtId7tNpAKj62+/TvUS3M4Nznr F8r7YyEK9j6hGY5u1wcoffohfLTlFa9RGtRNEuUBy688AignK3LZzxRIZr7+Psqq/l72 /S81AVGy4+wdITgA0V/WohC7LH5QKrfv8qzUg0njvX7X5/vdXzUem57DyfZ6US10barw eGmOiMLSbTpIEtC/HSKM3oxW639HEAt2UeDM6z26OqKq21QOkwint0OiY/Q46/Onwwlq eDMLf2PvJOWKS3aK0wB9+It3nWm5poH0e9l/cbvy78kORiIYfu1rIyY94XIESDuSuSF0 mqCQ== X-Gm-Message-State: ABy/qLbTs3oTbJUx0b4YEHpgS+VxxczveBuX92RV99tTqwQfRXO01T+o Nq1VKlyg81+SBy7FxU5mVqdt6BEJZ+rbMA== X-Google-Smtp-Source: APBJJlGISrIVHqtTzJmEoKkCFdkh312yOWIdofe97DP3L31i57YHpfR5kv90CKG6be0yr/8D4UtRMQ== X-Received: by 2002:a17:902:e88d:b0:1b0:f8:9b2d with SMTP id w13-20020a170902e88d00b001b000f89b2dmr19979122plg.29.1689753258128; Wed, 19 Jul 2023 00:54:18 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id i4-20020a17090332c400b001b1c3542f57sm3236847plr.103.2023.07.19.00.54.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:17 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 4/8] vdpa: Avoid using vhost_vdpa_net_load_*() outside vhost_vdpa_net_load() Date: Wed, 19 Jul 2023 15:53:49 +0800 Message-Id: <428a8fac2a29b37757fa15ca747be93c0226cb1f.1689748694.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=yin31149@gmail.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Next patches in this series will refactor vhost_vdpa_net_load_cmd() to iterate through the control commands shadow buffers, allowing QEMU to send CVQ state load commands in parallel at device startup. Considering that QEMU always forwards the CVQ command serialized outside of vhost_vdpa_net_load(), it is more elegant to send the CVQ commands directly without invoking vhost_vdpa_net_load_*() helpers. Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index dd71008e08..ae8f59adaa 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -1098,12 +1098,14 @@ static NetClientInfo net_vhost_vdpa_cvq_info = { */ static int vhost_vdpa_net_excessive_mac_filter_cvq_add(VhostVDPAState *s, VirtQueueElement *elem, - struct iovec *out) + struct iovec *out, + struct iovec *in) { struct virtio_net_ctrl_mac mac_data, *mac_ptr; struct virtio_net_ctrl_hdr *hdr_ptr; uint32_t cursor; ssize_t r; + uint8_t on = 1; /* parse the non-multicast MAC address entries from CVQ command */ cursor = sizeof(*hdr_ptr); @@ -1151,7 +1153,16 @@ static int vhost_vdpa_net_excessive_mac_filter_cvq_add(VhostVDPAState *s, * filter table to the vdpa device, it should send the * VIRTIO_NET_CTRL_RX_PROMISC CVQ command to enable promiscuous mode */ - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_PROMISC, 1); + cursor = 0; + hdr_ptr = out->iov_base; + out->iov_len = sizeof(*hdr_ptr) + sizeof(on); + assert(out->iov_len < vhost_vdpa_net_cvq_cmd_page_len()); + + hdr_ptr->class = VIRTIO_NET_CTRL_RX; + hdr_ptr->cmd = VIRTIO_NET_CTRL_RX_PROMISC; + cursor += sizeof(*hdr_ptr); + *(uint8_t *)(out->iov_base + cursor) = on; + r = vhost_vdpa_net_cvq_add(s, out, 1, in, 1); if (unlikely(r < 0)) { return r; } @@ -1264,7 +1275,7 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, * the CVQ command direclty. */ dev_written = vhost_vdpa_net_excessive_mac_filter_cvq_add(s, elem, - &out); + &out, &in); if (unlikely(dev_written < 0)) { goto out; } From patchwork Wed Jul 19 07:53:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68642C001DC for ; Wed, 19 Jul 2023 07:55:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM214-0005L3-AC; Wed, 19 Jul 2023 03:54:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM20v-0005H2-K7 for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:26 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM20t-0003Ox-W7 for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:25 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6686c74183cso6598356b3a.1 for ; Wed, 19 Jul 2023 00:54:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753262; x=1692345262; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TzUWFm6jl8VZ4HfXUTybQ0jJBYc+cOQnZhY89ahOgk4=; b=WH3c5vJUUVSkeYk31VYhuZEQq7qrcKVzApwL+HeSAAvcc+WvBAzf+9pTQSeL2pFMgR zdXEXn716wCUfta+QPlG9HIuZ0ymib2DUZ4K3AZYPbz6ukv+h6oC4u4ylVCWd2I+nQME CfIyNRkrxzhMxKDkPaI+NBDQMrbOcbEoxQOnnOoBk4e/v8QqkQKKlXNQeFv4geEemzpU QUZpCfg5V1E3FPCXTBqVPmveIzxD7g3IFYw7Qw3klRHZJvQNcNYwkX+p6AAvCfvLuAFJ G5LGX5pZ+EeYIoEw7vYNoeUuUmdXmp1Cukx6GHKEnGgvxxmrAVbi0/UDoQbUbBS2M5Lk WS+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753262; x=1692345262; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TzUWFm6jl8VZ4HfXUTybQ0jJBYc+cOQnZhY89ahOgk4=; b=B6qCB8LpcWUWq7eChbPaXDinfYfLfJ7+qKx1NupvZwSzhE3mUQx0CtBhuzb+CKCAsH 74ur27I0NJOjoaFrJbvWImHCIaUBobXkBl2rMs2JSlrPQuARrTXhiKRhqLbo6Opi8HwB e0+un4lkghaIi/pbXgCGzhbc3j5vNYrAxoE69DJ5HVumu9lcuec6zWowwZQK2+ckLvJV Yov6OFV9lgO88APHVDYafdtqg2cQgWfzsbBh+1igGoS1twz7iVmpzex9r15pvUYiW1gM ye/AjOUP/r0CMwFKNuWSHneSKW5wuAKwJn58WtX+kV9JRhL/D/sJcsq//cRIDkU0Dog/ 9bYw== X-Gm-Message-State: ABy/qLaLvvaz7lsCTpST6XDrs9U3TjFT1bjtiSRc5CJyi12E66mqfpkn y2ecT6tebeybGle0xBmGMVeYi1728C8iMQ== X-Google-Smtp-Source: APBJJlFMutHhHrJ1s3r/kJ8GjIlisRmVrNrnoCBdRonsGqlOc48DeduDFooUQQL+FG583nLfS9v4Qw== X-Received: by 2002:a05:6a20:2452:b0:137:2204:4ea6 with SMTP id t18-20020a056a20245200b0013722044ea6mr2186260pzc.27.1689753262512; Wed, 19 Jul 2023 00:54:22 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id a16-20020a62e210000000b006528d9080f6sm2685943pfi.9.2023.07.19.00.54.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:22 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 5/8] vdpa: Check device ack in vhost_vdpa_net_load_rx_mode() Date: Wed, 19 Jul 2023 15:53:50 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=yin31149@gmail.com; helo=mail-pf1-x42c.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Considering that vhost_vdpa_net_load_rx_mode() is only called within vhost_vdpa_net_load_rx() now, this patch refactors vhost_vdpa_net_load_rx_mode() to include a check for the device's ack, simplifying the code and improving its maintainability. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- net/vhost-vdpa.c | 76 ++++++++++++++++++++---------------------------- 1 file changed, 31 insertions(+), 45 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index ae8f59adaa..fe0ba19724 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -814,14 +814,24 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, .iov_base = &on, .iov_len = sizeof(on), }; - return vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_RX, - cmd, &data, 1); + ssize_t dev_written; + + dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_RX, + cmd, &data, 1); + if (unlikely(dev_written < 0)) { + return dev_written; + } + if (*s->status != VIRTIO_NET_OK) { + return -EIO; + } + + return 0; } static int vhost_vdpa_net_load_rx(VhostVDPAState *s, const VirtIONet *n) { - ssize_t dev_written; + ssize_t r; if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_CTRL_RX)) { return 0; @@ -846,13 +856,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (!n->mac_table.uni_overflow && !n->promisc) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_PROMISC, 0); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_PROMISC, 0); + if (unlikely(r < 0)) { + return r; } } @@ -874,13 +880,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->mac_table.multi_overflow || n->allmulti) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_ALLMULTI, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLMULTI, 1); + if (unlikely(r < 0)) { + return r; } } @@ -899,13 +901,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->alluni) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_ALLUNI, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLUNI, 1); + if (r < 0) { + return r; } } @@ -920,13 +918,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nomulti) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_NOMULTI, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOMULTI, 1); + if (r < 0) { + return r; } } @@ -941,13 +935,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nouni) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_NOUNI, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOUNI, 1); + if (r < 0) { + return r; } } @@ -962,13 +952,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nobcast) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_NOBCAST, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOBCAST, 1); + if (r < 0) { + return r; } } From patchwork Wed Jul 19 07:53:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318508 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71B9BEB64DA for ; Wed, 19 Jul 2023 07:55:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM21C-0005VQ-IN; Wed, 19 Jul 2023 03:54:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM210-0005Jm-QM for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:30 -0400 Received: from mail-oi1-x22f.google.com ([2607:f8b0:4864:20::22f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM20y-0003PX-5t for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:29 -0400 Received: by mail-oi1-x22f.google.com with SMTP id 5614622812f47-3a4875e65a3so5235b6e.2 for ; Wed, 19 Jul 2023 00:54:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753267; x=1692345267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dBd2+B9vCIc7vkdhzGUOSRbK80lFgHx/r36BXg7Em/I=; b=aBcAXbpELndhZ4uX7NTb4HcYTJZPEJ7/Bb1NuqkxoL57bn3lMfZHCluATzHYh5YUMS 37Mwk1liEOeIKMMJNzZTeZYClIcuTrcdwhFuxVDagg7HlVm9Dq05P1zhVko46WvdefM0 Mv36X1+h1NTgfpzUHu1UGdunqAnkDZS1dqTqsU3KnL4NIGnAshVl24T9saa2EmM6FAwE iKntoLMZHmNxaDshABzolo6x86JsDHYyB/ZPU75BAhdSqxyQgQBkmI+CUH8FEcfnmaKC LtNNNjXZcBNh5yPA3IgAN9w1JIGM5QjT8JezZrBRjrhx2ym77nwnxdhnKMbeYRajwUSB KeBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753267; x=1692345267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dBd2+B9vCIc7vkdhzGUOSRbK80lFgHx/r36BXg7Em/I=; b=Ddecs02oJhhS/wPMgCsVPnIBMPdQH4ajZIBGrAJooXcS724vx7z0V1Pl861Y4IARko k4GWh/wtnpi1svouG4TkU4jBc2aaJl+5tftTUAKDvthJK6EjZEB/EY5nVkOa8LTDNsgl jWI7/hi7iFqx1TyXwM8DYL/Y3LujEAS2lqlYDgTjNHRAMzzlcr+VSsH7SnXdPjXGMQfB vIuKja0Gk1+xvwuS/CRmaG2idtRbLSJBQPSS3Buwyl/S323BapZLF7KmzafbUULHf8zX /MSoDtVisRgbk/OhdqMX2Ufe0f6Lml7dcko05AfWUMnVZsVvuWe2KkeTNUAVcTdCdrX3 4JBg== X-Gm-Message-State: ABy/qLYqhzCMZ6IyTax34zupw68NKwHujG5eaxFiPmEHgHjP/ocKlUlO tgqYF2BpYMq5a0KmUpSLL2A= X-Google-Smtp-Source: APBJJlEwqdTaWGxQzygXj2FAt1UYIaAZGRIft7bwTBt+YsZwbCsO9bzTsJIyLhu+4Pi1+QfQgtoYXg== X-Received: by 2002:a05:6808:2186:b0:39c:a986:953a with SMTP id be6-20020a056808218600b0039ca986953amr2150258oib.34.1689753266767; Wed, 19 Jul 2023 00:54:26 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id e6-20020a17090a818600b00262e485156esm658511pjn.57.2023.07.19.00.54.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:26 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 6/8] vdpa: Move vhost_svq_poll() to the caller of vhost_vdpa_net_cvq_add() Date: Wed, 19 Jul 2023 15:53:51 +0800 Message-Id: <152177c4e7082236fba9d31d535e40f8c2984349.1689748694.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22f; envelope-from=yin31149@gmail.com; helo=mail-oi1-x22f.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This patch moves vhost_svq_poll() to the caller of vhost_vdpa_net_cvq_add() and introduces a helper funtion. By making this change, next patches in this series is able to refactor vhost_vdpa_net_load_x() only to delay the polling and checking process until either the SVQ is full or control commands shadow buffers are full. Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 50 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index fe0ba19724..d06f38403f 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -609,15 +609,21 @@ static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, qemu_log_mask(LOG_GUEST_ERROR, "%s: No space on device queue\n", __func__); } - return r; } - /* - * We can poll here since we've had BQL from the time we sent the - * descriptor. Also, we need to take the answer before SVQ pulls by itself, - * when BQL is released - */ - return vhost_svq_poll(svq, 1); + return r; +} + +/* + * Convenience wrapper to poll SVQ for multiple control commands. + * + * Caller should hold the BQL when invoking this function, and should take + * the answer before SVQ pulls by itself when BQL is released. + */ +static ssize_t vhost_vdpa_net_svq_poll(VhostVDPAState *s, size_t cmds_in_flight) +{ + VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); + return vhost_svq_poll(svq, cmds_in_flight); } /* Convenience wrapper to get number of available SVQ descriptors */ @@ -645,6 +651,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, .iov_base = s->status, .iov_len = sizeof(*s->status), }; + ssize_t r; assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); /* Each CVQ command has one out descriptor and one in descriptor */ @@ -657,7 +664,16 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, iov_to_buf(data_sg, data_num, 0, s->cvq_cmd_out_buffer + sizeof(ctrl), data_size); - return vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); + r = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); + if (unlikely(r < 0)) { + return r; + } + + /* + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + return vhost_vdpa_net_svq_poll(s, 1); } static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) @@ -1152,6 +1168,12 @@ static int vhost_vdpa_net_excessive_mac_filter_cvq_add(VhostVDPAState *s, if (unlikely(r < 0)) { return r; } + + /* + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + vhost_vdpa_net_svq_poll(s, 1); if (*s->status != VIRTIO_NET_OK) { return sizeof(*s->status); } @@ -1266,10 +1288,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, goto out; } } else { - dev_written = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); - if (unlikely(dev_written < 0)) { + ssize_t r; + r = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); + if (unlikely(r < 0)) { + dev_written = r; goto out; } + + /* + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + dev_written = vhost_vdpa_net_svq_poll(s, 1); } if (unlikely(dev_written < sizeof(status))) { From patchwork Wed Jul 19 07:53:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 861E0EB64DA for ; Wed, 19 Jul 2023 07:56:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM21E-0005cX-1X; Wed, 19 Jul 2023 03:54:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM214-0005PD-OG for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:38 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM212-0003QC-N8 for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:34 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6689430d803so4467272b3a.0 for ; Wed, 19 Jul 2023 00:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753271; x=1692345271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kax3kZAFUArGF7G0Eefgbu80wcduxB0sSZaLz99i4/w=; b=MwaT8sRLJxkkShb4xfnrHRXvq+R85TGXUr/A6xi2eegnjmmXIW/8FX49jl+aLs1mid SgWg5tEzIxCPFHtTLwSIP20OO4R7Tjyznrau5j+Y//S2TIFnso4OC2liMhqe3D8l+km+ L/eIMiqG0uv8BTmmOPMhhcATJOLYiM5aDZrhugbfw8yTxDG3owF68iFXLR4b73OnxNqz E9V5xUI6bmp1XC1k9CVDJrHbP3P0PFXHHCkeSjWILLS6M+Lnu5SUbtmt7MvwskVnUexF 5iJujhDNSdnnecKfjT3ZRkHRpg7jjAU9lN3CFJXJgK0gjd/WJi2bZ6mH9xBQRD30klSz I1Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753271; x=1692345271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kax3kZAFUArGF7G0Eefgbu80wcduxB0sSZaLz99i4/w=; b=P3tpskeYhb7ri6nqotJjk6Mo6Y8C65xRdBEP28oJcw80zy/AFfTFHpbUCZZBBzM2rq By+jWoI/5su/TLx54xi/cXOeshNAL2f5i2h4w4swB/okh1McDgay/9BJxrK73e3As2sX JTzesGY0gtn3lqa8hrTLRq6Mkdf4yZkslPIG04s+Ypj9IHJDYgUJviKAweKP4q8N2EPa YeXBey1ttPAfN/ORhqdI+vIF2pbx57b2wcXuilVWq84is7YXbmvldsQ8cEpICbqSWmdD 7NFaofXYh3Oh25RIMcTvVjmpifKz1DBmkH+izVMZoSZbx8TTHWu38ptAmWOFUxWSP++P bBLA== X-Gm-Message-State: ABy/qLb5QaGuwhaUaIUJdwEifn+FlgDWRA+gHe+q8Ap/puGxcx4lpotG b1rLXMoAhHqlePa2tyE4ZrI= X-Google-Smtp-Source: APBJJlFnR9xSsebEzXfPnwyKnus63+cClM7+nCUPU88UJzITZnh07tSzG6J3AQeBhZBMJtchPr57SQ== X-Received: by 2002:a05:6a00:b56:b0:67f:e74a:d309 with SMTP id p22-20020a056a000b5600b0067fe74ad309mr20469348pfo.30.1689753271060; Wed, 19 Jul 2023 00:54:31 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id j10-20020aa783ca000000b0063f2a5a59d1sm2635231pfn.190.2023.07.19.00.54.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:30 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 7/8] vdpa: Add cursors to vhost_vdpa_net_loadx() Date: Wed, 19 Jul 2023 15:53:52 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=yin31149@gmail.com; helo=mail-pf1-x429.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This patch adds `out_cursor` and `in_cursor` arguments to vhost_vdpa_net_loadx(). By making this change, next patches in this series can refactor vhost_vdpa_net_load_cmd() directly to iterate through the control commands shadow buffers, allowing QEMU to send CVQ state load commands in parallel at device startup. Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 79 ++++++++++++++++++++++++++++++------------------ 1 file changed, 50 insertions(+), 29 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index d06f38403f..795c9c1fd2 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -633,7 +633,8 @@ static uint16_t vhost_vdpa_net_svq_available_slots(VhostVDPAState *s) return vhost_svq_available_slots(svq); } -static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, +static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, void **out_cursor, + void **in_cursor, uint8_t class, uint8_t cmd, const struct iovec *data_sg, size_t data_num) { @@ -644,11 +645,11 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, size_t data_size = iov_size(data_sg, data_num); /* Buffers for the device */ struct iovec out = { - .iov_base = s->cvq_cmd_out_buffer, + .iov_base = *out_cursor, .iov_len = sizeof(ctrl) + data_size, }; struct iovec in = { - .iov_base = s->status, + .iov_base = *in_cursor, .iov_len = sizeof(*s->status), }; ssize_t r; @@ -658,11 +659,11 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, assert(vhost_vdpa_net_svq_available_slots(s) >= 2); /* pack the CVQ command header */ - memcpy(s->cvq_cmd_out_buffer, &ctrl, sizeof(ctrl)); + memcpy(out.iov_base, &ctrl, sizeof(ctrl)); /* pack the CVQ command command-specific-data */ iov_to_buf(data_sg, data_num, 0, - s->cvq_cmd_out_buffer + sizeof(ctrl), data_size); + out.iov_base + sizeof(ctrl), data_size); r = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); if (unlikely(r < 0)) { @@ -676,14 +677,16 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, return vhost_vdpa_net_svq_poll(s, 1); } -static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) +static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, + void **out_cursor, void **in_cursor) { if (virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_CTRL_MAC_ADDR)) { const struct iovec data = { .iov_base = (void *)n->mac, .iov_len = sizeof(n->mac), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_MAC, + ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MAC, VIRTIO_NET_CTRL_MAC_ADDR_SET, &data, 1); if (unlikely(dev_written < 0)) { @@ -735,7 +738,7 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) .iov_len = mul_macs_size, }, }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, + ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, VIRTIO_NET_CTRL_MAC, VIRTIO_NET_CTRL_MAC_TABLE_SET, data, ARRAY_SIZE(data)); @@ -750,7 +753,8 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) } static int vhost_vdpa_net_load_mq(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + void **out_cursor, void **in_cursor) { struct virtio_net_ctrl_mq mq; ssize_t dev_written; @@ -764,7 +768,8 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, .iov_base = &mq, .iov_len = sizeof(mq), }; - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_MQ, + dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MQ, VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, &data, 1); if (unlikely(dev_written < 0)) { @@ -778,7 +783,8 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, } static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + void **out_cursor, void **in_cursor) { uint64_t offloads; ssize_t dev_written; @@ -809,7 +815,8 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, .iov_base = &offloads, .iov_len = sizeof(offloads), }; - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_GUEST_OFFLOADS, + dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_GUEST_OFFLOADS, VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET, &data, 1); if (unlikely(dev_written < 0)) { @@ -823,6 +830,7 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, } static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, + void **out_cursor, void **in_cursor, uint8_t cmd, uint8_t on) { @@ -832,8 +840,8 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, }; ssize_t dev_written; - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_RX, - cmd, &data, 1); + dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX, cmd, &data, 1); if (unlikely(dev_written < 0)) { return dev_written; } @@ -845,7 +853,8 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, } static int vhost_vdpa_net_load_rx(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + void **out_cursor, void **in_cursor) { ssize_t r; @@ -872,7 +881,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (!n->mac_table.uni_overflow && !n->promisc) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_PROMISC, 0); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_PROMISC, 0); if (unlikely(r < 0)) { return r; } @@ -896,7 +906,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->mac_table.multi_overflow || n->allmulti) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLMULTI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_ALLMULTI, 1); if (unlikely(r < 0)) { return r; } @@ -917,7 +928,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->alluni) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLUNI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_ALLUNI, 1); if (r < 0) { return r; } @@ -934,7 +946,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nomulti) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOMULTI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_NOMULTI, 1); if (r < 0) { return r; } @@ -951,7 +964,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nouni) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOUNI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_NOUNI, 1); if (r < 0) { return r; } @@ -968,7 +982,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nobcast) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOBCAST, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_NOBCAST, 1); if (r < 0) { return r; } @@ -979,13 +994,15 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s, const VirtIONet *n, + void **out_cursor, void **in_cursor, uint16_t vid) { const struct iovec data = { .iov_base = &vid, .iov_len = sizeof(vid), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_VLAN, + ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_VLAN, VIRTIO_NET_CTRL_VLAN_ADD, &data, 1); if (unlikely(dev_written < 0)) { @@ -999,7 +1016,8 @@ static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s, } static int vhost_vdpa_net_load_vlan(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + void **out_cursor, void **in_cursor) { int r; @@ -1010,7 +1028,8 @@ static int vhost_vdpa_net_load_vlan(VhostVDPAState *s, for (int i = 0; i < MAX_VLAN >> 5; i++) { for (int j = 0; n->vlans[i] && j <= 0x1f; j++) { if (n->vlans[i] & (1U << j)) { - r = vhost_vdpa_net_load_single_vlan(s, n, (i << 5) + j); + r = vhost_vdpa_net_load_single_vlan(s, n, out_cursor, + in_cursor, (i << 5) + j); if (unlikely(r != 0)) { return r; } @@ -1028,6 +1047,8 @@ static int vhost_vdpa_net_load(NetClientState *nc) struct vhost_vdpa *v = &s->vhost_vdpa; const VirtIONet *n; int r; + void *out_cursor = s->cvq_cmd_out_buffer, + *in_cursor = s->status; assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA); @@ -1036,23 +1057,23 @@ static int vhost_vdpa_net_load(NetClientState *nc) } n = VIRTIO_NET(v->dev->vdev); - r = vhost_vdpa_net_load_mac(s, n); + r = vhost_vdpa_net_load_mac(s, n, &out_cursor, &in_cursor); if (unlikely(r < 0)) { return r; } - r = vhost_vdpa_net_load_mq(s, n); + r = vhost_vdpa_net_load_mq(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } - r = vhost_vdpa_net_load_offloads(s, n); + r = vhost_vdpa_net_load_offloads(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } - r = vhost_vdpa_net_load_rx(s, n); + r = vhost_vdpa_net_load_rx(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } - r = vhost_vdpa_net_load_vlan(s, n); + r = vhost_vdpa_net_load_vlan(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } From patchwork Wed Jul 19 07:53:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 13318510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21B83C001B0 for ; Wed, 19 Jul 2023 07:55:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qM21D-0005aK-Mz; Wed, 19 Jul 2023 03:54:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qM21A-0005VV-4U for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:42 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qM218-0003Qc-3X for qemu-devel@nongnu.org; Wed, 19 Jul 2023 03:54:39 -0400 Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-55b0e7efb1cso3806342a12.1 for ; Wed, 19 Jul 2023 00:54:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689753276; x=1692345276; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9hVkRZxgxwyHhUOFXwXUro/N/dlyZ2PieUblJXKdidQ=; b=i1afV9QtspKjBN0XBDmMR2ylTNo5KzTBnqB7wlR7T8F4vPZoiY12F4NyqiFuZc8l8L l47O5uZfmvytnFqjwAVQBDTIkLK1YzbU6TaRymFbCGBACeGcfSYKrtUnmls0YWkNR1iQ pwp9fqGDj5CNydXUPTtRPVtogP4/v7IsPeC4fYBWJz3fYJjqVhHtv0wul053GOODhPF1 qyNsM8ImpuPta7yEtI+TyfQ6M28JQilCg0GFr/MkoHgwrxItdLEc/9WHWYy2EzBppDS5 6UMIwx7hDzE4UDRijagJNfMIctnEJFC7oi5t5YGTixmxoeUyxXpmK09IGSnyrazkrtnx PRgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689753276; x=1692345276; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9hVkRZxgxwyHhUOFXwXUro/N/dlyZ2PieUblJXKdidQ=; b=bcZsW4Rqo0fGTAq3KDQDjXkYJSqCzwn1rn1zLKxtl+u++MANdEcP7NNLeTXphfjTrl RPvfbFJ8pax5MvSnCs6iaBloGo7i7AetXO11Zu40SC+pM4Hpqqaa+5d34xw5z/pziaPl lO4tWvB3Hii+hOOK0S1Y5LejABdwe5/XGakXeXNJMtffe+y+ZiU628nLY7b0bKcXWDSN 3Wxjj08zU0zgymAJaAskavqsYgbsOCr6fTezkfuisP0cz+a/KnR0Cvjyz6VLPXFbT8qE bamCUV/pAipEJVKoWWFlq8ESKUvEbB6KbMmJQnailvoh5DZ6HrsidjZ5SEqLZglkgdNj p5WA== X-Gm-Message-State: ABy/qLY+lE+7IDXzdix7Fqe2hDQzFXPlfDM46V7qu8zwR+OzmfIHq2+v StNEd4Fo0HL5ZWxZ1Jd2rBc= X-Google-Smtp-Source: APBJJlEdvkFVmDs7pgwnbs4ZmWJw4NMHDL2M0vvwwuKUjuB4sVUKXnTbyKcOcsoZyG+ObPzeR4WQBw== X-Received: by 2002:a17:90a:990e:b0:267:6586:f39a with SMTP id b14-20020a17090a990e00b002676586f39amr15089251pjp.47.1689753276475; Wed, 19 Jul 2023 00:54:36 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id y10-20020a17090a1f4a00b0026094c23d0asm736822pjy.17.2023.07.19.00.54.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jul 2023 00:54:36 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v3 8/8] vdpa: Send cvq state load commands in parallel Date: Wed, 19 Jul 2023 15:53:53 +0800 Message-Id: <3a002790e6c880af928c6470ecbf03e7c65a68bb.1689748694.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=yin31149@gmail.com; helo=mail-pg1-x52b.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This patch enables sending CVQ state load commands in parallel at device startup by following steps: * Refactor vhost_vdpa_net_load_cmd() to iterate through the control commands shadow buffers. This allows different CVQ state load commands to use their own unique buffers. * Delay the polling and checking of buffers until either the SVQ is full or control commands shadow buffers are full. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1578 Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 157 +++++++++++++++++++++++++++++------------------ 1 file changed, 96 insertions(+), 61 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 795c9c1fd2..1ebb58f7f6 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -633,6 +633,26 @@ static uint16_t vhost_vdpa_net_svq_available_slots(VhostVDPAState *s) return vhost_svq_available_slots(svq); } +/* + * Poll SVQ for multiple pending control commands and check the device's ack. + * + * Caller should hold the BQL when invoking this function. + */ +static ssize_t vhost_vdpa_net_svq_flush(VhostVDPAState *s, + size_t cmds_in_flight) +{ + vhost_vdpa_net_svq_poll(s, cmds_in_flight); + + /* Device should and must use only one byte ack each control command */ + assert(cmds_in_flight < vhost_vdpa_net_cvq_cmd_page_len()); + for (int i = 0; i < cmds_in_flight; ++i) { + if (s->status[i] != VIRTIO_NET_OK) { + return -EIO; + } + } + return 0; +} + static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, void **out_cursor, void **in_cursor, uint8_t class, uint8_t cmd, const struct iovec *data_sg, @@ -642,19 +662,41 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, void **out_cursor, .class = class, .cmd = cmd, }; - size_t data_size = iov_size(data_sg, data_num); + size_t data_size = iov_size(data_sg, data_num), + left_bytes = vhost_vdpa_net_cvq_cmd_page_len() - + (*out_cursor - s->cvq_cmd_out_buffer); /* Buffers for the device */ struct iovec out = { - .iov_base = *out_cursor, .iov_len = sizeof(ctrl) + data_size, }; struct iovec in = { - .iov_base = *in_cursor, .iov_len = sizeof(*s->status), }; ssize_t r; - assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); + if (sizeof(ctrl) > left_bytes || data_size > left_bytes - sizeof(ctrl) || + vhost_vdpa_net_svq_available_slots(s) < 2) { + /* + * It is time to flush all pending control commands if SVQ is full + * or control commands shadow buffers are full. + * + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + r = vhost_vdpa_net_svq_flush(s, *in_cursor - (void *)s->status); + if (unlikely(r < 0)) { + return r; + } + + *out_cursor = s->cvq_cmd_out_buffer; + *in_cursor = s->status; + left_bytes = vhost_vdpa_net_cvq_cmd_page_len(); + } + + out.iov_base = *out_cursor; + in.iov_base = *in_cursor; + + assert(data_size <= left_bytes - sizeof(ctrl)); /* Each CVQ command has one out descriptor and one in descriptor */ assert(vhost_vdpa_net_svq_available_slots(s) >= 2); @@ -670,11 +712,11 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, void **out_cursor, return r; } - /* - * We can poll here since we've had BQL from the time - * we sent the descriptor. - */ - return vhost_vdpa_net_svq_poll(s, 1); + /* iterate the cursors */ + *out_cursor += out.iov_len; + *in_cursor += in.iov_len; + + return 0; } static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, @@ -685,15 +727,12 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, .iov_base = (void *)n->mac, .iov_len = sizeof(n->mac), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_MAC, - VIRTIO_NET_CTRL_MAC_ADDR_SET, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + ssize_t r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MAC, + VIRTIO_NET_CTRL_MAC_ADDR_SET, + &data, 1); + if (unlikely(r < 0)) { + return r; } } @@ -738,15 +777,12 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, .iov_len = mul_macs_size, }, }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + ssize_t r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, VIRTIO_NET_CTRL_MAC, VIRTIO_NET_CTRL_MAC_TABLE_SET, data, ARRAY_SIZE(data)); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + if (unlikely(r < 0)) { + return r; } return 0; @@ -757,7 +793,7 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, void **out_cursor, void **in_cursor) { struct virtio_net_ctrl_mq mq; - ssize_t dev_written; + ssize_t r; if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_MQ)) { return 0; @@ -768,15 +804,12 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, .iov_base = &mq, .iov_len = sizeof(mq), }; - dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_MQ, - VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MQ, + VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, + &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -787,7 +820,7 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, void **out_cursor, void **in_cursor) { uint64_t offloads; - ssize_t dev_written; + ssize_t r; if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)) { @@ -815,15 +848,12 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, .iov_base = &offloads, .iov_len = sizeof(offloads), }; - dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_GUEST_OFFLOADS, - VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_GUEST_OFFLOADS, + VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET, + &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -838,15 +868,12 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, .iov_base = &on, .iov_len = sizeof(on), }; - ssize_t dev_written; + ssize_t r; - dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_RX, cmd, &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX, cmd, &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -1001,15 +1028,12 @@ static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s, .iov_base = &vid, .iov_len = sizeof(vid), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_VLAN, - VIRTIO_NET_CTRL_VLAN_ADD, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (unlikely(*s->status != VIRTIO_NET_OK)) { - return -EIO; + ssize_t r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_VLAN, + VIRTIO_NET_CTRL_VLAN_ADD, + &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -1078,6 +1102,17 @@ static int vhost_vdpa_net_load(NetClientState *nc) return r; } + /* + * We need to poll and check all pending device's used buffers. + * + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + r = vhost_vdpa_net_svq_flush(s, in_cursor - (void *)s->status); + if (unlikely(r)) { + return r; + } + return 0; }