From patchwork Tue Apr 18 22:56:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pei Li X-Patchwork-Id: 13216266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 05616C77B76 for ; Wed, 19 Apr 2023 00:40:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1povrU-0000ec-Ee; Tue, 18 Apr 2023 20:39:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pouFi-0001AP-Ht for qemu-devel@nongnu.org; Tue, 18 Apr 2023 18:56:46 -0400 Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pouFg-0006sT-P6 for qemu-devel@nongnu.org; Tue, 18 Apr 2023 18:56:46 -0400 Received: by mail-qv1-xf2c.google.com with SMTP id me15so10619214qvb.4 for ; Tue, 18 Apr 2023 15:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681858599; x=1684450599; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=QCFwWrX25kJSvBvwNyNTdbJhUgeLifkPPy7L2EURcFU=; b=pb+/mMg0swMc6/VKuDWzcU4D8nOFgCcDd8EPfg6McX8BI1bjwPmbvyKbVl4M80Q2kN 1szHKzv0Y68S0a5F2fQML3/MdzRbNzXAyv46Yg/V1/XHW6tFRZcUYRuEhj91ZG5o/LdE 1yPLtXjTBNdWrgfFskyefn9l21aV/l/O5cZDXU215LS87XQ1VNcvv5Y+an45axMApFlL Oroa99uq09hXZ1XtGKL3Yu3E61B9njsPRMWIXiciWACBVohqdaPWs+24xnqi87+mFhlo TNhdj8MGnmxIo1UTuaP7Nf+eXjKp4ozphcOxEaWLBOUrV/B20CQBGdRNpXyhylOz6VTF KrEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681858599; x=1684450599; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QCFwWrX25kJSvBvwNyNTdbJhUgeLifkPPy7L2EURcFU=; b=hOdLivuogUv04xGhDYLXlSTgANfjj7JQBevPUoAOLmQCg1qYemv3xYezUCV0X8LUW9 yy9IWvKHqYDSBo0Gz1oO9+iH04Vlpo9e9On2qvFH5wusldyFjLuMSUR40csmRlNQLRuV /NE+jl6P1U1cotIFf+UUAX85yvDmTv/y1pxYvRVtNcUAwmZkvnF+aFucAmPQEmfrOSkD 5TsJXAErlwEkQyVaVo3YlHS/ioEgcV3MyW48Phg5CpWh8FWcTkJP+axWWs2AGLIPv6Al mxrY1wHu0qS8S64uk0Y97skSByayXfxvji9bqYt0KPs54YqRiymSf8w/8o4CI6fgvwA2 9ZHw== X-Gm-Message-State: AAQBX9deGHV+xYpJELTHtSpeaKlL6VM4iTyMtZorR15Yv2u8T2l1tnnE DSHFvWrafOnF7DDxat1LhpNEiYOkgc8= X-Google-Smtp-Source: AKy350ZeTLBIyoq+/GiwPLZq+bS/A1Czd1BmoDtQ0yRnwIahXLvPtFNlQP8TCazUi+uM/tjhXwycvw== X-Received: by 2002:a05:6214:29e1:b0:5ef:8b22:69a1 with SMTP id jv1-20020a05621429e100b005ef8b2269a1mr9941418qvb.12.1681858598977; Tue, 18 Apr 2023 15:56:38 -0700 (PDT) Received: from entropy.pdl.local.cmu.edu (NAT1.PDL.CMU.EDU. [128.2.147.192]) by smtp.gmail.com with ESMTPSA id r2-20020a0cf802000000b005ef5b1006c5sm3163243qvn.38.2023.04.18.15.56.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 15:56:38 -0700 (PDT) From: peili.dev@gmail.com To: qemu-devel@nongnu.org Cc: eperezma@redhat.com, Pei Li Subject: [PATCH 1/2] Reduce vdpa initialization / startup overhead Date: Tue, 18 Apr 2023 18:56:37 -0400 Message-Id: <20230418225638.1467969-1-peili.dev@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f2c; envelope-from=peili.dev@gmail.com; helo=mail-qv1-xf2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 18 Apr 2023 20:39:50 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Pei Li Currently, part of the vdpa initialization / startup process needs to trigger many ioctls per vq, which is very inefficient and causing unnecessary context switch between user mode and kernel mode. This patch creates an additional ioctl() command, namely VHOST_VDPA_GET_VRING_GROUP_BATCH, that will batching commands of VHOST_VDPA_GET_VRING_GROUP into a single ioctl() call. Signed-off-by: Pei Li --- hw/virtio/vhost-vdpa.c | 31 +++++++++++++++----- include/standard-headers/linux/vhost_types.h | 3 ++ linux-headers/linux/vhost.h | 7 +++++ 3 files changed, 33 insertions(+), 8 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index bc6bad23d5..6d45ff8539 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -679,7 +679,8 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) uint64_t f = 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | - 0x1ULL << VHOST_BACKEND_F_SUSPEND; + 0x1ULL << VHOST_BACKEND_F_SUSPEND | + 0x1ULL << VHOST_BACKEND_F_IOCTL_BATCH; int r; if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { @@ -731,14 +732,28 @@ static int vhost_vdpa_get_vq_index(struct vhost_dev *dev, int idx) static int vhost_vdpa_set_vring_ready(struct vhost_dev *dev) { - int i; + int i, nvqs = dev->nvqs; + uint64_t backend_features = dev->backend_cap; + trace_vhost_vdpa_set_vring_ready(dev); - for (i = 0; i < dev->nvqs; ++i) { - struct vhost_vring_state state = { - .index = dev->vq_index + i, - .num = 1, - }; - vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state); + + if (!(backend_features & BIT_ULL(VHOST_BACKEND_F_IOCTL_BATCH))) { + for (i = 0; i < nvqs; ++i) { + struct vhost_vring_state state = { + .index = dev->vq_index + i, + .num = 1, + }; + vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state); + } + } else { + struct vhost_vring_state states[nvqs + 1]; + states[0].num = nvqs; + for (i = 1; i <= nvqs; ++i) { + states[i].index = dev->vq_index + i - 1; + states[i].num = 1; + } + + vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE_BATCH, &states[0]); } return 0; } diff --git a/include/standard-headers/linux/vhost_types.h b/include/standard-headers/linux/vhost_types.h index c41a73fe36..068d0e1ceb 100644 --- a/include/standard-headers/linux/vhost_types.h +++ b/include/standard-headers/linux/vhost_types.h @@ -164,4 +164,7 @@ struct vhost_vdpa_iova_range { /* Device can be suspended */ #define VHOST_BACKEND_F_SUSPEND 0x4 +/* IOCTL requests can be batched */ +#define VHOST_BACKEND_F_IOCTL_BATCH 0x6 + #endif diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h index f9f115a7c7..4c9ddd0a0e 100644 --- a/linux-headers/linux/vhost.h +++ b/linux-headers/linux/vhost.h @@ -180,4 +180,11 @@ */ #define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D) +/* Batch version of VHOST_VDPA_SET_VRING_ENABLE + * + * Enable/disable the ring while batching the commands. + */ +#define VHOST_VDPA_SET_VRING_ENABLE_BATCH _IOW(VHOST_VIRTIO, 0x7F, \ + struct vhost_vring_state) + #endif From patchwork Tue Apr 18 22:56:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pei Li X-Patchwork-Id: 13216267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B05C0C6FD18 for ; Wed, 19 Apr 2023 00:40:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1povrV-0000ed-Th; Tue, 18 Apr 2023 20:39:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pouFi-0001AQ-HG for qemu-devel@nongnu.org; Tue, 18 Apr 2023 18:56:46 -0400 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pouFg-0006sW-P1 for qemu-devel@nongnu.org; Tue, 18 Apr 2023 18:56:46 -0400 Received: by mail-qv1-xf35.google.com with SMTP id l17so10357486qvq.10 for ; Tue, 18 Apr 2023 15:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681858599; x=1684450599; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FiycUZJ0nXkvvRNvHas9yRNRXmm9AKIGo6at3LAejXU=; b=jKZ29ICHvQdVinMG9D2x4mD4Wb+Fbtp12cVBNUQHON9KOLbEYdndMfFVfzpV5htg1e dnKct4vWF78E3HbVpKt7F6rDwzJNuuZ8UZeThaMKhivCWkHXbHPaorSFQWaICzYeTzfm Xqof1njemQ6+/HB/97jCBPoe03vzKV9hsnrwR4GdcS+DDRW31qGCISW3+/+4LGRChiC1 Ye+QW9Y+89TQ+RbPq2XgjU/GDI3EI3n9xYfDSozyWLVTFramPlWzDLdWhcLhAlWlfjp+ bShL8dKnox0jBZMO1uvbtNeAnTJdQ9nTILR30+1/j8qTahBwKYswBmPxZwRhvHBMt35x +Lgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681858599; x=1684450599; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FiycUZJ0nXkvvRNvHas9yRNRXmm9AKIGo6at3LAejXU=; b=CDCBlVlXytRoCg8liY4yS8XrNO5Yf5BgKl4lrBZpADm2XLhSCw8bcD18i+Yxm9bu8S 6gb84z5vQXLjQqX+C6cciiWjvfoE538xik6NaBcAxS9NRgnob45MyRa2OyXcB2JZnS2z 155iJusLqZWDNF4sEtTkfTI+D4qITmA96xxyvGtQdJJ7AEFWDwFy8S4F5LT0g2fGHHbO XKCe1uLwz1tYC1zsxI3WjDHRfomSoS9KpfovSE2Z63nrreRQxl7YSneWvFYOIP/3i76v t0RDOwRuAZMM3a/k7nG28L3EmGGLPi4q9OYPLEDoJiEG7wRKIdjiPRzrT9IVQKpAq/GQ UdEQ== X-Gm-Message-State: AAQBX9ccZHdsE+q3qdp+IgkE7mVVVCCa+Yp2pbIAR/T4newkgpJlL9VO 2ZGwK8WGOtUDrJI2bqmi9KBzZZ9rLVg= X-Google-Smtp-Source: AKy350aDLR+IuTGXOnZnk2KNGb/51/PivzZTC8wZhFVVRBiU/NbMVqXlkT09FJN5rWdqtOe5VX9XeA== X-Received: by 2002:a05:6214:501b:b0:56e:bfda:a138 with SMTP id jo27-20020a056214501b00b0056ebfdaa138mr28667006qvb.10.1681858599536; Tue, 18 Apr 2023 15:56:39 -0700 (PDT) Received: from entropy.pdl.local.cmu.edu (NAT1.PDL.CMU.EDU. [128.2.147.192]) by smtp.gmail.com with ESMTPSA id r2-20020a0cf802000000b005ef5b1006c5sm3163243qvn.38.2023.04.18.15.56.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 15:56:39 -0700 (PDT) From: peili.dev@gmail.com To: qemu-devel@nongnu.org Cc: eperezma@redhat.com, Pei Li Subject: [PATCH 2/2] Reduce vdpa initialization / startup overhead Date: Tue, 18 Apr 2023 18:56:38 -0400 Message-Id: <20230418225638.1467969-2-peili.dev@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230418225638.1467969-1-peili.dev@gmail.com> References: <20230418225638.1467969-1-peili.dev@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::f35; envelope-from=peili.dev@gmail.com; helo=mail-qv1-xf35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 18 Apr 2023 20:39:48 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Pei Li Currently, part of the vdpa initialization / startup process needs to trigger many ioctls per vq, which is very inefficient and causing unnecessary context switch between user mode and kernel mode. This patch creates an additional ioctl() command, namely VHOST_VDPA_SET_VRING_ENABLE_BATCH, that will batching commands of VHOST_VDPA_SET_VRING_ENABLE_BATCH into a single ioctl() call. Signed-off-by: Pei Li --- linux-headers/linux/vhost.h | 10 ++++++ net/vhost-vdpa.c | 70 +++++++++++++++++++++++++++++++------ 2 files changed, 70 insertions(+), 10 deletions(-) diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h index 4c9ddd0a0e..f7cfa324c4 100644 --- a/linux-headers/linux/vhost.h +++ b/linux-headers/linux/vhost.h @@ -187,4 +187,14 @@ #define VHOST_VDPA_SET_VRING_ENABLE_BATCH _IOW(VHOST_VIRTIO, 0x7F, \ struct vhost_vring_state) +/* Batch version of VHOST_VDPA_GET_VRING_GROUP + * + * Get the group for a virtqueue: read index, write group in num, + * The virtqueue index is stored in the index field of + * vhost_vring_state. The group for this specific virtqueue is + * returned via num field of vhost_vring_state while batching commands. + */ +#define VHOST_VDPA_GET_VRING_GROUP_BATCH _IOWR(VHOST_VIRTIO, 0x82, \ + struct vhost_vring_state) + #endif diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 99904a0da7..ed4f2d5c49 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -377,6 +377,47 @@ static int64_t vhost_vdpa_get_vring_group(int device_fd, unsigned vq_index) return state.num; } +static int64_t vhost_vdpa_get_vring_group_batch(int device_fd, unsigned vq_index) +{ + int r; + struct vhost_vring_state states[vq_index + 1]; + int64_t cvq_group; + + states[0].num = vq_index; + + for (int i = 1; i <= vq_index; ++i) { + states[i].index = i - 1; + } + + r = ioctl(device_fd, VHOST_VDPA_GET_VRING_GROUP_BATCH, &states[0]); + + if (unlikely(r < 0)) { + error_report("Cannot get VQ %d group: %s", vq_index - 1, + g_strerror(errno)); + return r; + } + + cvq_group = states[vq_index].num; + + if (unlikely(cvq_group < 0)) { + return cvq_group; + } + + for (int i = 1; i < vq_index; ++i) { + int64_t group = states[i].num; + + if (unlikely(group < 0)) { + return group; + } + + if (group == cvq_group) { + return 0; + } + } + + return vq_index; +} + static int vhost_vdpa_set_address_space_id(struct vhost_vdpa *v, unsigned vq_group, unsigned asid_num) @@ -512,19 +553,28 @@ static int vhost_vdpa_net_cvq_start(NetClientState *nc) * than the last vq. VQ group of last group passed in cvq_group. */ cvq_index = v->dev->vq_index_end - 1; - cvq_group = vhost_vdpa_get_vring_group(v->device_fd, cvq_index); - if (unlikely(cvq_group < 0)) { - return cvq_group; - } - for (int i = 0; i < cvq_index; ++i) { - int64_t group = vhost_vdpa_get_vring_group(v->device_fd, i); - if (unlikely(group < 0)) { - return group; + if (! (backend_features & BIT_ULL(VHOST_BACKEND_F_IOCTL_BATCH))) { + cvq_group = vhost_vdpa_get_vring_group(v->device_fd, cvq_index); + if (unlikely(cvq_group < 0)) { + return cvq_group; } + for (int i = 0; i < cvq_index; ++i) { + int64_t group = vhost_vdpa_get_vring_group(v->device_fd, i); - if (group == cvq_group) { - return 0; + if (unlikely(group < 0)) { + return group; + } + + if (group == cvq_group) { + return 0; + } + } + } else { + cvq_group = vhost_vdpa_get_vring_group_batch(v->device_fd, cvq_index + 1); + + if (unlikely(cvq_group <= 0)) { + return cvq_group; } }