From patchwork Tue Dec 27 07:20:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Zhijian Li (Fujitsu)\" via" X-Patchwork-Id: 13082086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C8372C10F1B for ; Tue, 27 Dec 2022 07:21:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pA4Ge-0004md-7l; Tue, 27 Dec 2022 02:20:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pA4GQ-0004jo-HI for qemu-devel@nongnu.org; Tue, 27 Dec 2022 02:20:43 -0500 Received: from szxga02-in.huawei.com ([45.249.212.188]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pA4GL-00078z-Uo for qemu-devel@nongnu.org; Tue, 27 Dec 2022 02:20:39 -0500 Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Nh5cP2fPBzRqJr; Tue, 27 Dec 2022 15:19:01 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 27 Dec 2022 15:20:20 +0800 To: , , , CC: , , , , , , , Longpeng Subject: [PATCH v3 2/3] vhost: configure all host notifiers in a single MR transaction Date: Tue, 27 Dec 2022 15:20:14 +0800 Message-ID: <20221227072015.3134-3-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 In-Reply-To: <20221227072015.3134-1-longpeng2@huawei.com> References: <20221227072015.3134-1-longpeng2@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.188; envelope-from=longpeng2@huawei.com; helo=szxga02-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: "Longpeng(Mike)" X-Patchwork-Original-From: "Longpeng(Mike)" via From: "Zhijian Li (Fujitsu)\" via" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Longpeng This allows the vhost device to batch the setup of all its host notifiers. This significantly reduces the device starting time, e.g. the time spend on enabling notifiers reduce from 376ms to 9.1ms for a VM with 64 vCPUs and 3 vhost-vDPA generic devices (vdpa_sim_blk, 64vq per device) Signed-off-by: Longpeng Reviewed-by: Philippe Mathieu-Daudé --- hw/virtio/vhost.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 5994559da8..064d4abe5c 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1562,16 +1562,25 @@ int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev) return r; } + /* + * Batch all the host notifiers in a single transaction to avoid + * quadratic time complexity in address_space_update_ioeventfds(). + */ + memory_region_transaction_begin(); + for (i = 0; i < hdev->nvqs; ++i) { r = virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i, true); if (r < 0) { error_report("vhost VQ %d notifier binding failed: %d", i, -r); + memory_region_transaction_commit(); vhost_dev_disable_notifiers(hdev, vdev); return r; } } + memory_region_transaction_commit(); + return 0; } @@ -1585,6 +1594,12 @@ void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev) BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); int i, r; + /* + * Batch all the host notifiers in a single transaction to avoid + * quadratic time complexity in address_space_update_ioeventfds(). + */ + memory_region_transaction_begin(); + for (i = 0; i < hdev->nvqs; ++i) { r = virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i, false); @@ -1592,6 +1607,15 @@ void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev) error_report("vhost VQ %d notifier cleanup failed: %d", i, -r); } assert (r >= 0); + } + + /* + * The transaction expects the ioeventfds to be open when it + * commits. Do it now, before the cleanup loop. + */ + memory_region_transaction_commit(); + + for (i = 0; i < hdev->nvqs; ++i) { virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i); } virtio_device_release_ioeventfd(vdev);