From patchwork Sun Oct 8 09:12:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13412662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97671E95A82 for ; Sun, 8 Oct 2023 09:14:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qpPqO-0001nv-Mf; Sun, 08 Oct 2023 05:13:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qpPqN-0001nN-J3 for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:12:59 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qpPqK-0002Dr-ME for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:12:59 -0400 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c61bde0b4bso30754515ad.3 for ; Sun, 08 Oct 2023 02:12:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1696756375; x=1697361175; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KtjO1lS5KyQU3h+Hf7ZgeU0iuEx8e6d6jY/0St+hkG8=; b=v5cXyPRQkFpnHlsmB8Dn1Anoie+sHD0D8sUyHFT6/zzzKfF2uu9HSbwfB0n92453R4 7pYCQyPdxgNsorhuCiWKIwuF4Rfh9cjHE8g2AKyZYuZ90sVTVXhAqHb87RodAvpVBDZZ HibNppArgOM2o6PAc5s5xFlco8I4SUVbpiVCqvnTOCiUCqMkMMBITt3bHC++6zftqHTR 8yJ8EN8xXGucgRBVtyNV+7ub3lTJo84QXJIL4ZFhouWUWlOfzSRYi9FclbOo7Vuh2XMI oJXFo9AMmh6WRelJILre/CCWg9YeVHdzLcvNyGqcgrjOSQZWHY9bSSKtssxDh9L5VpoX 3g4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696756375; x=1697361175; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KtjO1lS5KyQU3h+Hf7ZgeU0iuEx8e6d6jY/0St+hkG8=; b=u5RabmRf2uR4DK8WvTFqbQQIyibHv7b9OIBmdGgbKAyVHrlzBdpFuGCWlkvEMEXSbJ qElPnIQn5XLe8/09mLrEcogDFzTeEuXilFVlCYpllbiTPumOlL+FAG+05abAAzS3OFpb B+esoTVXjRgzE8IPlWv+C6yAl3vnTUggV3TNxfMymNVqkXFX12EVHg1Fn94yUUZfRcbM wvh8JSILj0LgMNU1y2viJQ0VtaH8+xmE8Gjp/iwWW/Hcike/9EUBC3vf4ALj9k2FGF0k S9lmC1RTZL1n7g8Skjn9jCecyF/ZB8wnHFoscLLvUwYsVYslErSrPD0VFq8/cfcY/0PU HmyQ== X-Gm-Message-State: AOJu0Yz0jJK1QuKkb9YllpWiHCZsh/w2ZjqLvtuTJwsDvkduUL42CPBn ddBjSLIvQUeDQuE/Ll7ksfUXAw== X-Google-Smtp-Source: AGHT+IG8t8Y8W45PdvoJOlZM5uJkz7QgMahd7vmW1LTACA67FyV2QbQRKM888LaQiM+tsDTfzKZP4Q== X-Received: by 2002:a17:903:2611:b0:1c6:2fc0:9626 with SMTP id jd17-20020a170903261100b001c62fc09626mr11866290plb.6.1696756375321; Sun, 08 Oct 2023 02:12:55 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id e8-20020a17090301c800b001b8a3e2c241sm7118084plh.14.2023.10.08.02.12.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 02:12:54 -0700 (PDT) From: Li Feng To: Markus Armbruster , Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v7 1/5] vhost-user-common: send get_inflight_fd once Date: Sun, 8 Oct 2023 17:12:01 +0800 Message-ID: <20231008091220.870171-2-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231008091220.870171-1-fengli@smartx.com> References: <20231008091220.870171-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::62e; envelope-from=fengli@smartx.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Currently the get_inflight_fd will be sent every time the device is started, and the backend will allocate shared memory to save the inflight state. If the backend finds that it receives the second get_inflight_fd, it will release the previous shared memory, which breaks inflight working logic. This patch is a preparation for the following patches. Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/scsi/vhost-scsi-common.c | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a06f01af26..a61cd0e907 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -52,20 +52,28 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vsc->dev.acked_features = vdev->guest_features; - assert(vsc->inflight == NULL); - vsc->inflight = g_new0(struct vhost_inflight, 1); - ret = vhost_dev_get_inflight(&vsc->dev, - vs->conf.virtqueue_size, - vsc->inflight); + ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error get inflight: %d", -ret); + error_report("Error setting inflight format: %d", -ret); goto err_guest_notifiers; } - ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); - if (ret < 0) { - error_report("Error set inflight: %d", -ret); - goto err_guest_notifiers; + if (vsc->inflight) { + if (!vsc->inflight->addr) { + ret = vhost_dev_get_inflight(&vsc->dev, + vs->conf.virtqueue_size, + vsc->inflight); + if (ret < 0) { + error_report("Error getting inflight: %d", -ret); + goto err_guest_notifiers; + } + } + + ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); + if (ret < 0) { + error_report("Error setting inflight: %d", -ret); + goto err_guest_notifiers; + } } ret = vhost_dev_start(&vsc->dev, vdev, true); @@ -85,9 +93,6 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) return ret; err_guest_notifiers: - g_free(vsc->inflight); - vsc->inflight = NULL; - k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&vsc->dev, vdev); @@ -111,12 +116,6 @@ void vhost_scsi_common_stop(VHostSCSICommon *vsc) } assert(ret >= 0); - if (vsc->inflight) { - vhost_dev_free_inflight(vsc->inflight); - g_free(vsc->inflight); - vsc->inflight = NULL; - } - vhost_dev_disable_notifiers(&vsc->dev, vdev); } From patchwork Sun Oct 8 09:12:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13412658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9078AE95A82 for ; Sun, 8 Oct 2023 09:14:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qpPqS-0001oz-3K; Sun, 08 Oct 2023 05:13:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qpPqQ-0001oQ-AS for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:02 -0400 Received: from mail-oa1-x29.google.com ([2001:4860:4864:20::29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qpPqO-0002F9-JZ for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:02 -0400 Received: by mail-oa1-x29.google.com with SMTP id 586e51a60fabf-1dd0526b5easo2665156fac.2 for ; Sun, 08 Oct 2023 02:13:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1696756379; x=1697361179; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=flzDQE1QokZHW+hOGv8gceXlgLhnm4s9nj6hmmZpjyU=; b=kd4nd0bz7nwvnO8riBuxItF5pYns0BqmyEyEAXwB8QlO3uBSk9VZ/V4qA/l6xvM0Xj bqHOTdhxzwG6xAe+Cw5wT2CywZRETTVKwhNBxd2UozOKl3i4fssFwyFkkje56FCHycgw jt0dYuwfTW7EkKFxbDOUTEhwV8Y7/gXW6MscchLuewDITMzWiDHRIITii5jj+1vhU4Ta AIh5FdcpbnrDQ/3mKAWmh1zR1t5xLWHgXcKXmjZ9hbVuHt/WVMcX71KAr1djK+vLkyFu bRPZ6x14H3Ajd9xdWcM8Qfz0p2Wy9wYstm7csslKTItGPpiO6DtzC+QLwdx4+0tMAiB7 U6NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696756379; x=1697361179; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=flzDQE1QokZHW+hOGv8gceXlgLhnm4s9nj6hmmZpjyU=; b=ia9swhfO4xBNn88clKIR96xgqnBObU6kqTAvBS1sAraDcAGp0KpPq+AmpUTRkSNrJY cdJSlrIbIbDKiPE0J1I+k1aj8JIx1TliT/jnLdFYt8arP1GjUrplIBDWWZ1s+kD6pdra DiQflJfRAFXb7cu5YFJghAzlp56RHOkOt+OQv1VhR8yMRbfJ2LNk1eE8DI3Ny7dw4yEE 6qizIRNtLkAik6MgPR6DStPl+hW9bnyJn9CLPdwNFtBN1rZ+ZK/RINtl46ZoOAAqGZDy q9POCOnPY6vH70zSd0JpfboagPedB59HAr6vhiCdjuZ/qTcfPAn4CY0S7YVddbHruNff f9Iw== X-Gm-Message-State: AOJu0YzC1hr08QVt9AxtOutLOGGYbSxOboNeRnRTU55iwkl+v911lmYb CvPVKef7jM+noDjmnRNXEQa6Kg== X-Google-Smtp-Source: AGHT+IH3e7RcEtzZbdxvibUthCX9bMwDv9vVMTT4YRhgCPI//2hDfJbgomWS/BmuQvpGBbJ2ZgOYhg== X-Received: by 2002:a05:6870:898c:b0:1d7:1533:6869 with SMTP id f12-20020a056870898c00b001d715336869mr14792038oaq.31.1696756379070; Sun, 08 Oct 2023 02:12:59 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id e8-20020a17090301c800b001b8a3e2c241sm7118084plh.14.2023.10.08.02.12.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 02:12:58 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v7 2/5] vhost: move and rename the conn retry times Date: Sun, 8 Oct 2023 17:12:02 +0800 Message-ID: <20231008091220.870171-3-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231008091220.870171-1-fengli@smartx.com> References: <20231008091220.870171-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2001:4860:4864:20::29; envelope-from=fengli@smartx.com; helo=mail-oa1-x29.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Multiple devices need this macro, move it to a common header. Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/block/vhost-user-blk.c | 4 +--- hw/virtio/vhost-user-gpio.c | 3 +-- include/hw/virtio/vhost.h | 2 ++ 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index eecf3f7a81..3c69fa47d5 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -32,8 +32,6 @@ #include "sysemu/sysemu.h" #include "sysemu/runstate.h" -#define REALIZE_CONNECTION_RETRIES 3 - static const int user_feature_bits[] = { VIRTIO_BLK_F_SIZE_MAX, VIRTIO_BLK_F_SEG_MAX, @@ -482,7 +480,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) s->inflight = g_new0(struct vhost_inflight, 1); s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues); - retries = REALIZE_CONNECTION_RETRIES; + retries = VU_REALIZE_CONN_RETRIES; assert(!*errp); do { if (*errp) { diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c index 3d7fae3984..fc784e4213 100644 --- a/hw/virtio/vhost-user-gpio.c +++ b/hw/virtio/vhost-user-gpio.c @@ -15,7 +15,6 @@ #include "standard-headers/linux/virtio_ids.h" #include "trace.h" -#define REALIZE_CONNECTION_RETRIES 3 #define VHOST_NVQS 2 /* Features required from VirtIO */ @@ -365,7 +364,7 @@ static void vu_gpio_device_realize(DeviceState *dev, Error **errp) qemu_chr_fe_set_handlers(&gpio->chardev, NULL, NULL, vu_gpio_event, NULL, dev, NULL, true); - retries = REALIZE_CONNECTION_RETRIES; + retries = VU_REALIZE_CONN_RETRIES; g_assert(!*errp); do { if (*errp) { diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 6a173cb9fa..ca3131b1af 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -8,6 +8,8 @@ #define VHOST_F_DEVICE_IOTLB 63 #define VHOST_USER_F_PROTOCOL_FEATURES 30 +#define VU_REALIZE_CONN_RETRIES 3 + /* Generic structures common for any vhost based device. */ struct vhost_inflight { From patchwork Sun Oct 8 09:12:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13412659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95600E95A67 for ; Sun, 8 Oct 2023 09:14:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qpPqb-0001r2-VX; Sun, 08 Oct 2023 05:13:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qpPqZ-0001qA-WD for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:12 -0400 Received: from mail-oa1-x36.google.com ([2001:4860:4864:20::36]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qpPqT-0002GJ-IG for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:11 -0400 Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-1dd8304b980so2470659fac.2 for ; Sun, 08 Oct 2023 02:13:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1696756384; x=1697361184; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VHeGy5XmpcGI2kz/1DSIja8YSEs1srshIq8PJ1kE9ZE=; b=L+BvWSA1lXpjONr/cQs+4vS4IfWanfoaLyGZeYgIcld8X552i9256podLNUGBNsOVG lTBFJnE5imfZgWawlszeOJrIrzW8q83wk/ibkmyDGUPT26UhtVvb6lcXWxsRrqLfKL0d pcuTeW4RC4FRnLpvMfV56gWdHF8fgxE0jc80w802YVdA0y+LROX293v1XvJU/r7gjpwt 2yL+gip37OYv0FhxnecQLslW+c3q+3czI5U2CUgcYVzn6LNBDSIVxhjLpTEEQ+VDGzsq MfhBhHlSIZWx6xqhGUjN/ycYw0qCWmgobTMEnbhruvYcQ8uVxInctPdeqIMqt6sJL+4S R7Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696756384; x=1697361184; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VHeGy5XmpcGI2kz/1DSIja8YSEs1srshIq8PJ1kE9ZE=; b=XwDjBOSmetck2METVzkBPBZaViFEhGhIlTHcuZCmmqVIpkz6wJCqBeiQ2nKQJY+lgf OZ4Ay1YG1LG8Q17y7dXr25rzaT9rQK/E2A35YC5HhuiBYKpKPvKjrAnRvJNVnEPSsUv/ 6z1HF70rcW+4Xn/1Rq8cmCH8nK6rsOwe+vw8t/Qj6JXeWrdsO49ltss+V7cY2AzZ+dQ2 ARA4Swl8y1zGEqtuH5quXi0W2/rUe5XPaKSxuYZCiWGaBJ+OTHMc48oy9D5+KXK8t0zV ZXv4HDY3WSPMboJtqRUnzUdGHXnMH1pDfZHeUoSJOQ9GWj+wvnay5oeFF7pphGy0ut1D N9ag== X-Gm-Message-State: AOJu0Yxe6OCw1i0n3ZB6rF42eBVQY7E0kKzQj1Y4xf6cU32MYTREXGSY M9PZca0uT285M09E0VsDGcJ8EA== X-Google-Smtp-Source: AGHT+IE9TEJENT8H7s2LfGSIIx0IQtJXXC4xYts7SrJL0dTXCSAmPbIsfRUnShiWHRtLAJkn3WIt2w== X-Received: by 2002:a05:6870:819e:b0:1d6:162d:dc69 with SMTP id k30-20020a056870819e00b001d6162ddc69mr16136791oae.29.1696756384061; Sun, 08 Oct 2023 02:13:04 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id e8-20020a17090301c800b001b8a3e2c241sm7118084plh.14.2023.10.08.02.12.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 02:13:03 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v7 3/5] vhost-user-scsi: support reconnect to backend Date: Sun, 8 Oct 2023 17:12:03 +0800 Message-ID: <20231008091220.870171-4-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231008091220.870171-1-fengli@smartx.com> References: <20231008091220.870171-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2001:4860:4864:20::36; envelope-from=fengli@smartx.com; helo=mail-oa1-x36.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org If the backend crashes and restarts, the device is broken. This patch adds reconnect for vhost-user-scsi. This patch also improves the error messages, and reports some silent errors. Tested with spdk backend. Signed-off-by: Li Feng --- hw/scsi/vhost-scsi-common.c | 16 +- hw/scsi/vhost-scsi.c | 6 +- hw/scsi/vhost-user-scsi.c | 204 +++++++++++++++++++++++--- include/hw/virtio/vhost-scsi-common.h | 2 +- include/hw/virtio/vhost-user-scsi.h | 4 + 5 files changed, 202 insertions(+), 30 deletions(-) diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a61cd0e907..4c8637045d 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -16,6 +16,7 @@ */ #include "qemu/osdep.h" +#include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/module.h" #include "hw/virtio/vhost.h" @@ -25,7 +26,7 @@ #include "hw/virtio/virtio-access.h" #include "hw/fw-path-provider.h" -int vhost_scsi_common_start(VHostSCSICommon *vsc) +int vhost_scsi_common_start(VHostSCSICommon *vsc, Error **errp) { int ret, i; VirtIODevice *vdev = VIRTIO_DEVICE(vsc); @@ -35,18 +36,19 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) VirtIOSCSICommon *vs = (VirtIOSCSICommon *)vsc; if (!k->set_guest_notifiers) { - error_report("binding does not support guest notifiers"); + error_setg(errp, "binding does not support guest notifiers"); return -ENOSYS; } ret = vhost_dev_enable_notifiers(&vsc->dev, vdev); if (ret < 0) { + error_setg_errno(errp, -ret, "Error enabling host notifiers"); return ret; } ret = k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, true); if (ret < 0) { - error_report("Error binding guest notifier"); + error_setg_errno(errp, -ret, "Error binding guest notifier"); goto err_host_notifiers; } @@ -54,7 +56,7 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error setting inflight format: %d", -ret); + error_setg_errno(errp, -ret, "Error setting inflight format"); goto err_guest_notifiers; } @@ -64,21 +66,21 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vs->conf.virtqueue_size, vsc->inflight); if (ret < 0) { - error_report("Error getting inflight: %d", -ret); + error_setg_errno(errp, -ret, "Error getting inflight"); goto err_guest_notifiers; } } ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); if (ret < 0) { - error_report("Error setting inflight: %d", -ret); + error_setg_errno(errp, -ret, "Error setting inflight"); goto err_guest_notifiers; } } ret = vhost_dev_start(&vsc->dev, vdev, true); if (ret < 0) { - error_report("Error start vhost dev"); + error_setg_errno(errp, -ret, "Error starting vhost dev"); goto err_guest_notifiers; } diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 443f67daa4..95cadb93e7 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -75,6 +75,7 @@ static int vhost_scsi_start(VHostSCSI *s) int ret, abi_version; VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); const VhostOps *vhost_ops = vsc->dev.vhost_ops; + Error *local_err = NULL; ret = vhost_ops->vhost_scsi_get_abi_version(&vsc->dev, &abi_version); if (ret < 0) { @@ -88,14 +89,15 @@ static int vhost_scsi_start(VHostSCSI *s) return -ENOSYS; } - ret = vhost_scsi_common_start(vsc); + ret = vhost_scsi_common_start(vsc, &local_err); if (ret < 0) { + error_reportf_err(local_err, "Error starting vhost-scsi"); return ret; } ret = vhost_scsi_set_endpoint(s); if (ret < 0) { - error_report("Error setting vhost-scsi endpoint"); + error_reportf_err(local_err, "Error setting vhost-scsi endpoint"); vhost_scsi_common_stop(vsc); } diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index df6b66cc1a..5df24faff4 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -39,26 +39,56 @@ static const int user_feature_bits[] = { VHOST_INVALID_FEATURE_BIT }; +static int vhost_user_scsi_start(VHostUserSCSI *s, Error **errp) +{ + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + int ret; + + ret = vhost_scsi_common_start(vsc, errp); + s->started_vu = (ret < 0 ? false : true); + + return ret; +} + +static void vhost_user_scsi_stop(VHostUserSCSI *s) +{ + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + + if (!s->started_vu) { + return; + } + s->started_vu = false; + + vhost_scsi_common_stop(vsc); +} + static void vhost_user_scsi_set_status(VirtIODevice *vdev, uint8_t status) { VHostUserSCSI *s = (VHostUserSCSI *)vdev; + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - bool start = (status & VIRTIO_CONFIG_S_DRIVER_OK) && vdev->vm_running; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + bool should_start = virtio_device_should_start(vdev, status); + Error *local_err = NULL; + int ret; - if (vhost_dev_is_started(&vsc->dev) == start) { + if (!s->connected) { return; } - if (start) { - int ret; + if (vhost_dev_is_started(&vsc->dev) == should_start) { + return; + } - ret = vhost_scsi_common_start(vsc); + if (should_start) { + ret = vhost_user_scsi_start(s, &local_err); if (ret < 0) { - error_report("unable to start vhost-user-scsi: %s", strerror(-ret)); - exit(1); + error_reportf_err(local_err, "unable to start vhost-user-scsi: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&vs->conf.chardev); } } else { - vhost_scsi_common_stop(vsc); + vhost_user_scsi_stop(s); } } @@ -85,14 +115,127 @@ static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) { } +static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret = 0; + + if (s->connected) { + return 0; + } + s->connected = true; + + vsc->dev.num_queues = vs->conf.num_queues; + vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; + vsc->dev.vqs = s->vhost_vqs; + vsc->dev.vq_index = 0; + vsc->dev.backend_features = 0; + + ret = vhost_dev_init(&vsc->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0, + errp); + if (ret < 0) { + return ret; + } + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + ret = vhost_user_scsi_start(s, errp); + if (ret < 0) { + return ret; + } + } + + return 0; +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event); + +static void vhost_user_scsi_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + if (!s->connected) { + return; + } + s->connected = false; + + vhost_user_scsi_stop(s); + + vhost_dev_cleanup(&vsc->dev); + + /* Re-instate the event handler for new connections */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, dev, NULL, true); +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + Error *local_err = NULL; + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_scsi_connect(dev, &local_err) < 0) { + error_report_err(local_err); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + /* defer close until later to avoid circular close */ + vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, + vhost_user_scsi_disconnect); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static int vhost_user_scsi_realize_connect(VHostUserSCSI *s, Error **errp) +{ + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret; + + s->connected = false; + + ret = qemu_chr_fe_wait_connected(&vs->conf.chardev, errp); + if (ret < 0) { + return ret; + } + + ret = vhost_user_scsi_connect(dev, errp); + if (ret < 0) { + qemu_chr_fe_disconnect(&vs->conf.chardev); + return ret; + } + assert(s->connected); + + return 0; +} + static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) { + ERRP_GUARD(); VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = NULL; Error *err = NULL; int ret; + int retries = VU_REALIZE_CONN_RETRIES; if (!vs->conf.chardev.chr) { error_setg(errp, "vhost-user-scsi: missing chardev"); @@ -111,18 +254,28 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) goto free_virtio; } - vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; - vsc->dev.vqs = g_new0(struct vhost_virtqueue, vsc->dev.nvqs); - vsc->dev.vq_index = 0; - vsc->dev.backend_features = 0; - vqs = vsc->dev.vqs; + vsc->inflight = g_new0(struct vhost_inflight, 1); + s->vhost_vqs = g_new0(struct vhost_virtqueue, + VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues); + + assert(!*errp); + do { + if (*errp) { + error_prepend(errp, "Reconnecting after error: "); + error_report_err(*errp); + *errp = NULL; + } + ret = vhost_user_scsi_realize_connect(s, errp); + } while (ret < 0 && retries--); - ret = vhost_dev_init(&vsc->dev, &s->vhost_user, - VHOST_BACKEND_TYPE_USER, 0, errp); if (ret < 0) { goto free_vhost; } + /* we're fully initialized, now we can operate, so add the handler */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, (void *)dev, + NULL, true); /* Channel and lun both are 0 for bootable vhost-user-scsi disk */ vsc->channel = 0; vsc->lun = 0; @@ -131,8 +284,12 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; free_vhost: + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + g_free(vsc->inflight); + vsc->inflight = NULL; vhost_user_cleanup(&s->vhost_user); - g_free(vqs); + free_virtio: virtio_scsi_common_unrealize(dev); } @@ -142,16 +299,23 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) VirtIODevice *vdev = VIRTIO_DEVICE(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = vsc->dev.vqs; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); /* This will stop the vhost backend. */ vhost_user_scsi_set_status(vdev, 0); + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, NULL, NULL, NULL, + NULL, false); vhost_dev_cleanup(&vsc->dev); - g_free(vqs); + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + + vhost_dev_free_inflight(vsc->inflight); + g_free(vsc->inflight); + vsc->inflight = NULL; - virtio_scsi_common_unrealize(dev); vhost_user_cleanup(&s->vhost_user); + virtio_scsi_common_unrealize(dev); } static Property vhost_user_scsi_properties[] = { diff --git a/include/hw/virtio/vhost-scsi-common.h b/include/hw/virtio/vhost-scsi-common.h index 18f115527c..c5d2c09455 100644 --- a/include/hw/virtio/vhost-scsi-common.h +++ b/include/hw/virtio/vhost-scsi-common.h @@ -39,7 +39,7 @@ struct VHostSCSICommon { struct vhost_inflight *inflight; }; -int vhost_scsi_common_start(VHostSCSICommon *vsc); +int vhost_scsi_common_start(VHostSCSICommon *vsc, Error **errp); void vhost_scsi_common_stop(VHostSCSICommon *vsc); char *vhost_scsi_common_get_fw_dev_path(FWPathProvider *p, BusState *bus, DeviceState *dev); diff --git a/include/hw/virtio/vhost-user-scsi.h b/include/hw/virtio/vhost-user-scsi.h index 521b08e559..b405ec952a 100644 --- a/include/hw/virtio/vhost-user-scsi.h +++ b/include/hw/virtio/vhost-user-scsi.h @@ -29,6 +29,10 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserSCSI, VHOST_USER_SCSI) struct VHostUserSCSI { VHostSCSICommon parent_obj; VhostUserState vhost_user; + bool connected; + bool started_vu; + + struct vhost_virtqueue *vhost_vqs; }; #endif /* VHOST_USER_SCSI_H */ From patchwork Sun Oct 8 09:12:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13412657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9370DE95A8E for ; Sun, 8 Oct 2023 09:14:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qpPqc-0001rY-J5; Sun, 08 Oct 2023 05:13:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qpPqa-0001qV-Lm for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:12 -0400 Received: from mail-ot1-x332.google.com ([2607:f8b0:4864:20::332]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qpPqY-0002H5-7i for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:12 -0400 Received: by mail-ot1-x332.google.com with SMTP id 46e09a7af769-6c49f781855so2388092a34.3 for ; Sun, 08 Oct 2023 02:13:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1696756388; x=1697361188; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b9JY+QhbHyxCnkbfdAPQGdcFe4vb0ent2G3OkgeTOTY=; b=tIE++MCi58aXxV4EphPbrn2S8PzIbhdfMct16WVd3RT8WIYS5SCFWuG4HnTZX+3KVS mwRNKkvLchHATfRs3in/DaerQrMwKDDwqa8pQXL8f0udk7ZyND2Un3i/ACo14jVvOl7Y Qmj5Vi+jzY7UvjCrGbdFzfFFZNQSQSbo4WJDVv9I9VQle8orOLZWM3Y8RIgHUsDF9VZY NVzvxRP8rPXA4QTFOINdbEnjm0g05pJdrTC5t98/euf3kCL+chRIHw+oa8D1Sla0lJZk uDaKWwfYwqIIsbeL+vHNLJ3uTlfsG80s5HOwP50CrYXBSh5exsx5PyoQSS/SkhevwT1D tN/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696756388; x=1697361188; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b9JY+QhbHyxCnkbfdAPQGdcFe4vb0ent2G3OkgeTOTY=; b=cQwv83sqKLwNNz2dufYZdnMyk2oW5haoLQ1treHIJiykMts5QqLPfrvRvPk2+BjNPR AW8/PZcv7N3OxU3KxmR8Ir+JENk437Z5IdHYamSTyy0hQi222VY9XlJAW5u8M+fd6Ze0 iEhaBxpII6ZJY4LQC+mxutxoXH/O4UbpPaqTqTNCji2Y/CXsUUoIp7JKFqwvFmv+PjEJ IEBKzxGDbMla33Ex4tbsXDcvHsg7s9yVfQzMJxdXwy6ccBPEEVja1MEHgZ2ILfA0Q5fR D7HsKwkI3cC1U0/S5eMteU3ReiqtZ7h7KpgnQoiZtqe9v43jWu+T1PYlz6QwHhi2bx5p uJtg== X-Gm-Message-State: AOJu0YyOM1+vndMEXy8fRcoGB3sjgjM0btb3+lmNTjQwx0U2ym9dgloI mv+pu8wcI+UqMRKH7Gs6eN1eSA== X-Google-Smtp-Source: AGHT+IGvYT2b+Ug85U66Tpr72P6g6YvdjhFu/58CFO7RIOU2++nlY4bkorkxcA8xkY2KE/6vj151VA== X-Received: by 2002:a05:6871:7a7:b0:1bb:c50d:7437 with SMTP id o39-20020a05687107a700b001bbc50d7437mr15302280oap.53.1696756387995; Sun, 08 Oct 2023 02:13:07 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id e8-20020a17090301c800b001b8a3e2c241sm7118084plh.14.2023.10.08.02.13.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 02:13:07 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v7 4/5] vhost-user-scsi: start vhost when guest kicks Date: Sun, 8 Oct 2023 17:12:04 +0800 Message-ID: <20231008091220.870171-5-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231008091220.870171-1-fengli@smartx.com> References: <20231008091220.870171-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::332; envelope-from=fengli@smartx.com; helo=mail-ot1-x332.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Let's keep the same behavior as vhost-user-blk. Some old guests kick virtqueue before setting VIRTIO_CONFIG_S_DRIVER_OK. Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/scsi/vhost-user-scsi.c | 48 +++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 5df24faff4..5afb514398 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -111,8 +111,48 @@ static void vhost_user_scsi_reset(VirtIODevice *vdev) } } -static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) +static void vhost_user_scsi_handle_output(VirtIODevice *vdev, VirtQueue *vq) { + VHostUserSCSI *s = (VHostUserSCSI *)vdev; + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + Error *local_err = NULL; + int i, ret; + + if (!vdev->start_on_kick) { + return; + } + + if (!s->connected) { + return; + } + + if (vhost_dev_is_started(&vsc->dev)) { + return; + } + + /* + * Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start + * vhost here instead of waiting for .set_status(). + */ + ret = vhost_user_scsi_start(s, &local_err); + if (ret < 0) { + error_reportf_err(local_err, "vhost-user-scsi: vhost start failed: "); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + + /* Kick right away to begin processing requests already in vring */ + for (i = 0; i < vsc->dev.nvqs; i++) { + VirtQueue *kick_vq = virtio_get_queue(vdev, i); + + if (!virtio_queue_get_desc_addr(vdev, i)) { + continue; + } + event_notifier_set(virtio_queue_get_host_notifier(kick_vq)); + } } static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) @@ -242,9 +282,9 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; } - virtio_scsi_common_realize(dev, vhost_dummy_handle_output, - vhost_dummy_handle_output, - vhost_dummy_handle_output, &err); + virtio_scsi_common_realize(dev, vhost_user_scsi_handle_output, + vhost_user_scsi_handle_output, + vhost_user_scsi_handle_output, &err); if (err != NULL) { error_propagate(errp, err); return; From patchwork Sun Oct 8 09:12:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13412660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0FFCE95A67 for ; Sun, 8 Oct 2023 09:14:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qpPqf-0001sL-NU; Sun, 08 Oct 2023 05:13:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qpPqd-0001rf-Qn for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:15 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qpPqc-0002I7-4s for qemu-devel@nongnu.org; Sun, 08 Oct 2023 05:13:15 -0400 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2773f776f49so2616628a91.1 for ; Sun, 08 Oct 2023 02:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1696756393; x=1697361193; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rlgRVYVf3q+ye7htm+kTeAEWkvbvLFqoEkz9AMfOsY4=; b=ED1NvAVGwS371MM2ujB36h0bTC8YGCdfzhOcbL2PuGevH/vMYHn4L5MSYnCrev0cYp Spi0y+lQRnB4JraOyMo7kLiiXQSJtQEYqs5hI0kux+pwomHEIxcsrhXV+cnykWQp0Tgs x/GWU6bWL8AckwgJR9KntELJ3VjKn/rN/r8UBO2mAO7kC451y6QQ+uRSqmwkoZQwj3JX YwCmJW4JPIj3GfTDpKJulL/s119jUVAz0PGpS90xDR+EdsBuWFtcpqrIC/t7QTDc27rL 6vMlUFxWEJ7iwu313CWf9iS9h666EcT3PsCVI9tmqQg9FJZaCyVs0UAwyXdB6atp1Odh zZ8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696756393; x=1697361193; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rlgRVYVf3q+ye7htm+kTeAEWkvbvLFqoEkz9AMfOsY4=; b=l89G0z8Qp84uDe9S/hk0dcpzRqgCWs6ym9BVeHlKJCs60TGQ9sngeMMFLGi4NYtiFb 9ndQxqNSKhUPCUduhJbbIQ5ePIXUmJ95SKMMU7JHLutY5Rt/oueVj43WZNOQMmaOVk+7 TW0rxajd2WnGVH8sXN9BwnZVVWJFsyBRvkOG2CmzbxlUSIfx451Geys9qBIEIQwNwabj SNiobroTbhJuPwWEVn/QTXzEHebG327QeZc82eYY9BvSxnKaBAQnkC4LqW96uPRtFG1l mouCejFI4umGxbQi2QGbZl4DoRabKwfoZTOSqiKng5Y+kMURELIVVn3HHc0rOZalVoWy WpZg== X-Gm-Message-State: AOJu0Yx5RQUDQ6ViP4XulHfWmf+sLkdpGJGmULaDAStg98OiDX2AyZHO Db33z91FIg70ws4n7i4JkHIl/g== X-Google-Smtp-Source: AGHT+IEqdlUzRoWoJ5I/GQePkwBpyIDj7sGONOtB6PHKHKdTvBFMAmXczJSclMsEeji1qi9jDC4x4Q== X-Received: by 2002:a17:90b:4b84:b0:277:422d:3a0f with SMTP id lr4-20020a17090b4b8400b00277422d3a0fmr15132407pjb.17.1696756392669; Sun, 08 Oct 2023 02:13:12 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id e8-20020a17090301c800b001b8a3e2c241sm7118084plh.14.2023.10.08.02.13.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 02:13:12 -0700 (PDT) From: Li Feng To: Markus Armbruster , Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v7 5/5] vhost-user: fix lost reconnect Date: Sun, 8 Oct 2023 17:12:05 +0800 Message-ID: <20231008091220.870171-6-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231008091220.870171-1-fengli@smartx.com> References: <20231008091220.870171-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::1029; envelope-from=fengli@smartx.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When the vhost-user is reconnecting to the backend, and if the vhost-user fails at the get_features in vhost_dev_init(), then the reconnect will fail and it will not be retriggered forever. The reason is: When the vhost-user fails at get_features, the vhost_dev_cleanup will be called immediately. vhost_dev_cleanup calls 'memset(hdev, 0, sizeof(struct vhost_dev))'. The reconnect path is: vhost_user_blk_event vhost_user_async_close(.. vhost_user_blk_disconnect ..) qemu_chr_fe_set_handlers <----- clear the notifier callback schedule vhost_user_async_close_bh The vhost->vdev is null, so the vhost_user_blk_disconnect will not be called, then the event fd callback will not be reinstalled. All vhost-user devices have this issue, including vhost-user-blk/scsi. With this patch, if the vdev->vdev is null, the fd callback will still be reinstalled. Fixes: 71e076a07d ("hw/virtio: generalise CHR_EVENT_CLOSED handling") Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/block/vhost-user-blk.c | 2 +- hw/scsi/vhost-user-scsi.c | 3 ++- hw/virtio/vhost-user-gpio.c | 2 +- hw/virtio/vhost-user.c | 10 ++++++++-- include/hw/virtio/vhost-user.h | 3 ++- 5 files changed, 14 insertions(+), 6 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 3c69fa47d5..95c758200d 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -391,7 +391,7 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &s->chardev, &s->dev, - vhost_user_blk_disconnect); + vhost_user_blk_disconnect, vhost_user_blk_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 5afb514398..dbe864c0e5 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -234,7 +234,8 @@ static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, - vhost_user_scsi_disconnect); + vhost_user_scsi_disconnect, + vhost_user_scsi_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c index fc784e4213..aff2d7eff6 100644 --- a/hw/virtio/vhost-user-gpio.c +++ b/hw/virtio/vhost-user-gpio.c @@ -289,7 +289,7 @@ static void vu_gpio_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &gpio->chardev, &gpio->vhost_dev, - vu_gpio_disconnect); + vu_gpio_disconnect, vu_gpio_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 3766b415f8..7395bfc531 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -2765,6 +2765,7 @@ typedef struct { DeviceState *dev; CharBackend *cd; struct vhost_dev *vhost; + IOEventHandler *event_cb; } VhostAsyncCallback; static void vhost_user_async_close_bh(void *opaque) @@ -2779,7 +2780,10 @@ static void vhost_user_async_close_bh(void *opaque) */ if (vhost->vdev) { data->cb(data->dev); - } + } else if (data->event_cb) { + qemu_chr_fe_set_handlers(data->cd, NULL, NULL, data->event_cb, + NULL, data->dev, NULL, true); + } g_free(data); } @@ -2791,7 +2795,8 @@ static void vhost_user_async_close_bh(void *opaque) */ void vhost_user_async_close(DeviceState *d, CharBackend *chardev, struct vhost_dev *vhost, - vu_async_close_fn cb) + vu_async_close_fn cb, + IOEventHandler *event_cb) { if (!runstate_check(RUN_STATE_SHUTDOWN)) { /* @@ -2807,6 +2812,7 @@ void vhost_user_async_close(DeviceState *d, data->dev = d; data->cd = chardev; data->vhost = vhost; + data->event_cb = event_cb; /* Disable any further notifications on the chardev */ qemu_chr_fe_set_handlers(chardev, diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h index 9f9ddf878d..6b06ecb1bd 100644 --- a/include/hw/virtio/vhost-user.h +++ b/include/hw/virtio/vhost-user.h @@ -106,6 +106,7 @@ typedef void (*vu_async_close_fn)(DeviceState *cb); void vhost_user_async_close(DeviceState *d, CharBackend *chardev, struct vhost_dev *vhost, - vu_async_close_fn cb); + vu_async_close_fn cb, + IOEventHandler *event_cb); #endif