From patchwork Fri Jul 21 10:51:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13321853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 185A9EB64DD for ; Fri, 21 Jul 2023 10:54:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qMnmH-0001EQ-TB; Fri, 21 Jul 2023 06:54:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qMnmE-0001E6-J1 for qemu-devel@nongnu.org; Fri, 21 Jul 2023 06:54:26 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qMnmA-0004tN-K0 for qemu-devel@nongnu.org; Fri, 21 Jul 2023 06:54:26 -0400 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1b89600a37fso11573265ad.2 for ; Fri, 21 Jul 2023 03:52:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20221208.gappssmtp.com; s=20221208; t=1689936737; x=1690541537; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=6W8C7rFibqHDhrw6L5RpAzolrzeEfVWE3BtgA5JGR5w=; b=Qyey+dmrqrlX0ENCnHOe0epKbnANB5sig7gp5men4vosNeOd6WS3zMaoefUEH+u5V8 gHZ722gW0bWgOhk29Gv4X42CnxT9wxP+g0QEqNEtDbd8J0oD3VzXLdtFnCm0u10NoX4D bs6CfhuFc33yem6lxAuyMfWFm+YmB6tD9z0Z6F3oH/ESdUgsCQTdDWyFnENObPbjkusv bhOrwFoGSu25peaPcoC+YTFYeVIRM+PRfkJ7l1iG0redZG2Tf36V1EcvRn6Q4JZ5/5W6 +eI++R+Ep1V85nbG6G+XE7MKLXiqBTw4n/WJMdHTXXvqJKFP5Y+stlBdG+HrxEbqmgOV KjWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689936737; x=1690541537; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6W8C7rFibqHDhrw6L5RpAzolrzeEfVWE3BtgA5JGR5w=; b=Tr5bGr9JxhBTdUScwsG9ny/C+h6lpKyN5OEWvP2UMgidnCpazlKe+jdhL5w24j3rqK 0hCKvQpclnuyNkIlPvdPrXZQzkUl4VqIaI+9QuhqaGqZiGs6WmrTdHKt55Z3Ryy5YdU+ 0ghMGW7hN60mEI4s5VPwU4yWZghcONmVIbnV4casU2GKl5Jo6zC6w7LQsHyTRcezd8BO iOyuQD4zSn2rFbtGsjJtW969bir3gY2JyiZ2HWbBv9Dmkk9N48IziUTAY0hAjYpTDvYQ 9dxV9U+QzpsW+V/I6PnDoI1I012ZZvh20eunQm/uoWkhtl3BcqpoWbZLvNK5+y3dSqpQ h9Mw== X-Gm-Message-State: ABy/qLZdKP9Uhz4vYlPNfasXYBMgdjrDEKRoH66ZbzamyDq8SXVNjaWc 7s7oxlJNuSoiKavBafen9+Ittg== X-Google-Smtp-Source: APBJJlFsYEt4PN2pd00Yr5T8PE72AwblgARkCt+3onHI/ht3s1UV1YD/SC9LuwEY3wZPn7qa6CmEMw== X-Received: by 2002:a17:902:ec84:b0:1b9:de3e:7a59 with SMTP id x4-20020a170902ec8400b001b9de3e7a59mr1747300plg.10.1689936737179; Fri, 21 Jul 2023 03:52:17 -0700 (PDT) Received: from localhost.localdomain.gitgo.cc ([47.75.78.161]) by smtp.gmail.com with ESMTPSA id je15-20020a170903264f00b001b8ad8382a4sm3124254plb.216.2023.07.21.03.52.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 03:52:16 -0700 (PDT) From: Li Feng To: "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: lifeng1519@gmail.com, Li Feng Subject: [PATCH] vhost-user-scsi: support reconnect to backend Date: Fri, 21 Jul 2023 18:51:59 +0800 Message-ID: <20230721105205.1714449-1-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::632; envelope-from=fengli@smartx.com; helo=mail-pl1-x632.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org If the backend crashes and restarts, the device is broken. This patch adds reconnect for vhost-user-scsi. Tested with spdk backend. Signed-off-by: Li Feng --- hw/block/vhost-user-blk.c | 2 - hw/scsi/vhost-scsi-common.c | 27 ++--- hw/scsi/vhost-user-scsi.c | 163 +++++++++++++++++++++++++--- include/hw/virtio/vhost-user-scsi.h | 3 + include/hw/virtio/vhost.h | 2 + 5 files changed, 165 insertions(+), 32 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index eecf3f7a81..f250c740b5 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -32,8 +32,6 @@ #include "sysemu/sysemu.h" #include "sysemu/runstate.h" -#define REALIZE_CONNECTION_RETRIES 3 - static const int user_feature_bits[] = { VIRTIO_BLK_F_SIZE_MAX, VIRTIO_BLK_F_SEG_MAX, diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a06f01af26..08801886b8 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -52,16 +52,22 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vsc->dev.acked_features = vdev->guest_features; - assert(vsc->inflight == NULL); - vsc->inflight = g_new0(struct vhost_inflight, 1); - ret = vhost_dev_get_inflight(&vsc->dev, - vs->conf.virtqueue_size, - vsc->inflight); + ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error get inflight: %d", -ret); + error_report("Error setting inflight format: %d", -ret); goto err_guest_notifiers; } + if (!vsc->inflight->addr) { + ret = vhost_dev_get_inflight(&vsc->dev, + vs->conf.virtqueue_size, + vsc->inflight); + if (ret < 0) { + error_report("Error get inflight: %d", -ret); + goto err_guest_notifiers; + } + } + ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); if (ret < 0) { error_report("Error set inflight: %d", -ret); @@ -85,9 +91,6 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) return ret; err_guest_notifiers: - g_free(vsc->inflight); - vsc->inflight = NULL; - k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&vsc->dev, vdev); @@ -111,12 +114,6 @@ void vhost_scsi_common_stop(VHostSCSICommon *vsc) } assert(ret >= 0); - if (vsc->inflight) { - vhost_dev_free_inflight(vsc->inflight); - g_free(vsc->inflight); - vsc->inflight = NULL; - } - vhost_dev_disable_notifiers(&vsc->dev, vdev); } diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index ee99b19e7a..e0e88b0c42 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -89,14 +89,126 @@ static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) { } +static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret = 0; + + if (s->connected) { + return 0; + } + s->connected = true; + + vsc->dev.num_queues = vs->conf.num_queues; + vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; + vsc->dev.vqs = s->vhost_vqs; + vsc->dev.vq_index = 0; + vsc->dev.backend_features = 0; + + ret = vhost_dev_init(&vsc->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0, + errp); + if (ret < 0) { + return ret; + } + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + ret = vhost_scsi_common_start(vsc); + if (ret < 0) { + return ret; + } + } + + return 0; +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event); + +static void vhost_user_scsi_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + if (!s->connected) { + return; + } + s->connected = false; + + vhost_scsi_common_stop(vsc); + + vhost_dev_cleanup(&vsc->dev); + + /* Re-instate the event handler for new connections */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, dev, NULL, true); +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + Error *local_err = NULL; + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_scsi_connect(dev, &local_err) < 0) { + error_report_err(local_err); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + /* defer close until later to avoid circular close */ + vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, + vhost_user_scsi_disconnect); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static int vhost_user_scsi_realize_connect(VHostUserSCSI *s, Error **errp) +{ + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret; + + s->connected = false; + + ret = qemu_chr_fe_wait_connected(&vs->conf.chardev, errp); + if (ret < 0) { + return ret; + } + + ret = vhost_user_scsi_connect(dev, errp); + if (ret < 0) { + qemu_chr_fe_disconnect(&vs->conf.chardev); + return ret; + } + assert(s->connected); + + return 0; +} + static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) { VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = NULL; Error *err = NULL; int ret; + int retries = REALIZE_CONNECTION_RETRIES; if (!vs->conf.chardev.chr) { error_setg(errp, "vhost-user-scsi: missing chardev"); @@ -112,21 +224,31 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) } if (!vhost_user_init(&s->vhost_user, &vs->conf.chardev, errp)) { - goto free_virtio; + goto free_vhost; } - vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; - vsc->dev.vqs = g_new0(struct vhost_virtqueue, vsc->dev.nvqs); - vsc->dev.vq_index = 0; - vsc->dev.backend_features = 0; - vqs = vsc->dev.vqs; + vsc->inflight = g_new0(struct vhost_inflight, 1); + s->vhost_vqs = g_new0(struct vhost_virtqueue, + VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues); + + assert(!*errp); + do { + if (*errp) { + error_prepend(errp, "Reconnecting after error: "); + error_report_err(*errp); + *errp = NULL; + } + ret = vhost_user_scsi_realize_connect(s, errp); + } while (ret < 0 && retries--); - ret = vhost_dev_init(&vsc->dev, &s->vhost_user, - VHOST_BACKEND_TYPE_USER, 0, errp); if (ret < 0) { - goto free_vhost; + goto free_vqs; } + /* we're fully initialized, now we can operate, so add the handler */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, (void *)dev, + NULL, true); /* Channel and lun both are 0 for bootable vhost-user-scsi disk */ vsc->channel = 0; vsc->lun = 0; @@ -134,10 +256,15 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; +free_vqs: + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + g_free(vsc->inflight); + vsc->inflight = NULL; + free_vhost: vhost_user_cleanup(&s->vhost_user); - g_free(vqs); -free_virtio: + virtio_scsi_common_unrealize(dev); } @@ -146,16 +273,22 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) VirtIODevice *vdev = VIRTIO_DEVICE(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = vsc->dev.vqs; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); /* This will stop the vhost backend. */ vhost_user_scsi_set_status(vdev, 0); + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, NULL, NULL, NULL, + NULL, false); vhost_dev_cleanup(&vsc->dev); - g_free(vqs); + vhost_dev_free_inflight(vsc->inflight); + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + g_free(vsc->inflight); + vsc->inflight = NULL; - virtio_scsi_common_unrealize(dev); vhost_user_cleanup(&s->vhost_user); + virtio_scsi_common_unrealize(dev); } static Property vhost_user_scsi_properties[] = { diff --git a/include/hw/virtio/vhost-user-scsi.h b/include/hw/virtio/vhost-user-scsi.h index 521b08e559..c66acc68b7 100644 --- a/include/hw/virtio/vhost-user-scsi.h +++ b/include/hw/virtio/vhost-user-scsi.h @@ -29,6 +29,9 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserSCSI, VHOST_USER_SCSI) struct VHostUserSCSI { VHostSCSICommon parent_obj; VhostUserState vhost_user; + bool connected; + + struct vhost_virtqueue *vhost_vqs; }; #endif /* VHOST_USER_SCSI_H */ diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 6a173cb9fa..b904346fe1 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -8,6 +8,8 @@ #define VHOST_F_DEVICE_IOTLB 63 #define VHOST_USER_F_PROTOCOL_FEATURES 30 +#define REALIZE_CONNECTION_RETRIES 3 + /* Generic structures common for any vhost based device. */ struct vhost_inflight { From patchwork Tue Sep 19 11:40:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13391263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DCF2ACD54B6 for ; Tue, 19 Sep 2023 11:43:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qiZ8O-0007mK-3o; Tue, 19 Sep 2023 07:43:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiZ8L-0007kp-PQ for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:13 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qiZ8J-0004Fd-A9 for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:13 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-68fb2e9ebcdso4145739b3a.2 for ; Tue, 19 Sep 2023 04:41:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695123670; x=1695728470; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ak3cZicP6FXgjNSZA0EnmzY0v3FARPxaB+bcgiWIe/k=; b=PlhNhJ18J7/YO46NpeqxCYx04UYJ8bYnTehuSZ0CNkgLp37rxsHsmNgBNbHHdNzeb+ g9tiF5hxh84LKTEsF+j/cfACJsPzjgZ/Z2BRaQe1l085bnQWYTKrL6mEnBEPAHBIm6Dw vKfv6ygkXU1lncfKY4JhX5cTUo7C2sYl3pjDc+SJw8Q8k1vMK38Kt8XVx6HUX8ucxi0V 66eRvVkFOa1hKRymU1U51NzKBs27fbBAo89WWxslgL1dt9BfgQ5ztmB69BnWOr6W93Im 3VBXfoShZ1XDQt+1dqbtOOtyLGsXgFRwZpwVb0ky0YEQVIc1nWRtzOlASgJdvXu0cksj cZ5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695123670; x=1695728470; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ak3cZicP6FXgjNSZA0EnmzY0v3FARPxaB+bcgiWIe/k=; b=NbAZ7WeTQb+mAQznHEK1hAYOb0p8zHwYLu8/Vi//kXKzP+DVy7jkAbzjVEkAr8rjNq TI16ptToXoW6SPmCs3lHeeluHm0IeB+ZA73yNNVDaa1uRYgL6tyHnIBzfTAp+5aP7B7u wdB9G8ymO8JylLKO5b1mh+0cZiSVYJJFmgGlBsiCQS5dqJICPyoYdNzawOymEVxROReb tMAGGkTfbsnq5ZOG36w81EDbbfkzL+F5ND5a6P961shzoFl1dfFsOadh4r+lAb+vMmJ5 RPHH276F4oPeFUJLmuiHBN2J8abd8wPriUPG7DJbDKXOjlSx2CKXa1aXKZhtT8b8uh/I Kdnw== X-Gm-Message-State: AOJu0YzLiIGwz720xU6WhnG5llMh57mlDrkPYahfgPxVEHeY0cPp7T7K kboYuWfwFHNOxibbheYs6mvh3g== X-Google-Smtp-Source: AGHT+IFqQdE9zR7dJmn8PVkbIaZPsz/mS2smwyJ0mv4zrKrqbOLPPiWN+FJb8sgzJQ8KZkA1mkcBRg== X-Received: by 2002:a05:6a20:12c4:b0:159:b45e:1c7d with SMTP id v4-20020a056a2012c400b00159b45e1c7dmr12504989pzg.15.1695123669748; Tue, 19 Sep 2023 04:41:09 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id z24-20020aa791d8000000b006887be16675sm8487030pfa.205.2023.09.19.04.41.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 04:41:09 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v5 2/5] vhost: move and rename the conn retry times Date: Tue, 19 Sep 2023 19:40:23 +0800 Message-ID: <20230919114030.280414-3-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230919114030.280414-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230919114030.280414-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::42a; envelope-from=fengli@smartx.com; helo=mail-pf1-x42a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Multiple devices need this macro, move it to a common header. Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/block/vhost-user-blk.c | 4 +--- hw/virtio/vhost-user-gpio.c | 3 +-- include/hw/virtio/vhost.h | 2 ++ 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index eecf3f7a81..3c69fa47d5 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -32,8 +32,6 @@ #include "sysemu/sysemu.h" #include "sysemu/runstate.h" -#define REALIZE_CONNECTION_RETRIES 3 - static const int user_feature_bits[] = { VIRTIO_BLK_F_SIZE_MAX, VIRTIO_BLK_F_SEG_MAX, @@ -482,7 +480,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) s->inflight = g_new0(struct vhost_inflight, 1); s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues); - retries = REALIZE_CONNECTION_RETRIES; + retries = VU_REALIZE_CONN_RETRIES; assert(!*errp); do { if (*errp) { diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c index 3b013f2d0f..d9979aa5db 100644 --- a/hw/virtio/vhost-user-gpio.c +++ b/hw/virtio/vhost-user-gpio.c @@ -15,7 +15,6 @@ #include "standard-headers/linux/virtio_ids.h" #include "trace.h" -#define REALIZE_CONNECTION_RETRIES 3 #define VHOST_NVQS 2 /* Features required from VirtIO */ @@ -359,7 +358,7 @@ static void vu_gpio_device_realize(DeviceState *dev, Error **errp) qemu_chr_fe_set_handlers(&gpio->chardev, NULL, NULL, vu_gpio_event, NULL, dev, NULL, true); - retries = REALIZE_CONNECTION_RETRIES; + retries = VU_REALIZE_CONN_RETRIES; g_assert(!*errp); do { if (*errp) { diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 6a173cb9fa..ca3131b1af 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -8,6 +8,8 @@ #define VHOST_F_DEVICE_IOTLB 63 #define VHOST_USER_F_PROTOCOL_FEATURES 30 +#define VU_REALIZE_CONN_RETRIES 3 + /* Generic structures common for any vhost based device. */ struct vhost_inflight { From patchwork Tue Sep 19 11:40:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13391265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3A59CD54B7 for ; Tue, 19 Sep 2023 11:43:51 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qiZ8Q-0007ph-Mx; Tue, 19 Sep 2023 07:43:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiZ8O-0007n7-UF for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:16 -0400 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qiZ8M-0004G2-Tg for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:16 -0400 Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-577a98f78b2so4000318a12.3 for ; Tue, 19 Sep 2023 04:41:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695123673; x=1695728473; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+XH7wdtDBHjkJj1TRjhNqIG/+/Tyzm+qcFAhIy62ShA=; b=m+VHgC8GuF33xt1HtuIsSnrM2yMOh2e2RCXHVeFWNWvecP7qfppX7MzQgRrfNndIsx KYbpqwPSVsuNvaM7DMuc5sOtsKvbB9C8CiOhGKPM/djXd4YVpfTL1GPhQ/8vnJOt09ms kMW0LiqS4BtVJ+h2UrboEVRDuqaMuhflrf0LMIVxMocinujD+a233+VfhpobxeDrSFv9 k6h7ZUTQ+Moq8oglf0+7K2ykrC9/WwnQO2L0XnXI7xasanP/oXvVAlHA6+2u65tkCI2m Wg3FmrXEkqNIXUBxL0YZIr/kHgTSzBtyxumOM151f7VEAX2n/sRBaHvJz32xig65krCO mRbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695123673; x=1695728473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+XH7wdtDBHjkJj1TRjhNqIG/+/Tyzm+qcFAhIy62ShA=; b=PPiGUchNNWrEGqrxBr6c6DN6RDyddVbEN1vok+gO0gkh7EMRgPW1VrZAR+KChiWCH7 v31NTvwkCxpZB3WmhnRtyhaMiAwqDtgTZ2VoR+uIvHj9I1IG52e22k2iNq8VcqNW0Ybv kfzZ/2bHo9+M2luuynA9w5f0xzBbb5Bf+EfK+jP1VvhlqBarEf8U5qwuwmq519ZS11/F 0kAHZcMpxxBbZwgmQnELp7R8yrEI12I3slc68avZX3WokDktQKwc7T2e6D++AndzbxMy Y/kTlBdeFR/4/AG/lZh0riaAdWuhqDJn3y0HtGtiNOxi2SRd5Lv2PvaxPaPXcC7kAcge 7+CQ== X-Gm-Message-State: AOJu0YwxAqukLQvX4CKuNyJwoUDKuLmC2ozsF7IhSwA6yIDbKUAWjYVj oxUCQ3Yy2jg7ESGdF3WA/Xn2RA== X-Google-Smtp-Source: AGHT+IF3Gfj/mTjVpxJOB9WgXdMjLGeHU2hWgkHV/eOM6VvbIknyScBgvZDFZdYursL44iN+duHW4g== X-Received: by 2002:a05:6a20:104c:b0:155:5c28:ea74 with SMTP id gt12-20020a056a20104c00b001555c28ea74mr9989601pzc.12.1695123673346; Tue, 19 Sep 2023 04:41:13 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id z24-20020aa791d8000000b006887be16675sm8487030pfa.205.2023.09.19.04.41.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 04:41:12 -0700 (PDT) From: Li Feng To: Markus Armbruster , Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v5 3/5] vhost-user-scsi: support reconnect to backend Date: Tue, 19 Sep 2023 19:40:24 +0800 Message-ID: <20230919114030.280414-4-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230919114030.280414-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230919114030.280414-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::534; envelope-from=fengli@smartx.com; helo=mail-pg1-x534.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org If the backend crashes and restarts, the device is broken. This patch adds reconnect for vhost-user-scsi. This patch also improves the error messages, and reports some silent errors. Tested with spdk backend. Signed-off-by: Li Feng --- hw/scsi/vhost-scsi-common.c | 16 +- hw/scsi/vhost-scsi.c | 5 +- hw/scsi/vhost-user-scsi.c | 204 +++++++++++++++++++++++--- include/hw/virtio/vhost-scsi-common.h | 2 +- include/hw/virtio/vhost-user-scsi.h | 4 + 5 files changed, 201 insertions(+), 30 deletions(-) diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a61cd0e907..4c8637045d 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -16,6 +16,7 @@ */ #include "qemu/osdep.h" +#include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/module.h" #include "hw/virtio/vhost.h" @@ -25,7 +26,7 @@ #include "hw/virtio/virtio-access.h" #include "hw/fw-path-provider.h" -int vhost_scsi_common_start(VHostSCSICommon *vsc) +int vhost_scsi_common_start(VHostSCSICommon *vsc, Error **errp) { int ret, i; VirtIODevice *vdev = VIRTIO_DEVICE(vsc); @@ -35,18 +36,19 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) VirtIOSCSICommon *vs = (VirtIOSCSICommon *)vsc; if (!k->set_guest_notifiers) { - error_report("binding does not support guest notifiers"); + error_setg(errp, "binding does not support guest notifiers"); return -ENOSYS; } ret = vhost_dev_enable_notifiers(&vsc->dev, vdev); if (ret < 0) { + error_setg_errno(errp, -ret, "Error enabling host notifiers"); return ret; } ret = k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, true); if (ret < 0) { - error_report("Error binding guest notifier"); + error_setg_errno(errp, -ret, "Error binding guest notifier"); goto err_host_notifiers; } @@ -54,7 +56,7 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error setting inflight format: %d", -ret); + error_setg_errno(errp, -ret, "Error setting inflight format"); goto err_guest_notifiers; } @@ -64,21 +66,21 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vs->conf.virtqueue_size, vsc->inflight); if (ret < 0) { - error_report("Error getting inflight: %d", -ret); + error_setg_errno(errp, -ret, "Error getting inflight"); goto err_guest_notifiers; } } ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); if (ret < 0) { - error_report("Error setting inflight: %d", -ret); + error_setg_errno(errp, -ret, "Error setting inflight"); goto err_guest_notifiers; } } ret = vhost_dev_start(&vsc->dev, vdev, true); if (ret < 0) { - error_report("Error start vhost dev"); + error_setg_errno(errp, -ret, "Error starting vhost dev"); goto err_guest_notifiers; } diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 443f67daa4..01a3ab4277 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -75,6 +75,7 @@ static int vhost_scsi_start(VHostSCSI *s) int ret, abi_version; VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); const VhostOps *vhost_ops = vsc->dev.vhost_ops; + Error *local_err = NULL; ret = vhost_ops->vhost_scsi_get_abi_version(&vsc->dev, &abi_version); if (ret < 0) { @@ -88,14 +89,14 @@ static int vhost_scsi_start(VHostSCSI *s) return -ENOSYS; } - ret = vhost_scsi_common_start(vsc); + ret = vhost_scsi_common_start(vsc, &local_err); if (ret < 0) { return ret; } ret = vhost_scsi_set_endpoint(s); if (ret < 0) { - error_report("Error setting vhost-scsi endpoint"); + error_reportf_err(local_err, "Error setting vhost-scsi endpoint"); vhost_scsi_common_stop(vsc); } diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index ee99b19e7a..dc109154ad 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -43,26 +43,56 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13, }; +static int vhost_user_scsi_start(VHostUserSCSI *s, Error **errp) +{ + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + int ret; + + ret = vhost_scsi_common_start(vsc, errp); + s->started_vu = (ret < 0 ? false : true); + + return ret; +} + +static void vhost_user_scsi_stop(VHostUserSCSI *s) +{ + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + + if (!s->started_vu) { + return; + } + s->started_vu = false; + + vhost_scsi_common_stop(vsc); +} + static void vhost_user_scsi_set_status(VirtIODevice *vdev, uint8_t status) { VHostUserSCSI *s = (VHostUserSCSI *)vdev; + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - bool start = (status & VIRTIO_CONFIG_S_DRIVER_OK) && vdev->vm_running; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + bool should_start = virtio_device_should_start(vdev, status); + Error *local_err = NULL; + int ret; - if (vhost_dev_is_started(&vsc->dev) == start) { + if (!s->connected) { return; } - if (start) { - int ret; + if (vhost_dev_is_started(&vsc->dev) == should_start) { + return; + } - ret = vhost_scsi_common_start(vsc); + if (should_start) { + ret = vhost_user_scsi_start(s, &local_err); if (ret < 0) { - error_report("unable to start vhost-user-scsi: %s", strerror(-ret)); - exit(1); + error_reportf_err(local_err, "unable to start vhost-user-scsi: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&vs->conf.chardev); } } else { - vhost_scsi_common_stop(vsc); + vhost_user_scsi_stop(s); } } @@ -89,14 +119,127 @@ static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) { } +static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret = 0; + + if (s->connected) { + return 0; + } + s->connected = true; + + vsc->dev.num_queues = vs->conf.num_queues; + vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; + vsc->dev.vqs = s->vhost_vqs; + vsc->dev.vq_index = 0; + vsc->dev.backend_features = 0; + + ret = vhost_dev_init(&vsc->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0, + errp); + if (ret < 0) { + return ret; + } + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + ret = vhost_user_scsi_start(s, errp); + if (ret < 0) { + return ret; + } + } + + return 0; +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event); + +static void vhost_user_scsi_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + if (!s->connected) { + return; + } + s->connected = false; + + vhost_user_scsi_stop(s); + + vhost_dev_cleanup(&vsc->dev); + + /* Re-instate the event handler for new connections */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, dev, NULL, true); +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + Error *local_err = NULL; + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_scsi_connect(dev, &local_err) < 0) { + error_report_err(local_err); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + /* defer close until later to avoid circular close */ + vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, + vhost_user_scsi_disconnect); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static int vhost_user_scsi_realize_connect(VHostUserSCSI *s, Error **errp) +{ + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret; + + s->connected = false; + + ret = qemu_chr_fe_wait_connected(&vs->conf.chardev, errp); + if (ret < 0) { + return ret; + } + + ret = vhost_user_scsi_connect(dev, errp); + if (ret < 0) { + qemu_chr_fe_disconnect(&vs->conf.chardev); + return ret; + } + assert(s->connected); + + return 0; +} + static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) { + ERRP_GUARD(); VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = NULL; Error *err = NULL; int ret; + int retries = VU_REALIZE_CONN_RETRIES; if (!vs->conf.chardev.chr) { error_setg(errp, "vhost-user-scsi: missing chardev"); @@ -115,18 +258,28 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) goto free_virtio; } - vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; - vsc->dev.vqs = g_new0(struct vhost_virtqueue, vsc->dev.nvqs); - vsc->dev.vq_index = 0; - vsc->dev.backend_features = 0; - vqs = vsc->dev.vqs; + vsc->inflight = g_new0(struct vhost_inflight, 1); + s->vhost_vqs = g_new0(struct vhost_virtqueue, + VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues); + + assert(!*errp); + do { + if (*errp) { + error_prepend(errp, "Reconnecting after error: "); + error_report_err(*errp); + *errp = NULL; + } + ret = vhost_user_scsi_realize_connect(s, errp); + } while (ret < 0 && retries--); - ret = vhost_dev_init(&vsc->dev, &s->vhost_user, - VHOST_BACKEND_TYPE_USER, 0, errp); if (ret < 0) { goto free_vhost; } + /* we're fully initialized, now we can operate, so add the handler */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, (void *)dev, + NULL, true); /* Channel and lun both are 0 for bootable vhost-user-scsi disk */ vsc->channel = 0; vsc->lun = 0; @@ -135,8 +288,12 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; free_vhost: + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + g_free(vsc->inflight); + vsc->inflight = NULL; vhost_user_cleanup(&s->vhost_user); - g_free(vqs); + free_virtio: virtio_scsi_common_unrealize(dev); } @@ -146,16 +303,23 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) VirtIODevice *vdev = VIRTIO_DEVICE(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = vsc->dev.vqs; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); /* This will stop the vhost backend. */ vhost_user_scsi_set_status(vdev, 0); + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, NULL, NULL, NULL, + NULL, false); vhost_dev_cleanup(&vsc->dev); - g_free(vqs); + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + + vhost_dev_free_inflight(vsc->inflight); + g_free(vsc->inflight); + vsc->inflight = NULL; - virtio_scsi_common_unrealize(dev); vhost_user_cleanup(&s->vhost_user); + virtio_scsi_common_unrealize(dev); } static Property vhost_user_scsi_properties[] = { diff --git a/include/hw/virtio/vhost-scsi-common.h b/include/hw/virtio/vhost-scsi-common.h index 18f115527c..c5d2c09455 100644 --- a/include/hw/virtio/vhost-scsi-common.h +++ b/include/hw/virtio/vhost-scsi-common.h @@ -39,7 +39,7 @@ struct VHostSCSICommon { struct vhost_inflight *inflight; }; -int vhost_scsi_common_start(VHostSCSICommon *vsc); +int vhost_scsi_common_start(VHostSCSICommon *vsc, Error **errp); void vhost_scsi_common_stop(VHostSCSICommon *vsc); char *vhost_scsi_common_get_fw_dev_path(FWPathProvider *p, BusState *bus, DeviceState *dev); diff --git a/include/hw/virtio/vhost-user-scsi.h b/include/hw/virtio/vhost-user-scsi.h index 521b08e559..b405ec952a 100644 --- a/include/hw/virtio/vhost-user-scsi.h +++ b/include/hw/virtio/vhost-user-scsi.h @@ -29,6 +29,10 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserSCSI, VHOST_USER_SCSI) struct VHostUserSCSI { VHostSCSICommon parent_obj; VhostUserState vhost_user; + bool connected; + bool started_vu; + + struct vhost_virtqueue *vhost_vqs; }; #endif /* VHOST_USER_SCSI_H */ From patchwork Tue Sep 19 11:40:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13391266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56EAECD54B6 for ; Tue, 19 Sep 2023 11:43:53 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qiZ8U-0007rP-25; Tue, 19 Sep 2023 07:43:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiZ8R-0007q5-Qw for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:19 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qiZ8Q-0004GS-BX for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:19 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-68fac16ee5fso4740902b3a.1 for ; Tue, 19 Sep 2023 04:41:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695123677; x=1695728477; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HPRPD1pbMza8Qy14aglX4a9WyhzajqmGhD+mOHsf0R4=; b=sXDdvsIKOYI6WDGVfZbGo6THa5Udot+nWRZGQL3kY0cry4iQaaVSE0mWLSzk/RAzJo f5ndTPy6pUOe6tWSK2YL8LEEjoEkMy6el5UbgOyCam9BDdRSdMuCzvTVcRpg2rvxh1iB anxZgbAZd1xSumclwS7UYvkqLFz6bD4oz3upQM+hOvu641R1UZ0JUZiE74muWulCaW+h PoEuUK1LPm1SRik8rhnb+cuWbJ28yN3onvEZj9WKZf1ycRgYpkbtkoBqKrxOwlCSM+9o skmSnk+q4f6cFJT8O6zE11WXLYh/yXsxo8JVUXUTqwX0G0WMFcrcv3J7CWZIMYXJz47X FbKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695123677; x=1695728477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HPRPD1pbMza8Qy14aglX4a9WyhzajqmGhD+mOHsf0R4=; b=QqpY1mmvv4jks9/8bIm1F+eb4FT4XfTqR7mWDsKd93gNXMCCaoYps7siO0tvdBez5K oF/wEZyRqIYtVggUVH6e3fEV+q0CjUYMAAYF+PRQJ6rHdDfoOMjL45q9p8JE2cDzlM0p lqy3BrK3DVOaMdClNre+hrGdfKCQBXw7C7L1QM5JoPEUb8RplHeO0qVcZz9lbRKRhrW0 tkRfSog6RAh6LyiYbfEpVyMEtZ0oH/GRN3UfBNnPnqsDcLehg+BydArQ909hzQ/YAOwn bvf+8smzIamgvgLL7rIljeHt9StgvyZ6FIvVSGjcSwSaqMzoPhqJrwIiV515Z+/boO6X P5EQ== X-Gm-Message-State: AOJu0Yxs7pGMngI3DSWL/6ZJgM5c/5DjFrKC9unagx2gZeWKhDpkWLPE W/TAoExc9gOpYsCuc9X/Sq+fGdrYWHKdw5iTqRKJb62gU5Y= X-Google-Smtp-Source: AGHT+IGLhCT1WiQUD3MxhVzBqZJUtCW5aNOdZvJCjNJhqSVr1oHpo+i+po+25R7JkIPlvLRxpKzJVQ== X-Received: by 2002:a05:6a20:2446:b0:147:fd40:2482 with SMTP id t6-20020a056a20244600b00147fd402482mr11970103pzc.44.1695123676784; Tue, 19 Sep 2023 04:41:16 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id z24-20020aa791d8000000b006887be16675sm8487030pfa.205.2023.09.19.04.41.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 04:41:16 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v5 4/5] vhost-user-scsi: start vhost when guest kicks Date: Tue, 19 Sep 2023 19:40:25 +0800 Message-ID: <20230919114030.280414-5-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230919114030.280414-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230919114030.280414-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::434; envelope-from=fengli@smartx.com; helo=mail-pf1-x434.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Let's keep the same behavior as vhost-user-blk. Some old guests kick virtqueue before setting VIRTIO_CONFIG_S_DRIVER_OK. Signed-off-by: Li Feng --- hw/scsi/vhost-user-scsi.c | 48 +++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index dc109154ad..53a62c3170 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -115,8 +115,48 @@ static void vhost_user_scsi_reset(VirtIODevice *vdev) } } -static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) +static void vhost_user_scsi_handle_output(VirtIODevice *vdev, VirtQueue *vq) { + VHostUserSCSI *s = (VHostUserSCSI *)vdev; + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + Error *local_err = NULL; + int i, ret; + + if (!vdev->start_on_kick) { + return; + } + + if (!s->connected) { + return; + } + + if (vhost_dev_is_started(&vsc->dev)) { + return; + } + + /* + * Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start + * vhost here instead of waiting for .set_status(). + */ + ret = vhost_user_scsi_start(s, &local_err); + if (ret < 0) { + error_reportf_err(local_err, "vhost-user-scsi: vhost start failed: "); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + + /* Kick right away to begin processing requests already in vring */ + for (i = 0; i < vsc->dev.nvqs; i++) { + VirtQueue *kick_vq = virtio_get_queue(vdev, i); + + if (!virtio_queue_get_desc_addr(vdev, i)) { + continue; + } + event_notifier_set(virtio_queue_get_host_notifier(kick_vq)); + } } static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) @@ -246,9 +286,9 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; } - virtio_scsi_common_realize(dev, vhost_dummy_handle_output, - vhost_dummy_handle_output, - vhost_dummy_handle_output, &err); + virtio_scsi_common_realize(dev, vhost_user_scsi_handle_output, + vhost_user_scsi_handle_output, + vhost_user_scsi_handle_output, &err); if (err != NULL) { error_propagate(errp, err); return;