From patchwork Fri Sep 22 11:46:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13395649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EFAA0C04AAB for ; Fri, 22 Sep 2023 11:51:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qjeem-00084D-RH; Fri, 22 Sep 2023 07:49:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qjeek-00083N-9S for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:10 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qjeeh-0000d1-Bn for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:10 -0400 Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-274928c74b0so1384805a91.3 for ; Fri, 22 Sep 2023 04:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695383226; x=1695988026; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1eP8kuKM1qKZ8jHbFZXOAaKaPzBH+J+kNlsheIRdgb4=; b=wMfVj90Rveau3e7Op4ZCb/ekipFzkWBwuucV7VZO75RNJrLqeC3hIa52z+LQxNbq9p NUiDflVENJGc0oTm7kh7O6cO3fQmqpG92TBL5KSFNgj7rFC9wIevdhIB+Gaw0BCQM8NX KexMSjE0HQ9UVcApKPwk9P5X9HNzvpahOl5YE+kkEf23suYL+4KoX7F9DiiQ83DGQqUz XmRt4rxGIfBbdjp/HYgQvIntiwqfd9tO8yK5V7bi7cMZqVM8Tg7iQoBmPYbSLlVKBHan /HM/oyJKYzPC8lbJazaXwX/ZdDQBJRL/qPpXLoelbqlJbiebKzCoGW7VZaBx8itfurDz guGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695383226; x=1695988026; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1eP8kuKM1qKZ8jHbFZXOAaKaPzBH+J+kNlsheIRdgb4=; b=uTJlxeEZbYo9F2twkqH5UoTlq3k1TgtviYzCM/g1je3f6DtJh+LTt4X48HuO96chF6 bQCvkqo+erkV+3/9Owik+9pcRuuXI13VE4halUqpdlCuDcRk1drjl/VUPYr0DgcRUXnJ 7qxPQV8qEty/+lb3VUa8nbDFvrx8r736DjIG7sV4U2YScbIyfAEI9rdGDkjGXcyzsUVr o862Bjo0dGRtl8Qa376dy9uvuxj/Lerfori2wDAd5zTHJmYmeYBas/UG39Xd5Gc2f/q2 /oLiX6yPJBCSGKmorqNLnav6px6lM6sc0HezPpFPnWr651dVeQ+Sz4V4CJi372xr1qFA 6f0w== X-Gm-Message-State: AOJu0Yz90pJJnssFGanSq5TxGVCJhfopdZccYTz/gWOza7zQKtb3H+ls HBt1qGHWbgdAo2tHIYcX/KelAg== X-Google-Smtp-Source: AGHT+IEK7oFaVPM8D2Ye65pvi5ugoF6nWOtOEc9Npa8J9yGm9TKXCwgS4F1LTUmaXsCA8e/S8/l3GQ== X-Received: by 2002:a17:90a:4dc6:b0:268:1e51:3496 with SMTP id r6-20020a17090a4dc600b002681e513496mr8004197pjl.21.1695383225373; Fri, 22 Sep 2023 04:47:05 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id r6-20020a17090a438600b00263f41a655esm3303961pjg.43.2023.09.22.04.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 04:47:05 -0700 (PDT) From: Li Feng To: Markus Armbruster , Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v6 1/5] vhost-user-common: send get_inflight_fd once Date: Fri, 22 Sep 2023 19:46:11 +0800 Message-ID: <20230922114625.5786-2-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230922114625.5786-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230922114625.5786-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::1036; envelope-from=fengli@smartx.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Currently the get_inflight_fd will be sent every time the device is started, and the backend will allocate shared memory to save the inflight state. If the backend finds that it receives the second get_inflight_fd, it will release the previous shared memory, which breaks inflight working logic. This patch is a preparation for the following patches. Signed-off-by: Li Feng --- hw/scsi/vhost-scsi-common.c | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a06f01af26..a61cd0e907 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -52,20 +52,28 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vsc->dev.acked_features = vdev->guest_features; - assert(vsc->inflight == NULL); - vsc->inflight = g_new0(struct vhost_inflight, 1); - ret = vhost_dev_get_inflight(&vsc->dev, - vs->conf.virtqueue_size, - vsc->inflight); + ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error get inflight: %d", -ret); + error_report("Error setting inflight format: %d", -ret); goto err_guest_notifiers; } - ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); - if (ret < 0) { - error_report("Error set inflight: %d", -ret); - goto err_guest_notifiers; + if (vsc->inflight) { + if (!vsc->inflight->addr) { + ret = vhost_dev_get_inflight(&vsc->dev, + vs->conf.virtqueue_size, + vsc->inflight); + if (ret < 0) { + error_report("Error getting inflight: %d", -ret); + goto err_guest_notifiers; + } + } + + ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); + if (ret < 0) { + error_report("Error setting inflight: %d", -ret); + goto err_guest_notifiers; + } } ret = vhost_dev_start(&vsc->dev, vdev, true); @@ -85,9 +93,6 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) return ret; err_guest_notifiers: - g_free(vsc->inflight); - vsc->inflight = NULL; - k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&vsc->dev, vdev); @@ -111,12 +116,6 @@ void vhost_scsi_common_stop(VHostSCSICommon *vsc) } assert(ret >= 0); - if (vsc->inflight) { - vhost_dev_free_inflight(vsc->inflight); - g_free(vsc->inflight); - vsc->inflight = NULL; - } - vhost_dev_disable_notifiers(&vsc->dev, vdev); } From patchwork Fri Sep 22 11:46:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13395646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 12A70CD4F5B for ; Fri, 22 Sep 2023 11:50:03 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qjeer-00085p-3B; Fri, 22 Sep 2023 07:49:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qjeen-00084R-2u for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:13 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qjeek-0000ej-Vr for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:12 -0400 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-578b407045bso1552824a12.0 for ; Fri, 22 Sep 2023 04:47:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695383230; x=1695988030; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ak3cZicP6FXgjNSZA0EnmzY0v3FARPxaB+bcgiWIe/k=; b=W7nba+RQT7MwcJIz/2F5el851s0xIOEjzUih1/qyvLIKCDc+i+6K+rKChoxR08zym+ 4DEm7As+xL2supOUycpjh00wmJp8cFckTB6TUV+pJS5Ulq2q8FW7uCmz1exz6bxUVgf+ GtlrIzDblTpe5+OpC9PoZbrRuuY/Yri0MVW9bENq1Y4ydonTDWU/kdJyvP3d2PMNFRBZ AL+9XOjG7fjxtrr1/lB/IncAiY7joyeW/R1tg4tXoIIbxwPPaezvAw+hURc6oWFisHsA Y+OOXosZBtYsFnODSUJ/g3VKsc7YcfukJP8iv4b4MRCrAksGu1v9hoNOl66sBL9xQkPp XN5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695383230; x=1695988030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ak3cZicP6FXgjNSZA0EnmzY0v3FARPxaB+bcgiWIe/k=; b=G+s3NAH15LR5u8tGEf8EpGQBTOGd17uouI7Py0T5Hx756lCbwxg+0qLG9m47FQPnVh FuaRF3w08SusmQyhg5vzdAkho/MdpNc9dZF5DJ16fgT8oaUHyCqZvulOYi7TAkNI9H+N ZQcIw37ZY+ZwAg555JUqgeX5rogccaYCeVA/aYtkleJ81LfxXQ5E/Dn/g96KfElidcz5 IIkfmDLcSiZBd++K5jCJJ6L6CXq8JJgLlZSdA7EPdDbWg1ET0PMDA/8hNw982+4E37ey DLkehuzqEOKkHqQfF7uclltIt/1kDPUHpwtNDkgvjy7yfubM3t+3w15DD028150U/3Uy 31wA== X-Gm-Message-State: AOJu0Yw6woF85HTt45stT0Z2jge0aKsYbXnivz/Mx2X1GAzgog9tZh// wzNnVMY9Rxv80QDilvqFKNnezQ== X-Google-Smtp-Source: AGHT+IEdtggdijxsSdrzKJYLiQKj0JE0DMcNvELy95ai1DCaxGuMl2D5HMOYaDh876aeh8wj172NRQ== X-Received: by 2002:a17:90a:cf0f:b0:274:7725:ed9c with SMTP id h15-20020a17090acf0f00b002747725ed9cmr7386034pju.35.1695383229087; Fri, 22 Sep 2023 04:47:09 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id r6-20020a17090a438600b00263f41a655esm3303961pjg.43.2023.09.22.04.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 04:47:08 -0700 (PDT) From: Li Feng To: Markus Armbruster , Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v6 2/5] vhost: move and rename the conn retry times Date: Fri, 22 Sep 2023 19:46:12 +0800 Message-ID: <20230922114625.5786-3-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230922114625.5786-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230922114625.5786-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::52a; envelope-from=fengli@smartx.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Multiple devices need this macro, move it to a common header. Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/block/vhost-user-blk.c | 4 +--- hw/virtio/vhost-user-gpio.c | 3 +-- include/hw/virtio/vhost.h | 2 ++ 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index eecf3f7a81..3c69fa47d5 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -32,8 +32,6 @@ #include "sysemu/sysemu.h" #include "sysemu/runstate.h" -#define REALIZE_CONNECTION_RETRIES 3 - static const int user_feature_bits[] = { VIRTIO_BLK_F_SIZE_MAX, VIRTIO_BLK_F_SEG_MAX, @@ -482,7 +480,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) s->inflight = g_new0(struct vhost_inflight, 1); s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues); - retries = REALIZE_CONNECTION_RETRIES; + retries = VU_REALIZE_CONN_RETRIES; assert(!*errp); do { if (*errp) { diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c index 3b013f2d0f..d9979aa5db 100644 --- a/hw/virtio/vhost-user-gpio.c +++ b/hw/virtio/vhost-user-gpio.c @@ -15,7 +15,6 @@ #include "standard-headers/linux/virtio_ids.h" #include "trace.h" -#define REALIZE_CONNECTION_RETRIES 3 #define VHOST_NVQS 2 /* Features required from VirtIO */ @@ -359,7 +358,7 @@ static void vu_gpio_device_realize(DeviceState *dev, Error **errp) qemu_chr_fe_set_handlers(&gpio->chardev, NULL, NULL, vu_gpio_event, NULL, dev, NULL, true); - retries = REALIZE_CONNECTION_RETRIES; + retries = VU_REALIZE_CONN_RETRIES; g_assert(!*errp); do { if (*errp) { diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 6a173cb9fa..ca3131b1af 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -8,6 +8,8 @@ #define VHOST_F_DEVICE_IOTLB 63 #define VHOST_USER_F_PROTOCOL_FEATURES 30 +#define VU_REALIZE_CONN_RETRIES 3 + /* Generic structures common for any vhost based device. */ struct vhost_inflight { From patchwork Fri Sep 22 11:46:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13395648 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 211E8C04AAB for ; Fri, 22 Sep 2023 11:50:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qjeeu-00086x-Vf; Fri, 22 Sep 2023 07:49:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qjeer-00085r-9m for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:17 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qjeeo-0000f7-R8 for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:16 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-690bd8f89baso1803661b3a.2 for ; Fri, 22 Sep 2023 04:47:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695383233; x=1695988033; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+XH7wdtDBHjkJj1TRjhNqIG/+/Tyzm+qcFAhIy62ShA=; b=K5NFFG4g2gFKSDIkgImlRuVIxrxw4pI1vXYID/MpZDwb5/D9TQAg5VcvLJlpMSPLZA WCA4VvrZ+/0LO8FAQgSA5EB0UmuP8RfojULziXyVneaXrXkQMv14rXl+t+akxdEod8rv JBcEu/bvAJlX+NiVuDIEqDyoVGGppIoq8lFiHwPnonVNXYhgx5XzNMCg0WRKV75oojLz T17foPPs508pnEjH7jYU2fAEEg/Llfax3QcA4ObAOdpP4MGc+fwDRYyh5CVm774vXDNa PLNAon4hEaBqNSeghcSwM23Ow8XzAvPRLQrsx/5e7SxptKeW/rZL3Aihpxl1EHGZBFXU 0XoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695383233; x=1695988033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+XH7wdtDBHjkJj1TRjhNqIG/+/Tyzm+qcFAhIy62ShA=; b=TXIwJ1ME7Eee5P1Sa9Fook5BxnigLO7XMB6B7lsXcJRT5U9LLA2HBTMUZdjAXgSA+I 5WSCvW0RqPi5Gai777bdrRw4X1LTSrOQoIiGeFYwuHUnujK2ANmGozVv4B46XEIzZT0i DrxPeTc+YI9aWXzwSIKyHtYlcukAdVwCbqQgp25oFyLnrXlDPfixuem777hbrJGEgM9n 7EQfPvmyDczTnV3UoMDAM9ey90v6mI8Irm3ROmTeJj79sG8Fe5k4Xv+Vmu3Wt4geXca1 SlLXOK+gwvvjv7QhuZ+MJxLoyG8/Z5+KER61soPKQIsGR7hJ5oa0TqHwMGOIEuG9D7JG /5ow== X-Gm-Message-State: AOJu0YyZwFdNxsaKCCxBVqpDOcIoQ5fh0W+bVO29Lcb9N2HotIJAULbG ObSl3pqRnyOvuSoqoS71jTLOxA== X-Google-Smtp-Source: AGHT+IG9lzzsY25Yl8/1aZZth/473Qt4iMnZkb44DXRqc8NtVOk9spxsuiiIPXNNpVIR4YNoSvXBKg== X-Received: by 2002:a05:6a20:7d99:b0:154:9943:7320 with SMTP id v25-20020a056a207d9900b0015499437320mr8615623pzj.28.1695383232568; Fri, 22 Sep 2023 04:47:12 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id r6-20020a17090a438600b00263f41a655esm3303961pjg.43.2023.09.22.04.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 04:47:12 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v6 3/5] vhost-user-scsi: support reconnect to backend Date: Fri, 22 Sep 2023 19:46:13 +0800 Message-ID: <20230922114625.5786-4-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230922114625.5786-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230922114625.5786-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::42d; envelope-from=fengli@smartx.com; helo=mail-pf1-x42d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org If the backend crashes and restarts, the device is broken. This patch adds reconnect for vhost-user-scsi. This patch also improves the error messages, and reports some silent errors. Tested with spdk backend. Signed-off-by: Li Feng --- hw/scsi/vhost-scsi-common.c | 16 +- hw/scsi/vhost-scsi.c | 5 +- hw/scsi/vhost-user-scsi.c | 204 +++++++++++++++++++++++--- include/hw/virtio/vhost-scsi-common.h | 2 +- include/hw/virtio/vhost-user-scsi.h | 4 + 5 files changed, 201 insertions(+), 30 deletions(-) diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a61cd0e907..4c8637045d 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -16,6 +16,7 @@ */ #include "qemu/osdep.h" +#include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/module.h" #include "hw/virtio/vhost.h" @@ -25,7 +26,7 @@ #include "hw/virtio/virtio-access.h" #include "hw/fw-path-provider.h" -int vhost_scsi_common_start(VHostSCSICommon *vsc) +int vhost_scsi_common_start(VHostSCSICommon *vsc, Error **errp) { int ret, i; VirtIODevice *vdev = VIRTIO_DEVICE(vsc); @@ -35,18 +36,19 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) VirtIOSCSICommon *vs = (VirtIOSCSICommon *)vsc; if (!k->set_guest_notifiers) { - error_report("binding does not support guest notifiers"); + error_setg(errp, "binding does not support guest notifiers"); return -ENOSYS; } ret = vhost_dev_enable_notifiers(&vsc->dev, vdev); if (ret < 0) { + error_setg_errno(errp, -ret, "Error enabling host notifiers"); return ret; } ret = k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, true); if (ret < 0) { - error_report("Error binding guest notifier"); + error_setg_errno(errp, -ret, "Error binding guest notifier"); goto err_host_notifiers; } @@ -54,7 +56,7 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error setting inflight format: %d", -ret); + error_setg_errno(errp, -ret, "Error setting inflight format"); goto err_guest_notifiers; } @@ -64,21 +66,21 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vs->conf.virtqueue_size, vsc->inflight); if (ret < 0) { - error_report("Error getting inflight: %d", -ret); + error_setg_errno(errp, -ret, "Error getting inflight"); goto err_guest_notifiers; } } ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); if (ret < 0) { - error_report("Error setting inflight: %d", -ret); + error_setg_errno(errp, -ret, "Error setting inflight"); goto err_guest_notifiers; } } ret = vhost_dev_start(&vsc->dev, vdev, true); if (ret < 0) { - error_report("Error start vhost dev"); + error_setg_errno(errp, -ret, "Error starting vhost dev"); goto err_guest_notifiers; } diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 443f67daa4..01a3ab4277 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -75,6 +75,7 @@ static int vhost_scsi_start(VHostSCSI *s) int ret, abi_version; VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); const VhostOps *vhost_ops = vsc->dev.vhost_ops; + Error *local_err = NULL; ret = vhost_ops->vhost_scsi_get_abi_version(&vsc->dev, &abi_version); if (ret < 0) { @@ -88,14 +89,14 @@ static int vhost_scsi_start(VHostSCSI *s) return -ENOSYS; } - ret = vhost_scsi_common_start(vsc); + ret = vhost_scsi_common_start(vsc, &local_err); if (ret < 0) { return ret; } ret = vhost_scsi_set_endpoint(s); if (ret < 0) { - error_report("Error setting vhost-scsi endpoint"); + error_reportf_err(local_err, "Error setting vhost-scsi endpoint"); vhost_scsi_common_stop(vsc); } diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index ee99b19e7a..dc109154ad 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -43,26 +43,56 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_RESET_DEVICE = 13, }; +static int vhost_user_scsi_start(VHostUserSCSI *s, Error **errp) +{ + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + int ret; + + ret = vhost_scsi_common_start(vsc, errp); + s->started_vu = (ret < 0 ? false : true); + + return ret; +} + +static void vhost_user_scsi_stop(VHostUserSCSI *s) +{ + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + + if (!s->started_vu) { + return; + } + s->started_vu = false; + + vhost_scsi_common_stop(vsc); +} + static void vhost_user_scsi_set_status(VirtIODevice *vdev, uint8_t status) { VHostUserSCSI *s = (VHostUserSCSI *)vdev; + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - bool start = (status & VIRTIO_CONFIG_S_DRIVER_OK) && vdev->vm_running; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + bool should_start = virtio_device_should_start(vdev, status); + Error *local_err = NULL; + int ret; - if (vhost_dev_is_started(&vsc->dev) == start) { + if (!s->connected) { return; } - if (start) { - int ret; + if (vhost_dev_is_started(&vsc->dev) == should_start) { + return; + } - ret = vhost_scsi_common_start(vsc); + if (should_start) { + ret = vhost_user_scsi_start(s, &local_err); if (ret < 0) { - error_report("unable to start vhost-user-scsi: %s", strerror(-ret)); - exit(1); + error_reportf_err(local_err, "unable to start vhost-user-scsi: %s", + strerror(-ret)); + qemu_chr_fe_disconnect(&vs->conf.chardev); } } else { - vhost_scsi_common_stop(vsc); + vhost_user_scsi_stop(s); } } @@ -89,14 +119,127 @@ static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) { } +static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret = 0; + + if (s->connected) { + return 0; + } + s->connected = true; + + vsc->dev.num_queues = vs->conf.num_queues; + vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; + vsc->dev.vqs = s->vhost_vqs; + vsc->dev.vq_index = 0; + vsc->dev.backend_features = 0; + + ret = vhost_dev_init(&vsc->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0, + errp); + if (ret < 0) { + return ret; + } + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + ret = vhost_user_scsi_start(s, errp); + if (ret < 0) { + return ret; + } + } + + return 0; +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event); + +static void vhost_user_scsi_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + if (!s->connected) { + return; + } + s->connected = false; + + vhost_user_scsi_stop(s); + + vhost_dev_cleanup(&vsc->dev); + + /* Re-instate the event handler for new connections */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, dev, NULL, true); +} + +static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(vdev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + Error *local_err = NULL; + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_scsi_connect(dev, &local_err) < 0) { + error_report_err(local_err); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + /* defer close until later to avoid circular close */ + vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, + vhost_user_scsi_disconnect); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static int vhost_user_scsi_realize_connect(VHostUserSCSI *s, Error **errp) +{ + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + int ret; + + s->connected = false; + + ret = qemu_chr_fe_wait_connected(&vs->conf.chardev, errp); + if (ret < 0) { + return ret; + } + + ret = vhost_user_scsi_connect(dev, errp); + if (ret < 0) { + qemu_chr_fe_disconnect(&vs->conf.chardev); + return ret; + } + assert(s->connected); + + return 0; +} + static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) { + ERRP_GUARD(); VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = NULL; Error *err = NULL; int ret; + int retries = VU_REALIZE_CONN_RETRIES; if (!vs->conf.chardev.chr) { error_setg(errp, "vhost-user-scsi: missing chardev"); @@ -115,18 +258,28 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) goto free_virtio; } - vsc->dev.nvqs = VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; - vsc->dev.vqs = g_new0(struct vhost_virtqueue, vsc->dev.nvqs); - vsc->dev.vq_index = 0; - vsc->dev.backend_features = 0; - vqs = vsc->dev.vqs; + vsc->inflight = g_new0(struct vhost_inflight, 1); + s->vhost_vqs = g_new0(struct vhost_virtqueue, + VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues); + + assert(!*errp); + do { + if (*errp) { + error_prepend(errp, "Reconnecting after error: "); + error_report_err(*errp); + *errp = NULL; + } + ret = vhost_user_scsi_realize_connect(s, errp); + } while (ret < 0 && retries--); - ret = vhost_dev_init(&vsc->dev, &s->vhost_user, - VHOST_BACKEND_TYPE_USER, 0, errp); if (ret < 0) { goto free_vhost; } + /* we're fully initialized, now we can operate, so add the handler */ + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, + vhost_user_scsi_event, NULL, (void *)dev, + NULL, true); /* Channel and lun both are 0 for bootable vhost-user-scsi disk */ vsc->channel = 0; vsc->lun = 0; @@ -135,8 +288,12 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; free_vhost: + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + g_free(vsc->inflight); + vsc->inflight = NULL; vhost_user_cleanup(&s->vhost_user); - g_free(vqs); + free_virtio: virtio_scsi_common_unrealize(dev); } @@ -146,16 +303,23 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) VirtIODevice *vdev = VIRTIO_DEVICE(dev); VHostUserSCSI *s = VHOST_USER_SCSI(dev); VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); - struct vhost_virtqueue *vqs = vsc->dev.vqs; + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); /* This will stop the vhost backend. */ vhost_user_scsi_set_status(vdev, 0); + qemu_chr_fe_set_handlers(&vs->conf.chardev, NULL, NULL, NULL, NULL, NULL, + NULL, false); vhost_dev_cleanup(&vsc->dev); - g_free(vqs); + g_free(s->vhost_vqs); + s->vhost_vqs = NULL; + + vhost_dev_free_inflight(vsc->inflight); + g_free(vsc->inflight); + vsc->inflight = NULL; - virtio_scsi_common_unrealize(dev); vhost_user_cleanup(&s->vhost_user); + virtio_scsi_common_unrealize(dev); } static Property vhost_user_scsi_properties[] = { diff --git a/include/hw/virtio/vhost-scsi-common.h b/include/hw/virtio/vhost-scsi-common.h index 18f115527c..c5d2c09455 100644 --- a/include/hw/virtio/vhost-scsi-common.h +++ b/include/hw/virtio/vhost-scsi-common.h @@ -39,7 +39,7 @@ struct VHostSCSICommon { struct vhost_inflight *inflight; }; -int vhost_scsi_common_start(VHostSCSICommon *vsc); +int vhost_scsi_common_start(VHostSCSICommon *vsc, Error **errp); void vhost_scsi_common_stop(VHostSCSICommon *vsc); char *vhost_scsi_common_get_fw_dev_path(FWPathProvider *p, BusState *bus, DeviceState *dev); diff --git a/include/hw/virtio/vhost-user-scsi.h b/include/hw/virtio/vhost-user-scsi.h index 521b08e559..b405ec952a 100644 --- a/include/hw/virtio/vhost-user-scsi.h +++ b/include/hw/virtio/vhost-user-scsi.h @@ -29,6 +29,10 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserSCSI, VHOST_USER_SCSI) struct VHostUserSCSI { VHostSCSICommon parent_obj; VhostUserState vhost_user; + bool connected; + bool started_vu; + + struct vhost_virtqueue *vhost_vqs; }; #endif /* VHOST_USER_SCSI_H */ From patchwork Fri Sep 22 11:46:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13395647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3392BCD4F5B for ; Fri, 22 Sep 2023 11:50:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qjeew-00087W-05; Fri, 22 Sep 2023 07:49:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qjeet-000867-CO for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:19 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qjeer-0000fJ-QE for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:19 -0400 Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2746889aa89so1411840a91.2 for ; Fri, 22 Sep 2023 04:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695383236; x=1695988036; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HPRPD1pbMza8Qy14aglX4a9WyhzajqmGhD+mOHsf0R4=; b=U9PPXlgSSCnQ4ijxQsUGctJ/WWeHKI1D8t2SYOYJ7A174b21NYkVtjrc1qmhcg1gPr TUJVWfEsExxzU2hFVGl3F0Plz+dtFEnQrsZYdmQgRMkvXP5I86Pntz05AmC5C0H3QXsB p9/3nwTd993vDsYuXYINQRI22EOTM9ICXDF/5pj2Zufbk301WblTedjdhiNrX68VEM4u KLFzEZivhMCtDxr7faaIkfESz81Dv137phJLvnM0xK8HcQBUAnq2mRhRGCibHkcio4cR W8O9/2zBrgdOObF5pWl6h4/Efmm2FVKcWwAl97WoMGe+emIyCbY64vHder9gX4Cy5+VB GRqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695383236; x=1695988036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HPRPD1pbMza8Qy14aglX4a9WyhzajqmGhD+mOHsf0R4=; b=k+jX6QfoQ+Nq9Ylb47rYJ40JEuZbT5+ah+49WZB2xUCOj0nCwBqaNjmiN2mBFmtlth ak85gQaE5njRoKGFvvsdknXvJlySI+VzlXGI8/x8b83iexaVArGeVx4wR7LirH6BDnI4 9Lr1AM1ULxq9sqcNQWhwIeNtMmzim3QguR1IibM5uvkmVL3/sKS6hQTOtLQ4LB1lB1I+ wJCTD5fY/6YQunIq05bEhaz3vRUFFRBEl9yepoJZcLp53hkIeDecLvzYRC989WMOYs9M nuN9DQXNA9PGf+diaMC2/0OwyYBTi8i6KNmVN9twgCDHGus3oU8l4Lma7sf/mLHW4+3N syxw== X-Gm-Message-State: AOJu0Yx8cFmbxL/WLxy57OD5GhZTvY6aFT5VQfbLJD10CkJYrVmd7p77 bnsIvvUr9DL24BmcG1I7ifSAKZxdKeWS8T9Tb6F3LQI1cQI= X-Google-Smtp-Source: AGHT+IFbkJUqEYMMiLa7hpQvpDG8YePygkhZ3/l3nZvT5O/2TkZdiataQrR/Zh0Hvu8FWKfoZ92wAQ== X-Received: by 2002:a17:90a:f297:b0:274:e8e0:1503 with SMTP id fs23-20020a17090af29700b00274e8e01503mr8264491pjb.16.1695383236107; Fri, 22 Sep 2023 04:47:16 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id r6-20020a17090a438600b00263f41a655esm3303961pjg.43.2023.09.22.04.47.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 04:47:15 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v6 4/5] vhost-user-scsi: start vhost when guest kicks Date: Fri, 22 Sep 2023 19:46:14 +0800 Message-ID: <20230922114625.5786-5-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230922114625.5786-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230922114625.5786-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::1031; envelope-from=fengli@smartx.com; helo=mail-pj1-x1031.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Let's keep the same behavior as vhost-user-blk. Some old guests kick virtqueue before setting VIRTIO_CONFIG_S_DRIVER_OK. Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz --- hw/scsi/vhost-user-scsi.c | 48 +++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index dc109154ad..53a62c3170 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -115,8 +115,48 @@ static void vhost_user_scsi_reset(VirtIODevice *vdev) } } -static void vhost_dummy_handle_output(VirtIODevice *vdev, VirtQueue *vq) +static void vhost_user_scsi_handle_output(VirtIODevice *vdev, VirtQueue *vq) { + VHostUserSCSI *s = (VHostUserSCSI *)vdev; + DeviceState *dev = &s->parent_obj.parent_obj.parent_obj.parent_obj; + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + + Error *local_err = NULL; + int i, ret; + + if (!vdev->start_on_kick) { + return; + } + + if (!s->connected) { + return; + } + + if (vhost_dev_is_started(&vsc->dev)) { + return; + } + + /* + * Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start + * vhost here instead of waiting for .set_status(). + */ + ret = vhost_user_scsi_start(s, &local_err); + if (ret < 0) { + error_reportf_err(local_err, "vhost-user-scsi: vhost start failed: "); + qemu_chr_fe_disconnect(&vs->conf.chardev); + return; + } + + /* Kick right away to begin processing requests already in vring */ + for (i = 0; i < vsc->dev.nvqs; i++) { + VirtQueue *kick_vq = virtio_get_queue(vdev, i); + + if (!virtio_queue_get_desc_addr(vdev, i)) { + continue; + } + event_notifier_set(virtio_queue_get_host_notifier(kick_vq)); + } } static int vhost_user_scsi_connect(DeviceState *dev, Error **errp) @@ -246,9 +286,9 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) return; } - virtio_scsi_common_realize(dev, vhost_dummy_handle_output, - vhost_dummy_handle_output, - vhost_dummy_handle_output, &err); + virtio_scsi_common_realize(dev, vhost_user_scsi_handle_output, + vhost_user_scsi_handle_output, + vhost_user_scsi_handle_output, &err); if (err != NULL) { error_propagate(errp, err); return; From patchwork Fri Sep 22 11:46:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 13395645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B37CDC04AAB for ; Fri, 22 Sep 2023 11:49:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qjef5-00088E-Ud; Fri, 22 Sep 2023 07:49:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qjef2-00087n-Gy for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:28 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qjeev-0000fb-PP for qemu-devel@nongnu.org; Fri, 22 Sep 2023 07:49:28 -0400 Received: by mail-pj1-x1033.google.com with SMTP id 98e67ed59e1d1-2745cd2ba68so1403725a91.0 for ; Fri, 22 Sep 2023 04:47:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695383240; x=1695988040; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tZ2JAP3KP0J6f/lwwcf0OERy1rLupfPFuw/u9x6M7u8=; b=Pz12Lpda4RizgTPS2TMLLyxZ5xOraA8+vjRtatFLI/f25hIJopfV9Dlj9sInDU8quu DO7ZrDWUofXM2lKqrwgzkB3g2fdEBMElztjE+0yMEEZcyijLCSHgrwajtTh8hNsHAC+e J3/UmDpplC8kyWZcjnfrk5EWl0tvqXuBc8R8j63YAEZSqPGarh1s2S+Zt1wVsTqcW/+1 6JwhGGSsUpYAgooo9No0p7gozjP5t0MPpKQiBDKEaQm+PukFXjnYMMPDpxcuFlKRqE0j miYjzp0W0enoeAX/y1EAOQSCyb+mij98c8aT0H4Ila6ppN8lTz4oCOpxPnXnvzU7pREQ mMTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695383240; x=1695988040; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tZ2JAP3KP0J6f/lwwcf0OERy1rLupfPFuw/u9x6M7u8=; b=dMWabdvkOjZnOqMlkeVd6H8Zy6INFLUOKZ/UUK7WWzX1ny48I8ehfibqKyvXFkRBJb E1CjDv4185dAkxKbocuvZm+yWRbtcSCgYbv13YEzGqNcmb5H3e2g+52AIsc+cepmMIsh BwVUuEKYFz581b2SU5QCbinsi1GZZhav+UfpUGapmPNAuMT8TRJtbYwg+V8UyjFBbcSN 62Bq0R287fMxRYkhE4wqiLbiXTowzw6m4n7/jBKFlpxe4huC8w0XCCb/r8t/X3/zm+Sg UZulcSvw8yImv7LBNGQhf0gdlvdfMzSdHKXqQQs76im1dtDHouYd5jwI+Tv6xmtUm/LL pWDA== X-Gm-Message-State: AOJu0YxB5PfOU+2X61YwX+2lFqxXqGJXpLuHUGNhPjj/ClfS9NYijs6i xyPFmUItCRo1eLO1mCOPzI1ZjQ== X-Google-Smtp-Source: AGHT+IFFymD04GnjOx+/oAQChw64X7ONlsJFNcBlWajJKcqKmwTh6ub3Gx6EQoMZc3wKEIArftY4Wg== X-Received: by 2002:a17:90a:9bca:b0:26b:534e:234 with SMTP id b10-20020a17090a9bca00b0026b534e0234mr7338937pjw.35.1695383239787; Fri, 22 Sep 2023 04:47:19 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id r6-20020a17090a438600b00263f41a655esm3303961pjg.43.2023.09.22.04.47.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 04:47:19 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v6 5/5] vhost-user: fix lost reconnect Date: Fri, 22 Sep 2023 19:46:15 +0800 Message-ID: <20230922114625.5786-6-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230922114625.5786-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230922114625.5786-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::1033; envelope-from=fengli@smartx.com; helo=mail-pj1-x1033.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When the vhost-user is reconnecting to the backend, and if the vhost-user fails at the get_features in vhost_dev_init(), then the reconnect will fail and it will not be retriggered forever. The reason is: When the vhost-user fails at get_features, the vhost_dev_cleanup will be called immediately. vhost_dev_cleanup calls 'memset(hdev, 0, sizeof(struct vhost_dev))'. The reconnect path is: vhost_user_blk_event vhost_user_async_close(.. vhost_user_blk_disconnect ..) qemu_chr_fe_set_handlers <----- clear the notifier callback schedule vhost_user_async_close_bh The vhost->vdev is null, so the vhost_user_blk_disconnect will not be called, then the event fd callback will not be reinstalled. All vhost-user devices have this issue, including vhost-user-blk/scsi. With this patch, if the vdev->vdev is null, the fd callback will still be reinstalled. Fixes: 71e076a07d ("hw/virtio: generalise CHR_EVENT_CLOSED handling") Signed-off-by: Li Feng Reviewed-by: Raphael Norwitz Reviewed-by: Raphael Norwitz --- hw/block/vhost-user-blk.c | 2 +- hw/scsi/vhost-user-scsi.c | 3 ++- hw/virtio/vhost-user-gpio.c | 2 +- hw/virtio/vhost-user.c | 10 ++++++++-- include/hw/virtio/vhost-user.h | 3 ++- 5 files changed, 14 insertions(+), 6 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 3c69fa47d5..95c758200d 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -391,7 +391,7 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &s->chardev, &s->dev, - vhost_user_blk_disconnect); + vhost_user_blk_disconnect, vhost_user_blk_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 53a62c3170..0effbb4787 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -238,7 +238,8 @@ static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, - vhost_user_scsi_disconnect); + vhost_user_scsi_disconnect, + vhost_user_scsi_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c index d9979aa5db..04c2cc79f4 100644 --- a/hw/virtio/vhost-user-gpio.c +++ b/hw/virtio/vhost-user-gpio.c @@ -283,7 +283,7 @@ static void vu_gpio_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &gpio->chardev, &gpio->vhost_dev, - vu_gpio_disconnect); + vu_gpio_disconnect, vu_gpio_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 8dcf049d42..7344f57ba7 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -2643,6 +2643,7 @@ typedef struct { DeviceState *dev; CharBackend *cd; struct vhost_dev *vhost; + IOEventHandler *event_cb; } VhostAsyncCallback; static void vhost_user_async_close_bh(void *opaque) @@ -2657,7 +2658,10 @@ static void vhost_user_async_close_bh(void *opaque) */ if (vhost->vdev) { data->cb(data->dev); - } + } else if (data->event_cb) { + qemu_chr_fe_set_handlers(data->cd, NULL, NULL, data->event_cb, + NULL, data->dev, NULL, true); + } g_free(data); } @@ -2669,7 +2673,8 @@ static void vhost_user_async_close_bh(void *opaque) */ void vhost_user_async_close(DeviceState *d, CharBackend *chardev, struct vhost_dev *vhost, - vu_async_close_fn cb) + vu_async_close_fn cb, + IOEventHandler *event_cb) { if (!runstate_check(RUN_STATE_SHUTDOWN)) { /* @@ -2685,6 +2690,7 @@ void vhost_user_async_close(DeviceState *d, data->dev = d; data->cd = chardev; data->vhost = vhost; + data->event_cb = event_cb; /* Disable any further notifications on the chardev */ qemu_chr_fe_set_handlers(chardev, diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h index 191216a74f..649e9dd54f 100644 --- a/include/hw/virtio/vhost-user.h +++ b/include/hw/virtio/vhost-user.h @@ -84,6 +84,7 @@ typedef void (*vu_async_close_fn)(DeviceState *cb); void vhost_user_async_close(DeviceState *d, CharBackend *chardev, struct vhost_dev *vhost, - vu_async_close_fn cb); + vu_async_close_fn cb, + IOEventHandler *event_cb); #endif