From patchwork Mon Feb 18 10:27:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 10817733 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B1AF14E1 for ; Mon, 18 Feb 2019 10:29:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1610129E87 for ; Mon, 18 Feb 2019 10:29:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08FE129EAD; Mon, 18 Feb 2019 10:29:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 958E829E87 for ; Mon, 18 Feb 2019 10:29:41 +0000 (UTC) Received: from localhost ([127.0.0.1]:55975 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gvgBA-0008UQ-Bz for patchwork-qemu-devel@patchwork.kernel.org; Mon, 18 Feb 2019 05:29:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43289) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gvgA5-0008U9-MY for qemu-devel@nongnu.org; Mon, 18 Feb 2019 05:28:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gvg9t-00064G-8G for qemu-devel@nongnu.org; Mon, 18 Feb 2019 05:28:26 -0500 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:42042) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gvg9n-0005yp-0y for qemu-devel@nongnu.org; Mon, 18 Feb 2019 05:28:16 -0500 Received: by mail-pl1-x643.google.com with SMTP id s1so8579082plp.9 for ; Mon, 18 Feb 2019 02:28:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=2lifT8xRQAIXVndYq9hB4p3CBr9vEIkdp0ZucylPyhA=; b=lWS06OFD/Raxj12x8Xp3/HKOdQjV2mvCAqTVd9Ttb6AHNFfH2lVAu+G67IaZjdBD8q JK/O+yZ4YnpeR+qU/PV/ZiQmhjuA8K9vkOjD/BBbnb5tqBtz5zIhX+8XBq0ks1eln1kC Ruazp+mQkBXOvWbYRJb5hCGBl3LJ6KOv5eaCmVVjY1QWRg1dBQuU9y9kYj3dI6wxy9Sh bvNtn6BNlwu8ZmlWs88nbp8VWnTBER2ApYBumPh8GpZUtu/V1qOHNpmsmMh7dQ3j7/do PfRbju+/FcwtHWuxjR/5efBxKqbOJfIbvvKp/3fN7p3VfBGEAOVjpPM8+Nl7Qpd5guKB nQUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=2lifT8xRQAIXVndYq9hB4p3CBr9vEIkdp0ZucylPyhA=; b=gYI5uihn/PPihjKf4HsI6YDXjdSrcX/UWR+AVrqPeKuoRkuHpt+Pu7moucZewbM4re FcNVgWpaGh2/5wOlAo5+aHFKtDiSlFqjA3BVjfhPdcuRMQ+WTkGXU6BU0OJ88hP7HOj2 G0PNxap45hpy9vBWxJoHHWUJVw2cKA8kWoW9tCaa8tIq5uKMZjBbmpFnVF6GYWvl3WJr jxZwBsShPYRmrZ3fR99ENWt54jFlh8sFmecJuISb/Jivhk5Uq+ipqLT9oMOqPQYP3U98 h6oc61ff+ZUfTsanrtQPUYsfRzJrFJvP4oSVqs2AWH6sik3DgE2kzqaq3i2rPgKo0U4C R+Lg== X-Gm-Message-State: AHQUAuYmGeeo4oRAAbE3L+I6ZSp7b0pOskG5sm30rN7SdalD+l0sdhiE Q1Jt6brbmWNhQeyuMwL/BmY= X-Google-Smtp-Source: AHgI3IYlwAoo/yHGEM6Ra9v+7NfdlwYe4A+TDClrhgPj8qyqJHSSFjCmTE3SAq+3DxyZo/Tu5FOODA== X-Received: by 2002:a17:902:bcc7:: with SMTP id o7mr15991771pls.89.1550485692150; Mon, 18 Feb 2019 02:28:12 -0800 (PST) Received: from localhost ([116.247.112.152]) by smtp.gmail.com with ESMTPSA id p7sm19602609pfa.22.2019.02.18.02.28.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 18 Feb 2019 02:28:11 -0800 (PST) From: elohimes@gmail.com X-Google-Original-From: xieyongji@baidu.com To: mst@redhat.com, stefanha@gmail.com, marcandre.lureau@redhat.com, berrange@redhat.com, jasowang@redhat.com, maxime.coquelin@redhat.com, yury-kotov@yandex-team.ru, wrfsh@yandex-team.ru Date: Mon, 18 Feb 2019 18:27:41 +0800 Message-Id: <20190218102748.2242-1-xieyongji@baidu.com> X-Mailer: git-send-email 2.17.1 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::643 Subject: [Qemu-devel] [PATCH v6 0/7] vhost-user-blk: Add support for backend reconnecting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nixun@baidu.com, qemu-devel@nongnu.org, lilin24@baidu.com, zhangyu31@baidu.com, chaiwen@baidu.com, Xie Yongji Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Xie Yongji This patchset is aimed at supporting qemu to reconnect vhost-user-blk backend after vhost-user-blk backend crash or restart. The patch 1 introduces two new messages VHOST_USER_GET_INFLIGHT_FD and VHOST_USER_SET_INFLIGHT_FD to support transferring shared buffer between qemu and backend. The patch 2 deletes some redundant check in contrib/libvhost-user.c. The patch 3,4 are the corresponding libvhost-user patches of patch 1. Make libvhost-user support VHOST_USER_GET_INFLIGHT_FD and VHOST_USER_SET_INFLIGHT_FD. The patch 5 allows vhost-user-blk to use the two new messages to get/set inflight buffer from/to backend. The patch 6 supports vhost-user-blk to reconnect backend when connection closed. The patch 7 introduces VHOST_USER_PROTOCOL_F_SLAVE_SHMFD to vhost-user-blk backend which is used to tell qemu that we support reconnecting now. To use it, we could start qemu with: qemu-system-x86_64 \ -chardev socket,id=char0,path=/path/vhost.socket,reconnect=1, \ -device vhost-user-blk-pci,chardev=char0 \ and start vhost-user-blk backend with: vhost-user-blk -b /path/file -s /path/vhost.socket Then we can restart vhost-user-blk at any time during VM running. V5 to V6: - Document the layout in inflight buffer for packed virtqueue - Rework the layout in inflight buffer for split virtqueue - Remove version field in VhostUserInflight - Add a patch to remove some redundant check in contrib/libvhost-user.c - Document more details in vhost-user.txt V4 to V5: - Drop patch that enables "nowait" option on client sockets - Support resubmitting inflight I/O in order - Make inflight I/O tracking more robust - Remove align field and add queue size field in VhostUserInflight - Document more details in vhost-user.txt V3 to V4: - Drop messages VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD - Introduce two new messages VHOST_USER_GET_INFLIGHT_FD and VHOST_USER_SET_INFLIGHT_FD - Allocate inflight buffer in backend rather than in qemu - Document a recommended format for inflight buffer V2 to V3: - Using exisiting wait/nowait options to control connection on client sockets instead of introducing "disconnected" option. - Support the case that vhost-user backend restart during initialzation of vhost-user-blk device. V1 to V2: - Introduce "disconnected" option for chardev instead of reuse "wait" option - Support the case that QEMU starts before vhost-user backend - Drop message VHOST_USER_SET_VRING_INFLIGHT - Introduce two new messages VHOST_USER_GET_SHM_SIZE and VHOST_USER_SET_SHM_FD Xie Yongji (7): vhost-user: Support transferring inflight buffer between qemu and backend libvhost-user: Remove unnecessary FD flag check for event file descriptors libvhost-user: Introduce vu_queue_map_desc() libvhost-user: Support tracking inflight I/O in shared memory vhost-user-blk: Add support to get/set inflight buffer vhost-user-blk: Add support to reconnect backend contrib/vhost-user-blk: enable inflight I/O tracking Makefile | 2 +- contrib/libvhost-user/libvhost-user.c | 400 ++++++++++++++++++++---- contrib/libvhost-user/libvhost-user.h | 58 ++++ contrib/vhost-user-blk/vhost-user-blk.c | 3 +- docs/interop/vhost-user.txt | 264 ++++++++++++++++ hw/block/vhost-user-blk.c | 229 +++++++++++--- hw/virtio/vhost-user.c | 107 +++++++ hw/virtio/vhost.c | 96 ++++++ include/hw/virtio/vhost-backend.h | 10 + include/hw/virtio/vhost-user-blk.h | 5 + include/hw/virtio/vhost.h | 18 ++ 11 files changed, 1084 insertions(+), 108 deletions(-)