From patchwork Tue Apr 16 18:46:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Streetman X-Patchwork-Id: 10903833 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF45B161F for ; Tue, 16 Apr 2019 18:48:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C805628A00 for ; Tue, 16 Apr 2019 18:48:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B997028A0A; Tue, 16 Apr 2019 18:48:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5912C28A00 for ; Tue, 16 Apr 2019 18:48:14 +0000 (UTC) Received: from localhost ([127.0.0.1]:40951 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hGT7t-0002YR-Kc for patchwork-qemu-devel@patchwork.kernel.org; Tue, 16 Apr 2019 14:48:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52906) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hGT6U-0001fI-Rj for qemu-devel@nongnu.org; Tue, 16 Apr 2019 14:46:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hGT6T-0005bt-P8 for qemu-devel@nongnu.org; Tue, 16 Apr 2019 14:46:46 -0400 Received: from youngberry.canonical.com ([91.189.89.112]:48707) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hGT6T-0005YE-JA for qemu-devel@nongnu.org; Tue, 16 Apr 2019 14:46:45 -0400 Received: from mail-yw1-f71.google.com ([209.85.161.71]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1hGT6R-000072-JD for qemu-devel@nongnu.org; Tue, 16 Apr 2019 18:46:43 +0000 Received: by mail-yw1-f71.google.com with SMTP id 201so16283621ywr.13 for ; Tue, 16 Apr 2019 11:46:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JSL2reb0c3p5lrYQaX/Mundk1gFllcy7clmVFRRaQTg=; b=kkHjZArRtOV+Rhxt+Sh2GD3jGE2fqR/2APbiFnmq9F89AT/Lu9Ha42YM5s0z6Wu1if oxs24blKvkvITmrRt+hKntBkkdvEW5R4pfBrJDuUJoCxa3oN/8VwhVa1EE4oKpV+klFQ MV9gV24dw+vFVq1tstV6msaelDF/dnsltMnHnivp65p3L5AIBKTjofNp5tQQl5vtjY9Z 0rvxo47iZZbR07n334e+vE5kzcLt/uVPMtUupdpKF1WvS2mbj6iWlHR9QGEJWWsEMpYt 1mJXv+HVFgczOHUjq0pPfRJw3RNqrT1KKEuXJn7NueNj4VLbfbt8ZII3DAc0FIUWXcEc Ge0w== X-Gm-Message-State: APjAAAVu8HpmlHkSV0tPUbsc7EtpmsuIg6H66YEYMj1qf1m/+DTpgjZv BtRHYYKrgau+h+KtK1/y4Jg/JkXprIjZH7bTNsqR8r8t8N/mHq8hW3Ld6Hreigel1ZbdP8Zv6V2 D1MyWISaXB4knQyLZf4W8EGo+i/PuvMDa X-Received: by 2002:a81:234a:: with SMTP id j71mr64565787ywj.352.1555440402661; Tue, 16 Apr 2019 11:46:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqzElxArwhxx7jGTa6DtxDDsFeL4ORmUxkCDmPpf3NOGO7kzt5Yuxir4yIDFmDK1rsOAGpyERQ== X-Received: by 2002:a81:234a:: with SMTP id j71mr64565767ywj.352.1555440402378; Tue, 16 Apr 2019 11:46:42 -0700 (PDT) Received: from thorin.lan (45-27-90-188.lightspeed.rlghnc.sbcglobal.net. [45.27.90.188]) by smtp.gmail.com with ESMTPSA id c205sm17554833ywc.10.2019.04.16.11.46.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Apr 2019 11:46:39 -0700 (PDT) From: Dan Streetman To: "Michael S. Tsirkin" , Jason Wang , qemu-devel@nongnu.org, qemu-stable@nongnu.org Date: Tue, 16 Apr 2019 14:46:23 -0400 Message-Id: <20190416184624.15397-2-dan.streetman@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190416184624.15397-1-dan.streetman@canonical.com> References: <20190416184624.15397-1-dan.streetman@canonical.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 91.189.89.112 Subject: [Qemu-devel] [PATCH 1/2] add VirtIONet vhost_stopped flag to prevent multiple stops X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Dan Streetman Buglink: https://launchpad.net/bugs/1823458 There is a race condition when using the vhost-user driver, between a guest shutdown and the vhost-user interface being closed. This is explained in more detail at the bug link above; the short explanation is the vhost-user device can be closed while the main thread is in the middle of stopping the vhost_net. In this case, the main thread handling shutdown will enter virtio_net_vhost_status() and move into the n->vhost_started (else) block, and call vhost_net_stop(); while it is running that function, another thread is notified that the vhost-user device has been closed, and (indirectly) calls into virtio_net_vhost_status() also. Since the vhost_net status hasn't yet changed, the second thread also enters the n->vhost_started block, and also calls vhost_net_stop(). This causes problems for the second thread when it tries to stop the network that's already been stopped. This adds a flag to the struct that's atomically set to prevent more than one thread from calling vhost_net_stop(). The atomic_fetch_inc() is likely overkill and probably could be done with a simple check-and-set, but since it's a race condition there would still be a (very, very) small window without using an atomic to set it. Signed-off-by: Dan Streetman --- hw/net/virtio-net.c | 3 ++- include/hw/virtio/virtio-net.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index ffe0872fff..d36f50d5dd 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/iov.h" +#include "qemu/atomic.h" #include "hw/virtio/virtio.h" #include "net/net.h" #include "net/checksum.h" @@ -240,7 +241,7 @@ static void virtio_net_vhost_status(VirtIONet *n, uint8_t status) "falling back on userspace virtio", -r); n->vhost_started = 0; } - } else { + } else if (atomic_fetch_inc(&n->vhost_stopped) == 0) { vhost_net_stop(vdev, n->nic->ncs, queues); n->vhost_started = 0; } diff --git a/include/hw/virtio/virtio-net.h b/include/hw/virtio/virtio-net.h index b96f0c643f..d03fd933d0 100644 --- a/include/hw/virtio/virtio-net.h +++ b/include/hw/virtio/virtio-net.h @@ -164,6 +164,7 @@ struct VirtIONet { uint8_t nouni; uint8_t nobcast; uint8_t vhost_started; + int vhost_stopped; struct { uint32_t in_use; uint32_t first_multi; From patchwork Tue Apr 16 18:46:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Streetman X-Patchwork-Id: 10903831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 998FB17E0 for ; Tue, 16 Apr 2019 18:48:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7A21428A08 for ; Tue, 16 Apr 2019 18:48:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6E09828A0A; Tue, 16 Apr 2019 18:48:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1CBF528A0B for ; Tue, 16 Apr 2019 18:48:10 +0000 (UTC) Received: from localhost ([127.0.0.1]:40949 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hGT7p-0002XK-BH for patchwork-qemu-devel@patchwork.kernel.org; Tue, 16 Apr 2019 14:48:09 -0400 Received: from eggs.gnu.org ([209.51.188.92]:52909) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hGT6V-0001fc-7F for qemu-devel@nongnu.org; Tue, 16 Apr 2019 14:46:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hGT6U-0005dp-6B for qemu-devel@nongnu.org; Tue, 16 Apr 2019 14:46:47 -0400 Received: from youngberry.canonical.com ([91.189.89.112]:48710) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hGT6T-0005Zp-Tf for qemu-devel@nongnu.org; Tue, 16 Apr 2019 14:46:46 -0400 Received: from mail-yw1-f72.google.com ([209.85.161.72]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1hGT6S-00007J-TV for qemu-devel@nongnu.org; Tue, 16 Apr 2019 18:46:45 +0000 Received: by mail-yw1-f72.google.com with SMTP id g140so16214059ywb.12 for ; Tue, 16 Apr 2019 11:46:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hrTL3dZnHK74VlB2j2ilreuanyT4P+9dLL2Aizt0BCY=; b=O/s+b6qfY6aQHakbnW4TJQiKgSQsdTY4J2OyBi5X2Lame+8eZ9HxkmUBy9Tsdb7ipE cg6AlRL4OXzcwtYpgJnENLwy9QWAmGWjZs66vaOulRe45LFSTfIchRiKXR9tIropCq9X TZs9LBqa2C2+lrg5D0Ki6DW4YDorPlrVMAkscBDEQpECsLQC5Zs4OL+epGqEEWrtoS1Y PtrFWz2+BksQQytFWqwNwqMdBtLdtI+/ACmSe1P/gfm5liIv4pPMBjESA7jIWalun2ze V1NHJGl6xaW/Bk0q0Ik5ooVT6UEedysUTpus7IxLzXZvslTfmHvhW1vaPCnNcEJ5o5ou 9zPQ== X-Gm-Message-State: APjAAAVzvy1wBS+As1YskMs5ANflg95ygGwHOwZvh3ZfxWmVC5Bd1cQ3 BjZ47zGd2196VLHtEQedJoMh6fr6MuPhVs0CgffJ1ZUifSG+4EYb78Ce4CBDYrtZceT4Xh9TDqQ MTbAYhouJ0CRnfsmFZ+7Le/cTSB+8Ycnz X-Received: by 2002:a81:4f10:: with SMTP id d16mr65360375ywb.184.1555440404003; Tue, 16 Apr 2019 11:46:44 -0700 (PDT) X-Google-Smtp-Source: APXvYqxeCrXLAscbb7wcqKwiWcP5pkJ5JsWcDZjXB4Bk8/j9G+D0bU/sJvFlt4+sXlPDjVAbYoiMvg== X-Received: by 2002:a81:4f10:: with SMTP id d16mr65360351ywb.184.1555440403742; Tue, 16 Apr 2019 11:46:43 -0700 (PDT) Received: from thorin.lan (45-27-90-188.lightspeed.rlghnc.sbcglobal.net. [45.27.90.188]) by smtp.gmail.com with ESMTPSA id c205sm17554833ywc.10.2019.04.16.11.46.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Apr 2019 11:46:43 -0700 (PDT) From: Dan Streetman To: "Michael S. Tsirkin" , Jason Wang , qemu-devel@nongnu.org, qemu-stable@nongnu.org Date: Tue, 16 Apr 2019 14:46:24 -0400 Message-Id: <20190416184624.15397-3-dan.streetman@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190416184624.15397-1-dan.streetman@canonical.com> References: <20190416184624.15397-1-dan.streetman@canonical.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 91.189.89.112 Subject: [Qemu-devel] [PATCH 2/2] do not call vhost_net_cleanup() on running net from char user event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Dan Streetman Buglink: https://launchpad.net/bugs/1823458 Currently, a user CHR_EVENT_CLOSED event will cause net_vhost_user_event() to call vhost_user_cleanup(), which calls vhost_net_cleanup() for all its queues. However, vhost_net_cleanup() must never be called like this for fully-initialized nets; when other code later calls vhost_net_stop() - such as from virtio_net_vhost_status() - it will try to access the already-cleaned-up fields and fail with assertion errors or segfaults. The vhost_net_cleanup() will eventually be called from qemu_cleanup_net_client(). Signed-off-by: Dan Streetman --- net/vhost-user.c | 1 - 1 file changed, 1 deletion(-) diff --git a/net/vhost-user.c b/net/vhost-user.c index 5a26a24708..51921de443 100644 --- a/net/vhost-user.c +++ b/net/vhost-user.c @@ -236,7 +236,6 @@ static void chr_closed_bh(void *opaque) s = DO_UPCAST(NetVhostUserState, nc, ncs[0]); qmp_set_link(name, false, &err); - vhost_user_stop(queues, ncs); qemu_chr_fe_set_handlers(&s->chr, NULL, NULL, net_vhost_user_event, NULL, opaque, NULL, true);