From patchwork Thu Jan 24 10:28:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 10778695 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2FBF413BF for ; Thu, 24 Jan 2019 10:42:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1FB612EA97 for ; Thu, 24 Jan 2019 10:42:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D73D2EAB6; Thu, 24 Jan 2019 10:42:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 90C212EA97 for ; Thu, 24 Jan 2019 10:42:33 +0000 (UTC) Received: from localhost ([127.0.0.1]:51021 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmcSu-0003T9-PD for patchwork-qemu-devel@patchwork.kernel.org; Thu, 24 Jan 2019 05:42:32 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37842) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmcG3-0001Z8-LP for qemu-devel@nongnu.org; Thu, 24 Jan 2019 05:29:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gmcG2-0000Ph-Nb for qemu-devel@nongnu.org; Thu, 24 Jan 2019 05:29:15 -0500 Received: from mx1.redhat.com ([209.132.183.28]:45738) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gmcG0-0000Fc-AT; Thu, 24 Jan 2019 05:29:12 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 88A6D356D4; Thu, 24 Jan 2019 10:29:11 +0000 (UTC) Received: from localhost (ovpn-116-104.ams2.redhat.com [10.36.116.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAABBBA78; Thu, 24 Jan 2019 10:29:10 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Date: Thu, 24 Jan 2019 10:28:58 +0000 Message-Id: <20190124102900.20079-2-stefanha@redhat.com> In-Reply-To: <20190124102900.20079-1-stefanha@redhat.com> References: <20190124102900.20079-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 24 Jan 2019 10:29:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 1/3] throttle-groups: fix restart coroutine iothread race X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , Alberto Garcia , qemu-block@nongnu.org, Stefan Weil , Max Reitz , Stefan Hajnoczi , Kevin Wolf Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The following QMP command leads to a crash when iothreads are used: { 'execute': 'device_del', 'arguments': {'id': 'data'} } The backtrace involves the queue restart coroutine where tgm->throttle_state is a NULL pointer because throttle_group_unregister_tgm() has already been called: (gdb) bt full #0 0x00005585a7a3b378 in qemu_mutex_lock_impl (mutex=0xffffffffffffffd0, file=0x5585a7bb3d54 "block/throttle-groups.c", line=412) at util/qemu-thread-posix.c:64 err = __PRETTY_FUNCTION__ = "qemu_mutex_lock_impl" __func__ = "qemu_mutex_lock_impl" #1 0x00005585a79be074 in throttle_group_restart_queue_entry (opaque=0x5585a9de4eb0) at block/throttle-groups.c:412 _f = data = 0x5585a9de4eb0 tgm = 0x5585a9079440 ts = 0x0 tg = 0xffffffffffffff98 is_write = false empty_queue = 255 This coroutine should not execute in the iothread after the throttle group member has been unregistered! The root cause is that the device_del code path schedules the restart coroutine in the iothread while holding the AioContext lock. Therefore the iothread cannot execute the coroutine until after device_del releases the lock - by this time it's too late. This patch adds a reference count to ThrottleGroupMember so we can synchronously wait for restart coroutines to complete. Once they are done it is safe to unregister the ThrottleGroupMember. Signed-off-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20190114133257.30299-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- include/block/throttle-groups.h | 5 +++++ block/throttle-groups.c | 9 +++++++++ 2 files changed, 14 insertions(+) diff --git a/include/block/throttle-groups.h b/include/block/throttle-groups.h index e2fd0513c4..712a8e64b4 100644 --- a/include/block/throttle-groups.h +++ b/include/block/throttle-groups.h @@ -43,6 +43,11 @@ typedef struct ThrottleGroupMember { */ unsigned int io_limits_disabled; + /* Number of pending throttle_group_restart_queue_entry() coroutines. + * Accessed with atomic operations. + */ + unsigned int restart_pending; + /* The following fields are protected by the ThrottleGroup lock. * See the ThrottleGroup documentation for details. * throttle_state tells us if I/O limits are configured. */ diff --git a/block/throttle-groups.c b/block/throttle-groups.c index 5d8213a443..a5a2037924 100644 --- a/block/throttle-groups.c +++ b/block/throttle-groups.c @@ -415,6 +415,9 @@ static void coroutine_fn throttle_group_restart_queue_entry(void *opaque) } g_free(data); + + atomic_dec(&tgm->restart_pending); + aio_wait_kick(); } static void throttle_group_restart_queue(ThrottleGroupMember *tgm, bool is_write) @@ -430,6 +433,8 @@ static void throttle_group_restart_queue(ThrottleGroupMember *tgm, bool is_write * be no timer pending on this tgm at this point */ assert(!timer_pending(tgm->throttle_timers.timers[is_write])); + atomic_inc(&tgm->restart_pending); + co = qemu_coroutine_create(throttle_group_restart_queue_entry, rd); aio_co_enter(tgm->aio_context, co); } @@ -538,6 +543,7 @@ void throttle_group_register_tgm(ThrottleGroupMember *tgm, tgm->throttle_state = ts; tgm->aio_context = ctx; + atomic_set(&tgm->restart_pending, 0); qemu_mutex_lock(&tg->lock); /* If the ThrottleGroup is new set this ThrottleGroupMember as the token */ @@ -584,6 +590,9 @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm) return; } + /* Wait for throttle_group_restart_queue_entry() coroutines to finish */ + AIO_WAIT_WHILE(tgm->aio_context, atomic_read(&tgm->restart_pending) > 0); + qemu_mutex_lock(&tg->lock); for (i = 0; i < 2; i++) { assert(tgm->pending_reqs[i] == 0); From patchwork Thu Jan 24 10:28:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 10778683 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0600213B5 for ; Thu, 24 Jan 2019 10:33:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA4F32E9CB for ; Thu, 24 Jan 2019 10:33:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E8B9C2E9EB; Thu, 24 Jan 2019 10:33:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E573B2E9E6 for ; Thu, 24 Jan 2019 10:33:26 +0000 (UTC) Received: from localhost ([127.0.0.1]:50852 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmcK5-0004m7-Ql for patchwork-qemu-devel@patchwork.kernel.org; Thu, 24 Jan 2019 05:33:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37923) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmcGP-0001pV-Jd for qemu-devel@nongnu.org; Thu, 24 Jan 2019 05:29:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gmcGJ-0000uk-Kc for qemu-devel@nongnu.org; Thu, 24 Jan 2019 05:29:33 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41880) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gmcG4-0000TC-H6; Thu, 24 Jan 2019 05:29:16 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ADC88C058CBC; Thu, 24 Jan 2019 10:29:15 +0000 (UTC) Received: from localhost (ovpn-116-104.ams2.redhat.com [10.36.116.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id E59325D6A6; Thu, 24 Jan 2019 10:29:12 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Date: Thu, 24 Jan 2019 10:28:59 +0000 Message-Id: <20190124102900.20079-3-stefanha@redhat.com> In-Reply-To: <20190124102900.20079-1-stefanha@redhat.com> References: <20190124102900.20079-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 24 Jan 2019 10:29:15 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 2/3] iotests: add 238 for throttling tgm unregister iothread segfault X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , Alberto Garcia , qemu-block@nongnu.org, Stefan Weil , Max Reitz , Stefan Hajnoczi , Kevin Wolf Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hot-unplug a scsi-hd using an iothread. The previous patch fixes a segfault in this scenario. This patch adds a regression test. Suggested-by: Alberto Garcia Suggested-by: Kevin Wolf Signed-off-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Message-id: 20190114133257.30299-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi --- tests/qemu-iotests/238 | 47 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/238.out | 6 +++++ tests/qemu-iotests/group | 1 + 3 files changed, 54 insertions(+) create mode 100755 tests/qemu-iotests/238 create mode 100644 tests/qemu-iotests/238.out diff --git a/tests/qemu-iotests/238 b/tests/qemu-iotests/238 new file mode 100755 index 0000000000..f81ee1112f --- /dev/null +++ b/tests/qemu-iotests/238 @@ -0,0 +1,47 @@ +#!/usr/bin/env python +# +# Regression test for throttle group member unregister segfault with iothread +# +# Copyright (c) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import sys +import os +import iotests +from iotests import log + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'scripts')) + +from qemu import QEMUMachine + +if iotests.qemu_default_machine == 's390-ccw-virtio': + virtio_scsi_device = 'virtio-scsi-ccw' +else: + virtio_scsi_device = 'virtio-scsi-pci' + +vm = QEMUMachine(iotests.qemu_prog) +vm.add_args('-machine', 'accel=kvm') +vm.launch() + +log(vm.qmp('blockdev-add', node_name='hd0', driver='null-co')) +log(vm.qmp('object-add', qom_type='iothread', id='iothread0')) +log(vm.qmp('device_add', id='scsi0', driver=virtio_scsi_device, iothread='iothread0')) +log(vm.qmp('device_add', id='scsi-hd0', driver='scsi-hd', drive='hd0')) +log(vm.qmp('block_set_io_throttle', id='scsi-hd0', bps=0, bps_rd=0, bps_wr=0, + iops=1000, iops_rd=0, iops_wr=0, conv_keys=False)) +log(vm.qmp('device_del', id='scsi-hd0')) + +vm.shutdown() diff --git a/tests/qemu-iotests/238.out b/tests/qemu-iotests/238.out new file mode 100644 index 0000000000..4de840ba8c --- /dev/null +++ b/tests/qemu-iotests/238.out @@ -0,0 +1,6 @@ +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index f6b245917a..0f1c3f9cdf 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -234,3 +234,4 @@ 234 auto quick migration 235 auto quick 236 auto quick +238 auto quick From patchwork Thu Jan 24 10:29:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 10778723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D7F3E1850 for ; Thu, 24 Jan 2019 10:50:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C79382EA6C for ; Thu, 24 Jan 2019 10:50:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BA5A52EA81; Thu, 24 Jan 2019 10:50:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4EE922EA64 for ; Thu, 24 Jan 2019 10:50:14 +0000 (UTC) Received: from localhost ([127.0.0.1]:51194 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmcaL-0000gT-JP for patchwork-qemu-devel@patchwork.kernel.org; Thu, 24 Jan 2019 05:50:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37938) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gmcGS-0001sD-HR for qemu-devel@nongnu.org; Thu, 24 Jan 2019 05:29:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gmcGR-0001DY-J3 for qemu-devel@nongnu.org; Thu, 24 Jan 2019 05:29:40 -0500 Received: from mx1.redhat.com ([209.132.183.28]:45644) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gmcG6-0000Xd-Fs; Thu, 24 Jan 2019 05:29:18 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 78B78C070E2A; Thu, 24 Jan 2019 10:29:17 +0000 (UTC) Received: from localhost (ovpn-116-104.ams2.redhat.com [10.36.116.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 078CC5D6A6; Thu, 24 Jan 2019 10:29:16 +0000 (UTC) From: Stefan Hajnoczi To: qemu-devel@nongnu.org Date: Thu, 24 Jan 2019 10:29:00 +0000 Message-Id: <20190124102900.20079-4-stefanha@redhat.com> In-Reply-To: <20190124102900.20079-1-stefanha@redhat.com> References: <20190124102900.20079-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 24 Jan 2019 10:29:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 3/3] qemu-coroutine-sleep: drop CoSleepCB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Stefan Weil , Max Reitz , Stefan Hajnoczi , Kevin Wolf Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Vladimir Sementsov-Ogievskiy Drop CoSleepCB structure. It's actually unused. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190122143113.20331-1-vsementsov@virtuozzo.com Signed-off-by: Stefan Hajnoczi --- util/qemu-coroutine-sleep.c | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/util/qemu-coroutine-sleep.c b/util/qemu-coroutine-sleep.c index afb678fbe5..4bfdd30cbf 100644 --- a/util/qemu-coroutine-sleep.c +++ b/util/qemu-coroutine-sleep.c @@ -17,38 +17,31 @@ #include "qemu/timer.h" #include "block/aio.h" -typedef struct CoSleepCB { - QEMUTimer *ts; - Coroutine *co; -} CoSleepCB; - static void co_sleep_cb(void *opaque) { - CoSleepCB *sleep_cb = opaque; + Coroutine *co = opaque; /* Write of schedule protected by barrier write in aio_co_schedule */ - atomic_set(&sleep_cb->co->scheduled, NULL); - aio_co_wake(sleep_cb->co); + atomic_set(&co->scheduled, NULL); + aio_co_wake(co); } void coroutine_fn qemu_co_sleep_ns(QEMUClockType type, int64_t ns) { AioContext *ctx = qemu_get_current_aio_context(); - CoSleepCB sleep_cb = { - .co = qemu_coroutine_self(), - }; + QEMUTimer *ts; + Coroutine *co = qemu_coroutine_self(); - const char *scheduled = atomic_cmpxchg(&sleep_cb.co->scheduled, NULL, - __func__); + const char *scheduled = atomic_cmpxchg(&co->scheduled, NULL, __func__); if (scheduled) { fprintf(stderr, "%s: Co-routine was already scheduled in '%s'\n", __func__, scheduled); abort(); } - sleep_cb.ts = aio_timer_new(ctx, type, SCALE_NS, co_sleep_cb, &sleep_cb); - timer_mod(sleep_cb.ts, qemu_clock_get_ns(type) + ns); + ts = aio_timer_new(ctx, type, SCALE_NS, co_sleep_cb, co); + timer_mod(ts, qemu_clock_get_ns(type) + ns); qemu_coroutine_yield(); - timer_del(sleep_cb.ts); - timer_free(sleep_cb.ts); + timer_del(ts); + timer_free(ts); }