From patchwork Fri Apr 3 10:44:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11472521 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE115159A for ; Fri, 3 Apr 2020 10:45:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C2F9A20757 for ; Fri, 3 Apr 2020 10:45:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QR9WBvVN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2F9A20757 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53466 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jKJp3-00050V-IU for patchwork-qemu-devel@patchwork.kernel.org; Fri, 03 Apr 2020 06:45:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39559) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jKJoP-0003xW-PU for qemu-devel@nongnu.org; Fri, 03 Apr 2020 06:44:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jKJoO-0006So-Mb for qemu-devel@nongnu.org; Fri, 03 Apr 2020 06:44:33 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:37983 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jKJoO-0006SL-J7 for qemu-devel@nongnu.org; Fri, 03 Apr 2020 06:44:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1585910672; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cnqegz23nW1hpOaxoaYKoairv6E6mI41M3cjbrLkcuk=; b=QR9WBvVNg9vOKMD6Lo8XymVCHmZk+IlazP9co2xc527Uqtgt1d/VB3wExG4IjkgusUlvaU 7sN89N2jOeK1PW5sl7shZxb/TvSCnoR10PUf00f0FVgufjmZ8Aq84XEy0QoW0thgKPox14 dl7o70kyZM4ONX5HmOgxyuv+BeaXh/Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-58-RV6xt6xyO4uZDLQRYX4VfA-1; Fri, 03 Apr 2020 06:44:30 -0400 X-MC-Unique: RV6xt6xyO4uZDLQRYX4VfA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B57368018A1; Fri, 3 Apr 2020 10:44:28 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-113-132.ams2.redhat.com [10.36.113.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id EC1BB96F83; Fri, 3 Apr 2020 10:44:26 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH for-5.0 1/2] block: Don't blk_wait_while_drained() in blk_co_preadv/pwritev Date: Fri, 3 Apr 2020 12:44:14 +0200 Message-Id: <20200403104415.20963-2-kwolf@redhat.com> In-Reply-To: <20200403104415.20963-1-kwolf@redhat.com> References: <20200403104415.20963-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, s.reiter@proxmox.com, qemu-devel@nongnu.org, dietmar@proxmox.com, stefanha@redhat.com, mreitz@redhat.com, t.lamprecht@proxmox.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Calling blk_wait_while_drained() while blk->in_flight is increased for the current request is wrong because it will cause the drain operation to deadlock. In blk_co_preadv() and blk_co_pwritev_part(), this deadlock is easily fixed by simply removing the blk_wait_while_drained() call. We already wait in blk_aio_read_entry() and blk_aio_write_entry(), and if a request didn't wait there because it started basically at the same time as the drain, we can simply let it complete. We still do need the wait for emulating synchronous operations, which don't have a second call yet, so add blk_wait_while_drained() calls there. Fixes: cf3129323f900ef5ddbccbe86e4fa801e88c566e Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/block-backend.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 8b8f2a80a0..3124e367b3 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1154,8 +1154,6 @@ int coroutine_fn blk_co_preadv(BlockBackend *blk, int64_t offset, int ret; BlockDriverState *bs; - blk_wait_while_drained(blk); - /* Call blk_bs() only after waiting, the graph may have changed */ bs = blk_bs(blk); trace_blk_co_preadv(blk, bs, offset, bytes, flags); @@ -1186,8 +1184,6 @@ int coroutine_fn blk_co_pwritev_part(BlockBackend *blk, int64_t offset, int ret; BlockDriverState *bs; - blk_wait_while_drained(blk); - /* Call blk_bs() only after waiting, the graph may have changed */ bs = blk_bs(blk); trace_blk_co_pwritev(blk, bs, offset, bytes, flags); @@ -1234,6 +1230,7 @@ static void blk_read_entry(void *opaque) BlkRwCo *rwco = opaque; QEMUIOVector *qiov = rwco->iobuf; + blk_wait_while_drained(rwco->blk); rwco->ret = blk_co_preadv(rwco->blk, rwco->offset, qiov->size, qiov, rwco->flags); aio_wait_kick(); @@ -1244,6 +1241,7 @@ static void blk_write_entry(void *opaque) BlkRwCo *rwco = opaque; QEMUIOVector *qiov = rwco->iobuf; + blk_wait_while_drained(rwco->blk); rwco->ret = blk_co_pwritev(rwco->blk, rwco->offset, qiov->size, qiov, rwco->flags); aio_wait_kick(); From patchwork Fri Apr 3 10:44:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11472523 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B5604159A for ; Fri, 3 Apr 2020 10:47:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8B7BC2078C for ; Fri, 3 Apr 2020 10:47:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iFhdz3ou" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8B7BC2078C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jKJr3-0007Nv-Lm for patchwork-qemu-devel@patchwork.kernel.org; Fri, 03 Apr 2020 06:47:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39581) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jKJoR-0003zH-5t for qemu-devel@nongnu.org; Fri, 03 Apr 2020 06:44:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jKJoQ-0006Tx-13 for qemu-devel@nongnu.org; Fri, 03 Apr 2020 06:44:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:34956 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jKJoP-0006TY-Tb for qemu-devel@nongnu.org; Fri, 03 Apr 2020 06:44:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1585910673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hWJ3MgFYCUe9wil/UcV5TKzmCD6udz1cDHjGIDwnDVI=; b=iFhdz3oupyEHYZyhr24xksxpn5uM0jdR9CASKo9SAb/jQzSXD4n+uzzNp1nZ5lzfhQ7A2/ 4mx2fmqWpNhxV5Z51EmlvI8ASSbAt0tK8fOblM4LG7KCgQ0cvGxnspoucnGj3PY4yKTEC/ QhKHn0PyXuvTBnKg/lJ+mGmZGQOlMrw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-358-oTZH3IZaO9SXNHyz7cdoYA-1; Fri, 03 Apr 2020 06:44:32 -0400 X-MC-Unique: oTZH3IZaO9SXNHyz7cdoYA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BFF15800D5C; Fri, 3 Apr 2020 10:44:30 +0000 (UTC) Received: from linux.fritz.box.com (ovpn-113-132.ams2.redhat.com [10.36.113.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 06D6C8EA1A; Fri, 3 Apr 2020 10:44:28 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PATCH for-5.0 2/2] block: Fix blk->in_flight during blk_wait_while_drained() Date: Fri, 3 Apr 2020 12:44:15 +0200 Message-Id: <20200403104415.20963-3-kwolf@redhat.com> In-Reply-To: <20200403104415.20963-1-kwolf@redhat.com> References: <20200403104415.20963-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, s.reiter@proxmox.com, qemu-devel@nongnu.org, dietmar@proxmox.com, stefanha@redhat.com, mreitz@redhat.com, t.lamprecht@proxmox.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Calling blk_wait_while_drained() while blk->in_flight is increased for the current request is wrong because it will cause the drain operation to deadlock. Many callers of blk_wait_while_drained() have already increased blk->in_flight when called in a blk_aio_*() path, but can also be called in synchonous code paths where blk->in_flight isn't increased. This means that these calls of blk_wait_while_drained() are wrong at least in some cases. In order to fix this, increase blk->in_flight even for synchronous operations and temporarily decrease the counter again in blk_wait_while_drained(). Fixes: cf3129323f900ef5ddbccbe86e4fa801e88c566e Signed-off-by: Kevin Wolf --- block/block-backend.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index 3124e367b3..7bd16402b8 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1143,7 +1143,9 @@ static int blk_check_byte_request(BlockBackend *blk, int64_t offset, static void coroutine_fn blk_wait_while_drained(BlockBackend *blk) { if (blk->quiesce_counter && !blk->disable_request_queuing) { + blk_dec_in_flight(blk); qemu_co_queue_wait(&blk->queued_requests, NULL); + blk_inc_in_flight(blk); } } @@ -1260,6 +1262,7 @@ static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf, .ret = NOT_DONE, }; + blk_inc_in_flight(blk); if (qemu_in_coroutine()) { /* Fast-path if already in coroutine context */ co_entry(&rwco); @@ -1268,6 +1271,7 @@ static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf, bdrv_coroutine_enter(blk_bs(blk), co); BDRV_POLL_WHILE(blk_bs(blk), rwco.ret == NOT_DONE); } + blk_dec_in_flight(blk); return rwco.ret; } @@ -1386,9 +1390,7 @@ static void blk_aio_read_entry(void *opaque) QEMUIOVector *qiov = rwco->iobuf; if (rwco->blk->quiesce_counter) { - blk_dec_in_flight(rwco->blk); blk_wait_while_drained(rwco->blk); - blk_inc_in_flight(rwco->blk); } assert(qiov->size == acb->bytes); @@ -1404,9 +1406,7 @@ static void blk_aio_write_entry(void *opaque) QEMUIOVector *qiov = rwco->iobuf; if (rwco->blk->quiesce_counter) { - blk_dec_in_flight(rwco->blk); blk_wait_while_drained(rwco->blk); - blk_inc_in_flight(rwco->blk); } assert(!qiov || qiov->size == acb->bytes);