From patchwork Fri Dec 11 17:07:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1CDDC433FE for ; Fri, 11 Dec 2020 17:17:27 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1D70323DC2 for ; Fri, 11 Dec 2020 17:17:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1D70323DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44620 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm2n-0003T2-Qf for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:17:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49656) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlu4-0003OD-I5 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:24 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26329) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlu1-00021J-T6 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706500; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6xXTpU1H6DvmIaGvUCDdqUlVNd9M8yhJU7IteDKiJRk=; b=ho0R4788FMLAS4+LwMcFX35YzeX3tWxTfE1Sk4wWSu10uLVlrTIaZmL5BP4C2xAYO8Y0Uu b/gIhQFluSpNWko9oWl/Rj7WU5M8cL60iarVa9d+J7xR1EZmTYpiaf7Zpo90B6kfqPSU/f 2oe73ideqA2R32ERUsr4G48E9smUE/4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-217-CYtthzKNPUmhheOxlePl1Q-1; Fri, 11 Dec 2020 12:08:18 -0500 X-MC-Unique: CYtthzKNPUmhheOxlePl1Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EBBCE84E1F8; Fri, 11 Dec 2020 17:08:16 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB76E5D71D; Fri, 11 Dec 2020 17:08:15 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 01/34] block/accounting: Use lock guard macros Date: Fri, 11 Dec 2020 18:07:39 +0100 Message-Id: <20201211170812.228643-2-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Gan Qixin Replace manual lock()/unlock() calls with lock guard macros (QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/accounting. Signed-off-by: Gan Qixin Reviewed-by: Paolo Bonzini Message-Id: <20201203075055.127773-2-ganqixin@huawei.com> Signed-off-by: Kevin Wolf --- block/accounting.c | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/block/accounting.c b/block/accounting.c index 8d41c8a83a..2030851d79 100644 --- a/block/accounting.c +++ b/block/accounting.c @@ -199,29 +199,27 @@ static void block_account_one_io(BlockAcctStats *stats, BlockAcctCookie *cookie, return; } - qemu_mutex_lock(&stats->lock); - - if (failed) { - stats->failed_ops[cookie->type]++; - } else { - stats->nr_bytes[cookie->type] += cookie->bytes; - stats->nr_ops[cookie->type]++; - } + WITH_QEMU_LOCK_GUARD(&stats->lock) { + if (failed) { + stats->failed_ops[cookie->type]++; + } else { + stats->nr_bytes[cookie->type] += cookie->bytes; + stats->nr_ops[cookie->type]++; + } - block_latency_histogram_account(&stats->latency_histogram[cookie->type], - latency_ns); + block_latency_histogram_account(&stats->latency_histogram[cookie->type], + latency_ns); - if (!failed || stats->account_failed) { - stats->total_time_ns[cookie->type] += latency_ns; - stats->last_access_time_ns = time_ns; + if (!failed || stats->account_failed) { + stats->total_time_ns[cookie->type] += latency_ns; + stats->last_access_time_ns = time_ns; - QSLIST_FOREACH(s, &stats->intervals, entries) { - timed_average_account(&s->latency[cookie->type], latency_ns); + QSLIST_FOREACH(s, &stats->intervals, entries) { + timed_average_account(&s->latency[cookie->type], latency_ns); + } } } - qemu_mutex_unlock(&stats->lock); - cookie->type = BLOCK_ACCT_NONE; } From patchwork Fri Dec 11 17:07:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 035A0C433FE for ; Fri, 11 Dec 2020 17:17:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9252323DC2 for ; Fri, 11 Dec 2020 17:17:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9252323DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43560 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm2O-0002yF-Ta for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:17:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49680) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlu5-0003PQ-PS for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:26 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:56449) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlu4-00022i-Aj for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hIRKwnrx4PPsDE3REBIRH80Y3RMFY+GnMz9eOGwKwhM=; b=LIZUS3P1uJprHdEbfT/5cfPxNn4jFQ8V8YpGr9WUJJKDw0pHaxqgO+5FyeXUyWlay9QgEl VK5zJ1jtWHnq4zgWCg11bnc3Zo1/feH60Jz4XqeZ+MdVXrZkpnuvv4pJCFzS/Kjut5VJU5 42Gx3hA7MPNPHfPIEmwJqPHUP6KM9NE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-566-YPq9T5cRNb21-zucCOLBBA-1; Fri, 11 Dec 2020 12:08:19 -0500 X-MC-Unique: YPq9T5cRNb21-zucCOLBBA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 490B2800D55; Fri, 11 Dec 2020 17:08:18 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43BCA5D741; Fri, 11 Dec 2020 17:08:17 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 02/34] block/curl: Use lock guard macros Date: Fri, 11 Dec 2020 18:07:40 +0100 Message-Id: <20201211170812.228643-3-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Gan Qixin Replace manual lock()/unlock() calls with lock guard macros (QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/curl. Signed-off-by: Gan Qixin Reviewed-by: Paolo Bonzini Message-Id: <20201203075055.127773-3-ganqixin@huawei.com> Signed-off-by: Kevin Wolf --- block/curl.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/block/curl.c b/block/curl.c index 4f907c47be..d24a4c5897 100644 --- a/block/curl.c +++ b/block/curl.c @@ -564,23 +564,23 @@ static void curl_detach_aio_context(BlockDriverState *bs) BDRVCURLState *s = bs->opaque; int i; - qemu_mutex_lock(&s->mutex); - for (i = 0; i < CURL_NUM_STATES; i++) { - if (s->states[i].in_use) { - curl_clean_state(&s->states[i]); + WITH_QEMU_LOCK_GUARD(&s->mutex) { + for (i = 0; i < CURL_NUM_STATES; i++) { + if (s->states[i].in_use) { + curl_clean_state(&s->states[i]); + } + if (s->states[i].curl) { + curl_easy_cleanup(s->states[i].curl); + s->states[i].curl = NULL; + } + g_free(s->states[i].orig_buf); + s->states[i].orig_buf = NULL; } - if (s->states[i].curl) { - curl_easy_cleanup(s->states[i].curl); - s->states[i].curl = NULL; + if (s->multi) { + curl_multi_cleanup(s->multi); + s->multi = NULL; } - g_free(s->states[i].orig_buf); - s->states[i].orig_buf = NULL; - } - if (s->multi) { - curl_multi_cleanup(s->multi); - s->multi = NULL; } - qemu_mutex_unlock(&s->mutex); timer_del(&s->timer); } From patchwork Fri Dec 11 17:07:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7812C433FE for ; Fri, 11 Dec 2020 17:19:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 680CC23DC2 for ; Fri, 11 Dec 2020 17:19:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 680CC23DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50380 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm4h-000622-Ew for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:19:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49784) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluG-0003YY-Re for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:57639) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlu8-00024R-LN for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NnM6dYlYZRgd2ExH9w41QSV9/yrt19fOVsd+vEwELuw=; b=do8ve7+e+CRUSWnMCjuyp21GCTZo9KQg62hb9n47IqPZgFAm1CiS+HWZPqJcZilhaZgEab Z9PCth48zR9mCIG7m0rJ8OIfYC6xk2Ax60KMfr4Di1rNqR5ImwGSTWvCe3LfcjEsLEhExu GXnu4iFOZp1VGu1ea8AyDVna1ktGXy8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-534-JenYioVRM5GCif73l6RSbQ-1; Fri, 11 Dec 2020 12:08:23 -0500 X-MC-Unique: JenYioVRM5GCif73l6RSbQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B51CB800D53; Fri, 11 Dec 2020 17:08:22 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 945D05D71D; Fri, 11 Dec 2020 17:08:18 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 03/34] block/throttle-groups: Use lock guard macros Date: Fri, 11 Dec 2020 18:07:41 +0100 Message-Id: <20201211170812.228643-4-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Gan Qixin Replace manual lock()/unlock() calls with lock guard macros (QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/throttle-groups. Signed-off-by: Gan Qixin Message-Id: <20201203075055.127773-4-ganqixin@huawei.com> Signed-off-by: Kevin Wolf --- block/throttle-groups.c | 48 ++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/block/throttle-groups.c b/block/throttle-groups.c index e2f2813c0f..abd16ed9db 100644 --- a/block/throttle-groups.c +++ b/block/throttle-groups.c @@ -546,7 +546,7 @@ void throttle_group_register_tgm(ThrottleGroupMember *tgm, tgm->aio_context = ctx; qatomic_set(&tgm->restart_pending, 0); - qemu_mutex_lock(&tg->lock); + QEMU_LOCK_GUARD(&tg->lock); /* If the ThrottleGroup is new set this ThrottleGroupMember as the token */ for (i = 0; i < 2; i++) { if (!tg->tokens[i]) { @@ -565,8 +565,6 @@ void throttle_group_register_tgm(ThrottleGroupMember *tgm, qemu_co_mutex_init(&tgm->throttled_reqs_lock); qemu_co_queue_init(&tgm->throttled_reqs[0]); qemu_co_queue_init(&tgm->throttled_reqs[1]); - - qemu_mutex_unlock(&tg->lock); } /* Unregister a ThrottleGroupMember from its group, removing it from the list, @@ -594,25 +592,25 @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm) /* Wait for throttle_group_restart_queue_entry() coroutines to finish */ AIO_WAIT_WHILE(tgm->aio_context, qatomic_read(&tgm->restart_pending) > 0); - qemu_mutex_lock(&tg->lock); - for (i = 0; i < 2; i++) { - assert(tgm->pending_reqs[i] == 0); - assert(qemu_co_queue_empty(&tgm->throttled_reqs[i])); - assert(!timer_pending(tgm->throttle_timers.timers[i])); - if (tg->tokens[i] == tgm) { - token = throttle_group_next_tgm(tgm); - /* Take care of the case where this is the last tgm in the group */ - if (token == tgm) { - token = NULL; + WITH_QEMU_LOCK_GUARD(&tg->lock) { + for (i = 0; i < 2; i++) { + assert(tgm->pending_reqs[i] == 0); + assert(qemu_co_queue_empty(&tgm->throttled_reqs[i])); + assert(!timer_pending(tgm->throttle_timers.timers[i])); + if (tg->tokens[i] == tgm) { + token = throttle_group_next_tgm(tgm); + /* Take care of the case where this is the last tgm in the group */ + if (token == tgm) { + token = NULL; + } + tg->tokens[i] = token; } - tg->tokens[i] = token; } - } - /* remove the current tgm from the list */ - QLIST_REMOVE(tgm, round_robin); - throttle_timers_destroy(&tgm->throttle_timers); - qemu_mutex_unlock(&tg->lock); + /* remove the current tgm from the list */ + QLIST_REMOVE(tgm, round_robin); + throttle_timers_destroy(&tgm->throttle_timers); + } throttle_group_unref(&tg->ts); tgm->throttle_state = NULL; @@ -638,14 +636,14 @@ void throttle_group_detach_aio_context(ThrottleGroupMember *tgm) assert(qemu_co_queue_empty(&tgm->throttled_reqs[1])); /* Kick off next ThrottleGroupMember, if necessary */ - qemu_mutex_lock(&tg->lock); - for (i = 0; i < 2; i++) { - if (timer_pending(tt->timers[i])) { - tg->any_timer_armed[i] = false; - schedule_next_request(tgm, i); + WITH_QEMU_LOCK_GUARD(&tg->lock) { + for (i = 0; i < 2; i++) { + if (timer_pending(tt->timers[i])) { + tg->any_timer_armed[i] = false; + schedule_next_request(tgm, i); + } } } - qemu_mutex_unlock(&tg->lock); throttle_timers_detach_aio_context(tt); tgm->aio_context = NULL; From patchwork Fri Dec 11 17:07:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEE7EC433FE for ; Fri, 11 Dec 2020 17:25:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 591F223EF1 for ; Fri, 11 Dec 2020 17:25:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 591F223EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41620 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmA9-0006Lp-LA for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:25:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49816) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluJ-0003ZX-JI for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34693) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlu9-00025D-AM for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1GM0EIEy4CkX7YJmc8u/hwt0dMlsfVX2LCZh1LZVuEw=; b=PBpsI2KAADAaRFOIXr4YeHUd4EkZpfiixriej0QMxDAiLxlcvWqwPxZqf7l4JyPt3KgQwa 3AVo2j9TuRf4VfaaS/9ZgBIZdkoSxuL/J3miLXIAEUeea7dP4qF0li1UkOfvOFBFaZIiKo gEhr4clOb2UpvVk9XKjRYj+1D/Myg5A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-5-_6fgInrCPzGFeAhrTOXiZA-1; Fri, 11 Dec 2020 12:08:25 -0500 X-MC-Unique: _6fgInrCPzGFeAhrTOXiZA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0EDF2858182; Fri, 11 Dec 2020 17:08:24 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0CC515D731; Fri, 11 Dec 2020 17:08:22 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 04/34] block/iscsi: Use lock guard macros Date: Fri, 11 Dec 2020 18:07:42 +0100 Message-Id: <20201211170812.228643-5-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Gan Qixin Replace manual lock()/unlock() calls with lock guard macros (QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/iscsi. Signed-off-by: Gan Qixin Message-Id: <20201203075055.127773-5-ganqixin@huawei.com> Signed-off-by: Kevin Wolf --- block/iscsi.c | 50 ++++++++++++++++++++++++-------------------------- 1 file changed, 24 insertions(+), 26 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index e30a7e3606..7d4b3b56d5 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -322,25 +322,23 @@ iscsi_aio_cancel(BlockAIOCB *blockacb) IscsiAIOCB *acb = (IscsiAIOCB *)blockacb; IscsiLun *iscsilun = acb->iscsilun; - qemu_mutex_lock(&iscsilun->mutex); + WITH_QEMU_LOCK_GUARD(&iscsilun->mutex) { - /* If it was cancelled or completed already, our work is done here */ - if (acb->cancelled || acb->status != -EINPROGRESS) { - qemu_mutex_unlock(&iscsilun->mutex); - return; - } + /* If it was cancelled or completed already, our work is done here */ + if (acb->cancelled || acb->status != -EINPROGRESS) { + return; + } - acb->cancelled = true; + acb->cancelled = true; - qemu_aio_ref(acb); /* released in iscsi_abort_task_cb() */ + qemu_aio_ref(acb); /* released in iscsi_abort_task_cb() */ - /* send a task mgmt call to the target to cancel the task on the target */ - if (iscsi_task_mgmt_abort_task_async(iscsilun->iscsi, acb->task, - iscsi_abort_task_cb, acb) < 0) { - qemu_aio_unref(acb); /* since iscsi_abort_task_cb() won't be called */ + /* send a task mgmt call to the target to cancel the task on the target */ + if (iscsi_task_mgmt_abort_task_async(iscsilun->iscsi, acb->task, + iscsi_abort_task_cb, acb) < 0) { + qemu_aio_unref(acb); /* since iscsi_abort_task_cb() won't be called */ + } } - - qemu_mutex_unlock(&iscsilun->mutex); } static const AIOCBInfo iscsi_aiocb_info = { @@ -375,22 +373,22 @@ static void iscsi_timed_check_events(void *opaque) { IscsiLun *iscsilun = opaque; - qemu_mutex_lock(&iscsilun->mutex); + WITH_QEMU_LOCK_GUARD(&iscsilun->mutex) { + /* check for timed out requests */ + iscsi_service(iscsilun->iscsi, 0); - /* check for timed out requests */ - iscsi_service(iscsilun->iscsi, 0); + if (iscsilun->request_timed_out) { + iscsilun->request_timed_out = false; + iscsi_reconnect(iscsilun->iscsi); + } - if (iscsilun->request_timed_out) { - iscsilun->request_timed_out = false; - iscsi_reconnect(iscsilun->iscsi); + /* + * newer versions of libiscsi may return zero events. Ensure we are + * able to return to service once this situation changes. + */ + iscsi_set_events(iscsilun); } - /* newer versions of libiscsi may return zero events. Ensure we are able - * to return to service once this situation changes. */ - iscsi_set_events(iscsilun); - - qemu_mutex_unlock(&iscsilun->mutex); - timer_mod(iscsilun->event_timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME) + EVENT_INTERVAL); } From patchwork Fri Dec 11 17:07:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFA03C433FE for ; Fri, 11 Dec 2020 17:19:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8674F23DC2 for ; Fri, 11 Dec 2020 17:19:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8674F23DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm55-0006kO-MS for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:19:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49828) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluJ-0003Zw-8O for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28479) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluA-00025H-HN for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FQrHeWDNrFrtuZfxFSDLYFBPwtjd1f76AM7/tNRZpeY=; b=LE/5zKXHL9SG/H+/mI9dWIGwmE2/BTEOy5q7wHRD660yc2EaoWnLmHRJ4i7cogSESCUZSV OvYF9kES1NTjrQOYLh+ddvdaQplIQQvpFlBeaxVH8W2U8AvNH3t84es7GkJD5guywQX8B1 jhwBf9432s+7iDmq+5EykeP9dXGeu/s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-29-KwcjSYvUMnWCtythbLtJvQ-1; Fri, 11 Dec 2020 12:08:26 -0500 X-MC-Unique: KwcjSYvUMnWCtythbLtJvQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5AF75100C661; Fri, 11 Dec 2020 17:08:25 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5A2705D734; Fri, 11 Dec 2020 17:08:24 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 05/34] meson: Detect libfuse Date: Fri, 11 Dec 2020 18:07:43 +0100 Message-Id: <20201211170812.228643-6-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-2-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- meson_options.txt | 2 ++ configure | 7 +++++++ meson.build | 6 ++++++ 3 files changed, 15 insertions(+) diff --git a/meson_options.txt b/meson_options.txt index f6f64785fe..8f9f2e3df6 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -66,6 +66,8 @@ option('virtiofsd', type: 'feature', value: 'auto', description: 'build virtiofs daemon (virtiofsd)') option('vhost_user_blk_server', type: 'feature', value: 'auto', description: 'build vhost-user-blk server') +option('fuse', type: 'feature', value: 'auto', + description: 'FUSE block device export') option('capstone', type: 'combo', value: 'auto', choices: ['disabled', 'enabled', 'auto', 'system', 'internal'], diff --git a/configure b/configure index 18c26e0389..b2f96c0da2 100755 --- a/configure +++ b/configure @@ -449,6 +449,7 @@ meson="" ninja="" skip_meson=no gettext="" +fuse="auto" bogus_os="no" malloc_trim="auto" @@ -1525,6 +1526,10 @@ for opt do ;; --disable-libdaxctl) libdaxctl=no ;; + --enable-fuse) fuse="enabled" + ;; + --disable-fuse) fuse="disabled" + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information" @@ -1850,6 +1855,7 @@ disabled with --disable-FEATURE, default is enabled if available: xkbcommon xkbcommon support rng-none dummy RNG, avoid using /dev/(u)random and getrandom() libdaxctl libdaxctl support + fuse FUSE block device export NOTE: The object files are built at the place where configure is launched EOF @@ -7014,6 +7020,7 @@ NINJA=$ninja $meson setup \ -Diconv=$iconv -Dcurses=$curses -Dlibudev=$libudev\ -Ddocs=$docs -Dsphinx_build=$sphinx_build -Dinstall_blobs=$blobs \ -Dvhost_user_blk_server=$vhost_user_blk_server \ + -Dfuse=$fuse \ $cross_arg \ "$PWD" "$source_path" diff --git a/meson.build b/meson.build index 9ea05ab49f..6e8ef151d8 100644 --- a/meson.build +++ b/meson.build @@ -773,6 +773,10 @@ elif get_option('vhost_user_blk_server').disabled() or not have_system have_vhost_user_blk_server = false endif +fuse = dependency('fuse3', required: get_option('fuse'), + version: '>=3.1', method: 'pkg-config', + static: enable_static) + ################# # config-host.h # ################# @@ -807,6 +811,7 @@ config_host_data.set('CONFIG_KEYUTILS', keyutils.found()) config_host_data.set('CONFIG_GETTID', has_gettid) config_host_data.set('CONFIG_MALLOC_TRIM', has_malloc_trim) config_host_data.set('CONFIG_STATX', has_statx) +config_host_data.set('CONFIG_FUSE', fuse.found()) config_host_data.set('QEMU_VERSION', '"@0@"'.format(meson.project_version())) config_host_data.set('QEMU_VERSION_MAJOR', meson.project_version().split('.')[0]) config_host_data.set('QEMU_VERSION_MINOR', meson.project_version().split('.')[1]) @@ -2208,6 +2213,7 @@ endif summary_info += {'thread sanitizer': config_host.has_key('CONFIG_TSAN')} summary_info += {'rng-none': config_host.has_key('CONFIG_RNG_NONE')} summary_info += {'Linux keyring': config_host.has_key('CONFIG_SECRET_KEYRING')} +summary_info += {'FUSE exports': fuse.found()} summary(summary_info, bool_yn: true) if not supported_cpus.contains(cpu) From patchwork Fri Dec 11 17:07:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71F9CC433FE for ; Fri, 11 Dec 2020 17:27:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E49F323EF1 for ; Fri, 11 Dec 2020 17:27:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E49F323EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50538 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmCn-00022b-OG for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:27:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49960) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluR-0003hZ-Sv for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24770) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluC-00025n-QU for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706511; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M5pi+lqtSy6x09huXwtETuWlRVAFp3UAGT9VF9Rjo9g=; b=Ns9vbfNiS+q+2K83FhBWkp1+1KUdk0lZDiPI488j7imKJlYgBgApNICL7tJQzaVwoFpXdt nWd5+LKmrE8Sreepi20dyBoZrSUp6ScFRr8O/IvORANG7RnMds6htscNcobPM5nWOa0gfA QXBCbPSSj/5ETg5AvGkcXwRohnv5C9o= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-529-7kHaR_xhPsKfmrjrHqm9Lw-1; Fri, 11 Dec 2020 12:08:28 -0500 X-MC-Unique: 7kHaR_xhPsKfmrjrHqm9Lw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BEC62B8108; Fri, 11 Dec 2020 17:08:26 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id A7E525D734; Fri, 11 Dec 2020 17:08:25 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 06/34] fuse: Allow exporting BDSs via FUSE Date: Fri, 11 Dec 2020 18:07:44 +0100 Message-Id: <20201211170812.228643-7-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz block-export-add type=fuse allows mounting block graph nodes via FUSE on some existing regular file. That file should then appears like a raw disk image, and accesses to it result in accesses to the exported BDS. Right now, we only implement the necessary block export functions to set it up and shut it down. We do not implement any access functions, so accessing the mount point only results in errors. This will be addressed by a followup patch. We keep a hash table of exported mount points, because we want to be able to detect when users try to use a mount point twice. This is because we invoke stat() to check whether the given mount point is a regular file, but if that file is served by ourselves (because it is already used as a mount point), then this stat() would have to be served by ourselves, too, which is impossible to do while we (as the caller) are waiting for it to settle. Therefore, keep track of mount point paths to at least catch the most obvious instances of that problem. Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-3-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- qapi/block-export.json | 23 ++- include/block/fuse.h | 30 ++++ block.c | 1 + block/export/export.c | 4 + block/export/fuse.c | 295 +++++++++++++++++++++++++++++++++++++++ MAINTAINERS | 6 + block/export/meson.build | 2 + 7 files changed, 359 insertions(+), 2 deletions(-) create mode 100644 include/block/fuse.h create mode 100644 block/export/fuse.c diff --git a/qapi/block-export.json b/qapi/block-export.json index 4eeac7842d..430bc69f35 100644 --- a/qapi/block-export.json +++ b/qapi/block-export.json @@ -120,6 +120,21 @@ '*logical-block-size': 'size', '*num-queues': 'uint16'} } +## +# @BlockExportOptionsFuse: +# +# Options for exporting a block graph node on some (file) mountpoint +# as a raw image. +# +# @mountpoint: Path on which to export the block device via FUSE. +# This must point to an existing regular file. +# +# Since: 6.0 +## +{ 'struct': 'BlockExportOptionsFuse', + 'data': { 'mountpoint': 'str' }, + 'if': 'defined(CONFIG_FUSE)' } + ## # @NbdServerAddOptions: # @@ -222,11 +237,13 @@ # # @nbd: NBD export # @vhost-user-blk: vhost-user-blk export (since 5.2) +# @fuse: FUSE export (since: 6.0) # # Since: 4.2 ## { 'enum': 'BlockExportType', - 'data': [ 'nbd', 'vhost-user-blk' ] } + 'data': [ 'nbd', 'vhost-user-blk', + { 'name': 'fuse', 'if': 'defined(CONFIG_FUSE)' } ] } ## # @BlockExportOptions: @@ -267,7 +284,9 @@ 'discriminator': 'type', 'data': { 'nbd': 'BlockExportOptionsNbd', - 'vhost-user-blk': 'BlockExportOptionsVhostUserBlk' + 'vhost-user-blk': 'BlockExportOptionsVhostUserBlk', + 'fuse': { 'type': 'BlockExportOptionsFuse', + 'if': 'defined(CONFIG_FUSE)' } } } ## diff --git a/include/block/fuse.h b/include/block/fuse.h new file mode 100644 index 0000000000..ffa91fe364 --- /dev/null +++ b/include/block/fuse.h @@ -0,0 +1,30 @@ +/* + * Present a block device as a raw image through FUSE + * + * Copyright (c) 2020 Max Reitz + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; under version 2 or later of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see . + */ + +#ifndef BLOCK_FUSE_H +#define BLOCK_FUSE_H + +#ifdef CONFIG_FUSE + +#include "block/export.h" + +extern const BlockExportDriver blk_exp_fuse; + +#endif /* CONFIG_FUSE */ + +#endif diff --git a/block.c b/block.c index f1cedac362..eb16fb48c6 100644 --- a/block.c +++ b/block.c @@ -26,6 +26,7 @@ #include "block/trace.h" #include "block/block_int.h" #include "block/blockjob.h" +#include "block/fuse.h" #include "block/nbd.h" #include "block/qdict.h" #include "qemu/error-report.h" diff --git a/block/export/export.c b/block/export/export.c index bad6f21b1c..b716c1522c 100644 --- a/block/export/export.c +++ b/block/export/export.c @@ -17,6 +17,7 @@ #include "sysemu/block-backend.h" #include "sysemu/iothread.h" #include "block/export.h" +#include "block/fuse.h" #include "block/nbd.h" #include "qapi/error.h" #include "qapi/qapi-commands-block-export.h" @@ -31,6 +32,9 @@ static const BlockExportDriver *blk_exp_drivers[] = { #ifdef CONFIG_VHOST_USER_BLK_SERVER &blk_exp_vhost_user_blk, #endif +#ifdef CONFIG_FUSE + &blk_exp_fuse, +#endif }; /* Only accessed from the main thread */ diff --git a/block/export/fuse.c b/block/export/fuse.c new file mode 100644 index 0000000000..0553bcd630 --- /dev/null +++ b/block/export/fuse.c @@ -0,0 +1,295 @@ +/* + * Present a block device as a raw image through FUSE + * + * Copyright (c) 2020 Max Reitz + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; under version 2 or later of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, see . + */ + +#define FUSE_USE_VERSION 31 + +#include "qemu/osdep.h" +#include "block/aio.h" +#include "block/block.h" +#include "block/export.h" +#include "block/fuse.h" +#include "block/qapi.h" +#include "qapi/error.h" +#include "qapi/qapi-commands-block.h" +#include "sysemu/block-backend.h" + +#include +#include + + +/* Prevent overly long bounce buffer allocations */ +#define FUSE_MAX_BOUNCE_BYTES (MIN(BDRV_REQUEST_MAX_BYTES, 64 * 1024 * 1024)) + + +typedef struct FuseExport { + BlockExport common; + + struct fuse_session *fuse_session; + struct fuse_buf fuse_buf; + bool mounted, fd_handler_set_up; + + char *mountpoint; + bool writable; +} FuseExport; + +static GHashTable *exports; +static const struct fuse_lowlevel_ops fuse_ops; + +static void fuse_export_shutdown(BlockExport *exp); +static void fuse_export_delete(BlockExport *exp); + +static void init_exports_table(void); + +static int setup_fuse_export(FuseExport *exp, const char *mountpoint, + Error **errp); +static void read_from_fuse_export(void *opaque); + +static bool is_regular_file(const char *path, Error **errp); + + +static int fuse_export_create(BlockExport *blk_exp, + BlockExportOptions *blk_exp_args, + Error **errp) +{ + FuseExport *exp = container_of(blk_exp, FuseExport, common); + BlockExportOptionsFuse *args = &blk_exp_args->u.fuse; + int ret; + + assert(blk_exp_args->type == BLOCK_EXPORT_TYPE_FUSE); + + init_exports_table(); + + /* + * It is important to do this check before calling is_regular_file() -- + * that function will do a stat(), which we would have to handle if we + * already exported something on @mountpoint. But we cannot, because + * we are currently caught up here. + * (Note that ideally we would want to resolve relative paths here, + * but bdrv_make_absolute_filename() might do the wrong thing for + * paths that contain colons, and realpath() would resolve symlinks, + * which we do not want: The mount point is not going to be the + * symlink's destination, but the link itself.) + * So this will not catch all potential clashes, but hopefully at + * least the most common one of specifying exactly the same path + * string twice. + */ + if (g_hash_table_contains(exports, args->mountpoint)) { + error_setg(errp, "There already is a FUSE export on '%s'", + args->mountpoint); + ret = -EEXIST; + goto fail; + } + + if (!is_regular_file(args->mountpoint, errp)) { + ret = -EINVAL; + goto fail; + } + + exp->mountpoint = g_strdup(args->mountpoint); + exp->writable = blk_exp_args->writable; + + ret = setup_fuse_export(exp, args->mountpoint, errp); + if (ret < 0) { + goto fail; + } + + return 0; + +fail: + fuse_export_delete(blk_exp); + return ret; +} + +/** + * Allocates the global @exports hash table. + */ +static void init_exports_table(void) +{ + if (exports) { + return; + } + + exports = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); +} + +/** + * Create exp->fuse_session and mount it. + */ +static int setup_fuse_export(FuseExport *exp, const char *mountpoint, + Error **errp) +{ + const char *fuse_argv[4]; + char *mount_opts; + struct fuse_args fuse_args; + int ret; + + /* Needs to match what fuse_init() sets. Only max_read must be supplied. */ + mount_opts = g_strdup_printf("max_read=%zu", FUSE_MAX_BOUNCE_BYTES); + + fuse_argv[0] = ""; /* Dummy program name */ + fuse_argv[1] = "-o"; + fuse_argv[2] = mount_opts; + fuse_argv[3] = NULL; + fuse_args = (struct fuse_args)FUSE_ARGS_INIT(3, (char **)fuse_argv); + + exp->fuse_session = fuse_session_new(&fuse_args, &fuse_ops, + sizeof(fuse_ops), exp); + g_free(mount_opts); + if (!exp->fuse_session) { + error_setg(errp, "Failed to set up FUSE session"); + ret = -EIO; + goto fail; + } + + ret = fuse_session_mount(exp->fuse_session, mountpoint); + if (ret < 0) { + error_setg(errp, "Failed to mount FUSE session to export"); + ret = -EIO; + goto fail; + } + exp->mounted = true; + + g_hash_table_insert(exports, g_strdup(mountpoint), NULL); + + aio_set_fd_handler(exp->common.ctx, + fuse_session_fd(exp->fuse_session), true, + read_from_fuse_export, NULL, NULL, exp); + exp->fd_handler_set_up = true; + + return 0; + +fail: + fuse_export_shutdown(&exp->common); + return ret; +} + +/** + * Callback to be invoked when the FUSE session FD can be read from. + * (This is basically the FUSE event loop.) + */ +static void read_from_fuse_export(void *opaque) +{ + FuseExport *exp = opaque; + int ret; + + blk_exp_ref(&exp->common); + + do { + ret = fuse_session_receive_buf(exp->fuse_session, &exp->fuse_buf); + } while (ret == -EINTR); + if (ret < 0) { + goto out; + } + + fuse_session_process_buf(exp->fuse_session, &exp->fuse_buf); + +out: + blk_exp_unref(&exp->common); +} + +static void fuse_export_shutdown(BlockExport *blk_exp) +{ + FuseExport *exp = container_of(blk_exp, FuseExport, common); + + if (exp->fuse_session) { + fuse_session_exit(exp->fuse_session); + + if (exp->fd_handler_set_up) { + aio_set_fd_handler(exp->common.ctx, + fuse_session_fd(exp->fuse_session), true, + NULL, NULL, NULL, NULL); + exp->fd_handler_set_up = false; + } + } + + if (exp->mountpoint) { + /* + * Safe to drop now, because we will not handle any requests + * for this export anymore anyway. + */ + g_hash_table_remove(exports, exp->mountpoint); + } +} + +static void fuse_export_delete(BlockExport *blk_exp) +{ + FuseExport *exp = container_of(blk_exp, FuseExport, common); + + if (exp->fuse_session) { + if (exp->mounted) { + fuse_session_unmount(exp->fuse_session); + } + + fuse_session_destroy(exp->fuse_session); + } + + free(exp->fuse_buf.mem); + g_free(exp->mountpoint); +} + +/** + * Check whether @path points to a regular file. If not, put an + * appropriate message into *errp. + */ +static bool is_regular_file(const char *path, Error **errp) +{ + struct stat statbuf; + int ret; + + ret = stat(path, &statbuf); + if (ret < 0) { + error_setg_errno(errp, errno, "Failed to stat '%s'", path); + return false; + } + + if (!S_ISREG(statbuf.st_mode)) { + error_setg(errp, "'%s' is not a regular file", path); + return false; + } + + return true; +} + +/** + * A chance to set change some parameters supplied to FUSE_INIT. + */ +static void fuse_init(void *userdata, struct fuse_conn_info *conn) +{ + /* + * MIN_NON_ZERO() would not be wrong here, but what we set here + * must equal what has been passed to fuse_session_new(). + * Therefore, as long as max_read must be passed as a mount option + * (which libfuse claims will be changed at some point), we have + * to set max_read to a fixed value here. + */ + conn->max_read = FUSE_MAX_BOUNCE_BYTES; + + conn->max_write = MIN_NON_ZERO(BDRV_REQUEST_MAX_BYTES, conn->max_write); +} + +static const struct fuse_lowlevel_ops fuse_ops = { + .init = fuse_init, +}; + +const BlockExportDriver blk_exp_fuse = { + .type = BLOCK_EXPORT_TYPE_FUSE, + .instance_size = sizeof(FuseExport), + .create = fuse_export_create, + .delete = fuse_export_delete, + .request_shutdown = fuse_export_shutdown, +}; diff --git a/MAINTAINERS b/MAINTAINERS index aa39490a24..1950592d04 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3114,6 +3114,12 @@ F: include/qemu/vhost-user-server.h F: tests/qtest/libqos/vhost-user-blk.c F: util/vhost-user-server.c +FUSE block device exports +M: Max Reitz +L: qemu-block@nongnu.org +S: Supported +F: block/export/fuse.c + Replication M: Wen Congyang M: Xie Changlong diff --git a/block/export/meson.build b/block/export/meson.build index 135b356775..0a08e384c7 100644 --- a/block/export/meson.build +++ b/block/export/meson.build @@ -3,3 +3,5 @@ blockdev_ss.add(files('export.c')) if have_vhost_user_blk_server blockdev_ss.add(files('vhost-user-blk-server.c')) endif + +blockdev_ss.add(when: fuse, if_true: files('fuse.c')) From patchwork Fri Dec 11 17:07:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A1D9C433FE for ; Fri, 11 Dec 2020 17:22:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E04E23DC2 for ; Fri, 11 Dec 2020 17:22:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E04E23DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60560 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm7u-0001ze-Cq for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:22:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49908) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluO-0003eH-5B for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50221) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluD-00026J-7K for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706511; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0NAnf8ONMI16zDQq9Ngg351OMk8hdJE4xuxXJA0C5X0=; b=TcRE92tgItNxm355TSZnTLq7XpIIRQnX+05XNUzVudQJ48BdcPN9435zYCpwfQ8zb7hK5T Lar0FnFV1lyysSS5erOo7QoPnRrPFw7rXfJNKumwP3QgDycGk+9FMOx3yXS4RIovuTlSCD +8sBGHN7s1SwCvKCINMTfzkrVwS7Y8I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-301-Dl6XdjUSNOaDySuzR3DTvQ-1; Fri, 11 Dec 2020 12:08:29 -0500 X-MC-Unique: Dl6XdjUSNOaDySuzR3DTvQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1954459; Fri, 11 Dec 2020 17:08:28 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 174425D6A8; Fri, 11 Dec 2020 17:08:26 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 07/34] fuse: Implement standard FUSE operations Date: Fri, 11 Dec 2020 18:07:45 +0100 Message-Id: <20201211170812.228643-8-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz This makes the export actually useful instead of only producing errors whenever it is accessed. Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-4-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- block/export/fuse.c | 242 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 242 insertions(+) diff --git a/block/export/fuse.c b/block/export/fuse.c index 0553bcd630..d995829ab7 100644 --- a/block/export/fuse.c +++ b/block/export/fuse.c @@ -282,8 +282,250 @@ static void fuse_init(void *userdata, struct fuse_conn_info *conn) conn->max_write = MIN_NON_ZERO(BDRV_REQUEST_MAX_BYTES, conn->max_write); } +/** + * Let clients look up files. Always return ENOENT because we only + * care about the mountpoint itself. + */ +static void fuse_lookup(fuse_req_t req, fuse_ino_t parent, const char *name) +{ + fuse_reply_err(req, ENOENT); +} + +/** + * Let clients get file attributes (i.e., stat() the file). + */ +static void fuse_getattr(fuse_req_t req, fuse_ino_t inode, + struct fuse_file_info *fi) +{ + struct stat statbuf; + int64_t length, allocated_blocks; + time_t now = time(NULL); + FuseExport *exp = fuse_req_userdata(req); + mode_t mode; + + length = blk_getlength(exp->common.blk); + if (length < 0) { + fuse_reply_err(req, -length); + return; + } + + allocated_blocks = bdrv_get_allocated_file_size(blk_bs(exp->common.blk)); + if (allocated_blocks <= 0) { + allocated_blocks = DIV_ROUND_UP(length, 512); + } else { + allocated_blocks = DIV_ROUND_UP(allocated_blocks, 512); + } + + mode = S_IFREG | S_IRUSR; + if (exp->writable) { + mode |= S_IWUSR; + } + + statbuf = (struct stat) { + .st_ino = inode, + .st_mode = mode, + .st_nlink = 1, + .st_uid = getuid(), + .st_gid = getgid(), + .st_size = length, + .st_blksize = blk_bs(exp->common.blk)->bl.request_alignment, + .st_blocks = allocated_blocks, + .st_atime = now, + .st_mtime = now, + .st_ctime = now, + }; + + fuse_reply_attr(req, &statbuf, 1.); +} + +static int fuse_do_truncate(const FuseExport *exp, int64_t size, + bool req_zero_write, PreallocMode prealloc) +{ + uint64_t blk_perm, blk_shared_perm; + BdrvRequestFlags truncate_flags = 0; + int ret; + + if (req_zero_write) { + truncate_flags |= BDRV_REQ_ZERO_WRITE; + } + + blk_get_perm(exp->common.blk, &blk_perm, &blk_shared_perm); + + ret = blk_set_perm(exp->common.blk, blk_perm | BLK_PERM_RESIZE, + blk_shared_perm, NULL); + if (ret < 0) { + return ret; + } + + ret = blk_truncate(exp->common.blk, size, true, prealloc, + truncate_flags, NULL); + + /* Must succeed, because we are only giving up the RESIZE permission */ + blk_set_perm(exp->common.blk, blk_perm, blk_shared_perm, &error_abort); + + return ret; +} + +/** + * Let clients set file attributes. Only resizing is supported. + */ +static void fuse_setattr(fuse_req_t req, fuse_ino_t inode, struct stat *statbuf, + int to_set, struct fuse_file_info *fi) +{ + FuseExport *exp = fuse_req_userdata(req); + int ret; + + if (!exp->writable) { + fuse_reply_err(req, EACCES); + return; + } + + if (to_set & ~FUSE_SET_ATTR_SIZE) { + fuse_reply_err(req, ENOTSUP); + return; + } + + ret = fuse_do_truncate(exp, statbuf->st_size, true, PREALLOC_MODE_OFF); + if (ret < 0) { + fuse_reply_err(req, -ret); + return; + } + + fuse_getattr(req, inode, fi); +} + +/** + * Let clients open a file (i.e., the exported image). + */ +static void fuse_open(fuse_req_t req, fuse_ino_t inode, + struct fuse_file_info *fi) +{ + fuse_reply_open(req, fi); +} + +/** + * Handle client reads from the exported image. + */ +static void fuse_read(fuse_req_t req, fuse_ino_t inode, + size_t size, off_t offset, struct fuse_file_info *fi) +{ + FuseExport *exp = fuse_req_userdata(req); + int64_t length; + void *buf; + int ret; + + /* Limited by max_read, should not happen */ + if (size > FUSE_MAX_BOUNCE_BYTES) { + fuse_reply_err(req, EINVAL); + return; + } + + /** + * Clients will expect short reads at EOF, so we have to limit + * offset+size to the image length. + */ + length = blk_getlength(exp->common.blk); + if (length < 0) { + fuse_reply_err(req, -length); + return; + } + + if (offset + size > length) { + size = length - offset; + } + + buf = qemu_try_blockalign(blk_bs(exp->common.blk), size); + if (!buf) { + fuse_reply_err(req, ENOMEM); + return; + } + + ret = blk_pread(exp->common.blk, offset, buf, size); + if (ret >= 0) { + fuse_reply_buf(req, buf, size); + } else { + fuse_reply_err(req, -ret); + } + + qemu_vfree(buf); +} + +/** + * Handle client writes to the exported image. + */ +static void fuse_write(fuse_req_t req, fuse_ino_t inode, const char *buf, + size_t size, off_t offset, struct fuse_file_info *fi) +{ + FuseExport *exp = fuse_req_userdata(req); + int64_t length; + int ret; + + /* Limited by max_write, should not happen */ + if (size > BDRV_REQUEST_MAX_BYTES) { + fuse_reply_err(req, EINVAL); + return; + } + + if (!exp->writable) { + fuse_reply_err(req, EACCES); + return; + } + + /** + * Clients will expect short writes at EOF, so we have to limit + * offset+size to the image length. + */ + length = blk_getlength(exp->common.blk); + if (length < 0) { + fuse_reply_err(req, -length); + return; + } + + if (offset + size > length) { + size = length - offset; + } + + ret = blk_pwrite(exp->common.blk, offset, buf, size, 0); + if (ret >= 0) { + fuse_reply_write(req, size); + } else { + fuse_reply_err(req, -ret); + } +} + +/** + * Let clients fsync the exported image. + */ +static void fuse_fsync(fuse_req_t req, fuse_ino_t inode, int datasync, + struct fuse_file_info *fi) +{ + FuseExport *exp = fuse_req_userdata(req); + int ret; + + ret = blk_flush(exp->common.blk); + fuse_reply_err(req, ret < 0 ? -ret : 0); +} + +/** + * Called before an FD to the exported image is closed. (libfuse + * notes this to be a way to return last-minute errors.) + */ +static void fuse_flush(fuse_req_t req, fuse_ino_t inode, + struct fuse_file_info *fi) +{ + fuse_fsync(req, inode, 1, fi); +} + static const struct fuse_lowlevel_ops fuse_ops = { .init = fuse_init, + .lookup = fuse_lookup, + .getattr = fuse_getattr, + .setattr = fuse_setattr, + .open = fuse_open, + .read = fuse_read, + .write = fuse_write, + .flush = fuse_flush, + .fsync = fuse_fsync, }; const BlockExportDriver blk_exp_fuse = { From patchwork Fri Dec 11 17:07:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63819C4361B for ; Fri, 11 Dec 2020 17:24:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC25123EF1 for ; Fri, 11 Dec 2020 17:24:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC25123EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm9v-0006Be-QB for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:24:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49964) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluS-0003hw-0m for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:31220) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluD-00026j-SQ for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gfF+1ARM+6LlPmXbPh/K0dVaF34q6yc83sQISbGQHi4=; b=gI1+RyldLtZti06vhBiO2E86oN93CrxX+4O/mxqLyGH3ERsClBNiS4AljGpH4kHVxHVlI8 L5jKgsJdk5AJHV/Za3F9zj+xqK8bdsVj9aNQLW6bATUe5fU5g8KTp1Y3GR3E5ITAVabwIj rArNE+yD2gaQbBbzKhqpkDfT9Yx/tCM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-404-d4RFxrLTMciCXTcALK8gXw-1; Fri, 11 Dec 2020 12:08:30 -0500 X-MC-Unique: d4RFxrLTMciCXTcALK8gXw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74359107ACE8; Fri, 11 Dec 2020 17:08:29 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 64F015D735; Fri, 11 Dec 2020 17:08:28 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 08/34] fuse: Allow growable exports Date: Fri, 11 Dec 2020 18:07:46 +0100 Message-Id: <20201211170812.228643-9-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz These will behave more like normal files in that writes beyond the EOF will automatically grow the export size. As an optimization, keep the RESIZE permission for growable exports so we do not have to take it for every post-EOF write. (This permission is not released when the export is destroyed, because at that point the BlockBackend is destroyed altogether anyway.) Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-5-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- qapi/block-export.json | 6 +++++- block/export/fuse.c | 44 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 41 insertions(+), 9 deletions(-) diff --git a/qapi/block-export.json b/qapi/block-export.json index 430bc69f35..e819e70cac 100644 --- a/qapi/block-export.json +++ b/qapi/block-export.json @@ -129,10 +129,14 @@ # @mountpoint: Path on which to export the block device via FUSE. # This must point to an existing regular file. # +# @growable: Whether writes beyond the EOF should grow the block node +# accordingly. (default: false) +# # Since: 6.0 ## { 'struct': 'BlockExportOptionsFuse', - 'data': { 'mountpoint': 'str' }, + 'data': { 'mountpoint': 'str', + '*growable': 'bool' }, 'if': 'defined(CONFIG_FUSE)' } ## diff --git a/block/export/fuse.c b/block/export/fuse.c index d995829ab7..92d2f50bcc 100644 --- a/block/export/fuse.c +++ b/block/export/fuse.c @@ -45,6 +45,7 @@ typedef struct FuseExport { char *mountpoint; bool writable; + bool growable; } FuseExport; static GHashTable *exports; @@ -72,6 +73,19 @@ static int fuse_export_create(BlockExport *blk_exp, assert(blk_exp_args->type == BLOCK_EXPORT_TYPE_FUSE); + /* For growable exports, take the RESIZE permission */ + if (args->growable) { + uint64_t blk_perm, blk_shared_perm; + + blk_get_perm(exp->common.blk, &blk_perm, &blk_shared_perm); + + ret = blk_set_perm(exp->common.blk, blk_perm | BLK_PERM_RESIZE, + blk_shared_perm, errp); + if (ret < 0) { + return ret; + } + } + init_exports_table(); /* @@ -102,6 +116,7 @@ static int fuse_export_create(BlockExport *blk_exp, exp->mountpoint = g_strdup(args->mountpoint); exp->writable = blk_exp_args->writable; + exp->growable = args->growable; ret = setup_fuse_export(exp, args->mountpoint, errp); if (ret < 0) { @@ -349,19 +364,24 @@ static int fuse_do_truncate(const FuseExport *exp, int64_t size, truncate_flags |= BDRV_REQ_ZERO_WRITE; } - blk_get_perm(exp->common.blk, &blk_perm, &blk_shared_perm); + /* Growable exports have a permanent RESIZE permission */ + if (!exp->growable) { + blk_get_perm(exp->common.blk, &blk_perm, &blk_shared_perm); - ret = blk_set_perm(exp->common.blk, blk_perm | BLK_PERM_RESIZE, - blk_shared_perm, NULL); - if (ret < 0) { - return ret; + ret = blk_set_perm(exp->common.blk, blk_perm | BLK_PERM_RESIZE, + blk_shared_perm, NULL); + if (ret < 0) { + return ret; + } } ret = blk_truncate(exp->common.blk, size, true, prealloc, truncate_flags, NULL); - /* Must succeed, because we are only giving up the RESIZE permission */ - blk_set_perm(exp->common.blk, blk_perm, blk_shared_perm, &error_abort); + if (!exp->growable) { + /* Must succeed, because we are only giving up the RESIZE permission */ + blk_set_perm(exp->common.blk, blk_perm, blk_shared_perm, &error_abort); + } return ret; } @@ -482,7 +502,15 @@ static void fuse_write(fuse_req_t req, fuse_ino_t inode, const char *buf, } if (offset + size > length) { - size = length - offset; + if (exp->growable) { + ret = fuse_do_truncate(exp, offset + size, true, PREALLOC_MODE_OFF); + if (ret < 0) { + fuse_reply_err(req, -ret); + return; + } + } else { + size = length - offset; + } } ret = blk_pwrite(exp->common.blk, offset, buf, size, 0); From patchwork Fri Dec 11 17:07:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88886C4361B for ; Fri, 11 Dec 2020 17:29:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2012F23DC2 for ; Fri, 11 Dec 2020 17:29:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2012F23DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58828 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmEs-0005db-6p for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:29:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50024) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluW-0003m1-24 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:34786) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluE-00026z-Rd for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706514; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i7YYa3LFi0Y4wK4CNURxUtOEL+L50jCecL5ttIgFZE8=; b=QytCz3XhLIPTB0SCUNsea9m37EEEIGU84TUxcy56TNdpcrWzGQSs738a6AkVnQFOk4aV80 9aPmNfwcrkxfF9OkIs8mYIMKFo8779Cd0jB4HsfKI1m9GbTFIRTVSaLlDFj9fbBqgGEM5k dVq4rs2udBrUPXDwQHBXkK6npbxcZGY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-ontAnAgzMfaex2Hv-6JRfg-1; Fri, 11 Dec 2020 12:08:31 -0500 X-MC-Unique: ontAnAgzMfaex2Hv-6JRfg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B2521107ACF7; Fri, 11 Dec 2020 17:08:30 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id B11435D71D; Fri, 11 Dec 2020 17:08:29 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 09/34] fuse: (Partially) implement fallocate() Date: Fri, 11 Dec 2020 18:07:47 +0100 Message-Id: <20201211170812.228643-10-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz This allows allocating areas after the (old) EOF as part of a growing resize, writing zeroes, and discarding. Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-6-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- block/export/fuse.c | 84 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/block/export/fuse.c b/block/export/fuse.c index 92d2f50bcc..0b9d226b2f 100644 --- a/block/export/fuse.c +++ b/block/export/fuse.c @@ -521,6 +521,89 @@ static void fuse_write(fuse_req_t req, fuse_ino_t inode, const char *buf, } } +/** + * Let clients perform various fallocate() operations. + */ +static void fuse_fallocate(fuse_req_t req, fuse_ino_t inode, int mode, + off_t offset, off_t length, + struct fuse_file_info *fi) +{ + FuseExport *exp = fuse_req_userdata(req); + int64_t blk_len; + int ret; + + if (!exp->writable) { + fuse_reply_err(req, EACCES); + return; + } + + blk_len = blk_getlength(exp->common.blk); + if (blk_len < 0) { + fuse_reply_err(req, -blk_len); + return; + } + + if (mode & FALLOC_FL_KEEP_SIZE) { + length = MIN(length, blk_len - offset); + } + + if (mode & FALLOC_FL_PUNCH_HOLE) { + if (!(mode & FALLOC_FL_KEEP_SIZE)) { + fuse_reply_err(req, EINVAL); + return; + } + + do { + int size = MIN(length, BDRV_REQUEST_MAX_BYTES); + + ret = blk_pdiscard(exp->common.blk, offset, size); + offset += size; + length -= size; + } while (ret == 0 && length > 0); + } else if (mode & FALLOC_FL_ZERO_RANGE) { + if (!(mode & FALLOC_FL_KEEP_SIZE) && offset + length > blk_len) { + /* No need for zeroes, we are going to write them ourselves */ + ret = fuse_do_truncate(exp, offset + length, false, + PREALLOC_MODE_OFF); + if (ret < 0) { + fuse_reply_err(req, -ret); + return; + } + } + + do { + int size = MIN(length, BDRV_REQUEST_MAX_BYTES); + + ret = blk_pwrite_zeroes(exp->common.blk, + offset, size, 0); + offset += size; + length -= size; + } while (ret == 0 && length > 0); + } else if (!mode) { + /* We can only fallocate at the EOF with a truncate */ + if (offset < blk_len) { + fuse_reply_err(req, EOPNOTSUPP); + return; + } + + if (offset > blk_len) { + /* No preallocation needed here */ + ret = fuse_do_truncate(exp, offset, true, PREALLOC_MODE_OFF); + if (ret < 0) { + fuse_reply_err(req, -ret); + return; + } + } + + ret = fuse_do_truncate(exp, offset + length, true, + PREALLOC_MODE_FALLOC); + } else { + ret = -EOPNOTSUPP; + } + + fuse_reply_err(req, ret < 0 ? -ret : 0); +} + /** * Let clients fsync the exported image. */ @@ -552,6 +635,7 @@ static const struct fuse_lowlevel_ops fuse_ops = { .open = fuse_open, .read = fuse_read, .write = fuse_write, + .fallocate = fuse_fallocate, .flush = fuse_flush, .fsync = fuse_fsync, }; From patchwork Fri Dec 11 17:07:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 664A3C4361B for ; Fri, 11 Dec 2020 17:27:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E334023EF1 for ; Fri, 11 Dec 2020 17:27:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E334023EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmCe-0001fm-1u for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:27:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50010) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluV-0003ld-Dy for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:52 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45634) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluH-000295-Mp for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z2Zq6h7bkhiel0Yl+Jjlx614t8qHnw+60wBkeIOAUNc=; b=MyjA3lGeI+0WdcAZl78NyXggGU7Mss7u5pzT6YNbEHJktnzcY7o3tZ7nt3+e7Mycf1zktE hC7fLvdU3N7ILkf7F4FKMP0e2iz/LfelCj2b1tSva4t3vo0gJxQGzhFhdIAVj6I8VzwO9m Imq+9aN7fhUY56XpRo6ptZi/OOc+bQM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-367-eidId5rVM5OCiDDQl0t0Mw-1; Fri, 11 Dec 2020 12:08:33 -0500 X-MC-Unique: eidId5rVM5OCiDDQl0t0Mw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 08D19B8100; Fri, 11 Dec 2020 17:08:32 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 096DC5D731; Fri, 11 Dec 2020 17:08:30 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 10/34] fuse: Implement hole detection through lseek Date: Fri, 11 Dec 2020 18:07:48 +0100 Message-Id: <20201211170812.228643-11-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz This is a relatively new feature in libfuse (available since 3.8.0, which was released in November 2019), so we have to add a dedicated check whether it is available before making use of it. Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-7-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- meson_options.txt | 2 ++ configure | 8 ++++- block/export/fuse.c | 77 +++++++++++++++++++++++++++++++++++++++++++++ meson.build | 20 ++++++++++++ 4 files changed, 106 insertions(+), 1 deletion(-) diff --git a/meson_options.txt b/meson_options.txt index 8f9f2e3df6..74ac853548 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -68,6 +68,8 @@ option('vhost_user_blk_server', type: 'feature', value: 'auto', description: 'build vhost-user-blk server') option('fuse', type: 'feature', value: 'auto', description: 'FUSE block device export') +option('fuse_lseek', type : 'feature', value : 'auto', + description: 'SEEK_HOLE/SEEK_DATA support for FUSE exports') option('capstone', type: 'combo', value: 'auto', choices: ['disabled', 'enabled', 'auto', 'system', 'internal'], diff --git a/configure b/configure index b2f96c0da2..3f823ed163 100755 --- a/configure +++ b/configure @@ -450,6 +450,7 @@ ninja="" skip_meson=no gettext="" fuse="auto" +fuse_lseek="auto" bogus_os="no" malloc_trim="auto" @@ -1530,6 +1531,10 @@ for opt do ;; --disable-fuse) fuse="disabled" ;; + --enable-fuse-lseek) fuse_lseek="enabled" + ;; + --disable-fuse-lseek) fuse_lseek="disabled" + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information" @@ -1856,6 +1861,7 @@ disabled with --disable-FEATURE, default is enabled if available: rng-none dummy RNG, avoid using /dev/(u)random and getrandom() libdaxctl libdaxctl support fuse FUSE block device export + fuse-lseek SEEK_HOLE/SEEK_DATA support for FUSE exports NOTE: The object files are built at the place where configure is launched EOF @@ -7020,7 +7026,7 @@ NINJA=$ninja $meson setup \ -Diconv=$iconv -Dcurses=$curses -Dlibudev=$libudev\ -Ddocs=$docs -Dsphinx_build=$sphinx_build -Dinstall_blobs=$blobs \ -Dvhost_user_blk_server=$vhost_user_blk_server \ - -Dfuse=$fuse \ + -Dfuse=$fuse -Dfuse_lseek=$fuse_lseek \ $cross_arg \ "$PWD" "$source_path" diff --git a/block/export/fuse.c b/block/export/fuse.c index 0b9d226b2f..38f74c94da 100644 --- a/block/export/fuse.c +++ b/block/export/fuse.c @@ -627,6 +627,80 @@ static void fuse_flush(fuse_req_t req, fuse_ino_t inode, fuse_fsync(req, inode, 1, fi); } +#ifdef CONFIG_FUSE_LSEEK +/** + * Let clients inquire allocation status. + */ +static void fuse_lseek(fuse_req_t req, fuse_ino_t inode, off_t offset, + int whence, struct fuse_file_info *fi) +{ + FuseExport *exp = fuse_req_userdata(req); + + if (whence != SEEK_HOLE && whence != SEEK_DATA) { + fuse_reply_err(req, EINVAL); + return; + } + + while (true) { + int64_t pnum; + int ret; + + ret = bdrv_block_status_above(blk_bs(exp->common.blk), NULL, + offset, INT64_MAX, &pnum, NULL, NULL); + if (ret < 0) { + fuse_reply_err(req, -ret); + return; + } + + if (!pnum && (ret & BDRV_BLOCK_EOF)) { + int64_t blk_len; + + /* + * If blk_getlength() rounds (e.g. by sectors), then the + * export length will be rounded, too. However, + * bdrv_block_status_above() may return EOF at unaligned + * offsets. We must not let this become visible and thus + * always simulate a hole between @offset (the real EOF) + * and @blk_len (the client-visible EOF). + */ + + blk_len = blk_getlength(exp->common.blk); + if (blk_len < 0) { + fuse_reply_err(req, -blk_len); + return; + } + + if (offset > blk_len || whence == SEEK_DATA) { + fuse_reply_err(req, ENXIO); + } else { + fuse_reply_lseek(req, offset); + } + return; + } + + if (ret & BDRV_BLOCK_DATA) { + if (whence == SEEK_DATA) { + fuse_reply_lseek(req, offset); + return; + } + } else { + if (whence == SEEK_HOLE) { + fuse_reply_lseek(req, offset); + return; + } + } + + /* Safety check against infinite loops */ + if (!pnum) { + fuse_reply_err(req, ENXIO); + return; + } + + offset += pnum; + } +} +#endif + static const struct fuse_lowlevel_ops fuse_ops = { .init = fuse_init, .lookup = fuse_lookup, @@ -638,6 +712,9 @@ static const struct fuse_lowlevel_ops fuse_ops = { .fallocate = fuse_fallocate, .flush = fuse_flush, .fsync = fuse_fsync, +#ifdef CONFIG_FUSE_LSEEK + .lseek = fuse_lseek, +#endif }; const BlockExportDriver blk_exp_fuse = { diff --git a/meson.build b/meson.build index 6e8ef151d8..f344b25955 100644 --- a/meson.build +++ b/meson.build @@ -773,10 +773,28 @@ elif get_option('vhost_user_blk_server').disabled() or not have_system have_vhost_user_blk_server = false endif +if get_option('fuse').disabled() and get_option('fuse_lseek').enabled() + error('Cannot enable fuse-lseek while fuse is disabled') +endif + fuse = dependency('fuse3', required: get_option('fuse'), version: '>=3.1', method: 'pkg-config', static: enable_static) +fuse_lseek = not_found +if not get_option('fuse_lseek').disabled() + if fuse.version().version_compare('>=3.8') + # Dummy dependency + fuse_lseek = declare_dependency() + elif get_option('fuse_lseek').enabled() + if fuse.found() + error('fuse-lseek requires libfuse >=3.8, found ' + fuse.version()) + else + error('fuse-lseek requires libfuse, which was not found') + endif + endif +endif + ################# # config-host.h # ################# @@ -812,6 +830,7 @@ config_host_data.set('CONFIG_GETTID', has_gettid) config_host_data.set('CONFIG_MALLOC_TRIM', has_malloc_trim) config_host_data.set('CONFIG_STATX', has_statx) config_host_data.set('CONFIG_FUSE', fuse.found()) +config_host_data.set('CONFIG_FUSE_LSEEK', fuse_lseek.found()) config_host_data.set('QEMU_VERSION', '"@0@"'.format(meson.project_version())) config_host_data.set('QEMU_VERSION_MAJOR', meson.project_version().split('.')[0]) config_host_data.set('QEMU_VERSION_MINOR', meson.project_version().split('.')[1]) @@ -2214,6 +2233,7 @@ summary_info += {'thread sanitizer': config_host.has_key('CONFIG_TSAN')} summary_info += {'rng-none': config_host.has_key('CONFIG_RNG_NONE')} summary_info += {'Linux keyring': config_host.has_key('CONFIG_SECRET_KEYRING')} summary_info += {'FUSE exports': fuse.found()} +summary_info += {'FUSE lseek': fuse_lseek.found()} summary(summary_info, bool_yn: true) if not supported_cpus.contains(cpu) From patchwork Fri Dec 11 17:07:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0BBBC4361B for ; Fri, 11 Dec 2020 17:34:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4953423DC2 for ; Fri, 11 Dec 2020 17:34:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4953423DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40256 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmIp-0001q4-BR for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:33:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50070) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluZ-0003qX-8H for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:21054) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluI-00029E-3c for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NlNAGwbdtmYnWPDfJD2c+tjwRr6hd9k8Fdpcee5PByw=; b=CaiRJBzD2R0iNfd9AFVKTr9/Dhum4HlUWQNFFg162+QFfgmWg0zz5zJBfoADcclH3A6lQw 8K/PJ6q9Ddjk7630xynTBsOEOnd9u1CUt1lA/9qVNUADvo1Zk9Tudr43kVgRibgdyhfRHy 4ahNPrRyeROHHgJkIZf0wrWDyG7c7r8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-oFWaCkVvO3GcqnnKOCc10Q-1; Fri, 11 Dec 2020 12:08:34 -0500 X-MC-Unique: oFWaCkVvO3GcqnnKOCc10Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5A16E180E469; Fri, 11 Dec 2020 17:08:33 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 552FB5D731; Fri, 11 Dec 2020 17:08:32 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 11/34] iotests: Do not needlessly filter _make_test_img Date: Fri, 11 Dec 2020 18:07:49 +0100 Message-Id: <20201211170812.228643-12-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz In most cases, _make_test_img does not need a _filter_imgfmt on top. It does that by itself. (The exception is when IMGFMT has been overwritten but TEST_IMG has not. In such cases, we do need a _filter_imgfmt on top to filter the test's original IMGFMT from TEST_IMG.) Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-8-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/161 | 12 ++++++------ tests/qemu-iotests/175 | 6 +++--- tests/qemu-iotests/249 | 6 +++--- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tests/qemu-iotests/161 b/tests/qemu-iotests/161 index e270976d87..bbf7dbbc5c 100755 --- a/tests/qemu-iotests/161 +++ b/tests/qemu-iotests/161 @@ -48,9 +48,9 @@ _supported_os Linux IMG_SIZE=1M # Create the images -TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE | _filter_imgfmt -TEST_IMG="$TEST_IMG.int" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT | _filter_imgfmt -_make_test_img -b "$TEST_IMG.int" -F $IMGFMT -F $IMGFMT | _filter_imgfmt +TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE +TEST_IMG="$TEST_IMG.int" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT +_make_test_img -b "$TEST_IMG.int" -F $IMGFMT -F $IMGFMT # First test: reopen $TEST.IMG changing the detect-zeroes option on # its backing file ($TEST_IMG.int). @@ -105,9 +105,9 @@ echo echo "*** Commit and then change an option on the backing file" echo # Create the images again -TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE | _filter_imgfmt -TEST_IMG="$TEST_IMG.int" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT| _filter_imgfmt -_make_test_img -b "$TEST_IMG.int" -F $IMGFMT | _filter_imgfmt +TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE +TEST_IMG="$TEST_IMG.int" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT +_make_test_img -b "$TEST_IMG.int" -F $IMGFMT _launch_qemu -drive if=none,file="${TEST_IMG}" _send_qemu_cmd $QEMU_HANDLE \ diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175 index 00a626aa63..c3c2aed653 100755 --- a/tests/qemu-iotests/175 +++ b/tests/qemu-iotests/175 @@ -89,20 +89,20 @@ min_blocks=$(stat -c '%b' "$TEST_DIR/empty") echo echo "== creating image with default preallocation ==" -_make_test_img -o extent_size_hint=0 $size | _filter_imgfmt +_make_test_img -o extent_size_hint=0 $size stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size for mode in off full falloc; do echo echo "== creating image with preallocation $mode ==" - _make_test_img -o preallocation=$mode,extent_size_hint=0 $size | _filter_imgfmt + _make_test_img -o preallocation=$mode,extent_size_hint=0 $size stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $size done for new_size in 4096 1048576; do echo echo "== resize empty image with block_resize ==" - _make_test_img -o extent_size_hint=0 0 | _filter_imgfmt + _make_test_img -o extent_size_hint=0 0 _block_resize $TEST_IMG $new_size >/dev/null stat -c "size=%s, blocks=%b" $TEST_IMG | _filter_blocks $extra_blocks $min_blocks $new_size done diff --git a/tests/qemu-iotests/249 b/tests/qemu-iotests/249 index 68f13ed328..a9aa9303eb 100755 --- a/tests/qemu-iotests/249 +++ b/tests/qemu-iotests/249 @@ -48,9 +48,9 @@ _supported_os Linux IMG_SIZE=1M # Create the images: base <- int <- active -TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE | _filter_imgfmt -TEST_IMG="$TEST_IMG.int" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT | _filter_imgfmt -_make_test_img -b "$TEST_IMG.int" -F $IMGFMT | _filter_imgfmt +TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE +TEST_IMG="$TEST_IMG.int" _make_test_img -b "$TEST_IMG.base" -F $IMGFMT +_make_test_img -b "$TEST_IMG.int" -F $IMGFMT # Launch QEMU with these two drives: # none0: base (read-only) From patchwork Fri Dec 11 17:07:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B94C6C4361B for ; Fri, 11 Dec 2020 17:23:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33FC523EF1 for ; Fri, 11 Dec 2020 17:23:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33FC523EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35250 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm8b-0003Rz-22 for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:23:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50096) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluc-0003x6-JH for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:58 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:47754) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluJ-0002AP-3T for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:08:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706517; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tYM+0+uAjliHyr6fIWykWQTbOaWPQVG8yHBkWUuGhMY=; b=MjzgYHbsADhoownPxsOsmg/xMPO1AUllIkCSO7WaHsbwb/B4oY5l+LmrXUxgA80p1A2y9Y iIOuvT/IovItqakhWd30LGe+X69PdaOmCsd+QphZWD658FsrdsvlI97GTMjx3SQlpYgguC tJqLmX+L7FkF0P0qRfbY+LZWYUDAgAM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-410-hbiJOLAPOxGVKufHp9z4FQ-1; Fri, 11 Dec 2020 12:08:35 -0500 X-MC-Unique: hbiJOLAPOxGVKufHp9z4FQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A6364180E46E; Fri, 11 Dec 2020 17:08:34 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id A45CC5D731; Fri, 11 Dec 2020 17:08:33 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 12/34] iotests: Do not pipe _make_test_img Date: Fri, 11 Dec 2020 18:07:50 +0100 Message-Id: <20201211170812.228643-13-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Executing _make_test_img as part of a pipe will undo all variable changes it has done. As such, this could not work with FUSE (because we want to remember all of our exports and their qemu instances). Replace the pipe by a temporary file in 071 and 174 (the two tests that can run on FUSE). Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-9-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/071 | 19 +++++++++++++++---- tests/qemu-iotests/174 | 10 +++++++++- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071 index 88faebcc1d..18fe9054b0 100755 --- a/tests/qemu-iotests/071 +++ b/tests/qemu-iotests/071 @@ -61,8 +61,17 @@ echo echo "=== Testing blkverify through filename ===" echo -TEST_IMG="$TEST_IMG.base" IMGFMT="raw" _make_test_img --no-opts $IMG_SIZE |\ - _filter_imgfmt +# _make_test_img may set variables that we need to retain. Everything +# in a pipe is executed in a subshell, so doing so would throw away +# all changes. Therefore, we have to store the output in some temp +# file and filter that. +scratch_out="$TEST_DIR/img-create.out" + +TEST_IMG="$TEST_IMG.base" IMGFMT="raw" _make_test_img --no-opts $IMG_SIZE \ + >"$scratch_out" +_filter_imgfmt <"$scratch_out" +rm -f "$scratch_out" + _make_test_img $IMG_SIZE $QEMU_IO -c "open -o driver=raw,file.driver=blkverify,file.raw.filename=$TEST_IMG.base $TEST_IMG" \ -c 'read 0 512' -c 'write -P 42 0x38000 512' -c 'read -P 42 0x38000 512' | _filter_qemu_io @@ -76,8 +85,10 @@ echo echo "=== Testing blkverify through file blockref ===" echo -TEST_IMG="$TEST_IMG.base" IMGFMT="raw" _make_test_img --no-opts $IMG_SIZE |\ - _filter_imgfmt +TEST_IMG="$TEST_IMG.base" IMGFMT="raw" _make_test_img --no-opts $IMG_SIZE \ + >"$scratch_out" +_filter_imgfmt <"$scratch_out" + _make_test_img $IMG_SIZE $QEMU_IO -c "open -o driver=raw,file.driver=blkverify,file.raw.filename=$TEST_IMG.base,file.test.driver=$IMGFMT,file.test.file.filename=$TEST_IMG" \ -c 'read 0 512' -c 'write -P 42 0x38000 512' -c 'read -P 42 0x38000 512' | _filter_qemu_io diff --git a/tests/qemu-iotests/174 b/tests/qemu-iotests/174 index e2f14a38c6..1b0dd2e8b7 100755 --- a/tests/qemu-iotests/174 +++ b/tests/qemu-iotests/174 @@ -40,7 +40,15 @@ _unsupported_fmt raw size=256K -IMGFMT=raw IMGKEYSECRET= _make_test_img --no-opts $size | _filter_imgfmt + +# _make_test_img may set variables that we need to retain. Everything +# in a pipe is executed in a subshell, so doing so would throw away +# all changes. Therefore, we have to store the output in some temp +# file and filter that. +scratch_out="$TEST_DIR/img-create.out" +IMGFMT=raw IMGKEYSECRET= _make_test_img --no-opts $size >"$scratch_out" +_filter_imgfmt <"$scratch_out" +rm -f "$scratch_out" echo echo "== reading wrong format should fail ==" From patchwork Fri Dec 11 17:07:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1270CC433FE for ; Fri, 11 Dec 2020 17:29:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5CED23DC2 for ; Fri, 11 Dec 2020 17:29:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5CED23DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmEk-0005KG-I3 for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:29:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlui-00042F-BM for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:60368) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluL-0002Ai-Ty for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=taUdhaWhTrR05pDkOeeNsQ06+IhI3i10r0/TDU7Yldg=; b=ZpVMUMNF7PlXlcY3ScyI5sdpgqmMnRZ4CFAdTQuVfs8WcANiAKlURO48pXgGIiYRL7F71m jzhNIUbAKk2NhOiv2iIkMTcBPJv3pou3GWA7Iv7WDQM61e1Ha04q4KcdDryK+VcoX9pLHZ +F8IDp4BQPG5V3bAHEVk7Q+puvKPbtU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-504-Ac2Z56fTPYOdndC2LZ83bQ-1; Fri, 11 Dec 2020 12:08:37 -0500 X-MC-Unique: Ac2Z56fTPYOdndC2LZ83bQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F25FBB8102; Fri, 11 Dec 2020 17:08:35 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id F21865D731; Fri, 11 Dec 2020 17:08:34 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 13/34] iotests: Use convert -n in some cases Date: Fri, 11 Dec 2020 18:07:51 +0100 Message-Id: <20201211170812.228643-14-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz qemu-img convert (without -n) can often be replaced by a combination of _make_test_img + qemu-img convert -n. Doing so allows converting to protocols that do not allow direct file creation, such as FUSE exports. The only problem is that for formats other than qcow2 and qed (qcow1 at least), this may lead to high disk usage for some reason, so we cannot do it everywhere. But we can do it in 028 and 089, so let us do that so they can run on FUSE exports. Also, in 028 this allows us to remove a 9-line comment that used to explain why we cannot safely filter drive-backup's image creation output. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-10-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/028 | 14 ++++---------- tests/qemu-iotests/028.out | 3 +++ tests/qemu-iotests/089 | 3 ++- tests/qemu-iotests/089.out | 1 + 4 files changed, 10 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028 index 6dd3ae09a3..864dc4a4e2 100755 --- a/tests/qemu-iotests/028 +++ b/tests/qemu-iotests/028 @@ -116,16 +116,10 @@ else QEMU_COMM_TIMEOUT=1 fi -# Silence output since it contains the disk image path and QEMU's readline -# character echoing makes it very hard to filter the output. Plus, there -# is no telling how many times the command will repeat before succeeding. -# (Note that creating the image results in a "Formatting..." message over -# stdout, which is the same channel the monitor uses. We cannot reliably -# wait for it because the monitor output may interact with it in such a -# way that _timed_wait_for cannot read it. However, once the block job is -# done, we know that the "Formatting..." message must have appeared -# already, so the output is still deterministic.) -silent=y _send_qemu_cmd $h "drive_backup disk ${TEST_IMG}.copy" "(qemu)" +TEST_IMG="$TEST_IMG.copy" _make_test_img $image_size +_send_qemu_cmd $h "drive_backup -n disk ${TEST_IMG}.copy" "(qemu)" \ + | _filter_imgfmt + silent=y qemu_cmd_repeat=20 _send_qemu_cmd $h "info block-jobs" "No active jobs" _send_qemu_cmd $h "info block-jobs" "No active jobs" _send_qemu_cmd $h 'quit' "" diff --git a/tests/qemu-iotests/028.out b/tests/qemu-iotests/028.out index 5a68de5c46..e580488216 100644 --- a/tests/qemu-iotests/028.out +++ b/tests/qemu-iotests/028.out @@ -468,6 +468,9 @@ No errors were found on the image. block-backup +Formatting 'TEST_DIR/t.IMGFMT.copy', fmt=IMGFMT size=4294968832 +QEMU X.Y.Z monitor - type 'help' for more information +(qemu) drive_backup -n disk TEST_DIR/t.IMGFMT.copy (qemu) info block-jobs No active jobs === IO: pattern 195 diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089 index 66c5415abe..03a2ccf1e8 100755 --- a/tests/qemu-iotests/089 +++ b/tests/qemu-iotests/089 @@ -62,7 +62,8 @@ TEST_IMG="$TEST_IMG.base" _make_test_img $IMG_SIZE $QEMU_IO -c 'write -P 42 0 512' -c 'write -P 23 512 512' \ -c 'write -P 66 1024 512' "$TEST_IMG.base" | _filter_qemu_io -$QEMU_IMG convert -f raw -O $IMGFMT "$TEST_IMG.base" "$TEST_IMG" +_make_test_img $IMG_SIZE +$QEMU_IMG convert -f raw -O $IMGFMT -n "$TEST_IMG.base" "$TEST_IMG" $QEMU_IO_PROG --cache $CACHEMODE --aio $AIOMODE \ -c 'read -P 42 0 512' -c 'read -P 23 512 512' \ diff --git a/tests/qemu-iotests/089.out b/tests/qemu-iotests/089.out index 15682c2886..c53fc4823a 100644 --- a/tests/qemu-iotests/089.out +++ b/tests/qemu-iotests/089.out @@ -9,6 +9,7 @@ wrote 512/512 bytes at offset 512 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 512/512 bytes at offset 1024 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 read 512/512 bytes at offset 0 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 512/512 bytes at offset 512 From patchwork Fri Dec 11 17:07:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 650A0C4361B for ; Fri, 11 Dec 2020 17:32:46 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DB14B23DC2 for ; Fri, 11 Dec 2020 17:32:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DB14B23DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmHc-0000mt-5k for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:32:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlus-00048D-8c for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:14 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43286) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluN-0002Bs-Nk for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706521; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PztFHjihJ/Gl/R6RCt+wOksUHV6+NFI4Cm4G27ZvpfY=; b=fxX2otNcvtB1gWP7oC/A5ja4S6uSmMCcQ8+ARW+EPYBNS3t+3X5U5/ZfKVqhA8HUwkzE0E Zrn82gW6Fet6mRtlXY3GvaqVs+5k1wHbZGGpaW7GF8xXSurak2TlcW2stm+/s/jHmkmyaP 4+BOj0ESwdvj9ZISWDuIEY1M9YzLKTA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-23-lNyG-J1HPn6tcZD3-ZPVxg-1; Fri, 11 Dec 2020 12:08:38 -0500 X-MC-Unique: lNyG-J1HPn6tcZD3-ZPVxg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4AA03801817; Fri, 11 Dec 2020 17:08:37 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 49E0F5D731; Fri, 11 Dec 2020 17:08:36 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 14/34] iotests/046: Avoid renaming images Date: Fri, 11 Dec 2020 18:07:52 +0100 Message-Id: <20201211170812.228643-15-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz This generally does not work on non-file protocols. It is better to create the image with the final name from the start, and most tests do this already. Let 046 follow suit. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-11-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/046 | 5 +++-- tests/qemu-iotests/046.out | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046 index 88b3363c19..40a9f30087 100755 --- a/tests/qemu-iotests/046 +++ b/tests/qemu-iotests/046 @@ -47,6 +47,8 @@ size=128M echo echo "== creating backing file for COW tests ==" +TEST_IMG_SAVE=$TEST_IMG +TEST_IMG="$TEST_IMG.base" _make_test_img $size backing_io() @@ -67,8 +69,7 @@ backing_io() backing_io 0 32 write | $QEMU_IO "$TEST_IMG" | _filter_qemu_io -mv "$TEST_IMG" "$TEST_IMG.base" - +TEST_IMG=$TEST_IMG_SAVE _make_test_img -b "$TEST_IMG.base" -F $IMGFMT 6G echo diff --git a/tests/qemu-iotests/046.out b/tests/qemu-iotests/046.out index b022bcddd5..66ad987ab3 100644 --- a/tests/qemu-iotests/046.out +++ b/tests/qemu-iotests/046.out @@ -1,7 +1,7 @@ QA output created by 046 == creating backing file for COW tests == -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 65536/65536 bytes at offset 65536 From patchwork Fri Dec 11 17:07:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B8FDC4361B for ; Fri, 11 Dec 2020 17:37:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03E6423EF2 for ; Fri, 11 Dec 2020 17:37:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03E6423EF2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49356 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmMK-0005uo-4E for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:37:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50308) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluv-00048p-48 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:17 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:22466) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluN-0002C4-QI for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706522; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=goZmBIyniVyMLlSXxfgjCKnVNMJmhLBkaO5nJxyHp/A=; b=FPZ657PQ+X82jNq/hBtjZXweLKu6dZn3PvkAK6iqGG42DZgiCh1VJbdmyToQAYzwSkhYlT DMkblPQYoyAQDI/h4iP/RGk1/uHap/4wc5TvqC3/zK1AgF+lmVr/1bbns3dE07UZN3dmvl EhhN7fIdFGmQ5+ERsDPlxElKnBPef2o= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-48-lchqb4DTNouqX9dMbSSUuA-1; Fri, 11 Dec 2020 12:08:39 -0500 X-MC-Unique: lchqb4DTNouqX9dMbSSUuA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 97F9A100C660; Fri, 11 Dec 2020 17:08:38 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 96E835D731; Fri, 11 Dec 2020 17:08:37 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 15/34] iotests: Derive image names from $TEST_IMG Date: Fri, 11 Dec 2020 18:07:53 +0100 Message-Id: <20201211170812.228643-16-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Avoid creating images with custom filenames in $TEST_DIR, because non-file protocols may want to keep $TEST_IMG (and all other test images) in some other directory. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-12-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/200 | 3 +-- tests/qemu-iotests/200.out | 4 ++-- tests/qemu-iotests/229 | 3 +-- tests/qemu-iotests/229.out | 6 +++--- 4 files changed, 7 insertions(+), 9 deletions(-) diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200 index 59f7854b9f..a7aabbd032 100755 --- a/tests/qemu-iotests/200 +++ b/tests/qemu-iotests/200 @@ -44,8 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 qed _supported_proto file -BACKING_IMG="${TEST_DIR}/backing.img" -TEST_IMG="${TEST_DIR}/test.img" +BACKING_IMG="$TEST_IMG.base" TEST_IMG="$BACKING_IMG" _make_test_img 512M _make_test_img -F $IMGFMT -b "$BACKING_IMG" 512M diff --git a/tests/qemu-iotests/200.out b/tests/qemu-iotests/200.out index a6776070e4..5883f16ac3 100644 --- a/tests/qemu-iotests/200.out +++ b/tests/qemu-iotests/200.out @@ -1,6 +1,6 @@ QA output created by 200 -Formatting 'TEST_DIR/backing.img', fmt=IMGFMT size=536870912 -Formatting 'TEST_DIR/test.img', fmt=IMGFMT size=536870912 backing_file=TEST_DIR/backing.img backing_fmt=IMGFMT +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=536870912 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT wrote 314572800/314572800 bytes at offset 512 300 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/229 b/tests/qemu-iotests/229 index 89a5359f32..5f759fa587 100755 --- a/tests/qemu-iotests/229 +++ b/tests/qemu-iotests/229 @@ -51,8 +51,7 @@ _supported_os Linux _unsupported_imgopts data_file -DEST_IMG="$TEST_DIR/d.$IMGFMT" -TEST_IMG="$TEST_DIR/b.$IMGFMT" +DEST_IMG="$TEST_IMG.dest" BLKDEBUG_CONF="$TEST_DIR/blkdebug.conf" _make_test_img 2M diff --git a/tests/qemu-iotests/229.out b/tests/qemu-iotests/229.out index 4de6dfaa28..7eed393013 100644 --- a/tests/qemu-iotests/229.out +++ b/tests/qemu-iotests/229.out @@ -1,6 +1,6 @@ QA output created by 229 -Formatting 'TEST_DIR/b.IMGFMT', fmt=IMGFMT size=2097152 -Formatting 'TEST_DIR/d.IMGFMT', fmt=IMGFMT size=2097152 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152 +Formatting 'TEST_DIR/t.IMGFMT.dest', fmt=IMGFMT size=2097152 wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {'execute': 'qmp_capabilities'} @@ -8,7 +8,7 @@ wrote 2097152/2097152 bytes at offset 0 === Starting drive-mirror, causing error & stop === -{'execute': 'drive-mirror', 'arguments': {'device': 'testdisk', 'format': 'IMGFMT', 'target': 'blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/d.IMGFMT', 'sync': 'full', 'mode': 'existing', 'on-source-error': 'stop', 'on-target-error': 'stop' }} +{'execute': 'drive-mirror', 'arguments': {'device': 'testdisk', 'format': 'IMGFMT', 'target': 'blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT.dest', 'sync': 'full', 'mode': 'existing', 'on-source-error': 'stop', 'on-target-error': 'stop' }} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "testdisk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "testdisk"}} {"return": {}} From patchwork Fri Dec 11 17:07:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 352EAC4361B for ; Fri, 11 Dec 2020 17:36:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F44C23EF1 for ; Fri, 11 Dec 2020 17:36:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F44C23EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:46696 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmLX-0004iy-Nl for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:36:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlux-0004A4-Q9 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:19 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27533) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluN-0002CN-QU for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706523; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zn3uzvzjHLTFN8HuDvq8ZStsD36R6lONUzG018HcE+s=; b=TRQ0boAka0ayhGObwb4uTGYehEWuwvQttqJen+uTTlL5e6EAloY0OOWuvRWHuyG+77KVU1 jba4vmAbRDuZ1FrawwvBrT9ruSaC8PavNWeXWJ94mh30S4JU5YMovm0dNuK0cT3mRWSeXN leUDRFRadXuuTZS9gSyA1ch1QFoKQ8c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-289-lK9oX9KsOGy__q-d_4Iamg-1; Fri, 11 Dec 2020 12:08:41 -0500 X-MC-Unique: lK9oX9KsOGy__q-d_4Iamg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E5966B8105; Fri, 11 Dec 2020 17:08:39 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id E488D5D731; Fri, 11 Dec 2020 17:08:38 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 16/34] iotests/091: Use _cleanup_qemu instad of "wait" Date: Fri, 11 Dec 2020 18:07:54 +0100 Message-Id: <20201211170812.228643-17-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz If the test environment has some other child processes running (like a storage daemon that provides a FUSE export), then "wait" will never finish. Use wait=yes _cleanup_qemu instead. (We need to discard the output so there is no change to the reference output.) Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-13-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/091 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091 index 68fbfd777b..8a4ce5b7e2 100755 --- a/tests/qemu-iotests/091 +++ b/tests/qemu-iotests/091 @@ -96,7 +96,8 @@ _send_qemu_cmd $h2 'qemu-io disk flush' "(qemu)" _send_qemu_cmd $h2 'quit' "" _send_qemu_cmd $h1 'quit' "" -wait +wait=yes _cleanup_qemu >/dev/null + echo "Check image pattern" ${QEMU_IO} -c "read -P 0x22 0 4M" "${TEST_IMG}" | _filter_testdir | _filter_qemu_io From patchwork Fri Dec 11 17:07:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D7BBC4361B for ; Fri, 11 Dec 2020 17:40:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D864523EF1 for ; Fri, 11 Dec 2020 17:40:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D864523EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmPA-0000da-Pz for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:40:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50374) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knluz-0004Bj-Bu for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21783) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluR-0002Dk-Jf for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9bLvWWq8ylxzJiVSvMxxdy1clocGP94myryRr59obHs=; b=KQCrPcWQPsN/D/GgGrwiTVzUSjRPW5Nyb/cfIzCrrWQ28Mj7HI6VeczjrLIHJbi5qdw4gL o5Ndr3gN8Uof9Ig4EODTUHftQBqtjLzls1CoBuqKpYTmx+l4hSS7pCOjxv1OuDsqAqCt06 bb6hD2OsiolikiLmai2Ng76GTWidUH8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-429-FpLqifqOMPy_Ouh-PkIVFg-1; Fri, 11 Dec 2020 12:08:42 -0500 X-MC-Unique: FpLqifqOMPy_Ouh-PkIVFg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3ED5A107ACE6; Fri, 11 Dec 2020 17:08:41 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D2DB5D6A8; Fri, 11 Dec 2020 17:08:40 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 17/34] iotests: Restrict some Python tests to file Date: Fri, 11 Dec 2020 18:07:55 +0100 Message-Id: <20201211170812.228643-18-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Most Python tests are restricted to the file protocol (without explicitly saying so), but these are the ones that would break ./check -fuse -qcow2. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-14-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/206 | 3 ++- tests/qemu-iotests/242 | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206 index 11bc51f256..0a3ee5ef00 100755 --- a/tests/qemu-iotests/206 +++ b/tests/qemu-iotests/206 @@ -23,7 +23,8 @@ import iotests from iotests import imgfmt -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + supported_protocols=['file']) iotests.verify_working_luks() with iotests.FilePath('t.qcow2') as disk_path, \ diff --git a/tests/qemu-iotests/242 b/tests/qemu-iotests/242 index 64f1bd95e4..a16de3085f 100755 --- a/tests/qemu-iotests/242 +++ b/tests/qemu-iotests/242 @@ -24,7 +24,8 @@ import struct from iotests import qemu_img_create, qemu_io, qemu_img_pipe, \ file_path, img_info_log, log, filter_qemu_io -iotests.script_initialize(supported_fmts=['qcow2']) +iotests.script_initialize(supported_fmts=['qcow2'], + supported_protocols=['file']) disk = file_path('disk') chunk = 256 * 1024 From patchwork Fri Dec 11 17:07:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42C00C4361B for ; Fri, 11 Dec 2020 17:41:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BCDC223EF1 for ; Fri, 11 Dec 2020 17:41:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BCDC223EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59338 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmPq-0001vO-SW for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:41:14 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50390) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv0-0004Cf-CD for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53421) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluR-0002D8-IU for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XXQ1q4eGCs928B6Oq0fCOP/EJLg6jVmHtuPgWUDbyC8=; b=JdaHZOXenM2uLvnnAPi/UV+32YfWzAHJrEnpT/HnWNqUa/JXskVmEWXdKfYhlk9xJxNgEH 7dZyG6irHKTCKoJXMlZn98MPuns6VBZ+vwLi5zuzm5lLyvbjtMjalfz+W6BgGRb2ypLxkf TZol9zRrnU+fMP8/bCLAU1AVV6EcoAM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-11-2AlETZAmMK2FMjgYxJEEXQ-1; Fri, 11 Dec 2020 12:08:43 -0500 X-MC-Unique: 2AlETZAmMK2FMjgYxJEEXQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 88BED858180; Fri, 11 Dec 2020 17:08:42 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 89BF65D6A8; Fri, 11 Dec 2020 17:08:41 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 18/34] iotests: Let _make_test_img guess $TEST_IMG_FILE Date: Fri, 11 Dec 2020 18:07:56 +0100 Message-Id: <20201211170812.228643-19-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz When most iotests want to create a test image that is named differently from the default $TEST_IMG, they do something like this: TEST_IMG="$TEST_IMG.base" _make_test_img $options This works fine with the "file" protocol, but not so much for anything else: _make_test_img tries to create an image under $TEST_IMG_FILE first, and only under $TEST_IMG if the former is not set; and on everything but "file", $TEST_IMG_FILE is set. There are two ways we can fix this: First, we could make all tests adjust not only TEST_IMG, but also TEST_IMG_FILE if that is present (e.g. with something like _set_test_img_suffix $suffix that would affect not only TEST_IMG but also TEST_IMG_FILE, if necessary). This is a pretty clean solution, and this is maybe what we should have done from the start. But it would also require changes to most existing bash tests. So the alternative is this: Let _make_test_img see whether $TEST_IMG_FILE still points to the original value. If so, it is possible that the caller has adjusted $TEST_IMG but not $TEST_IMG_FILE. In such a case, we can (for most protocols) derive the corresponding $TEST_IMG_FILE value from $TEST_IMG value and thus work around what technically is the caller misbehaving. This second solution is less clean, but it is robust against people keeping their old habit of adjusting TEST_IMG only, and requires much less changes. So this patch implements it. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-15-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/common.rc | 40 +++++++++++++++++++++++++++++++++--- 1 file changed, 37 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 494490a272..23f46da2db 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -268,6 +268,7 @@ else TEST_IMG=$IMGPROTO:$TEST_DIR/t.$IMGFMT fi fi +ORIG_TEST_IMG_FILE=$TEST_IMG_FILE ORIG_TEST_IMG="$TEST_IMG" if [ -z "$TEST_DIR" ]; then @@ -330,6 +331,30 @@ _get_data_file() | sed -e "s#\\\$TEST_IMG#$1#" } +# Translate a $TEST_IMG to its corresponding $TEST_IMG_FILE for +# different protocols +_test_img_to_test_img_file() +{ + case "$IMGPROTO" in + file) + echo "$1" + ;; + + nfs) + echo "$1" | sed -e "s#nfs://127.0.0.1##" + ;; + + ssh) + echo "$1" | \ + sed -e "s#ssh://\\($USER@\\)\\?127.0.0.1\\(:[0-9]\\+\\)\\?##" + ;; + + *) + return 1 + ;; + esac +} + _make_test_img() { # extra qemu-img options can be added by tests @@ -343,10 +368,19 @@ _make_test_img() local opts_param=false local misc_params=() - if [ -n "$TEST_IMG_FILE" ]; then - img_name=$TEST_IMG_FILE - else + if [ -z "$TEST_IMG_FILE" ]; then img_name=$TEST_IMG + elif [ "$IMGOPTSSYNTAX" != "true" -a \ + "$TEST_IMG_FILE" = "$ORIG_TEST_IMG_FILE" ]; then + # Handle cases of tests only updating TEST_IMG, but not TEST_IMG_FILE + img_name=$(_test_img_to_test_img_file "$TEST_IMG") + if [ "$?" != 0 ]; then + img_name=$TEST_IMG_FILE + fi + else + # $TEST_IMG_FILE is not the default value, so it definitely has been + # modified by the test + img_name=$TEST_IMG_FILE fi if [ -n "$IMGOPTS" ]; then From patchwork Fri Dec 11 17:07:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0492DC4361B for ; Fri, 11 Dec 2020 17:28:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6535223EF1 for ; Fri, 11 Dec 2020 17:28:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6535223EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmDI-0002iR-Ey for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:28:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50418) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv2-0004Do-4o for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:23704) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluU-0002Eg-FT for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c6yUMGjkIw1U8jlbVVitY/0E/gdBSqbl/+tWD2FZ6RI=; b=EK90os/WE9lWxV+N269Ltjt5ou0x3GWQh4EO9yqW2yOWNvnv79yt9bnoijTwPUcYYKrcAb OR4kIsjqv11qMyeNS+PZCk3wktUC2E6hvPXjR8vwqEfMqECEO3Tlf3h4XsRYICEpq6gmxQ T9FAv33E7oLMQQ9U2Ex535m0CwZfywI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-tdwwJ-YFPv-UUkoIVkFYMw-1; Fri, 11 Dec 2020 12:08:45 -0500 X-MC-Unique: tdwwJ-YFPv-UUkoIVkFYMw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D5FA08015D0; Fri, 11 Dec 2020 17:08:43 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id D50EE5D6A8; Fri, 11 Dec 2020 17:08:42 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 19/34] iotests/287: Clean up subshell test image Date: Fri, 11 Dec 2020 18:07:57 +0100 Message-Id: <20201211170812.228643-20-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz 287 creates an image in a subshell (thanks to the pipe) to see whether that is possible with compression_type=zstd. If _make_test_img were to modify any global state, this global state would then be lost before we could cleanup the image. When using FUSE as the test protocol, this global state is important, so clean up the image before the state is lost. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-16-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/287 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287 index f98a4cadc1..036cc09e82 100755 --- a/tests/qemu-iotests/287 +++ b/tests/qemu-iotests/287 @@ -51,8 +51,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 CLUSTER_SIZE=65536 # Check if we can run this test. -if IMGOPTS='compression_type=zstd' _make_test_img 64M | - grep "Invalid parameter 'zstd'"; then +output=$(_make_test_img -o 'compression_type=zstd' 64M; _cleanup_test_img) +if echo "$output" | grep -q "Invalid parameter 'zstd'"; then _notrun "ZSTD is disabled" fi From patchwork Fri Dec 11 17:07:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 868CDC4167B for ; Fri, 11 Dec 2020 17:26:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F2E5E2404D for ; Fri, 11 Dec 2020 17:25:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2E5E2404D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmB4-0007SD-Rm for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:25:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50410) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv1-0004Cp-FR for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:23 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:55367) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluT-0002Ea-Bj for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706528; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ew2CPMIIsEJ5A0oBZ8T0MCj0xI9qGY9SF/yYazVPLvE=; b=IFUON1KHQEdOqGF4YVbAxqu4nYLhkEjRhpdXWqKW3zqMr3kcrVMB8RKIHHoStzQbe/JIEb ipvFOZFhseEBzq3VEN1CgJYlnn55ufYHWKAOAdc/9WOM3UbANLejnT6QosuuugrOIBvgoa FmnOXL20kqhO7mlN3avVdVidoS7OMBg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-162-k0n-sGk4NS2JQkMPhBOdKg-1; Fri, 11 Dec 2020 12:08:46 -0500 X-MC-Unique: k0n-sGk4NS2JQkMPhBOdKg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 41474180E473; Fri, 11 Dec 2020 17:08:45 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 402F55D734; Fri, 11 Dec 2020 17:08:44 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 20/34] storage-daemon: Call bdrv_close_all() on exit Date: Fri, 11 Dec 2020 18:07:58 +0100 Message-Id: <20201211170812.228643-21-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Otherwise, exports and block devices are not properly shut down and closed, unless the users explicitly issues blockdev-del and block-export-del commands for each of them. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-17-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- storage-daemon/qemu-storage-daemon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/storage-daemon/qemu-storage-daemon.c b/storage-daemon/qemu-storage-daemon.c index 7c914b0dc1..e0c87edbdd 100644 --- a/storage-daemon/qemu-storage-daemon.c +++ b/storage-daemon/qemu-storage-daemon.c @@ -314,6 +314,9 @@ int main(int argc, char *argv[]) main_loop_wait(false); } + bdrv_drain_all_begin(); + bdrv_close_all(); + monitor_cleanup(); qemu_chr_cleanup(); user_creatable_cleanup(); From patchwork Fri Dec 11 17:07:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76EB6C433FE for ; Fri, 11 Dec 2020 17:21:27 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 172F223DC2 for ; Fri, 11 Dec 2020 17:21:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 172F223DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm6f-00006y-Ub for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:21:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50452) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv4-0004Fh-CF for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:27 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37739) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluY-0002Fz-UL for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706532; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k1UzbfO3esCOtQrbuwUM6xMVpKO+Rjw+LLbfQ3Nxrv0=; b=BRsOCCWIJcwF8Losefx5/8l3zRD3/KiPANZWYWh8sLI+nViZEKCe8uciPOwKJbjOJhiMQf kP9W+cH3x3xCUAyDVz56prQCvTSr3XQwAsMK10haJIhh3fmBZi0T6gYNxv+QpZh25Cn0WF 3VQdLlwfKmGoc50cJa5j7dXZXdKDeEw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-XBEnQNNMMCKWmDU_l0x1pg-1; Fri, 11 Dec 2020 12:08:47 -0500 X-MC-Unique: XBEnQNNMMCKWmDU_l0x1pg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A0F7B107ACE3; Fri, 11 Dec 2020 17:08:46 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D0A75D6A8; Fri, 11 Dec 2020 17:08:45 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 21/34] iotests: Give access to the qemu-storage-daemon Date: Fri, 11 Dec 2020 18:07:59 +0100 Message-Id: <20201211170812.228643-22-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-18-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 11 +++++++++++ tests/qemu-iotests/common.rc | 17 +++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 678b6e4910..3c1fa4435a 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -644,6 +644,17 @@ if [ -z $QEMU_NBD_PROG ]; then fi export QEMU_NBD_PROG="$(type -p "$QEMU_NBD_PROG")" +if [ -z "$QSD_PROG" ]; then + if [ -x "$build_iotests/qemu-storage-daemon" ]; then + export QSD_PROG="$build_iotests/qemu-storage-daemon" + elif [ -x "$build_root/storage-daemon/qemu-storage-daemon" ]; then + export QSD_PROG="$build_root/storage-daemon/qemu-storage-daemon" + else + _init_error "qemu-storage-daemon not found" + fi +fi +export QSD_PROG="$(type -p "$QSD_PROG")" + if [ -x "$build_iotests/socket_scm_helper" ] then export SOCKET_SCM_HELPER="$build_iotests/socket_scm_helper" diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 23f46da2db..20589e59a5 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -124,6 +124,7 @@ fi : ${VALGRIND_QEMU_IMG=$VALGRIND_QEMU} : ${VALGRIND_QEMU_IO=$VALGRIND_QEMU} : ${VALGRIND_QEMU_NBD=$VALGRIND_QEMU} +: ${VALGRIND_QSD=$VALGRIND_QEMU} # The Valgrind own parameters may be set with # its environment variable VALGRIND_OPTS, e.g. @@ -211,6 +212,21 @@ _qemu_nbd_wrapper() return $RETVAL } +_qemu_storage_daemon_wrapper() +{ + local VALGRIND_LOGFILE="${TEST_DIR}"/$$.valgrind + ( + if [ -n "${QSD_NEED_PID}" ]; then + echo $BASHPID > "${QEMU_TEST_DIR}/qemu-storage-daemon.pid" + fi + VALGRIND_QEMU="${VALGRIND_QSD}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \ + "$QSD_PROG" $QSD_OPTIONS "$@" + ) + RETVAL=$? + _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL + return $RETVAL +} + # Valgrind bug #409141 https://bugs.kde.org/show_bug.cgi?id=409141 # Until valgrind 3.16+ is ubiquitous, we must work around a hang in # valgrind when issuing sigkill. Disable valgrind for this invocation. @@ -223,6 +239,7 @@ export QEMU=_qemu_wrapper export QEMU_IMG=_qemu_img_wrapper export QEMU_IO=_qemu_io_wrapper export QEMU_NBD=_qemu_nbd_wrapper +export QSD=_qemu_storage_daemon_wrapper if [ "$IMGOPTSSYNTAX" = "true" ]; then DRIVER="driver=$IMGFMT" From patchwork Fri Dec 11 17:08:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECEDCC4361B for ; Fri, 11 Dec 2020 17:30:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9242823EF1 for ; Fri, 11 Dec 2020 17:30:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9242823EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33548 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmFt-0006sG-Jg for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:30:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50484) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv6-0004Ga-3A for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51022) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluY-0002H3-Vs for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706534; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=USb2nBe0+scM0kjiNuae0MN90cBshxLU37cqU2q1P2E=; b=DMR09gETXJh0qgLOZmwSeu7990P+Wmu4LoNjIKrZU0N9Mi+tt/dohLEoFpC0v0dP/2sGla kebn+NowC4keaA2g+YXv1nSzpgLo23YScNIqXkLyKrWJtdZHgFIwBbjCoU23Lz/83Gsk5R Oing37LleTbMQfjB10OVcqXHbqPKOR4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-251-GHPBXmvsNVKK_7QF8AyQyA-1; Fri, 11 Dec 2020 12:08:50 -0500 X-MC-Unique: GHPBXmvsNVKK_7QF8AyQyA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EC73D800D53; Fri, 11 Dec 2020 17:08:47 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id EAF2D5D734; Fri, 11 Dec 2020 17:08:46 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 22/34] iotests: Allow testing FUSE exports Date: Fri, 11 Dec 2020 18:08:00 +0100 Message-Id: <20201211170812.228643-23-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz This pretends FUSE exports are a kind of protocol. As such, they are always tested under the format node. This is probably the best way to test them, actually, because this will generate more I/O load and more varied patterns. Signed-off-by: Max Reitz Message-Id: <20201027190600.192171-19-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/check | 6 ++ tests/qemu-iotests/common.filter | 5 +- tests/qemu-iotests/common.rc | 124 +++++++++++++++++++++++++++++++ 3 files changed, 134 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 3c1fa4435a..952762d5ed 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -270,6 +270,7 @@ image protocol options -rbd test rbd -sheepdog test sheepdog -nbd test nbd + -fuse test fuse -ssh test ssh -nfs test nfs @@ -382,6 +383,11 @@ testlist options xpand=false ;; + -fuse) + IMGPROTO=fuse + xpand=false + ;; + -ssh) IMGPROTO=ssh xpand=false diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 838ed15793..172ea5752e 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -44,7 +44,8 @@ _filter_qom_path() _filter_testdir() { $SED -e "s#$TEST_DIR/#TEST_DIR/#g" \ - -e "s#$SOCK_DIR/#SOCK_DIR/#g" + -e "s#$SOCK_DIR/#SOCK_DIR/#g" \ + -e "s#SOCK_DIR/fuse-#TEST_DIR/#g" } # replace occurrences of the actual IMGFMT value with IMGFMT @@ -127,6 +128,7 @@ _filter_img_create_filenames() -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ -e "s#$SOCK_DIR#SOCK_DIR#g" \ + -e 's#SOCK_DIR/fuse-#TEST_DIR/#g' \ -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd:127.0.0.1:[0-9]\\+#TEST_DIR/t.IMGFMT#g' \ -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' @@ -227,6 +229,7 @@ _filter_img_info() -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd://127.0.0.1:[0-9]\\+$#TEST_DIR/t.IMGFMT#g' \ -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' \ + -e 's#SOCK_DIR/fuse-#TEST_DIR/#g' \ -e "/encrypted: yes/d" \ -e "/cluster_size: [0-9]\\+/d" \ -e "/table_size: [0-9]\\+/d" \ diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 20589e59a5..29354654cc 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -257,6 +257,9 @@ if [ "$IMGOPTSSYNTAX" = "true" ]; then TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT TEST_IMG="$DRIVER,file.driver=nbd,file.type=unix" TEST_IMG="$TEST_IMG,file.path=$SOCK_DIR/nbd" + elif [ "$IMGPROTO" = "fuse" ]; then + TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT + TEST_IMG="$DRIVER,file.filename=$SOCK_DIR/fuse-t.$IMGFMT" elif [ "$IMGPROTO" = "ssh" ]; then TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT TEST_IMG="$DRIVER,file.driver=ssh,file.host=127.0.0.1,file.path=$TEST_IMG_FILE" @@ -273,6 +276,9 @@ else elif [ "$IMGPROTO" = "nbd" ]; then TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT TEST_IMG="nbd+unix:///?socket=$SOCK_DIR/nbd" + elif [ "$IMGPROTO" = "fuse" ]; then + TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT + TEST_IMG="$SOCK_DIR/fuse-t.$IMGFMT" elif [ "$IMGPROTO" = "ssh" ]; then TEST_IMG_FILE=$TEST_DIR/t.$IMGFMT REMOTE_TEST_DIR="ssh://\\($USER@\\)\\?127.0.0.1\\(:[0-9]\\+\\)\\?$TEST_DIR" @@ -288,6 +294,9 @@ fi ORIG_TEST_IMG_FILE=$TEST_IMG_FILE ORIG_TEST_IMG="$TEST_IMG" +FUSE_PIDS=() +FUSE_EXPORTS=() + if [ -z "$TEST_DIR" ]; then TEST_DIR=$PWD/scratch fi @@ -357,6 +366,10 @@ _test_img_to_test_img_file() echo "$1" ;; + fuse) + echo "$1" | sed -e "s#$SOCK_DIR/fuse-#$TEST_DIR/#" + ;; + nfs) echo "$1" | sed -e "s#nfs://127.0.0.1##" ;; @@ -385,6 +398,11 @@ _make_test_img() local opts_param=false local misc_params=() + if [[ $IMGPROTO == fuse && $TEST_IMG == $SOCK_DIR/fuse-* ]]; then + # The caller may be trying to overwrite an existing image + _rm_test_img "$TEST_IMG" + fi + if [ -z "$TEST_IMG_FILE" ]; then img_name=$TEST_IMG elif [ "$IMGOPTSSYNTAX" != "true" -a \ @@ -469,11 +487,105 @@ _make_test_img() eval "$QEMU_NBD -v -t -k '$SOCK_DIR/nbd' -f $IMGFMT -e 42 -x '' $TEST_IMG_FILE >/dev/null &" sleep 1 # FIXME: qemu-nbd needs to be listening before we continue fi + + if [ $IMGPROTO = "fuse" -a -f "$img_name" ]; then + local export_mp + local pid + local pidfile + local timeout + + export_mp=$(echo "$img_name" | sed -e "s#$TEST_DIR/#$SOCK_DIR/fuse-#") + if ! echo "$export_mp" | grep -q "^$SOCK_DIR"; then + echo 'Cannot use FUSE exports with images outside of TEST_DIR' >&2 + return 1 + fi + + touch "$export_mp" + rm -f "$SOCK_DIR/fuse-output" + + # Usually, users would export formatted nodes. But we present fuse as a + # protocol-level driver here, so we have to leave the format to the + # client. + QSD_NEED_PID=y $QSD \ + --blockdev file,node-name=export-node,filename=$img_name,discard=unmap \ + --export fuse,id=fuse-export,node-name=export-node,mountpoint="$export_mp",writable=on,growable=on \ + & + + pidfile="$QEMU_TEST_DIR/qemu-storage-daemon.pid" + + # Wait for the PID file + while [ ! -f "$pidfile" ]; do + sleep 0.5 + done + + pid=$(cat "$pidfile") + rm -f "$pidfile" + + FUSE_PIDS+=($pid) + FUSE_EXPORTS+=("$export_mp") + fi } _rm_test_img() { local img=$1 + + if [[ $IMGPROTO == fuse && $img == $SOCK_DIR/fuse-* ]]; then + # Drop a FUSE export + local df_output + local i + local image_file + local index='' + local timeout + + for i in "${!FUSE_EXPORTS[@]}"; do + if [ "${FUSE_EXPORTS[i]}" = "$img" ]; then + index=$i + break + fi + done + + if [ -z "$index" ]; then + # Probably gone already + return 0 + fi + + kill "${FUSE_PIDS[index]}" + + # Wait until the mount is gone + timeout=10 # *0.5 s + while true; do + # Will show the mount point; if the mount is still there, + # it will be $img. + df_output=$(df "$img" 2>/dev/null) + + # But df may also show an error ("Transpoint endpoint not + # connected"), so retry in such cases + if [ -n "$df_output" ]; then + if ! echo "$df_output" | grep -q "$img"; then + break + fi + fi + + sleep 0.5 + + timeout=$((timeout - 1)) + if [ "$timeout" = 0 ]; then + echo 'Failed to take down FUSE export' >&2 + return 1 + fi + done + + rm -f "$img" + + unset "FUSE_PIDS[$index]" + unset "FUSE_EXPORTS[$index]" + + image_file=$(echo "$img" | sed -e "s#$SOCK_DIR/fuse-#$TEST_DIR/#") + _rm_test_img "$image_file" + return + fi + if [ "$IMGFMT" = "vmdk" ]; then # Remove all the extents for vmdk "$QEMU_IMG" info "$img" 2>/dev/null | grep 'filename:' | cut -f 2 -d: \ @@ -496,6 +608,17 @@ _cleanup_test_img() rm -f "$TEST_IMG_FILE" ;; + fuse) + local mp + + for mp in "${FUSE_EXPORTS[@]}"; do + _rm_test_img "$mp" + done + + FUSE_PIDS=() + FUSE_EXPORTS=() + ;; + file) _rm_test_img "$TEST_DIR/t.$IMGFMT" _rm_test_img "$TEST_DIR/t.$IMGFMT.orig" @@ -562,6 +685,7 @@ _img_info() sed -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ + -e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \ -e "s#$IMGFMT#IMGFMT#g" \ -e "/^disk size:/ D" \ -e "/actual-size/ D" | \ From patchwork Fri Dec 11 17:08:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4631C433FE for ; Fri, 11 Dec 2020 17:23:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 19CAD23EF1 for ; Fri, 11 Dec 2020 17:23:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 19CAD23EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knm8n-0003wL-6p for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:23:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv9-0004IR-HI for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:54910) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluY-0002Gc-Vb for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706533; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vDUZUxN9kzzQ7NWfexWAv4H6iFzZkP0DZKgt9FaqIfk=; b=RnpOTGMGmw/9EkntJjOyyhVw5HkLzMsqIiXGtLEq15jBH++xwYt1nCqBlv77wgSlYuBAgu eSa9K7Vt8ZfP3CAyTuC4+dyBOFVtOWMwbFL9gmxrHTO6a3sKPXkoTurYtENLK53+fBm24z ccaHlSB2K0azEXqIXm0krvd/Fgf8kQo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-446-PsFZg1c1MM28QJ5OzQ-NvA-1; Fri, 11 Dec 2020 12:08:50 -0500 X-MC-Unique: PsFZg1c1MM28QJ5OzQ-NvA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A5EE10054FF; Fri, 11 Dec 2020 17:08:49 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5010E5D6A8; Fri, 11 Dec 2020 17:08:48 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 23/34] iotests: Enable fuse for many tests Date: Fri, 11 Dec 2020 18:08:01 +0100 Message-Id: <20201211170812.228643-24-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz Many tests (that do not support generic protocols) can run just fine with FUSE-exported images, so allow them to. Note that this is no attempt at being definitely complete. There are some tests that might be modified to run on FUSE, but this patch still skips them. This patch only tries to pick the rather low-hanging fruits. Note that 221 and 250 only pass when .lseek is correctly implemented, which is only possible with a libfuse that is 3.8 or newer. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-20-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/025 | 2 +- tests/qemu-iotests/026 | 2 +- tests/qemu-iotests/028 | 2 +- tests/qemu-iotests/031 | 2 +- tests/qemu-iotests/034 | 2 +- tests/qemu-iotests/036 | 2 +- tests/qemu-iotests/037 | 2 +- tests/qemu-iotests/038 | 2 +- tests/qemu-iotests/039 | 2 +- tests/qemu-iotests/046 | 2 +- tests/qemu-iotests/050 | 2 +- tests/qemu-iotests/054 | 2 +- tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/071 | 2 +- tests/qemu-iotests/079 | 2 +- tests/qemu-iotests/080 | 2 +- tests/qemu-iotests/089 | 2 +- tests/qemu-iotests/090 | 2 +- tests/qemu-iotests/091 | 2 +- tests/qemu-iotests/095 | 2 +- tests/qemu-iotests/097 | 2 +- tests/qemu-iotests/098 | 2 +- tests/qemu-iotests/102 | 2 +- tests/qemu-iotests/103 | 2 +- tests/qemu-iotests/106 | 2 +- tests/qemu-iotests/107 | 2 +- tests/qemu-iotests/108 | 2 +- tests/qemu-iotests/111 | 2 +- tests/qemu-iotests/112 | 2 +- tests/qemu-iotests/115 | 2 +- tests/qemu-iotests/117 | 2 +- tests/qemu-iotests/120 | 2 +- tests/qemu-iotests/121 | 2 +- tests/qemu-iotests/127 | 2 +- tests/qemu-iotests/133 | 2 +- tests/qemu-iotests/137 | 2 +- tests/qemu-iotests/138 | 2 +- tests/qemu-iotests/140 | 2 +- tests/qemu-iotests/154 | 2 +- tests/qemu-iotests/161 | 2 +- tests/qemu-iotests/171 | 2 +- tests/qemu-iotests/175 | 2 +- tests/qemu-iotests/176 | 2 +- tests/qemu-iotests/177 | 2 +- tests/qemu-iotests/179 | 2 +- tests/qemu-iotests/183 | 2 +- tests/qemu-iotests/186 | 2 +- tests/qemu-iotests/187 | 2 +- tests/qemu-iotests/191 | 2 +- tests/qemu-iotests/195 | 2 +- tests/qemu-iotests/200 | 2 +- tests/qemu-iotests/204 | 2 +- tests/qemu-iotests/214 | 2 +- tests/qemu-iotests/217 | 2 +- tests/qemu-iotests/220 | 2 +- tests/qemu-iotests/221 | 2 +- tests/qemu-iotests/229 | 2 +- tests/qemu-iotests/247 | 2 +- tests/qemu-iotests/249 | 2 +- tests/qemu-iotests/250 | 2 +- tests/qemu-iotests/252 | 2 +- tests/qemu-iotests/265 | 2 +- tests/qemu-iotests/268 | 2 +- tests/qemu-iotests/272 | 2 +- tests/qemu-iotests/273 | 2 +- tests/qemu-iotests/279 | 2 +- tests/qemu-iotests/286 | 2 +- tests/qemu-iotests/287 | 2 +- tests/qemu-iotests/289 | 2 +- tests/qemu-iotests/290 | 2 +- tests/qemu-iotests/291 | 2 +- tests/qemu-iotests/292 | 2 +- tests/qemu-iotests/293 | 2 +- tests/qemu-iotests/294 | 2 +- tests/qemu-iotests/305 | 2 +- 75 files changed, 75 insertions(+), 75 deletions(-) diff --git a/tests/qemu-iotests/025 b/tests/qemu-iotests/025 index e05d833452..1569d912f4 100755 --- a/tests/qemu-iotests/025 +++ b/tests/qemu-iotests/025 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.pattern _supported_fmt raw qcow2 qed luks -_supported_proto file sheepdog rbd nfs +_supported_proto file sheepdog rbd nfs fuse echo "=== Creating image" echo diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026 index b9713eb591..9ecc5880b1 100755 --- a/tests/qemu-iotests/026 +++ b/tests/qemu-iotests/026 @@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Currently only qcow2 supports rebasing _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _default_cache_mode writethrough _supported_cache_modes writethrough none # The refcount table tests expect a certain minimum width for refcount entries diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028 index 864dc4a4e2..57d34aae99 100755 --- a/tests/qemu-iotests/028 +++ b/tests/qemu-iotests/028 @@ -46,7 +46,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Any format supporting backing files except vmdk and qcow which do not support # smaller backing files. _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse _supported_os Linux # Choose a size that is not necessarily a cluster size multiple for image diff --git a/tests/qemu-iotests/031 b/tests/qemu-iotests/031 index 646ecd593f..2bcbc5886e 100755 --- a/tests/qemu-iotests/031 +++ b/tests/qemu-iotests/031 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # We want to test compat=0.10, which does not support external data # files or refcount widths other than 16 _unsupported_imgopts data_file 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034 index ac2d687c71..08f7aea6d5 100755 --- a/tests/qemu-iotests/034 +++ b/tests/qemu-iotests/034 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow qcow2 vmdk qed -_supported_proto file +_supported_proto file fuse _supported_os Linux _unsupported_imgopts "subformat=monolithicFlat" \ "subformat=twoGbMaxExtentFlat" \ diff --git a/tests/qemu-iotests/036 b/tests/qemu-iotests/036 index cf522de7a1..6b82638080 100755 --- a/tests/qemu-iotests/036 +++ b/tests/qemu-iotests/036 @@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # Only qcow2v3 and later supports feature bits; # qcow2.py does not support external data files; # this test requires a cluster size large enough for the feature table diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037 index e1187ac24a..bb893c43dc 100755 --- a/tests/qemu-iotests/037 +++ b/tests/qemu-iotests/037 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow qcow2 vmdk qed -_supported_proto file +_supported_proto file fuse _unsupported_imgopts "subformat=monolithicFlat" \ "subformat=twoGbMaxExtentFlat" \ "subformat=twoGbMaxExtentSparse" \ diff --git a/tests/qemu-iotests/038 b/tests/qemu-iotests/038 index a253231f5b..30f1f73c25 100755 --- a/tests/qemu-iotests/038 +++ b/tests/qemu-iotests/038 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse _supported_os Linux CLUSTER_SIZE=2M diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039 index 42f6503138..ad3867c3fc 100755 --- a/tests/qemu-iotests/039 +++ b/tests/qemu-iotests/039 @@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux _default_cache_mode writethrough _supported_cache_modes writethrough diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046 index 40a9f30087..ed6fae3529 100755 --- a/tests/qemu-iotests/046 +++ b/tests/qemu-iotests/046 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # data_file does not support compressed clusters _unsupported_imgopts data_file diff --git a/tests/qemu-iotests/050 b/tests/qemu-iotests/050 index 4b0a390c43..741bdb610e 100755 --- a/tests/qemu-iotests/050 +++ b/tests/qemu-iotests/050 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse echo echo "== Creating images ==" diff --git a/tests/qemu-iotests/054 b/tests/qemu-iotests/054 index a8905b60d0..40922db2b1 100755 --- a/tests/qemu-iotests/054 +++ b/tests/qemu-iotests/054 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse echo echo "creating too large image (1 EB)" diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 index 94c0d5accc..4b81d1aa51 100755 --- a/tests/qemu-iotests/060 +++ b/tests/qemu-iotests/060 @@ -46,7 +46,7 @@ _filter_io_error() # This tests qcow2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # These tests only work for compat=1.1 images without an external # data file with refcount_bits=16 diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071 index 18fe9054b0..49faae6684 100755 --- a/tests/qemu-iotests/071 +++ b/tests/qemu-iotests/071 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _require_drivers blkdebug blkverify # blkdebug can only inject errors on bs->file, not on the data_file, # so thie test does not work with external data files diff --git a/tests/qemu-iotests/079 b/tests/qemu-iotests/079 index 3642b51feb..0f0d94a2ac 100755 --- a/tests/qemu-iotests/079 +++ b/tests/qemu-iotests/079 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file nfs +_supported_proto file nfs fuse # Some containers (e.g. non-x86 on Travis) do not allow large files _require_large_file 4G diff --git a/tests/qemu-iotests/080 b/tests/qemu-iotests/080 index 7588c63b6c..bda8617c38 100755 --- a/tests/qemu-iotests/080 +++ b/tests/qemu-iotests/080 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # - Internal snapshots are (currently) impossible with refcount_bits=1, # and generally impossible with external data files diff --git a/tests/qemu-iotests/089 b/tests/qemu-iotests/089 index 03a2ccf1e8..f0929b64c0 100755 --- a/tests/qemu-iotests/089 +++ b/tests/qemu-iotests/089 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # Because anything other than 16 would change the output of qemu_io -c info _unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' diff --git a/tests/qemu-iotests/090 b/tests/qemu-iotests/090 index 1246e4f910..87e872ebf4 100755 --- a/tests/qemu-iotests/090 +++ b/tests/qemu-iotests/090 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file nfs +_supported_proto file nfs fuse # External data files do not support compressed clusters _unsupported_imgopts data_file diff --git a/tests/qemu-iotests/091 b/tests/qemu-iotests/091 index 8a4ce5b7e2..8dee168bf6 100755 --- a/tests/qemu-iotests/091 +++ b/tests/qemu-iotests/091 @@ -44,7 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux _supported_cache_modes writethrough none writeback _default_cache_mode none writeback diff --git a/tests/qemu-iotests/095 b/tests/qemu-iotests/095 index 77a5f0f3f5..7604ae6966 100755 --- a/tests/qemu-iotests/095 +++ b/tests/qemu-iotests/095 @@ -44,7 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse size_smaller=5M size_larger=100M diff --git a/tests/qemu-iotests/097 b/tests/qemu-iotests/097 index d910a8b107..1837d4e8e0 100755 --- a/tests/qemu-iotests/097 +++ b/tests/qemu-iotests/097 @@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Any format supporting backing files and bdrv_make_empty _supported_fmt qcow qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098 index f2ccdd7909..a35ce7205e 100755 --- a/tests/qemu-iotests/098 +++ b/tests/qemu-iotests/098 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.pattern _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # The code path we want to test here only works for compat=1.1 images; # blkdebug can only inject errors on bs->file, so external data files # do not work with this test diff --git a/tests/qemu-iotests/102 b/tests/qemu-iotests/102 index b898df436f..2cc3efd1ed 100755 --- a/tests/qemu-iotests/102 +++ b/tests/qemu-iotests/102 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse IMG_SIZE=64K diff --git a/tests/qemu-iotests/103 b/tests/qemu-iotests/103 index 8c1ebe0443..220481db4c 100755 --- a/tests/qemu-iotests/103 +++ b/tests/qemu-iotests/103 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file nfs +_supported_proto file nfs fuse # Internal snapshots are (currently) impossible with refcount_bits=1, # and generally impossible with external data files _unsupported_imgopts 'refcount_bits=1[^0-9]' data_file diff --git a/tests/qemu-iotests/106 b/tests/qemu-iotests/106 index a20659d443..20ad7bd5a2 100755 --- a/tests/qemu-iotests/106 +++ b/tests/qemu-iotests/106 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt raw -_supported_proto file +_supported_proto file fuse _supported_os Linux # in kB diff --git a/tests/qemu-iotests/107 b/tests/qemu-iotests/107 index 268ba27688..d24829ccf9 100755 --- a/tests/qemu-iotests/107 +++ b/tests/qemu-iotests/107 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file nfs +_supported_proto file nfs fuse IMG_SIZE=64K diff --git a/tests/qemu-iotests/108 b/tests/qemu-iotests/108 index 5f7076fba4..ba67748bdf 100755 --- a/tests/qemu-iotests/108 +++ b/tests/qemu-iotests/108 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # This test directly modifies a refblock so it relies on refcount_bits being 16; # and the low-level modification it performs are not tuned for external data diff --git a/tests/qemu-iotests/111 b/tests/qemu-iotests/111 index 3b43d1bd83..bd839a39f4 100755 --- a/tests/qemu-iotests/111 +++ b/tests/qemu-iotests/111 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qed qcow qcow2 vmdk -_supported_proto file +_supported_proto file fuse _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" _make_test_img -b "$TEST_IMG.inexistent" diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112 index 20ff5c224a..6e413f5651 100755 --- a/tests/qemu-iotests/112 +++ b/tests/qemu-iotests/112 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # This test will set refcount_bits on its own which would conflict with the # manual setting; compat will be overridden as well; # and external data files do not work well with our refcount testing diff --git a/tests/qemu-iotests/115 b/tests/qemu-iotests/115 index d254b18342..7f53987d1b 100755 --- a/tests/qemu-iotests/115 +++ b/tests/qemu-iotests/115 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # This test relies on refcounts being 64 bits wide (which does not work with # compat=0.10) _unsupported_imgopts 'refcount_bits=\([^6]\|.\([^4]\|$\)\)' 'compat=0.10' diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117 index f37b34f8b1..9039555ac4 100755 --- a/tests/qemu-iotests/117 +++ b/tests/qemu-iotests/117 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _make_test_img 64k diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120 index 2931a7550f..45c55c1c01 100755 --- a/tests/qemu-iotests/120 +++ b/tests/qemu-iotests/120 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic -_supported_proto file +_supported_proto file fuse _unsupported_fmt luks _require_drivers raw diff --git a/tests/qemu-iotests/121 b/tests/qemu-iotests/121 index 90ea0db737..8357ce089a 100755 --- a/tests/qemu-iotests/121 +++ b/tests/qemu-iotests/121 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # Refcount structures are used much differently with external data # files diff --git a/tests/qemu-iotests/127 b/tests/qemu-iotests/127 index e2ac2f5536..77fdfd0205 100755 --- a/tests/qemu-iotests/127 +++ b/tests/qemu-iotests/127 @@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _require_devices virtio-scsi scsi-hd diff --git a/tests/qemu-iotests/133 b/tests/qemu-iotests/133 index 4070fd9457..bc82d8ebd7 100755 --- a/tests/qemu-iotests/133 +++ b/tests/qemu-iotests/133 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse TEST_IMG="$TEST_IMG.base" _make_test_img 64M _make_test_img -b "$TEST_IMG.base" -F $IMGFMT diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137 index 7ae86892f7..de555a91c9 100755 --- a/tests/qemu-iotests/137 +++ b/tests/qemu-iotests/137 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # We are going to use lazy-refcounts _unsupported_imgopts 'compat=0.10' diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138 index 1d5b0bed6d..e87a64eb89 100755 --- a/tests/qemu-iotests/138 +++ b/tests/qemu-iotests/138 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This tests qcow2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # With an external data file, data clusters are not refcounted # (so qemu-img check would not do much); diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index 309b177e77..ff6b904fa0 100755 --- a/tests/qemu-iotests/140 +++ b/tests/qemu-iotests/140 @@ -44,7 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt generic -_supported_proto file +_supported_proto file fuse _supported_os Linux _make_test_img 64k diff --git a/tests/qemu-iotests/154 b/tests/qemu-iotests/154 index 7f1c0d9bd9..34a1c051b6 100755 --- a/tests/qemu-iotests/154 +++ b/tests/qemu-iotests/154 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux CLUSTER_SIZE=4k diff --git a/tests/qemu-iotests/161 b/tests/qemu-iotests/161 index bbf7dbbc5c..4fb7d0cbf0 100755 --- a/tests/qemu-iotests/161 +++ b/tests/qemu-iotests/161 @@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Any format implementing BlockDriver.bdrv_change_backing_file _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse _supported_os Linux IMG_SIZE=1M diff --git a/tests/qemu-iotests/171 b/tests/qemu-iotests/171 index 341064a1c6..f3582edb10 100755 --- a/tests/qemu-iotests/171 +++ b/tests/qemu-iotests/171 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt raw -_supported_proto file +_supported_proto file fuse _supported_os Linux diff --git a/tests/qemu-iotests/175 b/tests/qemu-iotests/175 index c3c2aed653..21a77a2bf5 100755 --- a/tests/qemu-iotests/175 +++ b/tests/qemu-iotests/175 @@ -71,7 +71,7 @@ EOF . ./common.filter _supported_fmt raw -_supported_proto file +_supported_proto file fuse _supported_os Linux _default_cache_mode none diff --git a/tests/qemu-iotests/176 b/tests/qemu-iotests/176 index 2565ff12ee..5ce3b27069 100755 --- a/tests/qemu-iotests/176 +++ b/tests/qemu-iotests/176 @@ -45,7 +45,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This test is specific to qcow2 _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # Persistent dirty bitmaps require compat=1.1; # Internal snapshots forbid using an external data file diff --git a/tests/qemu-iotests/177 b/tests/qemu-iotests/177 index 5d4a77a6ab..595bfd4236 100755 --- a/tests/qemu-iotests/177 +++ b/tests/qemu-iotests/177 @@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # tests specific to compat=1.1. _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse CLUSTER_SIZE=1M size=128M diff --git a/tests/qemu-iotests/179 b/tests/qemu-iotests/179 index 11a20cb1bf..7ada04c641 100755 --- a/tests/qemu-iotests/179 +++ b/tests/qemu-iotests/179 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # v2 images can't mark clusters as zero diff --git a/tests/qemu-iotests/183 b/tests/qemu-iotests/183 index acdbefa310..d889a3b19c 100755 --- a/tests/qemu-iotests/183 +++ b/tests/qemu-iotests/183 @@ -44,7 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_os Linux FreeBSD NetBSD _supported_fmt qcow2 raw qed quorum -_supported_proto file +_supported_proto file fuse size=64M _make_test_img $size diff --git a/tests/qemu-iotests/186 b/tests/qemu-iotests/186 index 3ea0442d44..0db25b0e68 100755 --- a/tests/qemu-iotests/186 +++ b/tests/qemu-iotests/186 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _require_drivers null-co if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then diff --git a/tests/qemu-iotests/187 b/tests/qemu-iotests/187 index c6e1dc57a0..f262d83e3a 100755 --- a/tests/qemu-iotests/187 +++ b/tests/qemu-iotests/187 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse size=64M _make_test_img $size diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191 index d17462e1e4..95a891350d 100755 --- a/tests/qemu-iotests/191 +++ b/tests/qemu-iotests/191 @@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # An external data file would change the query-named-block-nodes output _unsupported_imgopts data_file diff --git a/tests/qemu-iotests/195 b/tests/qemu-iotests/195 index 2351d55fe1..967af5b7b5 100755 --- a/tests/qemu-iotests/195 +++ b/tests/qemu-iotests/195 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse do_run_qemu() { diff --git a/tests/qemu-iotests/200 b/tests/qemu-iotests/200 index a7aabbd032..046539154f 100755 --- a/tests/qemu-iotests/200 +++ b/tests/qemu-iotests/200 @@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse BACKING_IMG="$TEST_IMG.base" diff --git a/tests/qemu-iotests/204 b/tests/qemu-iotests/204 index 6770fa9b96..536bb8b534 100755 --- a/tests/qemu-iotests/204 +++ b/tests/qemu-iotests/204 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # This test assumes that discard leaves zero clusters; see test 177 for # other tests that also work in older images _unsupported_imgopts 'compat=0.10' diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214 index af677d90b8..75ae7a14b5 100755 --- a/tests/qemu-iotests/214 +++ b/tests/qemu-iotests/214 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # Repairing the corrupted image requires qemu-img check to store a # refcount up to 3, which requires at least two refcount bits. diff --git a/tests/qemu-iotests/217 b/tests/qemu-iotests/217 index d89116ccad..7385342498 100755 --- a/tests/qemu-iotests/217 +++ b/tests/qemu-iotests/217 @@ -36,7 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This test is specific to qcow2 _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # This test needs clusters with at least a refcount of 2 so that # OFLAG_COPIED is not set. refcount_bits=1 is therefore unsupported. diff --git a/tests/qemu-iotests/220 b/tests/qemu-iotests/220 index a9259b7127..9ba3b3fdcb 100755 --- a/tests/qemu-iotests/220 +++ b/tests/qemu-iotests/220 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.pattern _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # To use a different refcount width but 16 bits we need compat=1.1, # and external data files do not support compressed clusters. diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 index 0e9096fec7..ca62b3baa1 100755 --- a/tests/qemu-iotests/221 +++ b/tests/qemu-iotests/221 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt raw -_supported_proto file +_supported_proto file fuse _supported_os Linux _default_cache_mode writeback diff --git a/tests/qemu-iotests/229 b/tests/qemu-iotests/229 index 5f759fa587..273ac2472d 100755 --- a/tests/qemu-iotests/229 +++ b/tests/qemu-iotests/229 @@ -44,7 +44,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Needs backing file and backing format support _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse _supported_os Linux # blkdebug can only inject errors on bs->file, so external data files # do not work with this test diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests/247 index 87e37b39e2..6cf2679750 100755 --- a/tests/qemu-iotests/247 +++ b/tests/qemu-iotests/247 @@ -41,7 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Requires backing files and .bdrv_change_backing_file support _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse size=128M diff --git a/tests/qemu-iotests/249 b/tests/qemu-iotests/249 index a9aa9303eb..29453b8c90 100755 --- a/tests/qemu-iotests/249 +++ b/tests/qemu-iotests/249 @@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Any format implementing BlockDriver.bdrv_change_backing_file _supported_fmt qcow2 qed -_supported_proto file +_supported_proto file fuse _supported_os Linux IMG_SIZE=1M diff --git a/tests/qemu-iotests/250 b/tests/qemu-iotests/250 index 9bb6b94d74..3df275c76b 100755 --- a/tests/qemu-iotests/250 +++ b/tests/qemu-iotests/250 @@ -37,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # This test does not make much sense with external data files _unsupported_imgopts data_file diff --git a/tests/qemu-iotests/252 b/tests/qemu-iotests/252 index 6662f4c9de..1d74afff99 100755 --- a/tests/qemu-iotests/252 +++ b/tests/qemu-iotests/252 @@ -42,7 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # zero cluster support _supported_fmt qcow2 _unsupported_imgopts 'compat=0.10' -_supported_proto file +_supported_proto file fuse _supported_os Linux CLUSTER_SIZE=65536 diff --git a/tests/qemu-iotests/265 b/tests/qemu-iotests/265 index 00f2ec769e..0e800fb524 100755 --- a/tests/qemu-iotests/265 +++ b/tests/qemu-iotests/265 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # qcow2-specific test _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux echo '--- Writing to the image ---' diff --git a/tests/qemu-iotests/268 b/tests/qemu-iotests/268 index 78c3f4db3a..ddf4312284 100755 --- a/tests/qemu-iotests/268 +++ b/tests/qemu-iotests/268 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse echo echo "== Required alignment larger than cluster size ==" diff --git a/tests/qemu-iotests/272 b/tests/qemu-iotests/272 index c2f782d47b..de475bf6f0 100755 --- a/tests/qemu-iotests/272 +++ b/tests/qemu-iotests/272 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This is a qcow2 regression test _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # External data files do not support compression; # We need an exact cluster size (2M) and refcount width (2) so we can diff --git a/tests/qemu-iotests/273 b/tests/qemu-iotests/273 index 41ffbf39c4..79b4ab4b05 100755 --- a/tests/qemu-iotests/273 +++ b/tests/qemu-iotests/273 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This is a qcow2 regression test _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # External data files would add nodes to the block graph, so it would # not match the reference output diff --git a/tests/qemu-iotests/279 b/tests/qemu-iotests/279 index 5515d4ed01..2a6315cf17 100755 --- a/tests/qemu-iotests/279 +++ b/tests/qemu-iotests/279 @@ -36,7 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Backing files are required... _supported_fmt qcow qcow2 vmdk qed -_supported_proto file +_supported_proto file fuse _supported_os Linux _unsupported_imgopts "subformat=monolithicFlat" \ "subformat=twoGbMaxExtentFlat" \ diff --git a/tests/qemu-iotests/286 b/tests/qemu-iotests/286 index f14445ba4a..f64e0eccea 100755 --- a/tests/qemu-iotests/286 +++ b/tests/qemu-iotests/286 @@ -35,7 +35,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # Internal snapshots are (currently) impossible with refcount_bits=1, # and generally impossible with external data files _unsupported_imgopts 'refcount_bits=1[^0-9]' data_file diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287 index 036cc09e82..3bb383fd4b 100755 --- a/tests/qemu-iotests/287 +++ b/tests/qemu-iotests/287 @@ -32,7 +32,7 @@ status=1 # failure is the default! # This tests qocw2-specific low-level functionality _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux _unsupported_imgopts 'compat=0.10' data_file diff --git a/tests/qemu-iotests/289 b/tests/qemu-iotests/289 index 1c11d4030e..fe69bde1eb 100755 --- a/tests/qemu-iotests/289 +++ b/tests/qemu-iotests/289 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.pattern _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse # This is a v3-exclusive test; # As for data_file, error paths often very much depend on whether # there is an external data file or not; so we create one exactly when diff --git a/tests/qemu-iotests/290 b/tests/qemu-iotests/290 index 01ee14dcfb..35c38d4f80 100755 --- a/tests/qemu-iotests/290 +++ b/tests/qemu-iotests/290 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux _unsupported_imgopts 'compat=0.10' refcount_bits data_file diff --git a/tests/qemu-iotests/291 b/tests/qemu-iotests/291 index ecef9eec62..bc70d5e389 100755 --- a/tests/qemu-iotests/291 +++ b/tests/qemu-iotests/291 @@ -36,7 +36,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.nbd _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux _require_command QEMU_NBD # compat=0.10 does not support bitmaps diff --git a/tests/qemu-iotests/292 b/tests/qemu-iotests/292 index 83ab19231d..3ae2772e3b 100755 --- a/tests/qemu-iotests/292 +++ b/tests/qemu-iotests/292 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux # We need qemu-img map to show the file where the data is allocated, # but with an external data file, it will show that instead of the diff --git a/tests/qemu-iotests/293 b/tests/qemu-iotests/293 index f86fe3b413..3363bf07f0 100755 --- a/tests/qemu-iotests/293 +++ b/tests/qemu-iotests/293 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 luks -_supported_proto file #TODO +_supported_proto file fuse #TODO _require_working_luks QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT diff --git a/tests/qemu-iotests/294 b/tests/qemu-iotests/294 index 9c95ed8c9a..87da35db49 100755 --- a/tests/qemu-iotests/294 +++ b/tests/qemu-iotests/294 @@ -34,7 +34,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt luks -_supported_proto file #TODO +_supported_proto file fuse #TODO QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT diff --git a/tests/qemu-iotests/305 b/tests/qemu-iotests/305 index 768818af4a..5a415eb2a4 100755 --- a/tests/qemu-iotests/305 +++ b/tests/qemu-iotests/305 @@ -38,7 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt qcow2 -_supported_proto file +_supported_proto file fuse _supported_os Linux _unsupported_imgopts cluster_size refcount_bits extended_l2 compat=0.10 data_file From patchwork Fri Dec 11 17:08:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14659C433FE for ; Fri, 11 Dec 2020 17:44:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AFFF722D03 for ; Fri, 11 Dec 2020 17:44:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFFF722D03 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40934 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmSc-0005x8-SZ for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:44:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50502) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv6-0004IB-TO for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:29 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46331) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluZ-0002HJ-Nw for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706535; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s8KcERNqQxzFl34w8yhz7YnvE6cyta9/a1xCZWPuoH8=; b=EVDq/Fa5cOdYMg72bpeQPf1mlocj8qas2P1yqCXxG/TO1RsuGo2RnBH7jNVHIDPXKVgxti +ECFeChdVji6yj8P1swPKMejTyfzbXOyXHPqkXH8r5EhsIVhe8zZavLWK2JgquQqAHUlix cFXB96m9SpC5RvMp1JBUxP+lxX7vilE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-431-jpfSp0_BPXm4Ulgal0tR3Q-1; Fri, 11 Dec 2020 12:08:52 -0500 X-MC-Unique: jpfSp0_BPXm4Ulgal0tR3Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C7054180E469; Fri, 11 Dec 2020 17:08:50 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id C64E25D6A8; Fri, 11 Dec 2020 17:08:49 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 24/34] iotests/308: Add test for FUSE exports Date: Fri, 11 Dec 2020 18:08:02 +0100 Message-Id: <20201211170812.228643-25-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz We have good coverage of the normal I/O paths now, but what remains is a test that tests some more special cases: Exporting an image on itself (thus turning a formatted image into a raw one), some error cases, and non-writable and non-growable exports. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Message-Id: <20201027190600.192171-21-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/308 | 339 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/308.out | 97 +++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 437 insertions(+) create mode 100755 tests/qemu-iotests/308 create mode 100644 tests/qemu-iotests/308.out diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 new file mode 100755 index 0000000000..b30f4400f6 --- /dev/null +++ b/tests/qemu-iotests/308 @@ -0,0 +1,339 @@ +#!/usr/bin/env bash +# +# Test FUSE exports (in ways that are not captured by the generic +# tests) +# +# Copyright (C) 2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq=$(basename "$0") +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_qemu + _cleanup_test_img + rmdir "$EXT_MP" 2>/dev/null + rm -f "$EXT_MP" + rm -f "$COPIED_IMG" +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.qemu + +# Generic format, but needs a plain filename +_supported_fmt generic +if [ "$IMGOPTSSYNTAX" = "true" ]; then + _unsupported_fmt $IMGFMT +fi +# We need the image to have exactly the specified size, and VPC does +# not allow that by default +_unsupported_fmt vpc + +_supported_proto file # We create the FUSE export manually +_supported_os Linux # We need /dev/urandom + +# $1: Export ID +# $2: Options (beyond the node-name and ID) +# $3: Expected return value (defaults to 'return') +# $4: Node to export (defaults to 'node-format') +fuse_export_add() +{ + _send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'block-export-add', + 'arguments': { + 'type': 'fuse', + 'id': '$1', + 'node-name': '${4:-node-format}', + $2 + } }" \ + "${3:-return}" \ + | _filter_imgfmt +} + +# $1: Export ID +fuse_export_del() +{ + _send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'block-export-del', + 'arguments': { + 'id': '$1' + } }" \ + 'return' + + _send_qemu_cmd $QEMU_HANDLE \ + '' \ + 'BLOCK_EXPORT_DELETED' +} + +# Return the length of the protocol file +# $1: Protocol node export mount point +# $2: Original file (to compare) +get_proto_len() +{ + len1=$(stat -c '%s' "$1") + len2=$(stat -c '%s' "$2") + + if [ "$len1" != "$len2" ]; then + echo 'ERROR: Length of export and original differ:' >&2 + echo "$len1 != $len2" >&2 + else + echo '(OK: Lengths of export and original are the same)' >&2 + fi + + echo "$len1" +} + +COPIED_IMG="$TEST_IMG.copy" +EXT_MP="$TEST_IMG.fuse" + +echo '=== Set up ===' + +# Create image with random data +_make_test_img 64M +$QEMU_IO -c 'write -s /dev/urandom 0 64M' "$TEST_IMG" | _filter_qemu_io + +_launch_qemu +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'qmp_capabilities'}" \ + 'return' + +# Separate blockdev-add calls for format and protocol so we can remove +# the format layer later on +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'blockdev-add', + 'arguments': { + 'driver': 'file', + 'node-name': 'node-protocol', + 'filename': '$TEST_IMG' + } }" \ + 'return' + +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'blockdev-add', + 'arguments': { + 'driver': '$IMGFMT', + 'node-name': 'node-format', + 'file': 'node-protocol' + } }" \ + 'return' + +echo +echo '=== Mountpoint not present ===' + +rmdir "$EXT_MP" 2>/dev/null +rm -f "$EXT_MP" +output=$(fuse_export_add 'export-err' "'mountpoint': '$EXT_MP'" error) + +if echo "$output" | grep -q "Invalid parameter 'fuse'"; then + _notrun 'No FUSE support' +fi + +echo "$output" + +echo +echo '=== Mountpoint is a directory ===' + +mkdir "$EXT_MP" +fuse_export_add 'export-err' "'mountpoint': '$EXT_MP'" error +rmdir "$EXT_MP" + +echo +echo '=== Mountpoint is a regular file ===' + +touch "$EXT_MP" +fuse_export_add 'export-mp' "'mountpoint': '$EXT_MP'" + +# Check that the export presents the same data as the original image +$QEMU_IMG compare -f raw -F $IMGFMT -U "$EXT_MP" "$TEST_IMG" + +echo +echo '=== Mount over existing file ===' + +# This is the coolest feature of FUSE exports: You can transparently +# make images in any format appear as raw images +fuse_export_add 'export-img' "'mountpoint': '$TEST_IMG'" + +# Accesses both exports at the same time, so we get a concurrency test +$QEMU_IMG compare -f raw -F raw -U "$EXT_MP" "$TEST_IMG" + +# Just to be sure, we later want to compare the data offline. Also, +# this allows us to see that cp works without complaining. +# (This is not a given, because cp will expect a short read at EOF. +# Internally, qemu does not allow short reads, so we have to check +# whether the FUSE export driver lets them work.) +cp "$TEST_IMG" "$COPIED_IMG" + +# $TEST_IMG will be in mode 0400 because it is read-only; we are going +# to write to the copy, so make it writable +chmod 0600 "$COPIED_IMG" + +echo +echo '=== Double export ===' + +# We have already seen that exporting a node twice works fine, but you +# cannot export anything twice on the same mount point. The reason is +# that qemu has to stat the given mount point, and this would have to +# be answered by the same qemu instance if it already has an export +# there. However, it cannot answer the stat because it is itself +# caught up in that same stat. +fuse_export_add 'export-err' "'mountpoint': '$EXT_MP'" error + +echo +echo '=== Remove export ===' + +# Double-check that $EXT_MP appears as a non-empty file (the raw image) +$QEMU_IMG info -f raw "$EXT_MP" | grep 'virtual size' + +fuse_export_del 'export-mp' + +# See that the file appears empty again +$QEMU_IMG info -f raw "$EXT_MP" | grep 'virtual size' + +echo +echo '=== Writable export ===' + +fuse_export_add 'export-mp' "'mountpoint': '$EXT_MP', 'writable': true" + +# Check that writing to the read-only export fails +$QEMU_IO -f raw -c 'write -P 42 1M 64k' "$TEST_IMG" | _filter_qemu_io + +# But here it should work +$QEMU_IO -f raw -c 'write -P 42 1M 64k' "$EXT_MP" | _filter_qemu_io + +# (Adjust the copy, too) +$QEMU_IO -f raw -c 'write -P 42 1M 64k' "$COPIED_IMG" | _filter_qemu_io + +echo +echo '=== Resizing exports ===' + +# Here, we need to export the protocol node -- the format layer may +# not be growable, simply because the format does not support it. + +# Remove all exports and the format node first so permissions will not +# get in the way +fuse_export_del 'export-mp' +fuse_export_del 'export-img' + +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'blockdev-del', + 'arguments': { + 'node-name': 'node-format' + } }" \ + 'return' + +# Now export the protocol node +fuse_export_add \ + 'export-mp' \ + "'mountpoint': '$EXT_MP', 'writable': true" \ + 'return' \ + 'node-protocol' + +echo +echo '--- Try growing non-growable export ---' + +# Get the current size so we can write beyond the EOF +orig_len=$(get_proto_len "$EXT_MP" "$TEST_IMG") +orig_disk_usage=$(stat -c '%b' "$TEST_IMG") + +# Should fail (exports are non-growable by default) +# (Note that qemu-io can never write beyond the EOF, so we have to use +# dd here) +dd if=/dev/zero of="$EXT_MP" bs=1 count=64k seek=$orig_len 2>&1 \ + | _filter_testdir | _filter_imgfmt + +echo +echo '--- Resize export ---' + +# But we can truncate it explicitly; even with fallocate +fallocate -o "$orig_len" -l 64k "$EXT_MP" + +new_len=$(get_proto_len "$EXT_MP" "$TEST_IMG") +if [ "$new_len" != "$((orig_len + 65536))" ]; then + echo 'ERROR: Unexpected post-truncate image size:' + echo "$new_len != $((orig_len + 65536))" +else + echo 'OK: Post-truncate image size is as expected' +fi + +new_disk_usage=$(stat -c '%b' "$TEST_IMG") +if [ "$new_disk_usage" -gt "$orig_disk_usage" ]; then + echo 'OK: Disk usage grew with fallocate' +else + echo 'ERROR: Disk usage did not grow despite fallocate:' + echo "$orig_disk_usage => $new_disk_usage" +fi + +echo +echo '--- Try growing growable export ---' + +# Now export as growable +fuse_export_del 'export-mp' +fuse_export_add \ + 'export-mp' \ + "'mountpoint': '$EXT_MP', 'writable': true, 'growable': true" \ + 'return' \ + 'node-protocol' + +# Now we should be able to write beyond the EOF +dd if=/dev/zero of="$EXT_MP" bs=1 count=64k seek=$new_len 2>&1 \ + | _filter_testdir | _filter_imgfmt + +new_len=$(get_proto_len "$EXT_MP" "$TEST_IMG") +if [ "$new_len" != "$((orig_len + 131072))" ]; then + echo 'ERROR: Unexpected post-grow image size:' + echo "$new_len != $((orig_len + 131072))" +else + echo 'OK: Post-grow image size is as expected' +fi + +echo +echo '--- Shrink export ---' + +# Now go back to the original size +truncate -s "$orig_len" "$EXT_MP" + +new_len=$(get_proto_len "$EXT_MP" "$TEST_IMG") +if [ "$new_len" != "$orig_len" ]; then + echo 'ERROR: Unexpected post-truncate image size:' + echo "$new_len != $orig_len" +else + echo 'OK: Post-truncate image size is as expected' +fi + +echo +echo '=== Tear down ===' + +_send_qemu_cmd $QEMU_HANDLE \ + "{'execute': 'quit'}" \ + 'return' + +wait=yes _cleanup_qemu + +echo +echo '=== Compare copy with original ===' + +$QEMU_IMG compare -f raw -F $IMGFMT "$COPIED_IMG" "$TEST_IMG" + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/308.out b/tests/qemu-iotests/308.out new file mode 100644 index 0000000000..b93aceed2e --- /dev/null +++ b/tests/qemu-iotests/308.out @@ -0,0 +1,97 @@ +QA output created by 308 +=== Set up === +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +wrote 67108864/67108864 bytes at offset 0 +64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +{'execute': 'qmp_capabilities'} +{"return": {}} +{'execute': 'blockdev-add', 'arguments': { 'driver': 'file', 'node-name': 'node-protocol', 'filename': 'TEST_DIR/t.IMGFMT' } } +{"return": {}} +{'execute': 'blockdev-add', 'arguments': { 'driver': 'IMGFMT', 'node-name': 'node-format', 'file': 'node-protocol' } } +{"return": {}} + +=== Mountpoint not present === +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-err', 'node-name': 'node-format', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse' } } +{"error": {"class": "GenericError", "desc": "Failed to stat 'TEST_DIR/t.IMGFMT.fuse': No such file or directory"}} + +=== Mountpoint is a directory === +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-err', 'node-name': 'node-format', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse' } } +{"error": {"class": "GenericError", "desc": "'TEST_DIR/t.IMGFMT.fuse' is not a regular file"}} + +=== Mountpoint is a regular file === +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-mp', 'node-name': 'node-format', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse' } } +{"return": {}} +Images are identical. + +=== Mount over existing file === +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-img', 'node-name': 'node-format', 'mountpoint': 'TEST_DIR/t.IMGFMT' } } +{"return": {}} +Images are identical. + +=== Double export === +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-err', 'node-name': 'node-format', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse' } } +{"error": {"class": "GenericError", "desc": "There already is a FUSE export on 'TEST_DIR/t.IMGFMT.fuse'"}} + +=== Remove export === +virtual size: 64 MiB (67108864 bytes) +{'execute': 'block-export-del', 'arguments': { 'id': 'export-mp' } } +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-mp"}} +virtual size: 0 B (0 bytes) + +=== Writable export === +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-mp', 'node-name': 'node-format', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse', 'writable': true } } +{"return": {}} +write failed: Permission denied +wrote 65536/65536 bytes at offset 1048576 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 65536/65536 bytes at offset 1048576 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Resizing exports === +{'execute': 'block-export-del', 'arguments': { 'id': 'export-mp' } } +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-mp"}} +{'execute': 'block-export-del', 'arguments': { 'id': 'export-img' } } +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-img"}} +{'execute': 'blockdev-del', 'arguments': { 'node-name': 'node-format' } } +{"return": {}} +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-mp', 'node-name': 'node-protocol', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse', 'writable': true } } +{"return": {}} + +--- Try growing non-growable export --- +(OK: Lengths of export and original are the same) +dd: error writing 'TEST_DIR/t.IMGFMT.fuse': Input/output error +1+0 records in +0+0 records out + +--- Resize export --- +(OK: Lengths of export and original are the same) +OK: Post-truncate image size is as expected +OK: Disk usage grew with fallocate + +--- Try growing growable export --- +{'execute': 'block-export-del', 'arguments': { 'id': 'export-mp' } } +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-mp"}} +{'execute': 'block-export-add', 'arguments': { 'type': 'fuse', 'id': 'export-mp', 'node-name': 'node-protocol', 'mountpoint': 'TEST_DIR/t.IMGFMT.fuse', 'writable': true, 'growable': true } } +{"return": {}} +65536+0 records in +65536+0 records out +(OK: Lengths of export and original are the same) +OK: Post-grow image size is as expected + +--- Shrink export --- +(OK: Lengths of export and original are the same) +OK: Post-truncate image size is as expected + +=== Tear down === +{'execute': 'quit'} +{"return": {}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-mp"}} + +=== Compare copy with original === +Images are identical. +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 2960dff728..9a8394b4cd 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -315,4 +315,5 @@ 304 rw quick 305 rw quick 307 rw quick export +308 rw 309 rw auto quick From patchwork Fri Dec 11 17:08:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1FDCC433FE for ; Fri, 11 Dec 2020 17:34:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1A3A923DC2 for ; Fri, 11 Dec 2020 17:34:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1A3A923DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41932 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmJD-0002bW-W6 for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:34:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlv7-0004IQ-S1 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:40600) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlua-0002HQ-4X for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706535; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l8azbxIqqkGL8GaTE59z7gRuMvfFxzCwcURPOIxUO40=; b=De0ffz70DaM5nBYkI+m4XCrRw4xrYd0wcoSGUMSrqrRKauggXOxaaNOoVnqfGrd2ICCqUt C4sXpUijrQl6TB/Dimrg2QdBB44fN1C0PKBirx3eQSdVoGo/LfLyfbpmCb/jegksdXxTHT PQRpqEDo88Ir6qEiMw9na6Hca/yQQnA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-178-0sP3k8JlMbG7-jVzEkscoA-1; Fri, 11 Dec 2020 12:08:53 -0500 X-MC-Unique: 0sP3k8JlMbG7-jVzEkscoA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1FB96107ACE3; Fri, 11 Dec 2020 17:08:52 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1EDF75D6A8; Fri, 11 Dec 2020 17:08:50 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 25/34] file-posix: check the use_lock before setting the file lock Date: Fri, 11 Dec 2020 18:08:03 +0100 Message-Id: <20201211170812.228643-26-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Li Feng The scenario is that when accessing a volume on an NFS filesystem without supporting the file lock, Qemu will complain "Failed to lock byte 100", even when setting the file.locking = off. We should do file lock related operations only when the file.locking is enabled, otherwise, the syscall of 'fcntl' will return non-zero. Signed-off-by: Li Feng Message-Id: <1607341446-85506-1-git-send-email-fengli@smartx.com> Signed-off-by: Kevin Wolf --- block/file-posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/file-posix.c b/block/file-posix.c index d5fd1dbcd2..806764f7e3 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -3104,7 +3104,7 @@ static int raw_check_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared, } /* Copy locks to the new fd */ - if (s->perm_change_fd) { + if (s->perm_change_fd && s->use_lock) { ret = raw_apply_lock_bytes(NULL, s->perm_change_fd, perm, ~shared, false, errp); if (ret < 0) { From patchwork Fri Dec 11 17:08:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D04A6C4361B for ; Fri, 11 Dec 2020 17:38:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7AF8023EF1 for ; Fri, 11 Dec 2020 17:38:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AF8023EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51224 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmMy-0006h9-B9 for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:38:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50574) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvA-0004JJ-8t for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:21089) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluc-0002IZ-Fo for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706536; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H1dq3GzuE742PJCu9bz/6je8AblWyWKe8uuIfOreqjs=; b=I0b4ub+W3VJjYtdVMhpKA2tT/vUMNOvZE1i8ablmk+iYFWdPxw7e1KdMQnZoG/Xow0Qjgk YokRVZnIKK+NeY9Vgt4zvrHTsFYdSqSf1rlmkxuzP7F3F5DHAhSt9hFAGeyHWKh04zkLSP fE4Q3zOvafqo/l1T1Qc5MtvT3eOuTzo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-296-I8bEIVzZPBi_HC4VVM-ttw-1; Fri, 11 Dec 2020 12:08:54 -0500 X-MC-Unique: I8bEIVzZPBi_HC4VVM-ttw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6E4C4B8100; Fri, 11 Dec 2020 17:08:53 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B3115D6A8; Fri, 11 Dec 2020 17:08:52 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 26/34] iotests/221: Discard image before qemu-img map Date: Fri, 11 Dec 2020 18:08:04 +0100 Message-Id: <20201211170812.228643-27-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Max Reitz See the new comment for why this should be done. I do not have a reproducer on master, but when using FUSE block exports, this test breaks depending on the underlying filesystem (for me, it works on tmpfs, but fails on xfs, because the block allocated by file-posix has 16 kB there instead of 4 kB). Suggested-by: Kevin Wolf Signed-off-by: Max Reitz Message-Id: <20201207152245.66987-1-mreitz@redhat.com> Signed-off-by: Kevin Wolf --- tests/qemu-iotests/221 | 7 +++++++ tests/qemu-iotests/221.out | 14 ++++++-------- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 index ca62b3baa1..7e6086b205 100755 --- a/tests/qemu-iotests/221 +++ b/tests/qemu-iotests/221 @@ -46,6 +46,13 @@ echo "=== Check mapping of unaligned raw image ===" echo _make_test_img 65537 # qemu-img create rounds size up + +# file-posix allocates the first block of any images when it is created; +# the size of this block depends on the host page size and the file +# system block size, none of which are constant. Discard the whole +# image so we will not see this allocation in qemu-img map's output. +$QEMU_IO -c 'discard 0 65537' "$TEST_IMG" | _filter_qemu_io + $QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map truncate --size=65537 "$TEST_IMG" # so we resize it and check again diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out index dca024a0c3..93846c7dab 100644 --- a/tests/qemu-iotests/221.out +++ b/tests/qemu-iotests/221.out @@ -3,18 +3,16 @@ QA output created by 221 === Check mapping of unaligned raw image === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65537 -[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] -[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 61952, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +discard 65537/65537 bytes at offset 0 +64.001 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 66048, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] wrote 1/1 bytes at offset 65536 1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, +[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] -[{ "start": 0, "length": 4096, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, -{ "start": 4096, "length": 61440, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, +[{ "start": 0, "length": 65536, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, { "start": 65536, "length": 1, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, { "start": 65537, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] *** done From patchwork Fri Dec 11 17:08:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3CDFC433FE for ; Fri, 11 Dec 2020 17:41:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8D6AB23EF1 for ; Fri, 11 Dec 2020 17:41:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D6AB23EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59722 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmQB-00024l-Dr for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:41:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50580) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvB-0004K1-PX for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:22112) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluc-0002Is-PS for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706538; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K+5lURr3H1iosGWhxxAwmPhxO97vAel5aKpCHk+8XmM=; b=BwlKbsPHjVBMU+QTcrz5q1vEafkNmlQY8suW0G6KG+JyCuqthTZlwiXAvtXDXV7bdFQzhX FNAoFi2ylorvv3Ligg866KqrNzqKocSqwnp0bLcEoYifPUtZFwVWTeLxe/Tk85U3gi3lEh NntrB0F9blQzSlgvwZXJv9iPQ0vD1hs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-297-cGCguWtBNXadWnOhByVNDg-1; Fri, 11 Dec 2020 12:08:55 -0500 X-MC-Unique: cGCguWtBNXadWnOhByVNDg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B9482800D55; Fri, 11 Dec 2020 17:08:54 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id B911E5D6A8; Fri, 11 Dec 2020 17:08:53 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 27/34] can-host: Fix crash when 'canbus' property is not set Date: Fri, 11 Dec 2020 18:08:05 +0100 Message-Id: <20201211170812.228643-28-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Providing the 'if' property, but not 'canbus' segfaults like this: #0 0x0000555555b0f14d in can_bus_insert_client (bus=0x0, client=0x555556aa9af0) at ../net/can/can_core.c:88 #1 0x00005555559c3803 in can_host_connect (ch=0x555556aa9ac0, errp=0x7fffffffd568) at ../net/can/can_host.c:62 #2 0x00005555559c386a in can_host_complete (uc=0x555556aa9ac0, errp=0x7fffffffd568) at ../net/can/can_host.c:72 #3 0x0000555555d52de9 in user_creatable_complete (uc=0x555556aa9ac0, errp=0x7fffffffd5c8) at ../qom/object_interfaces.c:23 Add the missing NULL check. Signed-off-by: Kevin Wolf Message-Id: <20201130105615.21799-5-kwolf@redhat.com> Signed-off-by: Kevin Wolf --- net/can/can_host.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/net/can/can_host.c b/net/can/can_host.c index be4547d913..ba7f163d0a 100644 --- a/net/can/can_host.c +++ b/net/can/can_host.c @@ -53,6 +53,11 @@ static void can_host_connect(CanHostState *ch, Error **errp) CanHostClass *chc = CAN_HOST_GET_CLASS(ch); Error *local_err = NULL; + if (ch->bus == NULL) { + error_setg(errp, "'canbus' property not set"); + return; + } + chc->connect(ch, &local_err); if (local_err) { error_propagate(errp, local_err); From patchwork Fri Dec 11 17:08:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0219BC433FE for ; Fri, 11 Dec 2020 17:49:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7D60022DFB for ; Fri, 11 Dec 2020 17:49:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D60022DFB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmXO-0001jx-Ek for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:49:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50688) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvP-0004Yu-F7 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47252) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlue-0002Jv-Op for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:45 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Si/VfFgD2E+KfaE6Uukp5i3Wn9g7s3KAAx8h/GEYWng=; b=gw6PDZ/dlJD6UcmIhZ/9ehHZvJAMHfKO15oVYQMQ8ShKBFNJrziJFHNF5lHpmXX95ktstx 0kka21uf4FIxgOJ6o+KFZtvfK93Ycf4sGumGBXn8AwpW7ebiCo2ZI2NY10p143al6GzuOv CBLeAFeYcI6mHWbmL4rD+5zRjhzc6n8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-457-Bj9Fx_eTMjmrFZ9s48aQGw-1; Fri, 11 Dec 2020 12:08:57 -0500 X-MC-Unique: Bj9Fx_eTMjmrFZ9s48aQGw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4057F107ACE6; Fri, 11 Dec 2020 17:08:56 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 111D15D6A8; Fri, 11 Dec 2020 17:08:54 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 28/34] block/file-posix: fix workaround in raw_do_pwrite_zeroes() Date: Fri, 11 Dec 2020 18:08:06 +0100 Message-Id: <20201211170812.228643-29-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We should not set overlap_bytes: 1. Don't worry: it is calculated by bdrv_mark_request_serialising() and will be equal to or greater than bytes anyway. 2. If the request was already aligned up to some greater alignment, than we may break things: we reduce overlap_bytes, and further bdrv_mark_request_serialising() may not help, as it will not restore old bigger alignment. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20201203222713.13507-2-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf --- block/file-posix.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/file-posix.c b/block/file-posix.c index 806764f7e3..9bee3d88d0 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -2952,7 +2952,6 @@ raw_do_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, end = INT64_MAX & -(uint64_t)bs->bl.request_alignment; req->bytes = end - req->offset; - req->overlap_bytes = req->bytes; bdrv_mark_request_serialising(req, bs->bl.request_alignment); } From patchwork Fri Dec 11 17:08:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21402C433FE for ; Fri, 11 Dec 2020 17:52:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B8B4223F37 for ; Fri, 11 Dec 2020 17:52:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8B4223F37 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59164 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmaT-0005Tu-LC for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:52:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvR-0004by-FF for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:49 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27978) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluh-0002Ke-Rh for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ggTN6lCQbe/i02k8oEth+jaxoBl+MComYxqCc7uAS60=; b=aF5HpSeT/VAAeWuteClGStvSkJs+3j2eFz+XMn8723BZy3LlAbKWsnXULDWtDtMYwppv2B M7enL+m1Cc1yGLOid13N4O8IsUkC5XHIfpm3EJI/bSgfC9edn45WqM2JmhvtIAgvRYCuUi Wap2j8KrFsL4NlvcQDckTPQ7n6vhlco= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-FmurCA-DNMWBEGJdPJiK5A-1; Fri, 11 Dec 2020 12:08:59 -0500 X-MC-Unique: FmurCA-DNMWBEGJdPJiK5A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 613C9107ACE3; Fri, 11 Dec 2020 17:08:57 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5ECE05D6A8; Fri, 11 Dec 2020 17:08:56 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 29/34] block/io: bdrv_refresh_limits(): use ERRP_GUARD Date: Fri, 11 Dec 2020 18:08:07 +0100 Message-Id: <20201211170812.228643-30-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy This simplifies following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20201203222713.13507-3-vsementsov@virtuozzo.com> Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf --- block/io.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block/io.c index ec5e152bb7..3e91074c9f 100644 --- a/block/io.c +++ b/block/io.c @@ -135,10 +135,10 @@ static void bdrv_merge_limits(BlockLimits *dst, const BlockLimits *src) void bdrv_refresh_limits(BlockDriverState *bs, Error **errp) { + ERRP_GUARD(); BlockDriver *drv = bs->drv; BdrvChild *c; bool have_limits; - Error *local_err = NULL; memset(&bs->bl, 0, sizeof(bs->bl)); @@ -156,9 +156,8 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp) QLIST_FOREACH(c, &bs->children, next) { if (c->role & (BDRV_CHILD_DATA | BDRV_CHILD_FILTERED | BDRV_CHILD_COW)) { - bdrv_refresh_limits(c->bs, &local_err); - if (local_err) { - error_propagate(errp, local_err); + bdrv_refresh_limits(c->bs, errp); + if (*errp) { return; } bdrv_merge_limits(&bs->bl, &c->bs->bl); From patchwork Fri Dec 11 17:08:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEA23C433FE for ; Fri, 11 Dec 2020 17:45:40 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51F6722D03 for ; Fri, 11 Dec 2020 17:45:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51F6722D03 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43422 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmU7-0006zr-4B for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:45:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50742) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvV-0004h1-7N for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28629) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlum-0002Lo-8Y for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cnCn3WK1WUMUBMIYMdE2Z3dfBeiDvXeSor63rfgxMPg=; b=cNpAgL1IEXEirKAw8z17iTMADwKW05k/FVWSGgjOKMN6RP8TodbloEJlwRlmfvlbHNILAB juwpTpNPS3h6nj0wOw1rY41veDHehnn5T9H0d3F9iDYBCWhfOiFZP4K61R7KGN0k7+de5m lsJtxwZAs+QbFn4TFfAbm1eX1HpntCg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-281-bz8SrvjcMaqPZAumFwshxQ-1; Fri, 11 Dec 2020 12:08:59 -0500 X-MC-Unique: bz8SrvjcMaqPZAumFwshxQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ABCA659; Fri, 11 Dec 2020 17:08:58 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB7745D6A8; Fri, 11 Dec 2020 17:08:57 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 30/34] block/io: bdrv_check_byte_request(): drop bdrv_is_inserted() Date: Fri, 11 Dec 2020 18:08:08 +0100 Message-Id: <20201211170812.228643-31-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Move bdrv_is_inserted() calls into callers. We are going to make bdrv_check_byte_request() a clean thing. bdrv_is_inserted() is not about checking the request, it's about checking the bs. So, it should be separate. With this patch we probably change error path for some failure scenarios. But depending on the fact that querying too big request on empty cdrom (or corrupted qcow2 node with no drv) will result in EIO and not ENOMEDIUM would be very strange. More over, we are going to move to 64bit requests, so larger requests will be allowed anyway. More over, keeping in mind that cdrom is the only driver that has .bdrv_is_inserted() handler it's strange that we should care so much about it in generic block layer, intuitively we should just do read and write, and cdrom driver should return correct errors if it is not inserted. But it's a work for another series. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20201203222713.13507-4-vsementsov@virtuozzo.com> Reviewed-by: Alberto Garcia Signed-off-by: Kevin Wolf --- block/io.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/block/io.c b/block/io.c index 3e91074c9f..ef75a5abb4 100644 --- a/block/io.c +++ b/block/io.c @@ -884,17 +884,12 @@ static bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self return waited; } -static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset, - size_t size) +static int bdrv_check_byte_request(int64_t offset, size_t size) { if (size > BDRV_REQUEST_MAX_BYTES) { return -EIO; } - if (!bdrv_is_inserted(bs)) { - return -ENOMEDIUM; - } - if (offset < 0) { return -EIO; } @@ -1642,7 +1637,11 @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child, trace_bdrv_co_preadv(bs, offset, bytes, flags); - ret = bdrv_check_byte_request(bs, offset, bytes); + if (!bdrv_is_inserted(bs)) { + return -ENOMEDIUM; + } + + ret = bdrv_check_byte_request(offset, bytes); if (ret < 0) { return ret; } @@ -2054,11 +2053,11 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child, trace_bdrv_co_pwritev(child->bs, offset, bytes, flags); - if (!bs->drv) { + if (!bdrv_is_inserted(bs)) { return -ENOMEDIUM; } - ret = bdrv_check_byte_request(bs, offset, bytes); + ret = bdrv_check_byte_request(offset, bytes); if (ret < 0) { return ret; } @@ -3045,10 +3044,10 @@ static int coroutine_fn bdrv_co_copy_range_internal( assert(!(read_flags & BDRV_REQ_NO_FALLBACK)); assert(!(write_flags & BDRV_REQ_NO_FALLBACK)); - if (!dst || !dst->bs) { + if (!dst || !dst->bs || !bdrv_is_inserted(dst->bs)) { return -ENOMEDIUM; } - ret = bdrv_check_byte_request(dst->bs, dst_offset, bytes); + ret = bdrv_check_byte_request(dst_offset, bytes); if (ret) { return ret; } @@ -3056,10 +3055,10 @@ static int coroutine_fn bdrv_co_copy_range_internal( return bdrv_co_pwrite_zeroes(dst, dst_offset, bytes, write_flags); } - if (!src || !src->bs) { + if (!src || !src->bs || !bdrv_is_inserted(src->bs)) { return -ENOMEDIUM; } - ret = bdrv_check_byte_request(src->bs, src_offset, bytes); + ret = bdrv_check_byte_request(src_offset, bytes); if (ret) { return ret; } From patchwork Fri Dec 11 17:08:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2430EC433FE for ; Fri, 11 Dec 2020 17:44:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9991222D06 for ; Fri, 11 Dec 2020 17:44:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9991222D06 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40316 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmSW-0005hr-NV for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:44:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvW-0004jS-Se for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:54 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:37725) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlum-0002M0-8F for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QUxoKtXvtL7mzdLc6eRUJf2djQkz67Cf2DjBCjlZ+wg=; b=A+G0+xMfYEesyMRfg3TEXIhtvrNA5BWpCWGPIDYSm1ufICRqYMdPIejTaMt0jwyqtEEwS5 pUj7aSTUVEYJiWNDF4fdW8VTL22+aVvo+vkUI43bHJvdwsSgh27aSSBTkVujWF+YG9onY4 C37+OjPAetZYZ/O7cauD7SLK+Hrns2Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-350-_CsMAFlsO-G3AFVN9x0UAA-1; Fri, 11 Dec 2020 12:09:01 -0500 X-MC-Unique: _CsMAFlsO-G3AFVN9x0UAA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 23F02800D55; Fri, 11 Dec 2020 17:09:00 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 039F85D6A8; Fri, 11 Dec 2020 17:08:58 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 31/34] block: introduce BDRV_MAX_LENGTH Date: Fri, 11 Dec 2020 18:08:09 +0100 Message-Id: <20201211170812.228643-32-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to modify block layer to work with 64bit requests. And first step is moving to int64_t type for both offset and bytes arguments in all block request related functions. It's mostly safe (when widening signed or unsigned int to int64_t), but switching from uint64_t is questionable. So, let's first establish the set of requests we want to work with. First signed int64_t should be enough, as off_t is signed anyway. Then, obviously offset + bytes should not overflow. And most interesting: (offset + bytes) being aligned up should not overflow as well. Aligned to what alignment? First thing that comes in mind is bs->bl.request_alignment, as we align up request to this alignment. But there is another thing: look at bdrv_mark_request_serialising(). It aligns request up to some given alignment. And this parameter may be bdrv_get_cluster_size(), which is often a lot greater than bs->bl.request_alignment. Note also, that bdrv_mark_request_serialising() uses signed int64_t for calculations. So, actually, we already depend on some restrictions. Happily, bdrv_get_cluster_size() returns int and bs->bl.request_alignment has 32bit unsigned type, but defined to be a power of 2 less than INT_MAX. So, we may establish, that INT_MAX is absolute maximum for any kind of alignment that may occur with the request. Note, that bdrv_get_cluster_size() is not documented to return power of 2, still bdrv_mark_request_serialising() behaves like it is. Also, backup uses bdi.cluster_size and is not prepared to it not being power of 2. So, let's establish that Qemu supports only power-of-2 clusters and alignments. So, alignment can't be greater than 2^30. Finally to be safe with calculations, to not calculate different maximums for different nodes (depending on cluster size and request_alignment), let's simply set QEMU_ALIGN_DOWN(INT64_MAX, 2^30) as absolute maximum bytes length for Qemu. Actually, it's not much less than INT64_MAX. OK, then, let's apply it to block/io. Let's consider all block/io entry points of offset/bytes: 4 bytes/offset interface functions: bdrv_co_preadv_part(), bdrv_co_pwritev_part(), bdrv_co_copy_range_internal() and bdrv_co_pdiscard() and we check them all with bdrv_check_request(). We also have one entry point with only offset: bdrv_co_truncate(). Check the offset. And one public structure: BdrvTrackedRequest. Happily, it has only three external users: file-posix.c: adopted by this patch write-threshold.c: only read fields test-write-threshold.c: sets obviously small constant values Better is to make the structure private and add corresponding interfaces.. Still it's not obvious what kind of interface is needed for file-posix.c. Let's keep it public but add corresponding assertions. After this patch we'll convert functions in block/io.c to int64_t bytes and offset parameters. We can assume that offset/bytes pair always satisfy new restrictions, and make corresponding assertions where needed. If we reach some offset/bytes point in block/io.c missing bdrv_check_request() it is considered a bug. As well, if block/io.c modifies a offset/bytes request, expanding it more then aligning up to request_alignment, it's a bug too. For all io requests except for discard we keep for now old restriction of 32bit request length. iotest 206 output error message changed, as now test disk size is larger than new limit. Add one more test case with new maximum disk size to cover too-big-L1 case. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20201203222713.13507-5-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf --- include/block/block.h | 10 +++++++ include/block/block_int.h | 8 ++++++ block.c | 17 +++++++++++- block/file-posix.c | 6 ++--- block/io.c | 51 +++++++++++++++++++++++++++++------- tests/test-write-threshold.c | 4 +++ tests/qemu-iotests/206 | 2 +- tests/qemu-iotests/206.out | 6 +++++ 8 files changed, 90 insertions(+), 14 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index c9d7c58765..5b81e33e94 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -142,6 +142,16 @@ typedef struct HDGeometry { INT_MAX >> BDRV_SECTOR_BITS) #define BDRV_REQUEST_MAX_BYTES (BDRV_REQUEST_MAX_SECTORS << BDRV_SECTOR_BITS) +/* + * We want allow aligning requests and disk length up to any 32bit alignment + * and don't afraid of overflow. + * To achieve it, and in the same time use some pretty number as maximum disk + * size, let's define maximum "length" (a limit for any offset/bytes request and + * for disk size) to be the greatest power of 2 less than INT64_MAX. + */ +#define BDRV_MAX_ALIGNMENT (1L << 30) +#define BDRV_MAX_LENGTH (QEMU_ALIGN_DOWN(INT64_MAX, BDRV_MAX_ALIGNMENT)) + /* * Allocation status flags for bdrv_block_status() and friends. * diff --git a/include/block/block_int.h b/include/block/block_int.h index 95d9333be1..1eeafc118c 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -70,6 +70,12 @@ enum BdrvTrackedRequestType { BDRV_TRACKED_TRUNCATE, }; +/* + * That is not quite good that BdrvTrackedRequest structure is public, + * as block/io.c is very careful about incoming offset/bytes being + * correct. Be sure to assert bdrv_check_request() succeeded after any + * modification of BdrvTrackedRequest object out of block/io.c + */ typedef struct BdrvTrackedRequest { BlockDriverState *bs; int64_t offset; @@ -87,6 +93,8 @@ typedef struct BdrvTrackedRequest { struct BdrvTrackedRequest *waiting_for; } BdrvTrackedRequest; +int bdrv_check_request(int64_t offset, int64_t bytes); + struct BlockDriver { const char *format_name; int instance_size; diff --git a/block.c b/block.c index eb16fb48c6..8f177504d4 100644 --- a/block.c +++ b/block.c @@ -962,6 +962,11 @@ int refresh_total_sectors(BlockDriverState *bs, int64_t hint) } bs->total_sectors = hint; + + if (bs->total_sectors * BDRV_SECTOR_SIZE > BDRV_MAX_LENGTH) { + return -EFBIG; + } + return 0; } @@ -5535,6 +5540,7 @@ void bdrv_get_backing_filename(BlockDriverState *bs, int bdrv_get_info(BlockDriverState *bs, BlockDriverInfo *bdi) { + int ret; BlockDriver *drv = bs->drv; /* if bs->drv == NULL, bs is closed, so there's nothing to do here */ if (!drv) { @@ -5548,7 +5554,16 @@ int bdrv_get_info(BlockDriverState *bs, BlockDriverInfo *bdi) return -ENOTSUP; } memset(bdi, 0, sizeof(*bdi)); - return drv->bdrv_get_info(bs, bdi); + ret = drv->bdrv_get_info(bs, bdi); + if (ret < 0) { + return ret; + } + + if (bdi->cluster_size > BDRV_MAX_ALIGNMENT) { + return -EINVAL; + } + + return 0; } ImageInfoSpecific *bdrv_get_specific_info(BlockDriverState *bs, diff --git a/block/file-posix.c b/block/file-posix.c index 9bee3d88d0..83e2cc5530 100644 --- a/block/file-posix.c +++ b/block/file-posix.c @@ -2926,7 +2926,6 @@ raw_do_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, #ifdef CONFIG_FALLOCATE if (offset + bytes > bs->total_sectors * BDRV_SECTOR_SIZE) { BdrvTrackedRequest *req; - uint64_t end; /* * This is a workaround for a bug in the Linux XFS driver, @@ -2950,8 +2949,9 @@ raw_do_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, assert(req->offset <= offset); assert(req->offset + req->bytes >= offset + bytes); - end = INT64_MAX & -(uint64_t)bs->bl.request_alignment; - req->bytes = end - req->offset; + req->bytes = BDRV_MAX_LENGTH - req->offset; + + assert(bdrv_check_request(req->offset, req->bytes) == 0); bdrv_mark_request_serialising(req, bs->bl.request_alignment); } diff --git a/block/io.c b/block/io.c index ef75a5abb4..6343d85476 100644 --- a/block/io.c +++ b/block/io.c @@ -176,6 +176,13 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp) /* Then let the driver override it */ if (drv->bdrv_refresh_limits) { drv->bdrv_refresh_limits(bs, errp); + if (*errp) { + return; + } + } + + if (bs->bl.request_alignment > BDRV_MAX_ALIGNMENT) { + error_setg(errp, "Driver requires too large request alignment"); } } @@ -884,13 +891,31 @@ static bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self return waited; } -static int bdrv_check_byte_request(int64_t offset, size_t size) +int bdrv_check_request(int64_t offset, int64_t bytes) { - if (size > BDRV_REQUEST_MAX_BYTES) { + if (offset < 0 || bytes < 0) { return -EIO; } - if (offset < 0) { + if (bytes > BDRV_MAX_LENGTH) { + return -EIO; + } + + if (offset > BDRV_MAX_LENGTH - bytes) { + return -EIO; + } + + return 0; +} + +static int bdrv_check_request32(int64_t offset, int64_t bytes) +{ + int ret = bdrv_check_request(offset, bytes); + if (ret < 0) { + return ret; + } + + if (bytes > BDRV_REQUEST_MAX_BYTES) { return -EIO; } @@ -1641,7 +1666,7 @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child, return -ENOMEDIUM; } - ret = bdrv_check_byte_request(offset, bytes); + ret = bdrv_check_request32(offset, bytes); if (ret < 0) { return ret; } @@ -2057,7 +2082,7 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child, return -ENOMEDIUM; } - ret = bdrv_check_byte_request(offset, bytes); + ret = bdrv_check_request32(offset, bytes); if (ret < 0) { return ret; } @@ -2787,8 +2812,9 @@ int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, return -EPERM; } - if (offset < 0 || bytes < 0 || bytes > INT64_MAX - offset) { - return -EIO; + ret = bdrv_check_request(offset, bytes); + if (ret < 0) { + return ret; } /* Do nothing if disabled. */ @@ -3047,7 +3073,7 @@ static int coroutine_fn bdrv_co_copy_range_internal( if (!dst || !dst->bs || !bdrv_is_inserted(dst->bs)) { return -ENOMEDIUM; } - ret = bdrv_check_byte_request(dst_offset, bytes); + ret = bdrv_check_request32(dst_offset, bytes); if (ret) { return ret; } @@ -3058,7 +3084,7 @@ static int coroutine_fn bdrv_co_copy_range_internal( if (!src || !src->bs || !bdrv_is_inserted(src->bs)) { return -ENOMEDIUM; } - ret = bdrv_check_byte_request(src_offset, bytes); + ret = bdrv_check_request32(src_offset, bytes); if (ret) { return ret; } @@ -3188,6 +3214,13 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, return -EINVAL; } + ret = bdrv_check_request(offset, 0); + if (ret < 0) { + error_setg(errp, "Required too big image size, it must be not greater " + "than %" PRId64, BDRV_MAX_LENGTH); + return ret; + } + old_size = bdrv_getlength(bs); if (old_size < 0) { error_setg_errno(errp, -old_size, "Failed to get old image size"); diff --git a/tests/test-write-threshold.c b/tests/test-write-threshold.c index 97ca12f710..4cf032652d 100644 --- a/tests/test-write-threshold.c +++ b/tests/test-write-threshold.c @@ -64,6 +64,8 @@ static void test_threshold_not_trigger(void) req.offset = 1024; req.bytes = 1024; + assert(bdrv_check_request(req.offset, req.bytes) == 0); + bdrv_write_threshold_set(&bs, threshold); amount = bdrv_write_threshold_exceeded(&bs, &req); g_assert_cmpuint(amount, ==, 0); @@ -82,6 +84,8 @@ static void test_threshold_trigger(void) req.offset = (4 * 1024 * 1024) - 1024; req.bytes = 2 * 1024; + assert(bdrv_check_request(req.offset, req.bytes) == 0); + bdrv_write_threshold_set(&bs, threshold); amount = bdrv_write_threshold_exceeded(&bs, &req); g_assert_cmpuint(amount, >=, 1024); diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206 index 0a3ee5ef00..d12d7cb566 100755 --- a/tests/qemu-iotests/206 +++ b/tests/qemu-iotests/206 @@ -202,7 +202,7 @@ with iotests.FilePath('t.qcow2') as disk_path, \ vm.launch() for size in [ 1234, 18446744073709551104, 9223372036854775808, - 9223372036854775296 ]: + 9223372036854775296, 9223372035781033984 ]: vm.blockdev_create({ 'driver': imgfmt, 'file': 'node0', 'size': size }) diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out index a100849fcb..e8a36de00b 100644 --- a/tests/qemu-iotests/206.out +++ b/tests/qemu-iotests/206.out @@ -180,6 +180,12 @@ Job failed: Could not resize image: Image size cannot be negative {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "qcow2", "file": "node0", "size": 9223372036854775296}}} {"return": {}} +Job failed: Could not resize image: Required too big image size, it must be not greater than 9223372035781033984 +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} + +{"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "qcow2", "file": "node0", "size": 9223372035781033984}}} +{"return": {}} Job failed: Could not resize image: Failed to grow the L1 table: File too large {"execute": "job-dismiss", "arguments": {"id": "job0"}} {"return": {}} From patchwork Fri Dec 11 17:08:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E054AC4361B for ; Fri, 11 Dec 2020 17:54:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5A51723D3C for ; Fri, 11 Dec 2020 17:54:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A51723D3C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39862 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmcw-0000lL-59 for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:54:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50796) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvX-0004kK-74 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37172) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluo-0002MK-53 for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706547; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NDh2ZCpLFeJcXJnq+h/IPAjdL93M8YBHVz8/G+2chms=; b=G0tUZMs37QpsJGnmEx9aUDjNwVwgbeJBlrndYEEXUVRt2nGbm95w9FmYrts/yoiP8avsBd ROT7NCWZz/KwnvlrCVCitkR2WQZzvlkTK0lk8i67j17E8m2d091cUUP8YU2Jg5GVeiEP92 qXHteU60l363klOwzpPTK70QfaVyxdI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-569-byBHYT6NPryhaGFrnJYRHA-1; Fri, 11 Dec 2020 12:09:03 -0500 X-MC-Unique: byBHYT6NPryhaGFrnJYRHA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 719B859; Fri, 11 Dec 2020 17:09:01 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 70ACA5D6A8; Fri, 11 Dec 2020 17:09:00 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 32/34] block: Simplify qmp_block_resize() error paths Date: Fri, 11 Dec 2020 18:08:10 +0100 Message-Id: <20201211170812.228643-33-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The only thing that happens after the 'out:' label is blk_unref(blk). However, blk = NULL in all of the error cases, so instead of jumping to 'out:', we can just return directly. Cc: qemu-stable@nongnu.org Signed-off-by: Kevin Wolf Message-Id: <20201203172311.68232-2-kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf --- blockdev.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/blockdev.c b/blockdev.c index 6c7be7c522..660c735c81 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2454,7 +2454,7 @@ void coroutine_fn qmp_block_resize(bool has_device, const char *device, int64_t size, Error **errp) { Error *local_err = NULL; - BlockBackend *blk = NULL; + BlockBackend *blk; BlockDriverState *bs; AioContext *old_ctx; @@ -2468,17 +2468,17 @@ void coroutine_fn qmp_block_resize(bool has_device, const char *device, if (size < 0) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size"); - goto out; + return; } if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_RESIZE, NULL)) { error_setg(errp, QERR_DEVICE_IN_USE, device); - goto out; + return; } blk = blk_new_with_bs(bs, BLK_PERM_RESIZE, BLK_PERM_ALL, errp); if (!blk) { - goto out; + return; } bdrv_drained_begin(bs); @@ -2487,7 +2487,6 @@ void coroutine_fn qmp_block_resize(bool has_device, const char *device, bdrv_co_leave(bs, old_ctx); bdrv_drained_end(bs); -out: bdrv_co_lock(bs); blk_unref(blk); bdrv_co_unlock(bs); From patchwork Fri Dec 11 17:08:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2236EC4361B for ; Fri, 11 Dec 2020 17:26:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9FCAA23EF3 for ; Fri, 11 Dec 2020 17:26:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9FCAA23EF3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:47102 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmBd-0000YZ-MJ for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:26:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50800) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvX-0004lf-NA for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:55 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:46832) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knluo-0002Mm-5l for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jndb6B5lBHrI91u/GDfiNUD87XHYMBIRDIRaz9pfsmo=; b=TbvqPMQyesyf0ToimHQPOjSHoclpvL/ppVU9yaU12mGfoUu1eCjCpPMzw9tNkjRUKtUmL5 UTCw6u66cMvmM9h3abaD2ThbTXvt95kWVdiFz1hhTAe8DyLpJ90ius0OyMr5qCSiPE1z1n x730aV50WJSF661F1I5acnJgs6zZnrk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-406-zT_5xmelMlCXcE3sl8G1RA-1; Fri, 11 Dec 2020 12:09:03 -0500 X-MC-Unique: zT_5xmelMlCXcE3sl8G1RA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C04751005504; Fri, 11 Dec 2020 17:09:02 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id BDE045D6A8; Fri, 11 Dec 2020 17:09:01 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 33/34] block: Fix locking in qmp_block_resize() Date: Fri, 11 Dec 2020 18:08:11 +0100 Message-Id: <20201211170812.228643-34-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The drain functions assume that we hold the AioContext lock of the drained block node. Make sure to actually take the lock. Cc: qemu-stable@nongnu.org Fixes: eb94b81a94bce112e6b206df846c1551aaf6cab6 Signed-off-by: Kevin Wolf Message-Id: <20201203172311.68232-3-kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf --- blockdev.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index 660c735c81..412354b4b6 100644 --- a/blockdev.c +++ b/blockdev.c @@ -2481,13 +2481,16 @@ void coroutine_fn qmp_block_resize(bool has_device, const char *device, return; } + bdrv_co_lock(bs); bdrv_drained_begin(bs); + bdrv_co_unlock(bs); + old_ctx = bdrv_co_enter(bs); blk_truncate(blk, size, false, PREALLOC_MODE_OFF, 0, errp); bdrv_co_leave(bs, old_ctx); - bdrv_drained_end(bs); bdrv_co_lock(bs); + bdrv_drained_end(bs); blk_unref(blk); bdrv_co_unlock(bs); } From patchwork Fri Dec 11 17:08:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 11968753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D694C433FE for ; Fri, 11 Dec 2020 17:28:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D847123DC2 for ; Fri, 11 Dec 2020 17:28:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D847123DC2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55346 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1knmDt-00049a-UB for qemu-devel@archiver.kernel.org; Fri, 11 Dec 2020 12:28:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50842) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1knlvY-0004oX-Tj for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:34929) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1knlum-0002M9-8g for qemu-devel@nongnu.org; Fri, 11 Dec 2020 12:09:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607706547; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LdeSNRayXVfV6tXPpk5japdh1GU4T81tza0hXxnHGNE=; b=RNSrWtFmU8ag+7dfGW2924zOdBOghKuINOzKEWcAK8+AWZJzjdYZbucy3gtkGR2cZREE7A NnEHCTgnej+tF5TeHq89J0KDzN7dUozy2o35jPPuWsl9n5q4hCRAdq+8Hlrm+d13WvNrbL pU5HyTUP/iRc01AfPAc7UlQu+z7/5+A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-428-AdF2lZiJNEmwNS9uq9JPOg-1; Fri, 11 Dec 2020 12:09:05 -0500 X-MC-Unique: AdF2lZiJNEmwNS9uq9JPOg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B8AB801817; Fri, 11 Dec 2020 17:09:04 +0000 (UTC) Received: from merkur.redhat.com (ovpn-114-231.ams2.redhat.com [10.36.114.231]) by smtp.corp.redhat.com (Postfix) with ESMTP id 177A85D6A8; Fri, 11 Dec 2020 17:09:02 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Subject: [PULL 34/34] block: Fix deadlock in bdrv_co_yield_to_drain() Date: Fri, 11 Dec 2020 18:08:12 +0100 Message-Id: <20201211170812.228643-35-kwolf@redhat.com> In-Reply-To: <20201211170812.228643-1-kwolf@redhat.com> References: <20201211170812.228643-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=kwolf@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, peter.maydell@linaro.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If bdrv_co_yield_to_drain() is called for draining a block node that runs in a different AioContext, it keeps that AioContext locked while it yields and schedules a BH in the AioContext to do the actual drain. As long as executing the BH is the very next thing that the event loop of the node's AioContext does, this actually happens to work, but when it tries to execute something else that wants to take the AioContext lock, it will deadlock. (In the bug report, this other thing is a virtio-scsi device running virtio_scsi_data_plane_handle_cmd().) Instead, always drop the AioContext lock across the yield and reacquire it only when the coroutine is reentered. The BH needs to unconditionally take the lock for itself now. This fixes the 'block_resize' QMP command on a block node that runs in an iothread. Cc: qemu-stable@nongnu.org Fixes: eb94b81a94bce112e6b206df846c1551aaf6cab6 Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1903511 Signed-off-by: Kevin Wolf Message-Id: <20201203172311.68232-4-kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf --- block/io.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/block/io.c b/block/io.c index 6343d85476..24205f5168 100644 --- a/block/io.c +++ b/block/io.c @@ -312,17 +312,7 @@ static void bdrv_co_drain_bh_cb(void *opaque) if (bs) { AioContext *ctx = bdrv_get_aio_context(bs); - AioContext *co_ctx = qemu_coroutine_get_aio_context(co); - - /* - * When the coroutine yielded, the lock for its home context was - * released, so we need to re-acquire it here. If it explicitly - * acquired a different context, the lock is still held and we don't - * want to lock it a second time (or AIO_WAIT_WHILE() would hang). - */ - if (ctx == co_ctx) { - aio_context_acquire(ctx); - } + aio_context_acquire(ctx); bdrv_dec_in_flight(bs); if (data->begin) { assert(!data->drained_end_counter); @@ -334,9 +324,7 @@ static void bdrv_co_drain_bh_cb(void *opaque) data->ignore_bds_parents, data->drained_end_counter); } - if (ctx == co_ctx) { - aio_context_release(ctx); - } + aio_context_release(ctx); } else { assert(data->begin); bdrv_drain_all_begin(); @@ -354,13 +342,16 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, int *drained_end_counter) { BdrvCoDrainData data; + Coroutine *self = qemu_coroutine_self(); + AioContext *ctx = bdrv_get_aio_context(bs); + AioContext *co_ctx = qemu_coroutine_get_aio_context(self); /* Calling bdrv_drain() from a BH ensures the current coroutine yields and * other coroutines run if they were queued by aio_co_enter(). */ assert(qemu_in_coroutine()); data = (BdrvCoDrainData) { - .co = qemu_coroutine_self(), + .co = self, .bs = bs, .done = false, .begin = begin, @@ -374,13 +365,29 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, if (bs) { bdrv_inc_in_flight(bs); } - replay_bh_schedule_oneshot_event(bdrv_get_aio_context(bs), - bdrv_co_drain_bh_cb, &data); + + /* + * Temporarily drop the lock across yield or we would get deadlocks. + * bdrv_co_drain_bh_cb() reaquires the lock as needed. + * + * When we yield below, the lock for the current context will be + * released, so if this is actually the lock that protects bs, don't drop + * it a second time. + */ + if (ctx != co_ctx) { + aio_context_release(ctx); + } + replay_bh_schedule_oneshot_event(ctx, bdrv_co_drain_bh_cb, &data); qemu_coroutine_yield(); /* If we are resumed from some other event (such as an aio completion or a * timer callback), it is a bug in the caller that should be fixed. */ assert(data.done); + + /* Reaquire the AioContext of bs if we dropped it */ + if (ctx != co_ctx) { + aio_context_acquire(ctx); + } } void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,