From patchwork Wed Sep 1 15:15:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCA36C4320A for ; Wed, 1 Sep 2021 15:18:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 29D8A6108E for ; Wed, 1 Sep 2021 15:18:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 29D8A6108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:39952 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS0H-00067O-9b for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:18:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48972) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyY-0002ji-N4 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55762) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyV-0002x4-Gf for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=izxbkLzhrkypD4EK6Jt6uqPpHUs2Xv2G8Vl0pZllRvs=; b=hqUCKXFSmH0SCIYSisE0w/nWDhFTGTxnIzOZnO32AWHiqXl3/ZZIXcHpe/KslTmJuPqsuY ucJ/gciPhV4SAvKNYM4GZ4wi+F/fRmvFKqY11ba21Zsd0IQTMR5aNQC1RLCGCVlKbplLB0 54TPtJR3wxxbO02/iHJXIyry5JdsZf8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-566-lmrtdt1sOKWlTU-kSadP6A-1; Wed, 01 Sep 2021 11:16:24 -0400 X-MC-Unique: lmrtdt1sOKWlTU-kSadP6A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DA4DA87D541; Wed, 1 Sep 2021 15:16:23 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7B7415B826; Wed, 1 Sep 2021 15:16:23 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 01/56] python: qemu: add timer parameter for qmp.accept socket Date: Wed, 1 Sep 2021 17:15:24 +0200 Message-Id: <20210901151619.689075-2-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Also add a new _qmp_timer field to the QEMUMachine class. Let's change the default socket timeout to None, so that if a subclass needs to add a timer, it can be done by modifying this private field. At the same time, restore the timer to be 15 seconds in iotests.py, to give an upper bound to the QMP monitor test command execution. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Acked-by: John Snow Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-2-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- python/qemu/machine/machine.py | 7 +++++-- python/qemu/machine/qtest.py | 5 +++-- tests/qemu-iotests/iotests.py | 3 ++- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 971ed7e8c6..14c4d17eca 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -97,7 +97,8 @@ def __init__(self, sock_dir: Optional[str] = None, drain_console: bool = False, console_log: Optional[str] = None, - log_dir: Optional[str] = None): + log_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): ''' Initialize a QEMUMachine @@ -112,6 +113,7 @@ def __init__(self, @param drain_console: (optional) True to drain console socket to buffer @param console_log: (optional) path to console log file @param log_dir: where to create and keep log files + @param qmp_timer: (optional) default QMP socket timeout @note: Qemu process is not started until launch() is used. ''' # pylint: disable=too-many-arguments @@ -121,6 +123,7 @@ def __init__(self, self._binary = binary self._args = list(args) self._wrapper = wrapper + self._qmp_timer = qmp_timer self._name = name or "qemu-%d" % os.getpid() self._base_temp_dir = base_temp_dir @@ -343,7 +346,7 @@ def _pre_launch(self) -> None: def _post_launch(self) -> None: if self._qmp_connection: - self._qmp.accept() + self._qmp.accept(self._qmp_timer) def _post_shutdown(self) -> None: """ diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py index d6d9c6a34a..592be263e0 100644 --- a/python/qemu/machine/qtest.py +++ b/python/qemu/machine/qtest.py @@ -115,7 +115,8 @@ def __init__(self, name: Optional[str] = None, base_temp_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None, - sock_dir: Optional[str] = None): + sock_dir: Optional[str] = None, + qmp_timer: Optional[float] = None): # pylint: disable=too-many-arguments if name is None: @@ -124,7 +125,7 @@ def __init__(self, sock_dir = base_temp_dir super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir, socket_scm_helper=socket_scm_helper, - sock_dir=sock_dir) + sock_dir=sock_dir, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 89663dac06..6b0db4ce54 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -570,10 +570,11 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) + timer = 15.0 super().__init__(qemu_prog, qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, - sock_dir=sock_dir) + sock_dir=sock_dir, qmp_timer=timer) self._num_drives = 0 def add_object(self, opts): From patchwork Wed Sep 1 15:15:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BD98C432BE for ; Wed, 1 Sep 2021 15:18:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E06146109E for ; Wed, 1 Sep 2021 15:18:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E06146109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:39708 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS0C-0005w1-LP for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:18:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49058) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyd-0002ld-Mk for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:60066) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyb-00035o-6r for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6fRA6xJhy1JphaXxcsjCwoETYeXi4Cc5KAH37O1t3U8=; b=faoJiPCTVWMJShyt6KsxUmT296LDwBOa+y4KF/QRHXdvW7VC3b7Ayqk5G3P/8Lt0uLdvED hj23LXWjlt4tzhqe8lcpdHTIYe7Kb3DKKkyr2SLNrGTjf3C8d6N0R1J7NtIj2v1wjz1oBr LUHFw1NywSMJjYp22mna8Py/6W+YkLY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-285-oBDy8ZaANb-od83D7_SjUg-1; Wed, 01 Sep 2021 11:16:27 -0400 X-MC-Unique: oBDy8ZaANb-od83D7_SjUg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 106AB51DF; Wed, 1 Sep 2021 15:16:26 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A0F601000358; Wed, 1 Sep 2021 15:16:25 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 02/56] python: Reduce strictness of pylint's duplicate-code check Date: Wed, 1 Sep 2021 17:15:25 +0200 Message-Id: <20210901151619.689075-3-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow Pylint prior to 2.8.3 (We pin at >= 2.8.0) includes function and method signatures as part of its duplicate checking algorithm. This check does not listen to pragmas, so the only way to disable it is to turn it off completely or increase the minimum duplicate lines so that it doesn't trigger for functions with long, multi-line signatures. When we decide to upgrade to pylint 2.8.3 or greater, we will be able to use 'ignore-signatures = true' to the config instead. I'd prefer not to keep us on the very bleeding edge of pylint if I can help it -- 2.8.3 came out only three days ago at time of writing. See: https://github.com/PyCQA/pylint/pull/4474 Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Acked-by: John Snow Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-3-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- python/setup.cfg | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/python/setup.cfg b/python/setup.cfg index 14bab90288..83909c1c97 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -105,6 +105,11 @@ good-names=i, # Ignore imports when computing similarities. ignore-imports=yes +# Minimum lines number of a similarity. +# TODO: Remove after we opt in to Pylint 2.8.3. See commit msg. +min-similarity-lines=6 + + [isort] force_grid_wrap=4 force_sort_within_sections=True From patchwork Wed Sep 1 15:15:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D79BC432BE for ; Wed, 1 Sep 2021 15:21:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 49A6A6108E for ; Wed, 1 Sep 2021 15:21:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 49A6A6108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:48454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS3Q-0003d3-DM for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:21:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49056) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyd-0002lc-6O for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50420) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyZ-00033d-7l for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P3nx/TTSaa2FxDC/CJgMyNnajeKC/ZL6o55tkoPitfA=; b=ZHD5ACiBR5nj2Sh4aN09PMY7UKu9U+QT9v0Ni5s2rCctw3IJQPrrv9mUY8iY4kfkpu8Xlz qkZqGvQnUfGtYnyvxuqOrwQ+njuQOlSr/jIrhkqfszW5P0LJkekAgVHst3nGDPtS1h5gKt RV8cP7z5G1g81IFN7wWPdRWwGyfm3Hw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-507-mjfueVP1Mg6M8G8YKbGUaQ-1; Wed, 01 Sep 2021 11:16:29 -0400 X-MC-Unique: mjfueVP1Mg6M8G8YKbGUaQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1551380198A; Wed, 1 Sep 2021 15:16:28 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AFA425B826; Wed, 1 Sep 2021 15:16:27 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 03/56] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine Date: Wed, 1 Sep 2021 17:15:26 +0200 Message-Id: <20210901151619.689075-4-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz Acked-by: John Snow Message-Id: <20210809090114.64834-4-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- python/qemu/machine/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py index 592be263e0..395cc8fbfe 100644 --- a/python/qemu/machine/qtest.py +++ b/python/qemu/machine/qtest.py @@ -112,6 +112,7 @@ class QEMUQtestMachine(QEMUMachine): def __init__(self, binary: str, args: Sequence[str] = (), + wrapper: Sequence[str] = (), name: Optional[str] = None, base_temp_dir: str = "/var/tmp", socket_scm_helper: Optional[str] = None, @@ -123,7 +124,8 @@ def __init__(self, name = "qemu-%d" % os.getpid() if sock_dir is None: sock_dir = base_temp_dir - super().__init__(binary, args, name=name, base_temp_dir=base_temp_dir, + super().__init__(binary, args, wrapper=wrapper, name=name, + base_temp_dir=base_temp_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None From patchwork Wed Sep 1 15:15:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCA13C432BE for ; Wed, 1 Sep 2021 15:21:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FCAF6108E for ; Wed, 1 Sep 2021 15:21:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8FCAF6108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:50160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS3i-0004k9-Qb for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:21:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49112) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyg-0002u9-U4 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51966) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRye-00038x-4i for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lj3P9uBZnOrjHaaQi5jEmlHPpsmLMJluI+Vm5eQy2jo=; b=OXaFa3Gf2fePrMh5wu+l9kqHDglMqjW6XFAdgbm7rewWc6RRy0Q7my9sim+gCcYpYAWKrH oAjLGfI7V3Q1b68Z0GQKoCc28ESdGSzLudlg9FzKDFAOMCp7uuDKT7ClHw/Q9O44w1Kal5 3merqsskaOWWAdsEhLkND0c+iWa4uIs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-415-DfhkJ6hcOiiptrzHAKkggg-1; Wed, 01 Sep 2021 11:16:32 -0400 X-MC-Unique: DfhkJ6hcOiiptrzHAKkggg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 38A9D801A93; Wed, 1 Sep 2021 15:16:30 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BC4E85C1BB; Wed, 1 Sep 2021 15:16:29 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 04/56] docs/devel/testing: add debug section to the QEMU iotests chapter Date: Wed, 1 Sep 2021 17:15:27 +0200 Message-Id: <20210901151619.689075-5-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Introduce the "Debugging a test case" section, in preparation to the additional flags that will be added in the next patches. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-5-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- docs/devel/testing.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 8a9cda33a5..8359f2ae37 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -224,6 +224,14 @@ another application on the host may have locked the file, possibly leading to a test failure. If using such devices are explicitly desired, consider adding ``locking=off`` option to disable image locking. +Debugging a test case +----------------------- +The following options to the ``check`` script can be useful when debugging +a failing test: + +* ``-d`` (debug) just increases the logging verbosity, showing + for example the QMP commands and answers. + Test case groups ---------------- From patchwork Wed Sep 1 15:15:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6975CC432BE for ; Wed, 1 Sep 2021 15:24:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 060406108E for ; Wed, 1 Sep 2021 15:24:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 060406108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:58660 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS6R-0001z3-OI for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:24:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49130) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyh-0002xE-Qv for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28258) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyd-00038T-M6 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uIs9rxU61soopvf8VzrwNZd2JmqVmoErzhDV3dukOCo=; b=A2Mwy6XAd9Bg1CRA15PoRga3DLbv+lhcqjlfW3iM4XUFFO4b5gZGMznkQ3mij2AOm0HGvS cTO0IRvZOwp8j6gncOKhK/R19+tgqzCQJbzRQD0aU37hb1FOdJFjXLp1Ohigb0+wubtLs+ aJfKbFdIZVnhnbc0RFIyy/oZuozK+xU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-330-mv0c9OpNNyWOlmT6HcQWDQ-1; Wed, 01 Sep 2021 11:16:33 -0400 X-MC-Unique: mv0c9OpNNyWOlmT6HcQWDQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7745E80198A; Wed, 1 Sep 2021 15:16:32 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 17C1B60C05; Wed, 1 Sep 2021 15:16:31 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 05/56] qemu-iotests: add option to attach gdbserver Date: Wed, 1 Sep 2021 17:15:28 +0200 Message-Id: <20210901151619.689075-6-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Define -gdb flag and GDB_OPTIONS environment variable to python tests to attach a gdbserver to each qemu instance. This patch only adds and parses this flag, it does not yet add the implementation for it. if -gdb is not provided but $GDB_OPTIONS is set, ignore the environment variable. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-6-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/check | 6 +++++- tests/qemu-iotests/iotests.py | 5 +++++ tests/qemu-iotests/testenv.py | 17 +++++++++++++++-- 3 files changed, 25 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 2dd529eb75..4365bb8066 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -36,6 +36,9 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-gdb', action='store_true', + help="start gdbserver with $GDB_OPTIONS options \ + ('localhost:12345' if $GDB_OPTIONS is empty)") p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'], @@ -114,7 +117,8 @@ if __name__ == '__main__': env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto, aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, - debug=args.debug, valgrind=args.valgrind) + debug=args.debug, valgrind=args.valgrind, + gdb=args.gdb) if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--': if not args.tests: diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6b0db4ce54..c86f239d81 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -74,6 +74,11 @@ qemu_prog = os.environ.get('QEMU_PROG', 'qemu') qemu_opts = os.environ.get('QEMU_OPTIONS', '').strip().split(' ') +gdb_qemu_env = os.environ.get('GDB_OPTIONS') +qemu_gdb = [] +if gdb_qemu_env: + qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' ') + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.') diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 0c3fe75636..8501c6caf5 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -27,6 +27,7 @@ import glob from typing import List, Dict, Any, Optional, ContextManager +DEF_GDB_OPTIONS = 'localhost:12345' def isxfile(path: str) -> bool: return os.path.isfile(path) and os.access(path, os.X_OK) @@ -72,7 +73,8 @@ class TestEnv(ContextManager['TestEnv']): 'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', - 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_'] + 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', + 'GDB_OPTIONS'] def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: if self.debug: @@ -178,7 +180,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, imgopts: Optional[str] = None, misalign: bool = False, debug: bool = False, - valgrind: bool = False) -> None: + valgrind: bool = False, + gdb: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode @@ -186,6 +189,15 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if gdb: + self.gdb_options = os.getenv('GDB_OPTIONS', DEF_GDB_OPTIONS) + if not self.gdb_options: + # cover the case 'export GDB_OPTIONS=' + self.gdb_options = DEF_GDB_OPTIONS + elif 'GDB_OPTIONS' in os.environ: + # to not propagate it in prepare_subprocess() + del os.environ['GDB_OPTIONS'] + if valgrind: self.valgrind_qemu = 'y' @@ -285,6 +297,7 @@ def print_env(self) -> None: TEST_DIR -- {TEST_DIR} SOCK_DIR -- {SOCK_DIR} SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} +GDB_OPTIONS -- {GDB_OPTIONS} """ args = collections.defaultdict(str, self.get_env()) From patchwork Wed Sep 1 15:15:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1677C432BE for ; Wed, 1 Sep 2021 15:24:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 43D446109E for ; Wed, 1 Sep 2021 15:24:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 43D446109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:57736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS6G-0001MA-DF for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:24:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49134) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyi-0002yQ-5G for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22091) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyg-0003Az-KW for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509398; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jQk6BsaXNkWlJxsKb48WisoVo789BTAu7eRy0A6U7s0=; b=Iu4ztI49LGpREcfyKq8w0fMD3yS2VrtMZk5WaCMpIM1Zbme6MDz+z7pRlc/e+7lyIkaJT+ ZLyHRhpFgb9lq0ohzzTCR/q76qR2o0woRDKwMC4Wghpy5t3liujmLvfXsO5NciW1G0Csnv mdg4Rkxh0q+yw11PgWepfV04eBeCciU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-334-xNUHHjpcOJ2x281MIZtNww-1; Wed, 01 Sep 2021 11:16:35 -0400 X-MC-Unique: xNUHHjpcOJ2x281MIZtNww-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D7ED87D541; Wed, 1 Sep 2021 15:16:34 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 236915B826; Wed, 1 Sep 2021 15:16:33 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 06/56] qemu-iotests: delay QMP socket timers Date: Wed, 1 Sep 2021 17:15:29 +0200 Message-Id: <20210901151619.689075-7-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Attaching gdbserver implies that the qmp socket should wait indefinitely for an answer from QEMU. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-7-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index c86f239d81..e176a84620 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -477,10 +477,14 @@ def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg def __enter__(self): + if qemu_gdb: + return self signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self def __exit__(self, exc_type, value, traceback): + if qemu_gdb: + return False signal.setitimer(signal.ITIMER_REAL, 0) return False def timeout(self, signum, frame): @@ -575,7 +579,7 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - timer = 15.0 + timer = 15.0 if not qemu_gdb else None super().__init__(qemu_prog, qemu_opts, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, From patchwork Wed Sep 1 15:15:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10626C432BE for ; Wed, 1 Sep 2021 15:19:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9394A6108E for ; Wed, 1 Sep 2021 15:18:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9394A6108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:40690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS0w-0006hA-P3 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:18:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49168) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyl-00037k-6b for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:30655) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyh-0003CQ-Qa for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5LKg41Zsyc5xPXQ6eEzc6BgpuTA833NynUCK2i519kE=; b=XMLY4risxPQskUt1TzZAQwM70SkhtQPwNBvuTmbMF5Q102+J24L2yZmNaLhhdZowZLgpNF SzqtV/Mx4ikEFSKTioTiHaXwDxsCw2X7to/x5+/LU5YdBAn5ZCOKzxcZ5erGqTatFKoL/t HS2P3X+f+mR5sfMC8Soetc9cQ221N7c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-355-vJteUQ7DNGm_q36bMZAnxA-1; Wed, 01 Sep 2021 11:16:37 -0400 X-MC-Unique: vJteUQ7DNGm_q36bMZAnxA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9970B51D5; Wed, 1 Sep 2021 15:16:36 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 288C35D9DC; Wed, 1 Sep 2021 15:16:36 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 07/56] qemu_iotests: insert gdbserver command line as wrapper for qemu binary Date: Wed, 1 Sep 2021 17:15:30 +0200 Message-Id: <20210901151619.689075-8-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-8-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e176a84620..e7e3d92d3e 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -580,7 +580,8 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) timer = 15.0 if not qemu_gdb else None - super().__init__(qemu_prog, qemu_opts, name=name, + super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, + name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir, qmp_timer=timer) From patchwork Wed Sep 1 15:15:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A456C432BE for ; Wed, 1 Sep 2021 15:21:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A203D6108E for ; Wed, 1 Sep 2021 15:21:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A203D6108E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:49194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS3Y-000470-Ov for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:21:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49190) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRym-0003A3-Rk for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30147) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyj-0003E9-RZ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509401; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+ojSnW/SHxutLR83p5we6T0lcUNnGO0+vA1u0c32Srg=; b=ic7ybeYvOOlMZeq/8E/N9AkKWycvBTJi4JgSrIrJbydPiE3Oi5/IJs2cnmiZO0sWaVe3Nm oHBuuqiugR+HLUXyvX72u+waZrFtq/Ama0JNbfDt4oOTHMnKGVXdFnb7X9/NqOPX7RB7Rj VluMZmK7VC94pi6BA72ClhNFm/3IjAY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-466-hS3AEusaOXiyUm6TblwOww-1; Wed, 01 Sep 2021 11:16:39 -0400 X-MC-Unique: hS3AEusaOXiyUm6TblwOww-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D3D2A107ACC7; Wed, 1 Sep 2021 15:16:38 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 67DB360C05; Wed, 1 Sep 2021 15:16:38 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 08/56] qemu-iotests: add gdbserver option to script tests too Date: Wed, 1 Sep 2021 17:15:31 +0200 Message-Id: <20210901151619.689075-9-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Remove read timer in test script when GDB_OPTIONS are set, so that the bash tests won't timeout while running gdb. The only limitation here is that running a script with gdbserver will make the test output mismatch with the expected results, making the test fail. Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20210809090114.64834-9-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/common.qemu | 7 ++++++- tests/qemu-iotests/common.rc | 8 +++++++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu index 0fc52d20d7..0f1fecc68e 100644 --- a/tests/qemu-iotests/common.qemu +++ b/tests/qemu-iotests/common.qemu @@ -85,7 +85,12 @@ _timed_wait_for() timeout=yes QEMU_STATUS[$h]=0 - while IFS= read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]} + read_timeout="-t ${QEMU_COMM_TIMEOUT}" + if [ -n "${GDB_OPTIONS}" ]; then + read_timeout= + fi + + while IFS= read ${read_timeout} resp <&${QEMU_OUT[$h]} do if [ -n "$capture_events" ]; then capture=0 diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 609d82de89..d8582454de 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -166,8 +166,14 @@ _qemu_wrapper() if [ -n "${QEMU_NEED_PID}" ]; then echo $BASHPID > "${QEMU_TEST_DIR}/qemu-${_QEMU_HANDLE}.pid" fi + + GDB="" + if [ -n "${GDB_OPTIONS}" ]; then + GDB="gdbserver ${GDB_OPTIONS}" + fi + VALGRIND_QEMU="${VALGRIND_QEMU_VM}" _qemu_proc_exec "${VALGRIND_LOGFILE}" \ - "$QEMU_PROG" $QEMU_OPTIONS "$@" + $GDB "$QEMU_PROG" $QEMU_OPTIONS "$@" ) RETVAL=$? _qemu_proc_valgrind_log "${VALGRIND_LOGFILE}" $RETVAL From patchwork Wed Sep 1 15:15:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F864C432BE for ; Wed, 1 Sep 2021 15:26:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0A3756109E for ; Wed, 1 Sep 2021 15:26:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0A3756109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:39546 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS8d-00083p-98 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:26:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49314) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyt-0003XD-1f for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44839) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyq-0003K7-D3 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509407; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g8251mwDnxi+Q0NunZx+JdqKG9qRr6JC3+IlvC5Z94c=; b=gdykQXCz4hMqQE/wWIyNZH0iosBak+Q2RGgbsKZNCz+N+dCbKPX8hOCuJCHy1TKvfz9jFx K+OH3NMH40URPWOkIM6bF3irNyWrWb7WdwPAzswgsd2La/gJRar9NAM/n4qqA7m9RC/WpU VTy8eT3I2k0nmi3HV90nnAk0Ee66XE4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-119-Il5Ox5loNbOmFRiELW1ZIA-1; Wed, 01 Sep 2021 11:16:41 -0400 X-MC-Unique: Il5Ox5loNbOmFRiELW1ZIA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D725B106BC6C; Wed, 1 Sep 2021 15:16:40 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7906640AC; Wed, 1 Sep 2021 15:16:40 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 09/56] docs/devel/testing: add -gdb option to the debugging section of QEMU iotests Date: Wed, 1 Sep 2021 17:15:32 +0200 Message-Id: <20210901151619.689075-10-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-10-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- docs/devel/testing.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 8359f2ae37..01e1919873 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -229,6 +229,17 @@ Debugging a test case The following options to the ``check`` script can be useful when debugging a failing test: +* ``-gdb`` wraps every QEMU invocation in a ``gdbserver``, which waits for a + connection from a gdb client. The options given to ``gdbserver`` (e.g. the + address on which to listen for connections) are taken from the ``$GDB_OPTIONS`` + environment variable. By default (if ``$GDB_OPTIONS`` is empty), it listens on + ``localhost:12345``. + It is possible to connect to it for example with + ``gdb -iex "target remote $addr"``, where ``$addr`` is the address + ``gdbserver`` listens on. + If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored, + regardless of whether it is set or not. + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers. From patchwork Wed Sep 1 15:15:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21C47C4320A for ; Wed, 1 Sep 2021 15:21:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D297360FDC for ; Wed, 1 Sep 2021 15:21:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D297360FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:48632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS3S-0003kf-Vz for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:21:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49258) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyq-0003MV-3n for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57297) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyn-0003Hq-Qk for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509405; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c1z9HCaH6U+d724bb2/qXmwPN5UXMZJsc1CFTAsc3Ss=; b=KV9O79BE3cFZlqus9d2Rjbx59H6EX0/1iOJlTqcqt5vid0nllWF06qTupKHwrNcVgL7lFC swZQeubRakRonVXtSn1+C8986lSC1b3+UVSnukqJXFaRD2z/jrlWpfmtML9ysyYn6ITPi2 cKFG0TeFRnzdGqauIK6YJVfZQ882gYo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-390-RFM0Fa96NRucS7AqFRkBkg-1; Wed, 01 Sep 2021 11:16:43 -0400 X-MC-Unique: RFM0Fa96NRucS7AqFRkBkg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DAB3B107ACCA; Wed, 1 Sep 2021 15:16:42 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8314C40AC; Wed, 1 Sep 2021 15:16:42 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 10/56] qemu-iotests: extend the check script to prepare supporting valgrind for python tests Date: Wed, 1 Sep 2021 17:15:33 +0200 Message-Id: <20210901151619.689075-11-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Currently, the check script only parses the option and sets the VALGRIND_QEMU environmental variable to "y". Add another local python variable that prepares the command line, identical to the one provided in the test scripts. Because the python script does not know in advance the valgrind PID to assign to the log file name, use the "%p" flag in valgrind log file name that automatically puts the process PID at runtime. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20210809090114.64834-11-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/check | 7 ++++--- tests/qemu-iotests/iotests.py | 11 +++++++++++ tests/qemu-iotests/testenv.py | 1 + 3 files changed, 16 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 4365bb8066..ebd27946db 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -39,6 +39,10 @@ def make_argparser() -> argparse.ArgumentParser: p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_OPTIONS options \ ('localhost:12345' if $GDB_OPTIONS is empty)") + p.add_argument('-valgrind', action='store_true', + help='use valgrind, sets VALGRIND_QEMU environment ' + 'variable') + p.add_argument('-misalign', action='store_true', help='misalign memory allocations') p.add_argument('--color', choices=['on', 'off', 'auto'], @@ -88,9 +92,6 @@ def make_argparser() -> argparse.ArgumentParser: g_bash.add_argument('-o', dest='imgopts', help='options to pass to qemu-img create/convert, ' 'sets IMGOPTS environment variable') - g_bash.add_argument('-valgrind', action='store_true', - help='use valgrind, sets VALGRIND_QEMU environment ' - 'variable') g_sel = p.add_argument_group('test selecting options', 'The following options specify test set ' diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index e7e3d92d3e..6aa1dc48ba 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -96,6 +96,17 @@ sys.stderr.write('Please run this test via the "check" script\n') sys.exit(os.EX_USAGE) +qemu_valgrind = [] +if os.environ.get('VALGRIND_QEMU') == "y" and \ + os.environ.get('NO_VALGRIND') != "y": + valgrind_logfile = "--log-file=" + test_dir + # %p allows to put the valgrind process PID, since + # we don't know it a priori (subprocess.Popen is + # not yet invoked) + valgrind_logfile += "/%p.valgrind" + + qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99'] + socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') luks_default_secret_object = 'secret,id=keysec0,data=' + \ diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 8501c6caf5..8bf154376f 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -298,6 +298,7 @@ def print_env(self) -> None: SOCK_DIR -- {SOCK_DIR} SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_OPTIONS -- {GDB_OPTIONS} +VALGRIND_QEMU -- {VALGRIND_QEMU} """ args = collections.defaultdict(str, self.get_env()) From patchwork Wed Sep 1 15:15:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 544D1C43216 for ; Wed, 1 Sep 2021 15:24:31 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C1A56109E for ; Wed, 1 Sep 2021 15:24:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0C1A56109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:57908 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS6I-0001TL-5G for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:24:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49326) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyt-0003ZP-Oq for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:31865) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRys-0003LR-38 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D6GKSXwD6XrQy3Kddsw4aBJAa6yq8J4eApyNE+Xkpk0=; b=W58Q7T5Y9GKwJciYl6SWj5vKW2oBwYUo6f9GuxrIIabeBFgsBrjWkbPMRi5f39pk7iFRNw yYR80z8vB9N8Hhu7vTtjxoeOA2xjdrUQLKKw90i2JgG2UEpwRxds1DsGLl23s20AQHOkzb PBMdawYoEY0ryCy2Pb29iQCM/OCqJJc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-279-3WcSN87_NmyF7wE95Mhrtw-1; Wed, 01 Sep 2021 11:16:46 -0400 X-MC-Unique: 3WcSN87_NmyF7wE95Mhrtw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F3D6010054F6; Wed, 1 Sep 2021 15:16:44 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A16F260864; Wed, 1 Sep 2021 15:16:44 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 11/56] qemu-iotests: extend QMP socket timeout when using valgrind Date: Wed, 1 Sep 2021 17:15:34 +0200 Message-Id: <20210901151619.689075-12-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout and the generic class Timeout in iotests.py timeouts too soon. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-12-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 6aa1dc48ba..26c580f9e7 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -488,13 +488,13 @@ def __init__(self, seconds, errmsg="Timeout"): self.seconds = seconds self.errmsg = errmsg def __enter__(self): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: return self signal.signal(signal.SIGALRM, self.timeout) signal.setitimer(signal.ITIMER_REAL, self.seconds) return self def __exit__(self, exc_type, value, traceback): - if qemu_gdb: + if qemu_gdb or qemu_valgrind: return False signal.setitimer(signal.ITIMER_REAL, 0) return False @@ -590,7 +590,7 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) - timer = 15.0 if not qemu_gdb else None + timer = 15.0 if not (qemu_gdb or qemu_valgrind) else None super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, name=name, base_temp_dir=test_dir, From patchwork Wed Sep 1 15:15:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E044EC432BE for ; Wed, 1 Sep 2021 15:29:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8B1FA61059 for ; Wed, 1 Sep 2021 15:29:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8B1FA61059 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:48590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSBM-0005ea-P0 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:29:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49344) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyu-0003cc-Ec for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48892) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRys-0003Li-99 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509409; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sh6ny78MxCS8p1zcfkl8MHFzVS565tvTjjayif6A9hU=; b=DlzE2Hs8bFtdfekDXjdT56q6JPoqBxtvZtw7vlt5yeqNPemKua2d67mKwyoVLRouraFMgy KkUvU0S763GYAQ8EwMzGR6Qsco1NO48CZBfP+WqrGqiDWqt8WaXWdKFEUrrdVUoGd+i6nM Tt3ffCEslBKPmRisCK/aTEMEujIBn1g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-587-OZhXQODwMl6bGb4vagubQA-1; Wed, 01 Sep 2021 11:16:48 -0400 X-MC-Unique: OZhXQODwMl6bGb4vagubQA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D4BA80198A; Wed, 1 Sep 2021 15:16:47 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B6AA25D9DC; Wed, 1 Sep 2021 15:16:46 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 12/56] qemu-iotests: allow valgrind to read/delete the generated log file Date: Wed, 1 Sep 2021 17:15:35 +0200 Message-Id: <20210901151619.689075-13-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito When using -valgrind on the script tests, it generates a log file in $TEST_DIR that is either read (if valgrind finds problems) or otherwise deleted. Provide the same exact behavior when using -valgrind on the python tests. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-13-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 26c580f9e7..85d8c0abbb 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -598,6 +598,17 @@ def __init__(self, path_suffix=''): sock_dir=sock_dir, qmp_timer=timer) self._num_drives = 0 + def _post_shutdown(self) -> None: + super()._post_shutdown() + if not qemu_valgrind or not self._popen: + return + valgrind_filename = f"{test_dir}/{self._popen.pid}.valgrind" + if self.exitcode() == 99: + with open(valgrind_filename) as f: + print(f.read()) + else: + os.remove(valgrind_filename) + def add_object(self, opts): self._args.append('-object') self._args.append(opts) From patchwork Wed Sep 1 15:15:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37733C432BE for ; Wed, 1 Sep 2021 15:26:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C977461075 for ; Wed, 1 Sep 2021 15:26:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C977461075 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:38426 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS8Q-0007Ko-Uj for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:26:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49362) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyv-0003hq-Vd for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27771) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyu-0003Nc-8Q for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q5a91vemobpHCDV1r1gv9jGMMkXJJEkkMVubXjwa9PE=; b=Y9yPP63aTMBi6sNjlLiiyYDUeoGBrWWnB7VVaLRu1KT3tq5yut/FUPpo0QSeU8OQqckNSA oonR/k+ZUKGTrBdSbPIfjuKYHd1RpNnYPr7jI2nq7pISEwf84HpjFpLwf6QO5SnOv6mFO2 vmD2DQbUyooyGyOm20lTX5IDVJtCMn4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-233-8QH-omThNV-2Bew0oaLRww-1; Wed, 01 Sep 2021 11:16:50 -0400 X-MC-Unique: 8QH-omThNV-2Bew0oaLRww-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2F56751DF; Wed, 1 Sep 2021 15:16:49 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BF1116A8F2; Wed, 1 Sep 2021 15:16:48 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 13/56] qemu-iotests: insert valgrind command line as wrapper for qemu binary Date: Wed, 1 Sep 2021 17:15:36 +0200 Message-Id: <20210901151619.689075-14-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito If -gdb and -valgrind are both defined, return an error. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-14-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 85d8c0abbb..74fa56840d 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -591,7 +591,11 @@ class VM(qtest.QEMUQtestMachine): def __init__(self, path_suffix=''): name = "qemu%s-%d" % (path_suffix, os.getpid()) timer = 15.0 if not (qemu_gdb or qemu_valgrind) else None - super().__init__(qemu_prog, qemu_opts, wrapper=qemu_gdb, + if qemu_gdb and qemu_valgrind: + sys.stderr.write('gdb and valgrind are mutually exclusive\n') + sys.exit(1) + wrapper = qemu_gdb if qemu_gdb else qemu_valgrind + super().__init__(qemu_prog, qemu_opts, wrapper=wrapper, name=name, base_temp_dir=test_dir, socket_scm_helper=socket_scm_helper, From patchwork Wed Sep 1 15:15:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2ACA8C432BE for ; Wed, 1 Sep 2021 15:29:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DC94C61059 for ; Wed, 1 Sep 2021 15:29:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DC94C61059 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:46988 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSB5-0004Zl-4t for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:29:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49396) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz0-0003vd-At for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36647) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRyy-0003RF-I0 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509415; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hC9uZEGVIHQAJ2DALP6sOzhlz9RV8KYywa8xGPVEw5g=; b=gOpXrvICzwv2iuG5LvxYbpm57CXiEh+DTa5XItxsm4PDYLDFWTvbkjGORcXn0jhr4Tr1gl jqoZJhhtHxSoOX6Vr7IujE+SLD1jgmZo5CZilJn0jL2PC82LwWWFnzHOxVZ0Y2d39wJpaw s5V/sSGlpDpHIr2EwSNfNYyRTpr3zV4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-178-TEzzAdSGPZ6KKL9KNqIzBQ-1; Wed, 01 Sep 2021 11:16:52 -0400 X-MC-Unique: TEzzAdSGPZ6KKL9KNqIzBQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 67A8F106BC6C; Wed, 1 Sep 2021 15:16:51 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E423D1F046; Wed, 1 Sep 2021 15:16:50 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 14/56] docs/devel/testing: add -valgrind option to the debug section of QEMU iotests Date: Wed, 1 Sep 2021 17:15:37 +0200 Message-Id: <20210901151619.689075-15-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Message-Id: <20210809090114.64834-15-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- docs/devel/testing.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 01e1919873..8ebcf85a31 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -240,6 +240,12 @@ a failing test: If the ``-gdb`` option is not used, ``$GDB_OPTIONS`` is ignored, regardless of whether it is set or not. +* ``-valgrind`` attaches a valgrind instance to QEMU. If it detects + warnings, it will print and save the log in + ``$TEST_DIR/.valgrind``. + The final command line will be ``valgrind --log-file=$TEST_DIR/ + .valgrind --error-exitcode=99 $QEMU ...`` + * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers. From patchwork Wed Sep 1 15:15:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DE65C432BE for ; Wed, 1 Sep 2021 15:33:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A0BCD60231 for ; Wed, 1 Sep 2021 15:33:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A0BCD60231 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:55396 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSEV-0001tr-P7 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:32:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz1-000407-TJ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36616) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz0-0003SB-0t for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509417; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hVT+YwY/Moo0QoS3SUa0beD2xBj6O23Womdyqag2NiI=; b=JPZMw1qxtzVWna32DHiYpfXj66h0YtPSJFaBIwWtlDxRQFUFfNLEi9+xIEkWiAmqeUSWdn gMnDcJURgCMaq8ptBa9sTAipwFn/h1kAsU8RWJRPGHooZfkJ+xl97ZSs+ZPU4fcnjnmldi GaeDc6fAA/RDx+4odW8yRxF/rXdB3bk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-54-rBSoTP_RMbuIpZuK87z1-Q-1; Wed, 01 Sep 2021 11:16:54 -0400 X-MC-Unique: rBSoTP_RMbuIpZuK87z1-Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 640D2106BC83; Wed, 1 Sep 2021 15:16:53 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1074360864; Wed, 1 Sep 2021 15:16:52 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 15/56] qemu-iotests: add option to show qemu binary logs on stdout Date: Wed, 1 Sep 2021 17:15:38 +0200 Message-Id: <20210901151619.689075-16-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Using the flag -p, allow the qemu binary to print to stdout. Also create the common function _close_qemu_log_file() to avoid accessing machine.py private fields directly and have duplicate code. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-16-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- python/qemu/machine/machine.py | 9 ++++++--- tests/qemu-iotests/check | 4 +++- tests/qemu-iotests/iotests.py | 8 ++++++++ tests/qemu-iotests/testenv.py | 9 +++++++-- 4 files changed, 24 insertions(+), 6 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 14c4d17eca..8b935813e9 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -348,6 +348,11 @@ def _post_launch(self) -> None: if self._qmp_connection: self._qmp.accept(self._qmp_timer) + def _close_qemu_log_file(self) -> None: + if self._qemu_log_file is not None: + self._qemu_log_file.close() + self._qemu_log_file = None + def _post_shutdown(self) -> None: """ Called to cleanup the VM instance after the process has exited. @@ -360,9 +365,7 @@ def _post_shutdown(self) -> None: self._qmp.close() self._qmp_connection = None - if self._qemu_log_file is not None: - self._qemu_log_file.close() - self._qemu_log_file = None + self._close_qemu_log_file() self._load_io_log() diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index ebd27946db..da1bfb839e 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -36,6 +36,8 @@ def make_argparser() -> argparse.ArgumentParser: help='pretty print output for make check') p.add_argument('-d', dest='debug', action='store_true', help='debug') + p.add_argument('-p', dest='print', action='store_true', + help='redirects qemu\'s stdout and stderr to the test output') p.add_argument('-gdb', action='store_true', help="start gdbserver with $GDB_OPTIONS options \ ('localhost:12345' if $GDB_OPTIONS is empty)") @@ -119,7 +121,7 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb) + gdb=args.gdb, qprint=args.print) if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--': if not args.tests: diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 74fa56840d..2cf5ff965b 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -79,6 +79,8 @@ if gdb_qemu_env: qemu_gdb = ['gdbserver'] + gdb_qemu_env.strip().split(' ') +qemu_print = os.environ.get('PRINT_QEMU', False) + imgfmt = os.environ.get('IMGFMT', 'raw') imgproto = os.environ.get('IMGPROTO', 'file') output_dir = os.environ.get('OUTPUT_DIR', '.') @@ -613,6 +615,12 @@ def _post_shutdown(self) -> None: else: os.remove(valgrind_filename) + def _pre_launch(self) -> None: + super()._pre_launch() + if qemu_print: + # set QEMU binary output to stdout + self._close_qemu_log_file() + def add_object(self, opts): self._args.append('-object') self._args.append(opts) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 8bf154376f..70da0d60c8 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -74,7 +74,7 @@ class TestEnv(ContextManager['TestEnv']): 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', 'CACHEMODE_IS_DEFAULT', 'IMGFMT_GENERIC', 'IMGOPTSSYNTAX', 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', - 'GDB_OPTIONS'] + 'GDB_OPTIONS', 'PRINT_QEMU'] def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: if self.debug: @@ -181,7 +181,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, misalign: bool = False, debug: bool = False, valgrind: bool = False, - gdb: bool = False) -> None: + gdb: bool = False, + qprint: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode @@ -189,6 +190,9 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, self.misalign = misalign self.debug = debug + if qprint: + self.print_qemu = 'y' + if gdb: self.gdb_options = os.getenv('GDB_OPTIONS', DEF_GDB_OPTIONS) if not self.gdb_options: @@ -299,6 +303,7 @@ def print_env(self) -> None: SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_OPTIONS -- {GDB_OPTIONS} VALGRIND_QEMU -- {VALGRIND_QEMU} +PRINT_QEMU_OUTPUT -- {PRINT_QEMU} """ args = collections.defaultdict(str, self.get_env()) From patchwork Wed Sep 1 15:15:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E92AC432BE for ; Wed, 1 Sep 2021 15:35:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 02B2061057 for ; Wed, 1 Sep 2021 15:35:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 02B2061057 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:35326 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSHJ-0007ZN-0A for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:35:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz2-00041W-9f for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:27719) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz0-0003SI-L2 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:16:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509418; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2usmsG2NMXyCm3Z8vG6OJ9Z1sEtTNOVRGMIcpXu7YC8=; b=apD+L0rMsmCrF/feEVupZO2UHR2wuAnTngSKbdwfBwQ9sxTTL0zE+AwaSjVszpAL/p5Eff JKnASDwoGvn8A/u/gR6kLhzz/l88MSvJJVx3tgwxOeSnsYazGt4YL8WXNlUgvKlBTxxWBX 5C7HFzkPaXZqkK9la0sB4DL5Mp7Icpg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-310-FCJiay8GOza2DLUWf9bn8Q-1; Wed, 01 Sep 2021 11:16:56 -0400 X-MC-Unique: FCJiay8GOza2DLUWf9bn8Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 80335801A93; Wed, 1 Sep 2021 15:16:55 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 154C46A8F5; Wed, 1 Sep 2021 15:16:54 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 16/56] docs/devel/testing: add -p option to the debug section of QEMU iotests Date: Wed, 1 Sep 2021 17:15:39 +0200 Message-Id: <20210901151619.689075-17-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210809090114.64834-17-eesposit@redhat.com> Signed-off-by: Hanna Reitz --- docs/devel/testing.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 8ebcf85a31..4a0abbf23d 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -249,6 +249,10 @@ a failing test: * ``-d`` (debug) just increases the logging verbosity, showing for example the QMP commands and answers. +* ``-p`` (print) redirects QEMU’s stdout and stderr to the test output, + instead of saving it into a log file in + ``$TEST_DIR/qemu-machine-``. + Test case groups ---------------- From patchwork Wed Sep 1 15:15:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82720C432BE for ; Wed, 1 Sep 2021 15:24:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2007660FDC for ; Wed, 1 Sep 2021 15:24:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2007660FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:60456 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS6k-00039G-8b for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:24:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49472) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz4-00049m-Sk for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24781) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz2-0003UQ-Eq for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509419; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/dWOu3tH/Dl5NzUP0V6ShDHO9gBC97s69qGQfrVxU44=; b=ioR7vw5+wE6Xd0oa6L/F55s8/m6T1wuGUVpNW36dVypwvAQbECRS/bS9SDLLEsP1KEjWlj dRiQ/Y7seDo+ZZgTvggRn4QvcfWxfeP9GCszQEfS3G43+Ll8/H84aJ7O4N3lf8EZz/kVZJ 94adRqCyByjDgR4SfZra3APT/8DkeSk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-254-eMG7yyU5N4uSVwmGiqseLg-1; Wed, 01 Sep 2021 11:16:58 -0400 X-MC-Unique: eMG7yyU5N4uSVwmGiqseLg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AD9EA87D543; Wed, 1 Sep 2021 15:16:57 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 55C7446; Wed, 1 Sep 2021 15:16:57 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 17/56] block/monitor: Consolidate hmp_handle_error calls to reduce redundant code Date: Wed, 1 Sep 2021 17:15:40 +0200 Message-Id: <20210901151619.689075-18-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Mao Zhongyi Signed-off-by: Mao Zhongyi Message-Id: <20210802062507.347555-1-maozhongyi@cmss.chinamobile.com> Signed-off-by: Hanna Reitz --- block/monitor/block-hmp-cmds.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c index 3e6670c963..2ac4aedfff 100644 --- a/block/monitor/block-hmp-cmds.c +++ b/block/monitor/block-hmp-cmds.c @@ -251,10 +251,10 @@ void hmp_drive_mirror(Monitor *mon, const QDict *qdict) if (!filename) { error_setg(&err, QERR_MISSING_PARAMETER, "target"); - hmp_handle_error(mon, err); - return; + goto end; } qmp_drive_mirror(&mirror, &err); +end: hmp_handle_error(mon, err); } @@ -281,11 +281,11 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict) if (!filename) { error_setg(&err, QERR_MISSING_PARAMETER, "target"); - hmp_handle_error(mon, err); - return; + goto end; } qmp_drive_backup(&backup, &err); +end: hmp_handle_error(mon, err); } @@ -356,8 +356,7 @@ void hmp_snapshot_blkdev(Monitor *mon, const QDict *qdict) * will be taken internally. Today it's actually required. */ error_setg(&err, QERR_MISSING_PARAMETER, "snapshot-file"); - hmp_handle_error(mon, err); - return; + goto end; } mode = reuse ? NEW_IMAGE_MODE_EXISTING : NEW_IMAGE_MODE_ABSOLUTE_PATHS; @@ -365,6 +364,7 @@ void hmp_snapshot_blkdev(Monitor *mon, const QDict *qdict) filename, false, NULL, !!format, format, true, mode, &err); +end: hmp_handle_error(mon, err); } From patchwork Wed Sep 1 15:15:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A01DC432BE for ; Wed, 1 Sep 2021 15:33:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D01EE60231 for ; Wed, 1 Sep 2021 15:33:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D01EE60231 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:57642 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSFB-0003Sv-0f for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:33:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49510) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz8-0004NA-O3 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36324) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz6-0003Yy-Tj for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0QRPJ2sCMvYzVCegQ2/2TxlcXzRemOgzbmSlUjg472s=; b=S8SWQwgGq0amO06OmUGQkh/8TsVQiJbdApCWShYAatgbEXJ2XVLRNI6rAUn+GIjp0jMcjc ZwuEu7PbgWnJyOF75W0GDuj2MByLpuaEEKVvBFbx4tejwLryy0ky9Qsv899QSIh1hFK4Xy ER16isJq6Pvr0XVcdNU6mhPKbtIapZA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-34-dyhVIbpPNB-236uqqm14gQ-1; Wed, 01 Sep 2021 11:17:00 -0400 X-MC-Unique: dyhVIbpPNB-236uqqm14gQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CB38287D541; Wed, 1 Sep 2021 15:16:59 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 712305C1BB; Wed, 1 Sep 2021 15:16:59 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 18/56] raw-format: drop WRITE and RESIZE child perms when possible Date: Wed, 1 Sep 2021 17:15:41 +0200 Message-Id: <20210901151619.689075-19-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Stefan Hajnoczi The following command-line fails due to a permissions conflict: $ qemu-storage-daemon \ --blockdev driver=nvme,node-name=nvme0,device=0000:08:00.0,namespace=1 \ --blockdev driver=raw,node-name=l1-1,file=nvme0,offset=0,size=1073741824 \ --blockdev driver=raw,node-name=l1-2,file=nvme0,offset=1073741824,size=1073741824 \ --nbd-server addr.type=unix,addr.path=/tmp/nbd.sock,max-connections=2 \ --export type=nbd,id=nbd-l1-1,node-name=l1-1,name=l1-1,writable=on \ --export type=nbd,id=nbd-l1-2,node-name=l1-2,name=l1-2,writable=on qemu-storage-daemon: --export type=nbd,id=nbd-l1-1,node-name=l1-1,name=l1-1,writable=on: Permission conflict on node 'nvme0': permissions 'resize' are both required by node 'l1-1' (uses node 'nvme0' as 'file' child) and unshared by node 'l1-2' (uses node 'nvme0' as 'file' child). The problem is that block/raw-format.c relies on bdrv_default_perms() to set permissions on the nvme node. The default permissions add RESIZE in anticipation of a format driver like qcow2 that needs to grow the image file. This fails because RESIZE is unshared, so we cannot get the RESIZE permission. Max Reitz pointed out that block/crypto.c already handles this case by implementing a custom ->bdrv_child_perm() function that adjusts the result of bdrv_default_perms(). This patch takes the same approach in block/raw-format.c so that RESIZE is only required if it's actually necessary (e.g. the parent is qcow2). Cc: Max Reitz Cc: Kevin Wolf Signed-off-by: Stefan Hajnoczi Message-Id: <20210726122839.822900-1-stefanha@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Hanna Reitz --- block/raw-format.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/block/raw-format.c b/block/raw-format.c index 7717578ed6..c26f493688 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -580,6 +580,25 @@ static void raw_cancel_in_flight(BlockDriverState *bs) bdrv_cancel_in_flight(bs->file->bs); } +static void raw_child_perm(BlockDriverState *bs, BdrvChild *c, + BdrvChildRole role, + BlockReopenQueue *reopen_queue, + uint64_t parent_perm, uint64_t parent_shared, + uint64_t *nperm, uint64_t *nshared) +{ + bdrv_default_perms(bs, c, role, reopen_queue, parent_perm, + parent_shared, nperm, nshared); + + /* + * bdrv_default_perms() may add WRITE and/or RESIZE (see comment in + * bdrv_default_perms_for_storage() for an explanation) but we only need + * them if they are in parent_perm. Drop WRITE and RESIZE whenever possible + * to avoid permission conflicts. + */ + *nperm &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); + *nperm |= parent_perm & (BLK_PERM_WRITE | BLK_PERM_RESIZE); +} + BlockDriver bdrv_raw = { .format_name = "raw", .instance_size = sizeof(BDRVRawState), @@ -588,7 +607,7 @@ BlockDriver bdrv_raw = { .bdrv_reopen_commit = &raw_reopen_commit, .bdrv_reopen_abort = &raw_reopen_abort, .bdrv_open = &raw_open, - .bdrv_child_perm = bdrv_default_perms, + .bdrv_child_perm = raw_child_perm, .bdrv_co_create_opts = &raw_co_create_opts, .bdrv_co_preadv = &raw_co_preadv, .bdrv_co_pwritev = &raw_co_pwritev, From patchwork Wed Sep 1 15:15:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C823C432BE for ; Wed, 1 Sep 2021 15:36:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C6636600EF for ; Wed, 1 Sep 2021 15:36:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C6636600EF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:37910 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSHr-0000ul-SL for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:36:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzA-0004SQ-7D for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31067) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRz7-0003ZY-Ud for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5PmbfJAkkFd/cgbNK2VepFueoG9SKGX3CAA9C+LMYNA=; b=gvndENIJHRSFWBib99n4/t4x6jTg42vC8+4G4wfeYBDaYMRjqxQ4BZaRfwcjnRHcG0cPzP aSJryNBXVbRSM0ybcqGqPMI6MyBaTpNQpsUQVmzN539dC7dTd2Zvz/OawFRynpflxQ40GU XPM/ztdYgpYL4nd4Z13V6Ow8b9QxVP4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-fqMvawGIPO2lr_oFvLDy5w-1; Wed, 01 Sep 2021 11:17:04 -0400 X-MC-Unique: fqMvawGIPO2lr_oFvLDy5w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BF4AA520E; Wed, 1 Sep 2021 15:17:01 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6C7A66A05B; Wed, 1 Sep 2021 15:17:01 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 19/56] iotests: use with-statement for open() calls Date: Wed, 1 Sep 2021 17:15:42 +0200 Message-Id: <20210901151619.689075-20-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow Silences a new pylint warning. The dangers of *not* doing this are somewhat unclear; I believe the file object gets garbage collected eventually, but possibly the way in which it happens is non-deterministic. Maybe this is a valid warning, but if there are consequences of not doing it, I am not aware of them at present. Signed-off-by: John Snow Message-Id: <20210720173336.1876937-2-jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 2cf5ff965b..2ad7a15c8b 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -1120,7 +1120,8 @@ def notrun(reason): # Each test in qemu-iotests has a number ("seq") seq = os.path.basename(sys.argv[0]) - open('%s/%s.notrun' % (output_dir, seq), 'w').write(reason + '\n') + with open('%s/%s.notrun' % (output_dir, seq), 'w') as outfile: + outfile.write(reason + '\n') logger.warning("%s not run: %s", seq, reason) sys.exit(0) @@ -1133,8 +1134,8 @@ def case_notrun(reason): # Each test in qemu-iotests has a number ("seq") seq = os.path.basename(sys.argv[0]) - open('%s/%s.casenotrun' % (output_dir, seq), 'a').write( - ' [case not run] ' + reason + '\n') + with open('%s/%s.casenotrun' % (output_dir, seq), 'a') as outfile: + outfile.write(' [case not run] ' + reason + '\n') def _verify_image_format(supported_fmts: Sequence[str] = (), unsupported_fmts: Sequence[str] = ()) -> None: From patchwork Wed Sep 1 15:15:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DFEEC432BE for ; Wed, 1 Sep 2021 15:38:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A44FD6109E for ; Wed, 1 Sep 2021 15:38:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A44FD6109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:47118 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSKC-0007JX-RK for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:38:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49594) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzD-0004dQ-5z for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57221) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzB-0003cW-Cz for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509428; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F8AaHfQRnn/TzuyV/qk/3Rp5lhp+NGLCu5i6cugLt1A=; b=RdB/vOk501OXDuDgHK1gs6NPWLma5H9MDXUbW0vCCKJFd4aaYdROng7Tnz6p9xbg8BHin6 daQ3tz4hndUbqZ/m5Vf02Olo7kUcCDqy3qbjAm7a5C8Qvtz/XEf+662u/EIYwtVzZWsDHR TNSB+es3pzgSE880oNX6OhK8IGbP5Uc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-498-onRFq3psP_Kbi0hw9U9lmg-1; Wed, 01 Sep 2021 11:17:04 -0400 X-MC-Unique: onRFq3psP_Kbi0hw9U9lmg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DA48E1B2C980; Wed, 1 Sep 2021 15:17:03 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 83E0D6608B; Wed, 1 Sep 2021 15:17:03 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 20/56] iotests: use subprocess.DEVNULL instead of open("/dev/null") Date: Wed, 1 Sep 2021 17:15:43 +0200 Message-Id: <20210901151619.689075-21-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: John Snow Avoids a warning from pylint not to use open() outside of a with-statement, and is ... probably more portable anyway. Not that I think we care too much about running tests *on* Windows, but... eh. Signed-off-by: John Snow Message-Id: <20210720173336.1876937-3-jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 2ad7a15c8b..4c8971d946 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -237,18 +237,18 @@ def qemu_io_silent(*args): default_args = qemu_io_args args = default_args + list(args) - exitcode = subprocess.call(args, stdout=open('/dev/null', 'w')) - if exitcode < 0: + result = subprocess.run(args, stdout=subprocess.DEVNULL, check=False) + if result.returncode < 0: sys.stderr.write('qemu-io received signal %i: %s\n' % - (-exitcode, ' '.join(args))) - return exitcode + (-result.returncode, ' '.join(args))) + return result.returncode def qemu_io_silent_check(*args): '''Run qemu-io and return the true if subprocess returned 0''' args = qemu_io_args + list(args) - exitcode = subprocess.call(args, stdout=open('/dev/null', 'w'), - stderr=subprocess.STDOUT) - return exitcode == 0 + result = subprocess.run(args, stdout=subprocess.DEVNULL, + stderr=subprocess.STDOUT, check=False) + return result.returncode == 0 class QemuIoInteractive: def __init__(self, *args): From patchwork Wed Sep 1 15:15:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42EEDC432BE for ; Wed, 1 Sep 2021 15:38:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D879F61059 for ; Wed, 1 Sep 2021 15:38:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D879F61059 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:45874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSJw-0006UL-0f for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:38:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49598) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzD-0004fy-Rb for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55670) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzB-0003cQ-00 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509428; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J3Wzs6PgdR5xFLD7xKbCyaLcwPJaATPbnIu6vqV2Zps=; b=ANjfiOqmJ4LLNSowFLDGJZnp2vp1vXaorK8XDLqvnBEVFndsVZUJvBoDm4mH6vDoLFYU8M 6GPfzxvceNQyz9UCEfrAdaFclV1M0dWR8Ge7RfuNXGpR6QX/KLG8o9MNV0vtgVV+xJ4tJo CHlHRba4oo5j51QxDHh5w+S87InKolk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-564-5ro6mnzQOWiQYZ8DcrptGQ-1; Wed, 01 Sep 2021 11:17:06 -0400 X-MC-Unique: 5ro6mnzQOWiQYZ8DcrptGQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D7D39802921; Wed, 1 Sep 2021 15:17:05 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8327A5C23A; Wed, 1 Sep 2021 15:17:05 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 21/56] block: introduce bdrv_replace_child_bs() Date: Wed, 1 Sep 2021 17:15:44 +0200 Message-Id: <20210901151619.689075-22-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Add function to transactionally replace bs inside BdrvChild. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-2-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- include/block/block.h | 2 ++ block.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/include/block/block.h b/include/block/block.h index 3477290f9a..740038a892 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp); int bdrv_drop_filter(BlockDriverState *bs, Error **errp); diff --git a/block.c b/block.c index e97ce0b1c8..b2b66263f9 100644 --- a/block.c +++ b/block.c @@ -5048,6 +5048,37 @@ out: return ret; } +/* Not for empty child */ +int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs, + Error **errp) +{ + int ret; + Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + bdrv_ref(old_bs); + bdrv_drained_begin(old_bs); + bdrv_drained_begin(new_bs); + + bdrv_replace_child_tran(child, new_bs, tran); + + found = g_hash_table_new(NULL, NULL); + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs); + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs); + + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp); + + tran_finalize(tran, ret); + + bdrv_drained_end(old_bs); + bdrv_drained_end(new_bs); + bdrv_unref(old_bs); + + return ret; +} + static void bdrv_delete(BlockDriverState *bs) { assert(bdrv_op_blocker_is_empty(bs)); From patchwork Wed Sep 1 15:15:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37E00C432BE for ; Wed, 1 Sep 2021 15:31:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E2C3160231 for ; Wed, 1 Sep 2021 15:31:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E2C3160231 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:52158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSCy-00082i-1x for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:31:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49820) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzT-0005Ua-OM for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32201) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzH-0003hs-RP for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m9V2SWgW8MNBHXmqIE1NGL40KoUoUxdHOsKBpy8sl3U=; b=AuGC/GxCYfjmcMweRXwhpJgY/tjbmImGXMqIT5HmcW3dfmKGRVuWv5bFbzimEqL1bF2wR2 jGeHzZ3/azunoidmBQFH+cF+wnAgD3d8nfVjmkiLOWH2pqbLzebxce1NJ8a5+izmIYmzz/ vcer9BdiXy52/6bUG+iEeWmq5o2RyLA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-105-HI5O-Gd_PyOpGNX7GGxgmg-1; Wed, 01 Sep 2021 11:17:09 -0400 X-MC-Unique: HI5O-Gd_PyOpGNX7GGxgmg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 09233106BC6A; Wed, 1 Sep 2021 15:17:08 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A911660864; Wed, 1 Sep 2021 15:17:07 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 22/56] block: introduce blk_replace_bs Date: Wed, 1 Sep 2021 17:15:45 +0200 Message-Id: <20210901151619.689075-23-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Add function to change bs inside blk. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-3-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- include/sysemu/block-backend.h | 1 + block/block-backend.c | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h index 9ac5f7bbd3..29d4fdbf63 100644 --- a/include/sysemu/block-backend.h +++ b/include/sysemu/block-backend.h @@ -102,6 +102,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public); BlockDriverState *blk_bs(BlockBackend *blk); void blk_remove_bs(BlockBackend *blk); int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp); +int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp); bool bdrv_has_blk(BlockDriverState *bs); bool bdrv_is_root_node(BlockDriverState *bs); int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm, diff --git a/block/block-backend.c b/block/block-backend.c index deb55c272e..6140d133e2 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -869,6 +869,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp) return 0; } +/* + * Change BlockDriverState associated with @blk. + */ +int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp) +{ + return bdrv_replace_child_bs(blk->root, new_bs, errp); +} + /* * Sets the permission bitmasks that the user of the BlockBackend needs. */ From patchwork Wed Sep 1 15:15:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B36F7C432BE for ; Wed, 1 Sep 2021 15:27:53 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 62D8161059 for ; Wed, 1 Sep 2021 15:27:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 62D8161059 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:42430 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS9Y-0001ZL-G1 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:27:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49642) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzJ-0004xv-LP for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24657) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzG-0003gd-Gz for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509433; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XWDYnbKgonoREXH/0EMaG7/oPM54UHGH1QuXlWRfC8U=; b=PKfyVmwHH8PYRvxQxo31gSEChAF5FM7AGp1a1W/yncCVSXcFGYZI3rNYaUPaWGzxXpBS7F Uv13azBMoVRRWP1t9kiSSMu+zoeoEuAQuxbX7K4OuvExKIo5ULhEC74Eu5xwnyNT0/6Z24 +e+kecN05UcGHjek1RVjVz3S8xTIn/8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-256-68cURg9vNPusmnKUqpqliQ-1; Wed, 01 Sep 2021 11:17:12 -0400 X-MC-Unique: 68cURg9vNPusmnKUqpqliQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A0DEF87D545; Wed, 1 Sep 2021 15:17:11 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AFDD55C1BB; Wed, 1 Sep 2021 15:17:09 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 23/56] qdev-properties: PropertyInfo: add realized_set_allowed field Date: Wed, 1 Sep 2021 17:15:46 +0200 Message-Id: <20210901151619.689075-24-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Add field, so property can declare support for setting the property when device is realized. To be used in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-4-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- include/hw/qdev-properties.h | 1 + hw/core/qdev-properties.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h index 0ef97d60ce..f7925f67d0 100644 --- a/include/hw/qdev-properties.h +++ b/include/hw/qdev-properties.h @@ -32,6 +32,7 @@ struct PropertyInfo { const char *name; const char *description; const QEnumLookup *enum_table; + bool realized_set_allowed; /* allow setting property on realized device */ int (*print)(Object *obj, Property *prop, char *dest, size_t len); void (*set_default_value)(ObjectProperty *op, const Property *prop); ObjectProperty *(*create)(ObjectClass *oc, const char *name, diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c index 50f40949f5..c34aac6ebc 100644 --- a/hw/core/qdev-properties.c +++ b/hw/core/qdev-properties.c @@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name, /* returns: true if property is allowed to be set, false otherwise */ static bool qdev_prop_allow_set(Object *obj, const char *name, - Error **errp) + const PropertyInfo *info, Error **errp) { DeviceState *dev = DEVICE(obj); - if (dev->realized) { + if (dev->realized && !info->realized_set_allowed) { qdev_prop_set_after_realize(dev, name, errp); return false; } @@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name, { Property *prop = opaque; - if (!qdev_prop_allow_set(obj, name, errp)) { + if (!qdev_prop_allow_set(obj, name, prop->info, errp)) { return; } From patchwork Wed Sep 1 15:15:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FDA0C432BE for ; Wed, 1 Sep 2021 15:42:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D05660FDC for ; Wed, 1 Sep 2021 15:42:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0D05660FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:55420 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSNH-0004mZ-7c for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:42:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49684) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzL-00056z-TB for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59790) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzI-0003ie-SC for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ds/cH/bSkilGbLJqatLi75oQmmFFOHug94uAZG1ACDg=; b=eB7qbWnhcdX9LKJtiHvfNh+1DzVoo79ymClq2XocF3uQWLC5veVFMf1hBMu6knqtgpVH86 T8jK5HBQ6wrg7taXHXdjNDvVgxglMkrQAXp+pksIBvtlqh8MZvVJjGf0if3UqOZyJK8cVg UqLKZtddc2G82IB6RTVnaoA7JH2pBfI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-600-L5jkTyTgNASD5QiHvcvqQg-1; Wed, 01 Sep 2021 11:17:14 -0400 X-MC-Unique: L5jkTyTgNASD5QiHvcvqQg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EBA1F801A92; Wed, 1 Sep 2021 15:17:13 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 929205D9DC; Wed, 1 Sep 2021 15:17:13 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 24/56] qdev: allow setting drive property for realized device Date: Wed, 1 Sep 2021 17:15:47 +0200 Message-Id: <20210901151619.689075-25-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set. Intended usage: Assume there is a node A that is attached to some guest device. 1. blockdev-add to create a filter node B that has A as its child. 2. qom-set to change the node attached to the guest device’s BlockBackend from A to B. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-5-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- hw/core/qdev-properties-system.c | 43 +++++++++++++++++++++++--------- 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c index 2760c21f11..e71f5d64d1 100644 --- a/hw/core/qdev-properties-system.c +++ b/hw/core/qdev-properties-system.c @@ -36,11 +36,11 @@ static bool check_prop_still_unset(Object *obj, const char *name, const void *old_val, const char *new_val, - Error **errp) + bool allow_override, Error **errp) { const GlobalProperty *prop = qdev_find_global_prop(obj, name); - if (!old_val) { + if (!old_val || (!prop && allow_override)) { return true; } @@ -93,16 +93,34 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name, BlockBackend *blk; bool blk_created = false; int ret; + BlockDriverState *bs; + AioContext *ctx; if (!visit_type_str(v, name, &str, errp)) { return; } - /* - * TODO Should this really be an error? If no, the old value - * needs to be released before we store the new one. - */ - if (!check_prop_still_unset(obj, name, *ptr, str, errp)) { + if (!check_prop_still_unset(obj, name, *ptr, str, true, errp)) { + return; + } + + if (*ptr) { + /* BlockBackend alread exists. So, we want to change attached node */ + blk = *ptr; + ctx = blk_get_aio_context(blk); + bs = bdrv_lookup_bs(NULL, str, errp); + if (!bs) { + return; + } + + if (ctx != bdrv_get_aio_context(bs)) { + error_setg(errp, "Different aio context is not supported for new " + "node"); + } + + aio_context_acquire(ctx); + blk_replace_bs(blk, bs, errp); + aio_context_release(ctx); return; } @@ -114,7 +132,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name, blk = blk_by_name(str); if (!blk) { - BlockDriverState *bs = bdrv_lookup_bs(NULL, str, NULL); + bs = bdrv_lookup_bs(NULL, str, NULL); if (bs) { /* * If the device supports iothreads, it will make sure to move the @@ -123,8 +141,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name, * aware of iothreads require their BlockBackends to be in the main * AioContext. */ - AioContext *ctx = iothread ? bdrv_get_aio_context(bs) : - qemu_get_aio_context(); + ctx = iothread ? bdrv_get_aio_context(bs) : qemu_get_aio_context(); blk = blk_new(ctx, 0, BLK_PERM_ALL); blk_created = true; @@ -196,6 +213,7 @@ static void release_drive(Object *obj, const char *name, void *opaque) const PropertyInfo qdev_prop_drive = { .name = "str", .description = "Node name or ID of a block device to use as a backend", + .realized_set_allowed = true, .get = get_drive, .set = set_drive, .release = release_drive, @@ -204,6 +222,7 @@ const PropertyInfo qdev_prop_drive = { const PropertyInfo qdev_prop_drive_iothread = { .name = "str", .description = "Node name or ID of a block device to use as a backend", + .realized_set_allowed = true, .get = get_drive, .set = set_drive_iothread, .release = release_drive, @@ -238,7 +257,7 @@ static void set_chr(Object *obj, Visitor *v, const char *name, void *opaque, * TODO Should this really be an error? If no, the old value * needs to be released before we store the new one. */ - if (!check_prop_still_unset(obj, name, be->chr, str, errp)) { + if (!check_prop_still_unset(obj, name, be->chr, str, false, errp)) { return; } @@ -408,7 +427,7 @@ static void set_netdev(Object *obj, Visitor *v, const char *name, * TODO Should this really be an error? If no, the old value * needs to be released before we store the new one. */ - if (!check_prop_still_unset(obj, name, ncs[i], str, errp)) { + if (!check_prop_still_unset(obj, name, ncs[i], str, false, errp)) { goto out; } From patchwork Wed Sep 1 15:15:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79C1AC432BE for ; Wed, 1 Sep 2021 15:44:27 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F1E4B60F3A for ; Wed, 1 Sep 2021 15:44:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F1E4B60F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:35636 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSPa-00027d-2F for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:44:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49720) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzO-0005H9-SY for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:39780) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzL-0003l6-S4 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509439; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qsa9vVTNz8uOC+0s7PqS4ZC9kZqJlEP89F2LBajet8g=; b=an+vHe3hLFkxaiZ9VU8aDkhJqEw8Fr7ouqSBrhLfMNmB9SChHUHKzCdOuJLF1djxUcSJvz HwZlaOvgkb4wUQfXDpOn/HEhfi7GHJAwG23fWgqGPxw/pO67pTFKJ48+y//PSzHp0W3Jce zflYT2bBrrz5oNGdSN8uUOGwvWUgyGM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-182-5RDFsOGcOKyFYYOPD-PyEg-1; Wed, 01 Sep 2021 11:17:17 -0400 X-MC-Unique: 5RDFsOGcOKyFYYOPD-PyEg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4B209106BC6A; Wed, 1 Sep 2021 15:17:16 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 99DC46B541; Wed, 1 Sep 2021 15:17:15 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 25/56] block: rename backup-top to copy-before-write Date: Wed, 1 Sep 2021 17:15:48 +0200 Message-Id: <20210901151619.689075-26-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does". While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode. Note that this change may be considered as incompatible interface change, as backup-top filter format name was visible through query-block and query-named-block-nodes. Still, consider the following reasoning: 1. backup-top was never documented, so if someone depends on format name (for driver that can't be used other than it is automatically inserted on backup job start), it's a kind of "undocumented feature use". So I think we are free to change it. 2. There is a hope, that there is no such users: it's a lot more native to give a good node-name to backup-top filter if need to operate with it somehow, and don't touch format name. 3. Another "incompatible" change in further commit would be moving copy-before-write filter from using backing child to file child. And this is even more reasonable than renaming: for now all public filters are file-child based. So, it's a risky change, but risk seems small and good interface worth it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-6-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/{backup-top.h => copy-before-write.h} | 28 +++--- block/backup.c | 22 ++--- block/{backup-top.c => copy-before-write.c} | 100 ++++++++++---------- MAINTAINERS | 4 +- block/meson.build | 2 +- tests/qemu-iotests/283 | 35 +++---- tests/qemu-iotests/283.out | 4 +- 7 files changed, 95 insertions(+), 100 deletions(-) rename block/{backup-top.h => copy-before-write.h} (56%) rename block/{backup-top.c => copy-before-write.c} (62%) diff --git a/block/backup-top.h b/block/copy-before-write.h similarity index 56% rename from block/backup-top.h rename to block/copy-before-write.h index b28b0031c4..5977b7aa31 100644 --- a/block/backup-top.h +++ b/block/copy-before-write.h @@ -1,10 +1,10 @@ /* - * backup-top filter driver + * copy-before-write filter driver * * The driver performs Copy-Before-Write (CBW) operation: it is injected above * some node, and before each write it copies _old_ data to the target node. * - * Copyright (c) 2018-2019 Virtuozzo International GmbH. + * Copyright (c) 2018-2021 Virtuozzo International GmbH. * * Author: * Sementsov-Ogievskiy Vladimir @@ -23,20 +23,20 @@ * along with this program. If not, see . */ -#ifndef BACKUP_TOP_H -#define BACKUP_TOP_H +#ifndef COPY_BEFORE_WRITE_H +#define COPY_BEFORE_WRITE_H #include "block/block_int.h" #include "block/block-copy.h" -BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, - BlockDriverState *target, - const char *filter_node_name, - uint64_t cluster_size, - BackupPerf *perf, - BdrvRequestFlags write_flags, - BlockCopyState **bcs, - Error **errp); -void bdrv_backup_top_drop(BlockDriverState *bs); +BlockDriverState *bdrv_cbw_append(BlockDriverState *source, + BlockDriverState *target, + const char *filter_node_name, + uint64_t cluster_size, + BackupPerf *perf, + BdrvRequestFlags write_flags, + BlockCopyState **bcs, + Error **errp); +void bdrv_cbw_drop(BlockDriverState *bs); -#endif /* BACKUP_TOP_H */ +#endif /* COPY_BEFORE_WRITE_H */ diff --git a/block/backup.c b/block/backup.c index bd3614ce70..ac91821b08 100644 --- a/block/backup.c +++ b/block/backup.c @@ -27,13 +27,13 @@ #include "qemu/bitmap.h" #include "qemu/error-report.h" -#include "block/backup-top.h" +#include "block/copy-before-write.h" #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) typedef struct BackupBlockJob { BlockJob common; - BlockDriverState *backup_top; + BlockDriverState *cbw; BlockDriverState *source_bs; BlockDriverState *target_bs; @@ -104,7 +104,7 @@ static void backup_clean(Job *job) { BackupBlockJob *s = container_of(job, BackupBlockJob, common.job); block_job_remove_all_bdrv(&s->common); - bdrv_backup_top_drop(s->backup_top); + bdrv_cbw_drop(s->cbw); } void backup_do_checkpoint(BlockJob *job, Error **errp) @@ -408,7 +408,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, BackupBlockJob *job = NULL; int64_t cluster_size; BdrvRequestFlags write_flags; - BlockDriverState *backup_top = NULL; + BlockDriverState *cbw = NULL; BlockCopyState *bcs = NULL; assert(bs); @@ -521,22 +521,22 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) | (compress ? BDRV_REQ_WRITE_COMPRESSED : 0), - backup_top = bdrv_backup_top_append(bs, target, filter_node_name, + cbw = bdrv_cbw_append(bs, target, filter_node_name, cluster_size, perf, write_flags, &bcs, errp); - if (!backup_top) { + if (!cbw) { goto error; } /* job->len is fixed, so we can't allow resize */ - job = block_job_create(job_id, &backup_job_driver, txn, backup_top, + job = block_job_create(job_id, &backup_job_driver, txn, cbw, 0, BLK_PERM_ALL, speed, creation_flags, cb, opaque, errp); if (!job) { goto error; } - job->backup_top = backup_top; + job->cbw = cbw; job->source_bs = bs; job->target_bs = target; job->on_source_error = on_source_error; @@ -552,7 +552,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, block_copy_set_progress_meter(bcs, &job->common.job.progress); block_copy_set_speed(bcs, speed); - /* Required permissions are already taken by backup-top target */ + /* Required permissions are taken by copy-before-write filter target */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, &error_abort); @@ -562,8 +562,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, if (sync_bitmap) { bdrv_reclaim_dirty_bitmap(sync_bitmap, NULL); } - if (backup_top) { - bdrv_backup_top_drop(backup_top); + if (cbw) { + bdrv_cbw_drop(cbw); } return NULL; diff --git a/block/backup-top.c b/block/copy-before-write.c similarity index 62% rename from block/backup-top.c rename to block/copy-before-write.c index 425e3778be..0dc5a107cf 100644 --- a/block/backup-top.c +++ b/block/copy-before-write.c @@ -1,10 +1,10 @@ /* - * backup-top filter driver + * copy-before-write filter driver * * The driver performs Copy-Before-Write (CBW) operation: it is injected above * some node, and before each write it copies _old_ data to the target node. * - * Copyright (c) 2018-2019 Virtuozzo International GmbH. + * Copyright (c) 2018-2021 Virtuozzo International GmbH. * * Author: * Sementsov-Ogievskiy Vladimir @@ -32,25 +32,25 @@ #include "block/qdict.h" #include "block/block-copy.h" -#include "block/backup-top.h" +#include "block/copy-before-write.h" -typedef struct BDRVBackupTopState { +typedef struct BDRVCopyBeforeWriteState { BlockCopyState *bcs; BdrvChild *target; int64_t cluster_size; -} BDRVBackupTopState; +} BDRVCopyBeforeWriteState; -static coroutine_fn int backup_top_co_preadv( +static coroutine_fn int cbw_co_preadv( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags); } -static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset, - uint64_t bytes, BdrvRequestFlags flags) +static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, BdrvRequestFlags flags) { - BDRVBackupTopState *s = bs->opaque; + BDRVCopyBeforeWriteState *s = bs->opaque; uint64_t off, end; if (flags & BDRV_REQ_WRITE_UNCHANGED) { @@ -63,10 +63,10 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset, return block_copy(s->bcs, off, end - off, true); } -static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs, - int64_t offset, int bytes) +static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs, + int64_t offset, int bytes) { - int ret = backup_top_cbw(bs, offset, bytes, 0); + int ret = cbw_do_copy_before_write(bs, offset, bytes, 0); if (ret < 0) { return ret; } @@ -74,10 +74,10 @@ static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs, return bdrv_co_pdiscard(bs->backing, offset, bytes); } -static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs, +static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs, int64_t offset, int bytes, BdrvRequestFlags flags) { - int ret = backup_top_cbw(bs, offset, bytes, flags); + int ret = cbw_do_copy_before_write(bs, offset, bytes, flags); if (ret < 0) { return ret; } @@ -85,12 +85,12 @@ static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs, return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags); } -static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs, - uint64_t offset, - uint64_t bytes, - QEMUIOVector *qiov, int flags) +static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, int flags) { - int ret = backup_top_cbw(bs, offset, bytes, flags); + int ret = cbw_do_copy_before_write(bs, offset, bytes, flags); if (ret < 0) { return ret; } @@ -98,7 +98,7 @@ static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs, return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags); } -static int coroutine_fn backup_top_co_flush(BlockDriverState *bs) +static int coroutine_fn cbw_co_flush(BlockDriverState *bs) { if (!bs->backing) { return 0; @@ -107,7 +107,7 @@ static int coroutine_fn backup_top_co_flush(BlockDriverState *bs) return bdrv_co_flush(bs->backing->bs); } -static void backup_top_refresh_filename(BlockDriverState *bs) +static void cbw_refresh_filename(BlockDriverState *bs) { if (bs->backing == NULL) { /* @@ -120,11 +120,11 @@ static void backup_top_refresh_filename(BlockDriverState *bs) bs->backing->bs->filename); } -static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c, - BdrvChildRole role, - BlockReopenQueue *reopen_queue, - uint64_t perm, uint64_t shared, - uint64_t *nperm, uint64_t *nshared) +static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, + BdrvChildRole role, + BlockReopenQueue *reopen_queue, + uint64_t perm, uint64_t shared, + uint64_t *nperm, uint64_t *nshared) { if (!(role & BDRV_CHILD_FILTERED)) { /* @@ -149,41 +149,41 @@ static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c, } } -BlockDriver bdrv_backup_top_filter = { - .format_name = "backup-top", - .instance_size = sizeof(BDRVBackupTopState), +BlockDriver bdrv_cbw_filter = { + .format_name = "copy-before-write", + .instance_size = sizeof(BDRVCopyBeforeWriteState), - .bdrv_co_preadv = backup_top_co_preadv, - .bdrv_co_pwritev = backup_top_co_pwritev, - .bdrv_co_pwrite_zeroes = backup_top_co_pwrite_zeroes, - .bdrv_co_pdiscard = backup_top_co_pdiscard, - .bdrv_co_flush = backup_top_co_flush, + .bdrv_co_preadv = cbw_co_preadv, + .bdrv_co_pwritev = cbw_co_pwritev, + .bdrv_co_pwrite_zeroes = cbw_co_pwrite_zeroes, + .bdrv_co_pdiscard = cbw_co_pdiscard, + .bdrv_co_flush = cbw_co_flush, - .bdrv_refresh_filename = backup_top_refresh_filename, + .bdrv_refresh_filename = cbw_refresh_filename, - .bdrv_child_perm = backup_top_child_perm, + .bdrv_child_perm = cbw_child_perm, .is_filter = true, }; -BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, - BlockDriverState *target, - const char *filter_node_name, - uint64_t cluster_size, - BackupPerf *perf, - BdrvRequestFlags write_flags, - BlockCopyState **bcs, - Error **errp) +BlockDriverState *bdrv_cbw_append(BlockDriverState *source, + BlockDriverState *target, + const char *filter_node_name, + uint64_t cluster_size, + BackupPerf *perf, + BdrvRequestFlags write_flags, + BlockCopyState **bcs, + Error **errp) { ERRP_GUARD(); int ret; - BDRVBackupTopState *state; + BDRVCopyBeforeWriteState *state; BlockDriverState *top; bool appended = false; assert(source->total_sectors == target->total_sectors); - top = bdrv_new_open_driver(&bdrv_backup_top_filter, filter_node_name, + top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name, BDRV_O_RDWR, errp); if (!top) { return NULL; @@ -210,7 +210,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, ret = bdrv_append(top, source, errp); if (ret < 0) { - error_prepend(errp, "Cannot append backup-top filter: "); + error_prepend(errp, "Cannot append copy-before-write filter: "); goto fail; } appended = true; @@ -231,7 +231,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, fail: if (appended) { - bdrv_backup_top_drop(top); + bdrv_cbw_drop(top); } else { bdrv_unref(top); } @@ -241,9 +241,9 @@ fail: return NULL; } -void bdrv_backup_top_drop(BlockDriverState *bs) +void bdrv_cbw_drop(BlockDriverState *bs) { - BDRVBackupTopState *s = bs->opaque; + BDRVCopyBeforeWriteState *s = bs->opaque; bdrv_drop_filter(bs, &error_abort); diff --git a/MAINTAINERS b/MAINTAINERS index dffcb651f4..973cecc70e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2342,8 +2342,8 @@ F: block/mirror.c F: qapi/job.json F: block/block-copy.c F: include/block/block-copy.c -F: block/backup-top.h -F: block/backup-top.c +F: block/copy-before-write.h +F: block/copy-before-write.c F: include/block/aio_task.h F: block/aio_task.c F: util/qemu-co-shared-resource.c diff --git a/block/meson.build b/block/meson.build index 0450914c7a..66ee11e62c 100644 --- a/block/meson.build +++ b/block/meson.build @@ -4,7 +4,7 @@ block_ss.add(files( 'aio_task.c', 'amend.c', 'backup.c', - 'backup-top.c', + 'copy-before-write.c', 'blkdebug.c', 'blklogwrites.c', 'blkverify.c', diff --git a/tests/qemu-iotests/283 b/tests/qemu-iotests/283 index 010c22f0a2..a09e0183ae 100755 --- a/tests/qemu-iotests/283 +++ b/tests/qemu-iotests/283 @@ -1,7 +1,7 @@ #!/usr/bin/env python3 # group: auto quick # -# Test for backup-top filter permission activation failure +# Test for copy-before-write filter permission conflict # # Copyright (c) 2019 Virtuozzo International GmbH. # @@ -31,13 +31,13 @@ size = 1024 * 1024 """ Test description When performing a backup, all writes on the source subtree must go through the -backup-top filter so it can copy all data to the target before it is changed. -backup-top filter is appended above source node, to achieve this thing, so all -parents of source node are handled. A configuration with side parents of source -sub-tree with write permission is unsupported (we'd have append several -backup-top filter like nodes to handle such parents). The test create an -example of such configuration and checks that a backup is then not allowed -(blockdev-backup command should fail). +copy-before-write filter so it can copy all data to the target before it is +changed. copy-before-write filter is appended above source node, to achieve +this thing, so all parents of source node are handled. A configuration with +side parents of source sub-tree with write permission is unsupported (we'd have +append several copy-before-write filter like nodes to handle such parents). The +test create an example of such configuration and checks that a backup is then +not allowed (blockdev-backup command should fail). The configuration: @@ -57,11 +57,10 @@ The configuration: │ base │ ◀──────────── │ other │ └─────────────┘ └───────┘ -On activation (see .active field of backup-top state in block/backup-top.c), -backup-top is going to unshare write permission on its source child. Write -unsharing will be propagated to the "source->base" link and will conflict with -other node write permission. So permission update will fail and backup job will -not be started. +copy-before-write filter wants to unshare write permission on its source child. +Write unsharing will be propagated to the "source->base" link and will conflict +with other node write permission. So permission update will fail and backup job +will not be started. Note, that the only thing which prevents backup of running on such configuration is default permission propagation scheme. It may be altered by @@ -99,13 +98,9 @@ vm.qmp_log('blockdev-backup', sync='full', device='source', target='target') vm.shutdown() -print('\n=== backup-top should be gone after job-finalize ===\n') +print('\n=== copy-before-write filter should be gone after job-finalize ===\n') -# Check that the backup-top node is gone after job-finalize. -# -# During finalization, the node becomes inactive and can no longer -# function. If it is still present, new parents might be attached, and -# there would be no meaningful way to handle their I/O requests. +# Check that the copy-before-write node is gone after job-finalize. vm = iotests.VM() vm.launch() @@ -131,7 +126,7 @@ vm.qmp_log('blockdev-backup', vm.event_wait('BLOCK_JOB_PENDING', 5.0) -# The backup-top filter should still be present prior to finalization +# The copy-before-write filter should still be present prior to finalization assert vm.node_info('backup-filter') is not None vm.qmp_log('job-finalize', id='backup') diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out index c6e12b15c5..f2b7219632 100644 --- a/tests/qemu-iotests/283.out +++ b/tests/qemu-iotests/283.out @@ -5,9 +5,9 @@ {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}} {"return": {}} {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}} -{"error": {"class": "GenericError", "desc": "Cannot append backup-top filter: Permission conflict on node 'base': permissions 'write' are both required by node 'other' (uses node 'base' as 'image' child) and unshared by node 'source' (uses node 'base' as 'image' child)."}} +{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Permission conflict on node 'base': permissions 'write' are both required by node 'other' (uses node 'base' as 'image' child) and unshared by node 'source' (uses node 'base' as 'image' child)."}} -=== backup-top should be gone after job-finalize === +=== copy-before-write filter should be gone after job-finalize === {"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name": "source"}} {"return": {}} From patchwork Wed Sep 1 15:15:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AA1CC432BE for ; Wed, 1 Sep 2021 15:42:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 051A760F3A for ; Wed, 1 Sep 2021 15:42:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 051A760F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56530 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSNX-0005Wg-Sr for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:42:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49750) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzQ-0005MB-BG for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33140) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzO-0003mM-1m for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pA8KdcqXP2kmo4Kd3McR9IkUAKKssIl/xZYVTCDPy1Y=; b=UTvZkj+CjSmrgRN6C8RSpwf9JCmAcC2QcMIPkkVhQQYUoFRXTTcYmysCXLi8diwPs/gINu d/Adi2yrGiJRFqbepZvR+fwZEQFMbet22KK2i+GpSwmRF9uX7UL1FlzJ3sTtC2bFOD6ngN vTp92ozd0emD2jDGIhjArfcyGhT3/E4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-542-14o84Fn_P1u9VL8XUfOfvA-1; Wed, 01 Sep 2021 11:17:19 -0400 X-MC-Unique: 14o84Fn_P1u9VL8XUfOfvA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70451801A92; Wed, 1 Sep 2021 15:17:18 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E88236A05B; Wed, 1 Sep 2021 15:17:17 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 26/56] block-copy: move detecting fleecing scheme to block-copy Date: Wed, 1 Sep 2021 17:15:49 +0200 Message-Id: <20210901151619.689075-27-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We want to simplify initialization interface of copy-before-write filter as we are going to make it public. So, let's detect fleecing scheme exactly in block-copy code, to not pass this information through extra levels. Why not just set BDRV_REQ_SERIALISING unconditionally: because we are going to implement new more efficient fleecing scheme which will not rely on backing feature. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Reitz Message-Id: <20210824083856.17408-7-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.h | 2 +- include/block/block-copy.h | 3 +-- block/backup.c | 21 +-------------------- block/block-copy.c | 24 +++++++++++++++++++++--- block/copy-before-write.c | 4 ++-- 5 files changed, 26 insertions(+), 28 deletions(-) diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 5977b7aa31..f37e2249ae 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -34,7 +34,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, const char *filter_node_name, uint64_t cluster_size, BackupPerf *perf, - BdrvRequestFlags write_flags, + bool compress, BlockCopyState **bcs, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 5c8278895c..734389d32a 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -26,8 +26,7 @@ typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, bool use_copy_range, - BdrvRequestFlags write_flags, - Error **errp); + bool compress, Error **errp); void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm); diff --git a/block/backup.c b/block/backup.c index ac91821b08..84f9a5f02c 100644 --- a/block/backup.c +++ b/block/backup.c @@ -407,7 +407,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, int64_t len, target_len; BackupBlockJob *job = NULL; int64_t cluster_size; - BdrvRequestFlags write_flags; BlockDriverState *cbw = NULL; BlockCopyState *bcs = NULL; @@ -504,26 +503,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } - /* - * If source is in backing chain of target assume that target is going to be - * used for "image fleecing", i.e. it should represent a kind of snapshot of - * source at backup-start point in time. And target is going to be read by - * somebody (for example, used as NBD export) during backup job. - * - * In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid - * intersection of backup writes and third party reads from target, - * otherwise reading from target we may occasionally read already updated by - * guest data. - * - * For more information see commit f8d59dfb40bb and test - * tests/qemu-iotests/222 - */ - write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) | - (compress ? BDRV_REQ_WRITE_COMPRESSED : 0), - cbw = bdrv_cbw_append(bs, target, filter_node_name, - cluster_size, perf, - write_flags, &bcs, errp); + cluster_size, perf, compress, &bcs, errp); if (!cbw) { goto error; } diff --git a/block/block-copy.c b/block/block-copy.c index 0becad52da..58b4345a5a 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -317,10 +317,11 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target) BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, bool use_copy_range, - BdrvRequestFlags write_flags, Error **errp) + bool compress, Error **errp) { BlockCopyState *s; BdrvDirtyBitmap *copy_bitmap; + bool is_fleecing; copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL, errp); @@ -329,6 +330,22 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, } bdrv_disable_dirty_bitmap(copy_bitmap); + /* + * If source is in backing chain of target assume that target is going to be + * used for "image fleecing", i.e. it should represent a kind of snapshot of + * source at backup-start point in time. And target is going to be read by + * somebody (for example, used as NBD export) during backup job. + * + * In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid + * intersection of backup writes and third party reads from target, + * otherwise reading from target we may occasionally read already updated by + * guest data. + * + * For more information see commit f8d59dfb40bb and test + * tests/qemu-iotests/222 + */ + is_fleecing = bdrv_chain_contains(target->bs, source->bs); + s = g_new(BlockCopyState, 1); *s = (BlockCopyState) { .source = source, @@ -336,7 +353,8 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, .copy_bitmap = copy_bitmap, .cluster_size = cluster_size, .len = bdrv_dirty_bitmap_size(copy_bitmap), - .write_flags = write_flags, + .write_flags = (is_fleecing ? BDRV_REQ_SERIALISING : 0) | + (compress ? BDRV_REQ_WRITE_COMPRESSED : 0), .mem = shres_create(BLOCK_COPY_MAX_MEM), .max_transfer = QEMU_ALIGN_DOWN( block_copy_max_transfer(source, target), @@ -351,7 +369,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, * behalf). */ s->method = COPY_READ_WRITE_CLUSTER; - } else if (write_flags & BDRV_REQ_WRITE_COMPRESSED) { + } else if (compress) { /* Compression supports only cluster-size writes and no copy-range. */ s->method = COPY_READ_WRITE_CLUSTER; } else { diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 0dc5a107cf..4337076c1c 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -171,7 +171,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, const char *filter_node_name, uint64_t cluster_size, BackupPerf *perf, - BdrvRequestFlags write_flags, + bool compress, BlockCopyState **bcs, Error **errp) { @@ -218,7 +218,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->cluster_size = cluster_size; state->bcs = block_copy_state_new(top->backing, state->target, cluster_size, perf->use_copy_range, - write_flags, errp); + compress, errp); if (!state->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); goto fail; From patchwork Wed Sep 1 15:15:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74E7CC432BE for ; Wed, 1 Sep 2021 15:31:13 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EAE9E601FF for ; Wed, 1 Sep 2021 15:31:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org EAE9E601FF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:51190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSCm-0007MF-5M for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:31:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzR-0005R3-K7 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28146) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzP-0003o0-KN for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ptmW75KyIzcJGjJ/kFpJ8re3VNKXO76q5hbA08JudWY=; b=EJgwC63z3GzfFZhBYyQ+HsMC65WxnYsVluJuBjhZmpSiUObaFifJDrIYI08f4qolm/b5dN XM1+CMIhpP0o5AWS2YJ73P/Qz5LlWhJT/U8+/IKWWwGXNPHUOuXpFCk9hfXA2r7l4yW7by Gtl2xK6zIir5Reb1tiB3q8afwD1Xfvs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-283-PP8YfaIFPqyhL1xZ0Lp3Wg-1; Wed, 01 Sep 2021 11:17:21 -0400 X-MC-Unique: PP8YfaIFPqyhL1xZ0Lp3Wg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 983B410053E7; Wed, 1 Sep 2021 15:17:20 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4161A60C9F; Wed, 1 Sep 2021 15:17:20 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 27/56] block/block-copy: introduce block_copy_set_copy_opts() Date: Wed, 1 Sep 2021 17:15:50 +0200 Message-Id: <20210901151619.689075-28-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We'll need a possibility to set compress and use_copy_range options after initialization of the state. So make corresponding part of block_copy_state_new() separate and public. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20210824083856.17408-8-vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz Signed-off-by: Hanna Reitz --- include/block/block-copy.h | 3 +++ block/block-copy.c | 49 ++++++++++++++++++++++---------------- 2 files changed, 32 insertions(+), 20 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 734389d32a..dca6c4ce36 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -28,6 +28,9 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, bool use_copy_range, bool compress, Error **errp); +/* Function should be called prior any actual copy request */ +void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range, + bool compress); void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm); void block_copy_state_free(BlockCopyState *s); diff --git a/block/block-copy.c b/block/block-copy.c index 58b4345a5a..e83870ff81 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -315,6 +315,33 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target) target->bs->bl.max_transfer)); } +void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range, + bool compress) +{ + /* Keep BDRV_REQ_SERIALISING set (or not set) in block_copy_state_new() */ + s->write_flags = (s->write_flags & BDRV_REQ_SERIALISING) | + (compress ? BDRV_REQ_WRITE_COMPRESSED : 0); + + if (s->max_transfer < s->cluster_size) { + /* + * copy_range does not respect max_transfer. We don't want to bother + * with requests smaller than block-copy cluster size, so fallback to + * buffered copying (read and write respect max_transfer on their + * behalf). + */ + s->method = COPY_READ_WRITE_CLUSTER; + } else if (compress) { + /* Compression supports only cluster-size writes and no copy-range. */ + s->method = COPY_READ_WRITE_CLUSTER; + } else { + /* + * If copy range enabled, start with COPY_RANGE_SMALL, until first + * successful copy_range (look at block_copy_do_copy). + */ + s->method = use_copy_range ? COPY_RANGE_SMALL : COPY_READ_WRITE; + } +} + BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, bool use_copy_range, bool compress, Error **errp) @@ -353,32 +380,14 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, .copy_bitmap = copy_bitmap, .cluster_size = cluster_size, .len = bdrv_dirty_bitmap_size(copy_bitmap), - .write_flags = (is_fleecing ? BDRV_REQ_SERIALISING : 0) | - (compress ? BDRV_REQ_WRITE_COMPRESSED : 0), + .write_flags = (is_fleecing ? BDRV_REQ_SERIALISING : 0), .mem = shres_create(BLOCK_COPY_MAX_MEM), .max_transfer = QEMU_ALIGN_DOWN( block_copy_max_transfer(source, target), cluster_size), }; - if (s->max_transfer < cluster_size) { - /* - * copy_range does not respect max_transfer. We don't want to bother - * with requests smaller than block-copy cluster size, so fallback to - * buffered copying (read and write respect max_transfer on their - * behalf). - */ - s->method = COPY_READ_WRITE_CLUSTER; - } else if (compress) { - /* Compression supports only cluster-size writes and no copy-range. */ - s->method = COPY_READ_WRITE_CLUSTER; - } else { - /* - * If copy range enabled, start with COPY_RANGE_SMALL, until first - * successful copy_range (look at block_copy_do_copy). - */ - s->method = use_copy_range ? COPY_RANGE_SMALL : COPY_READ_WRITE; - } + block_copy_set_copy_opts(s, use_copy_range, compress); ratelimit_init(&s->rate_limit); qemu_co_mutex_init(&s->lock); From patchwork Wed Sep 1 15:15:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BDD0C432BE for ; Wed, 1 Sep 2021 15:28:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1E4261059 for ; Wed, 1 Sep 2021 15:28:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B1E4261059 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:43656 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLS9l-0002N4-Td for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:28:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49810) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzT-0005Th-3F for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31779) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzR-0003pj-DM for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509444; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I132Md6AGfhZspt6ikCA4tUZKfOj7M6peueHsyGzFvQ=; b=B21Ns8KAT4HCRkqglJrhUWNfiMpnlIbfRZR8u7kEYAtK0iGsEy+UQEMFmB82uDNtCGpsLg StgAQbYhPQ5Ds0uv9K+Vt7xXxfAiLEsvPtp2t22Tu6d17VmEeNar+Bh5ir7B6A8fwZvPQX GkYGUPmqA0nhmoVxdWEfTvrIu6LBT04= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-96-Bxq2NmWKNuqqp3oLsjuipw-1; Wed, 01 Sep 2021 11:17:23 -0400 X-MC-Unique: Bxq2NmWKNuqqp3oLsjuipw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8F47A1B2C986; Wed, 1 Sep 2021 15:17:22 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3712E69CBC; Wed, 1 Sep 2021 15:17:22 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 28/56] block/backup: set copy_range and compress after filter insertion Date: Wed, 1 Sep 2021 17:15:51 +0200 Message-Id: <20210901151619.689075-29-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to publish copy-before-write filter, so it would be initialized through options. Still we don't want to publish compress and copy-range options, as 1. Modern way to enable compression is to use compress filter. 2. For copy-range it's unclean how to make proper interface: - it's has experimental prefix for backup job anyway - the whole BackupPerf structure doesn't make sense for the filter So, let's just add copy-range possibility to the filter later if needed. Still, we are going to continue support for compression and experimental copy-range in backup job. So, set these options after filter creation. Note, that we can drop "compress" argument of bdrv_cbw_append() now, as well as "perf". The only reason not doing so is that now, when I prepare this patch the big series around it is already reviewed and I want to avoid extra rebase conflicts to simplify review of the following version. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Reitz Message-Id: <20210824083856.17408-9-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.h | 1 - block/backup.c | 3 ++- block/copy-before-write.c | 4 +--- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/block/copy-before-write.h b/block/copy-before-write.h index f37e2249ae..538aab8bdb 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -33,7 +33,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, - BackupPerf *perf, bool compress, BlockCopyState **bcs, Error **errp); diff --git a/block/backup.c b/block/backup.c index 84f9a5f02c..b31fd99ab3 100644 --- a/block/backup.c +++ b/block/backup.c @@ -504,7 +504,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, } cbw = bdrv_cbw_append(bs, target, filter_node_name, - cluster_size, perf, compress, &bcs, errp); + cluster_size, false, &bcs, errp); if (!cbw) { goto error; } @@ -530,6 +530,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, job->len = len; job->perf = *perf; + block_copy_set_copy_opts(bcs, perf->use_copy_range, compress); block_copy_set_progress_meter(bcs, &job->common.job.progress); block_copy_set_speed(bcs, speed); diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 4337076c1c..235251a640 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -170,7 +170,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, - BackupPerf *perf, bool compress, BlockCopyState **bcs, Error **errp) @@ -217,8 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->cluster_size = cluster_size; state->bcs = block_copy_state_new(top->backing, state->target, - cluster_size, perf->use_copy_range, - compress, errp); + cluster_size, false, compress, errp); if (!state->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); goto fail; From patchwork Wed Sep 1 15:15:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EE95C432BE for ; Wed, 1 Sep 2021 15:49:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 19C926056C for ; Wed, 1 Sep 2021 15:49:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 19C926056C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:48530 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSUO-0002Yt-6w for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:49:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49860) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzW-0005Z2-OV for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42492) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzU-0003ry-1w for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509447; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XvlOWq+VufylovclDus/sYZnt3plpncNQIyQq6RZ4h4=; b=GeFgcz0dhsqCs9lK33VXTLRRG9lkHefmsME5gt3isN32nOclj+jYGT7/JED0OToJIP/lCe Hb7aKuzmyfrt26+9RptAbiqHEXuLLEb+2HjGXw4TrrldVkxSaXrHMBLDZJDqiYcl5z+u0B WqAqqotzK9XpQNV6i1HqXvvNhKVFB/o= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-287--keQTqelPjuCBM6jvKOfkA-1; Wed, 01 Sep 2021 11:17:25 -0400 X-MC-Unique: -keQTqelPjuCBM6jvKOfkA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C5C241B2C981; Wed, 1 Sep 2021 15:17:24 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3D4055D9DC; Wed, 1 Sep 2021 15:17:24 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 29/56] block/backup: move cluster size calculation to block-copy Date: Wed, 1 Sep 2021 17:15:52 +0200 Message-Id: <20210901151619.689075-30-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy The main consumer of cluster-size is block-copy. Let's calculate it here instead of passing through backup-top. We are going to publish copy-before-write filter soon, so it will be created through options. But we don't want for now to make explicit option for cluster-size, let's continue to calculate it automatically. So, now is the time to get rid of cluster_size argument for bdrv_cbw_append(). Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-10-vsementsov@virtuozzo.com> [hreitz: Add qemu/error-report.h include to block/block-copy.c] Signed-off-by: Hanna Reitz --- block/copy-before-write.h | 1 - include/block/block-copy.h | 5 +-- block/backup.c | 62 ++++++-------------------------------- block/block-copy.c | 52 +++++++++++++++++++++++++++++++- block/copy-before-write.c | 10 +++--- 5 files changed, 67 insertions(+), 63 deletions(-) diff --git a/block/copy-before-write.h b/block/copy-before-write.h index 538aab8bdb..b386fd8f01 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -32,7 +32,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, - uint64_t cluster_size, bool compress, BlockCopyState **bcs, Error **errp); diff --git a/include/block/block-copy.h b/include/block/block-copy.h index dca6c4ce36..b8a2d63545 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -25,8 +25,8 @@ typedef struct BlockCopyState BlockCopyState; typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, - int64_t cluster_size, bool use_copy_range, - bool compress, Error **errp); + bool use_copy_range, bool compress, + Error **errp); /* Function should be called prior any actual copy request */ void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range, @@ -91,6 +91,7 @@ void block_copy_kick(BlockCopyCallState *call_state); void block_copy_call_cancel(BlockCopyCallState *call_state); BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); +int64_t block_copy_cluster_size(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); #endif /* BLOCK_COPY_H */ diff --git a/block/backup.c b/block/backup.c index b31fd99ab3..83516297cb 100644 --- a/block/backup.c +++ b/block/backup.c @@ -29,8 +29,6 @@ #include "block/copy-before-write.h" -#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) - typedef struct BackupBlockJob { BlockJob common; BlockDriverState *cbw; @@ -354,43 +352,6 @@ static const BlockJobDriver backup_job_driver = { .set_speed = backup_set_speed, }; -static int64_t backup_calculate_cluster_size(BlockDriverState *target, - Error **errp) -{ - int ret; - BlockDriverInfo bdi; - bool target_does_cow = bdrv_backing_chain_next(target); - - /* - * If there is no backing file on the target, we cannot rely on COW if our - * backup cluster size is smaller than the target cluster size. Even for - * targets with a backing file, try to avoid COW if possible. - */ - ret = bdrv_get_info(target, &bdi); - if (ret == -ENOTSUP && !target_does_cow) { - /* Cluster size is not defined */ - warn_report("The target block device doesn't provide " - "information about the block size and it doesn't have a " - "backing file. The default block size of %u bytes is " - "used. If the actual block size of the target exceeds " - "this default, the backup may be unusable", - BACKUP_CLUSTER_SIZE_DEFAULT); - return BACKUP_CLUSTER_SIZE_DEFAULT; - } else if (ret < 0 && !target_does_cow) { - error_setg_errno(errp, -ret, - "Couldn't determine the cluster size of the target image, " - "which has no backing file"); - error_append_hint(errp, - "Aborting, since this may create an unusable destination image\n"); - return ret; - } else if (ret < 0 && target_does_cow) { - /* Not fatal; just trudge on ahead. */ - return BACKUP_CLUSTER_SIZE_DEFAULT; - } - - return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); -} - BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, BlockDriverState *target, int64_t speed, MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap, @@ -448,11 +409,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, return NULL; } - cluster_size = backup_calculate_cluster_size(target, errp); - if (cluster_size < 0) { - goto error; - } - if (perf->max_workers < 1) { error_setg(errp, "max-workers must be greater than zero"); return NULL; @@ -464,13 +420,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, return NULL; } - if (perf->max_chunk && perf->max_chunk < cluster_size) { - error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup " - "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size); - return NULL; - } - - if (sync_bitmap) { /* If we need to write to this bitmap, check that we can: */ if (bitmap_mode != BITMAP_SYNC_MODE_NEVER && @@ -503,12 +452,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } - cbw = bdrv_cbw_append(bs, target, filter_node_name, - cluster_size, false, &bcs, errp); + cbw = bdrv_cbw_append(bs, target, filter_node_name, false, &bcs, errp); if (!cbw) { goto error; } + cluster_size = block_copy_cluster_size(bcs); + + if (perf->max_chunk && perf->max_chunk < cluster_size) { + error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup " + "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size); + goto error; + } + /* job->len is fixed, so we can't allow resize */ job = block_job_create(job_id, &backup_job_driver, txn, cbw, 0, BLK_PERM_ALL, diff --git a/block/block-copy.c b/block/block-copy.c index e83870ff81..5d0076ac93 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -21,12 +21,14 @@ #include "qemu/units.h" #include "qemu/coroutine.h" #include "block/aio_task.h" +#include "qemu/error-report.h" #define BLOCK_COPY_MAX_COPY_RANGE (16 * MiB) #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) #define BLOCK_COPY_MAX_WORKERS 64 #define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */ +#define BLOCK_COPY_CLUSTER_SIZE_DEFAULT (1 << 16) typedef enum { COPY_READ_WRITE_CLUSTER, @@ -342,14 +344,57 @@ void block_copy_set_copy_opts(BlockCopyState *s, bool use_copy_range, } } +static int64_t block_copy_calculate_cluster_size(BlockDriverState *target, + Error **errp) +{ + int ret; + BlockDriverInfo bdi; + bool target_does_cow = bdrv_backing_chain_next(target); + + /* + * If there is no backing file on the target, we cannot rely on COW if our + * backup cluster size is smaller than the target cluster size. Even for + * targets with a backing file, try to avoid COW if possible. + */ + ret = bdrv_get_info(target, &bdi); + if (ret == -ENOTSUP && !target_does_cow) { + /* Cluster size is not defined */ + warn_report("The target block device doesn't provide " + "information about the block size and it doesn't have a " + "backing file. The default block size of %u bytes is " + "used. If the actual block size of the target exceeds " + "this default, the backup may be unusable", + BLOCK_COPY_CLUSTER_SIZE_DEFAULT); + return BLOCK_COPY_CLUSTER_SIZE_DEFAULT; + } else if (ret < 0 && !target_does_cow) { + error_setg_errno(errp, -ret, + "Couldn't determine the cluster size of the target image, " + "which has no backing file"); + error_append_hint(errp, + "Aborting, since this may create an unusable destination image\n"); + return ret; + } else if (ret < 0 && target_does_cow) { + /* Not fatal; just trudge on ahead. */ + return BLOCK_COPY_CLUSTER_SIZE_DEFAULT; + } + + return MAX(BLOCK_COPY_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); +} + BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, - int64_t cluster_size, bool use_copy_range, + bool use_copy_range, bool compress, Error **errp) { BlockCopyState *s; + int64_t cluster_size; BdrvDirtyBitmap *copy_bitmap; bool is_fleecing; + cluster_size = block_copy_calculate_cluster_size(target->bs, errp); + if (cluster_size < 0) { + return NULL; + } + copy_bitmap = bdrv_create_dirty_bitmap(source->bs, cluster_size, NULL, errp); if (!copy_bitmap) { @@ -960,6 +1005,11 @@ BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) return s->copy_bitmap; } +int64_t block_copy_cluster_size(BlockCopyState *s) +{ + return s->cluster_size; +} + void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip) { qatomic_set(&s->skip_unallocated, skip); diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 235251a640..a7996d54db 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -37,7 +37,6 @@ typedef struct BDRVCopyBeforeWriteState { BlockCopyState *bcs; BdrvChild *target; - int64_t cluster_size; } BDRVCopyBeforeWriteState; static coroutine_fn int cbw_co_preadv( @@ -52,13 +51,14 @@ static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs, { BDRVCopyBeforeWriteState *s = bs->opaque; uint64_t off, end; + int64_t cluster_size = block_copy_cluster_size(s->bcs); if (flags & BDRV_REQ_WRITE_UNCHANGED) { return 0; } - off = QEMU_ALIGN_DOWN(offset, s->cluster_size); - end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size); + off = QEMU_ALIGN_DOWN(offset, cluster_size); + end = QEMU_ALIGN_UP(offset + bytes, cluster_size); return block_copy(s->bcs, off, end - off, true); } @@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = { BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, - uint64_t cluster_size, bool compress, BlockCopyState **bcs, Error **errp) @@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } appended = true; - state->cluster_size = cluster_size; state->bcs = block_copy_state_new(top->backing, state->target, - cluster_size, false, compress, errp); + false, compress, errp); if (!state->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); goto fail; From patchwork Wed Sep 1 15:15:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EA83C432BE for ; Wed, 1 Sep 2021 15:35:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 42C9260231 for ; Wed, 1 Sep 2021 15:35:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 42C9260231 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:33616 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSGh-0006Oj-EM for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49894) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzZ-0005a0-AD for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22673) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzX-0003v2-Rw for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509451; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mKztRHW663aUlIh2EJJMrzrttdTTCRai5eK7RpunmXE=; b=Ejy3i26wizJIDzsgg7rlyjf4Ofi5ue1Ghie5XBMEruoYfK9+K5L4f7APuZSVtrtGQFsZoT jk8g8Q3GSgZHtGfhDND0VLDfLYY1aIt6ly6K5ZleNc7f7ITaOXMKpEW89NoA+jHNxxK+Bi f30j9EXElMDVwegw0zBYIk9W937EREs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-5-eO8qske2NWuOsRbGTEwpQw-1; Wed, 01 Sep 2021 11:17:28 -0400 X-MC-Unique: eO8qske2NWuOsRbGTEwpQw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EFC81801B3D; Wed, 1 Sep 2021 15:17:26 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9472D6788D; Wed, 1 Sep 2021 15:17:26 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 30/56] block/copy-before-write: relax permission requirements when no parents Date: Wed, 1 Sep 2021 17:15:53 +0200 Message-Id: <20210901151619.689075-31-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed. Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent. Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-11-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 8 +++++--- tests/qemu-iotests/283.out | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index a7996d54db..2a51cc64e4 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared); - if (perm & BLK_PERM_WRITE) { - *nperm = *nperm | BLK_PERM_CONSISTENT_READ; + if (!QLIST_EMPTY(&bs->parents)) { + if (perm & BLK_PERM_WRITE) { + *nperm = *nperm | BLK_PERM_CONSISTENT_READ; + } + *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } - *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } } diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out index f2b7219632..5bb75952ef 100644 --- a/tests/qemu-iotests/283.out +++ b/tests/qemu-iotests/283.out @@ -5,7 +5,7 @@ {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}} {"return": {}} {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}} -{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Permission conflict on node 'base': permissions 'write' are both required by node 'other' (uses node 'base' as 'image' child) and unshared by node 'source' (uses node 'base' as 'image' child)."}} +{"error": {"class": "GenericError", "desc": "Permission conflict on node 'base': permissions 'write' are both required by node 'other' (uses node 'base' as 'image' child) and unshared by node 'source' (uses node 'base' as 'image' child)."}} === copy-before-write filter should be gone after job-finalize === From patchwork Wed Sep 1 15:15:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84B7CC432BE for ; Wed, 1 Sep 2021 15:53:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 244CD60F3A for ; Wed, 1 Sep 2021 15:53:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 244CD60F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:58554 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSYF-0000wv-8i for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:53:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49914) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzd-0005a7-CZ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45832) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzX-0003vI-VY for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509451; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UnRxZ0EvCWF/R05qBnIV+U/5Ek8nQK1L8oPU8Cstgs0=; b=hUFiYMX/fCfXFyaApWfXa3r5v3/iLgRA6l1QQmAVm+9GNUMTU0AOm38/6k+CHHF+/N+rPD YLsSvEhDeY/6FETERWm3Qo2lItKTz3fzJ9cwCZGF0Yrw2SEpVQh4UN6gYIMdm7nPKykzgU Nd8f/dI64LPKh2jBnIuiSfYHsV4eH3k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-70-PhM8wleLMAmD1GygQEt--g-1; Wed, 01 Sep 2021 11:17:29 -0400 X-MC-Unique: PhM8wleLMAmD1GygQEt--g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F0CBF1B2C980; Wed, 1 Sep 2021 15:17:28 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 996E7380; Wed, 1 Sep 2021 15:17:28 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 31/56] block/copy-before-write: drop extra bdrv_unref on failure path Date: Wed, 1 Sep 2021 17:15:54 +0200 Message-Id: <20210901151619.689075-32-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy bdrv_attach_child() do bdrv_unref() on failure, so we shouldn't do it by hand here. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-12-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 2a51cc64e4..945d9340f4 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -201,7 +201,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->target = bdrv_attach_child(top, target, "target", &child_of_bds, BDRV_CHILD_DATA, errp); if (!state->target) { - bdrv_unref(target); bdrv_unref(top); return NULL; } From patchwork Wed Sep 1 15:15:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7442C432BE for ; Wed, 1 Sep 2021 15:37:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 39F73600EF for ; Wed, 1 Sep 2021 15:37:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 39F73600EF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:42578 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSJ6-0004E4-EM for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:37:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49966) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzh-0005cI-Db for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:27036) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRza-0003xX-Tn for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509454; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yWI7HfOc4ZaN7Ip2VvLJzPpi6fGTcUvJwalJHHSt93k=; b=UnkYmCgYM7LSdvKbUq8i0Rc+tV4iqSmc1MqM34Fpu23GT/b0gunphDRqOeauMjeBXoQ00P QGXyIMdIN3hz9g0Qr9BxR3EVSf91/pEHQCYXmLboWTyc34jOlNrDGfEDOHKyUNsdmT2ZX3 Kx3tqnW6oboqRFvMEJSLCKai+CCB120= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-15-loDVQI27OwmBw3WGAAbB0A-1; Wed, 01 Sep 2021 11:17:32 -0400 X-MC-Unique: loDVQI27OwmBw3WGAAbB0A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0280CFC81; Wed, 1 Sep 2021 15:17:31 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9AB2760C05; Wed, 1 Sep 2021 15:17:30 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 32/56] block/copy-before-write: use file child instead of backing Date: Wed, 1 Sep 2021 17:15:55 +0200 Message-Id: <20210901151619.689075-33-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-13-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 945d9340f4..7a6c15f141 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -43,7 +43,7 @@ static coroutine_fn int cbw_co_preadv( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags); + return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); } static coroutine_fn int cbw_do_copy_before_write(BlockDriverState *bs, @@ -71,7 +71,7 @@ static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs, return ret; } - return bdrv_co_pdiscard(bs->backing, offset, bytes); + return bdrv_co_pdiscard(bs->file, offset, bytes); } static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs, @@ -82,7 +82,7 @@ static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs, return ret; } - return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags); + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); } static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs, @@ -95,29 +95,22 @@ static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs, return ret; } - return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags); + return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); } static int coroutine_fn cbw_co_flush(BlockDriverState *bs) { - if (!bs->backing) { + if (!bs->file) { return 0; } - return bdrv_co_flush(bs->backing->bs); + return bdrv_co_flush(bs->file->bs); } static void cbw_refresh_filename(BlockDriverState *bs) { - if (bs->backing == NULL) { - /* - * we can be here after failed bdrv_attach_child in - * bdrv_set_backing_hd - */ - return; - } pstrcpy(bs->exact_filename, sizeof(bs->exact_filename), - bs->backing->bs->filename); + bs->file->bs->filename); } static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, @@ -186,6 +179,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name, BDRV_O_RDWR, errp); if (!top) { + error_prepend(errp, "Cannot open driver: "); return NULL; } @@ -201,21 +195,32 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->target = bdrv_attach_child(top, target, "target", &child_of_bds, BDRV_CHILD_DATA, errp); if (!state->target) { + error_prepend(errp, "Cannot attach target child: "); + bdrv_unref(top); + return NULL; + } + + bdrv_ref(source); + top->file = bdrv_attach_child(top, source, "file", &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + errp); + if (!top->file) { + error_prepend(errp, "Cannot attach file child: "); bdrv_unref(top); return NULL; } bdrv_drained_begin(source); - ret = bdrv_append(top, source, errp); + ret = bdrv_replace_node(source, top, errp); if (ret < 0) { error_prepend(errp, "Cannot append copy-before-write filter: "); goto fail; } appended = true; - state->bcs = block_copy_state_new(top->backing, state->target, - false, compress, errp); + state->bcs = block_copy_state_new(top->file, state->target, false, compress, + errp); if (!state->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); goto fail; From patchwork Wed Sep 1 15:15:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD831C4320E for ; Wed, 1 Sep 2021 15:43:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 86A4960FDC for ; Wed, 1 Sep 2021 15:43:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 86A4960FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:60352 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSOZ-00088j-J1 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:43:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzr-0005vu-FB for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54456) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzc-0003zy-0H for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9OIJu2C6R8G1A7dgQhlDuoWRimyYxMU47fUlvcQm6qw=; b=DLB0UKpvWdfmcYrLszNGJTajo/e/hjn+L3TI5+EF67ySwge2f2z6i5j8orZfcUEq8pWsRY lcu7ePczYP3iaUeobVw/kxzJRbKS8+VnyyHPaYiXrO6XqVDesdeDGhHBbmV/RbgWTg2usI 7+dHtnS5a5mnpJ1qJKfuVp8YdXt1bQA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-526-1hc-Q8kZPlqPScDGMPoDEQ-1; Wed, 01 Sep 2021 11:17:34 -0400 X-MC-Unique: 1hc-Q8kZPlqPScDGMPoDEQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3DBD31B2C986; Wed, 1 Sep 2021 15:17:33 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C9A8160C05; Wed, 1 Sep 2021 15:17:32 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 33/56] block/copy-before-write: bdrv_cbw_append(): replace child at last Date: Wed, 1 Sep 2021 17:15:56 +0200 Message-Id: <20210901151619.689075-34-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Refactor the function to replace child at last. Thus we don't need to revert it and code is simplified. block-copy state initialization being done before replacing the child doesn't need any drained section. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-14-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 33 +++++++++++---------------------- 1 file changed, 11 insertions(+), 22 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 7a6c15f141..adbbc64aa9 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -172,7 +172,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, int ret; BDRVCopyBeforeWriteState *state; BlockDriverState *top; - bool appended = false; assert(source->total_sectors == target->total_sectors); @@ -196,8 +195,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BDRV_CHILD_DATA, errp); if (!state->target) { error_prepend(errp, "Cannot attach target child: "); - bdrv_unref(top); - return NULL; + goto fail; } bdrv_ref(source); @@ -206,18 +204,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, errp); if (!top->file) { error_prepend(errp, "Cannot attach file child: "); - bdrv_unref(top); - return NULL; - } - - bdrv_drained_begin(source); - - ret = bdrv_replace_node(source, top, errp); - if (ret < 0) { - error_prepend(errp, "Cannot append copy-before-write filter: "); goto fail; } - appended = true; state->bcs = block_copy_state_new(top->file, state->target, false, compress, errp); @@ -225,21 +213,22 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, error_prepend(errp, "Cannot create block-copy-state: "); goto fail; } - *bcs = state->bcs; + bdrv_drained_begin(source); + ret = bdrv_replace_node(source, top, errp); bdrv_drained_end(source); + if (ret < 0) { + error_prepend(errp, "Cannot append copy-before-write filter: "); + goto fail; + } + + *bcs = state->bcs; return top; fail: - if (appended) { - bdrv_cbw_drop(top); - } else { - bdrv_unref(top); - } - - bdrv_drained_end(source); - + block_copy_state_free(state->bcs); + bdrv_unref(top); return NULL; } From patchwork Wed Sep 1 15:15:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EA90C432BE for ; Wed, 1 Sep 2021 15:44:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC1A360F3A for ; Wed, 1 Sep 2021 15:44:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org CC1A360F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:36462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSPh-0002gO-Tw for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:44:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49960) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzg-0005cH-7H for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52823) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRze-00040y-7N for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pbtRkyCBuEz6z3CiRjmQtrZUxsSXudROhZETeyLgJks=; b=XvyEJu6eQ9Ig9o4jgxDznuvAsznipp4rAU5x3kVBXPwhuBkWb+9EyMe6hFFETZQyMIoONF MeCsGH85THvJLa92kRRVKzrhKZzkl0hFdNa7w5VBCG+z9tFMaplI32qx1xzq0dmGFWWmzJ VbxLPBHO25hR7x9I6aM2S24z0nLOsHM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-120-wPJ2U6UhN5qufrPixdJ-ow-1; Wed, 01 Sep 2021 11:17:36 -0400 X-MC-Unique: wPJ2U6UhN5qufrPixdJ-ow-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 500F71B2C980; Wed, 1 Sep 2021 15:17:35 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CE5175C1BB; Wed, 1 Sep 2021 15:17:34 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 34/56] block/copy-before-write: introduce cbw_init() Date: Wed, 1 Sep 2021 17:15:57 +0200 Message-Id: <20210901151619.689075-35-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Move part of bdrv_cbw_append() to new function cbw_open(). It's an intermediate step for adding normal .bdrv_open() handler to the filter. With this commit no logic is changed, but we have a function which will be turned into .bdrv_open() handler in future commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-15-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 69 +++++++++++++++++++++++---------------- 1 file changed, 41 insertions(+), 28 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index adbbc64aa9..a4fee645fd 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -144,6 +144,45 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } +static int cbw_init(BlockDriverState *top, BlockDriverState *source, + BlockDriverState *target, bool compress, Error **errp) +{ + BDRVCopyBeforeWriteState *state = top->opaque; + + top->total_sectors = source->total_sectors; + top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & source->supported_write_flags); + top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & + source->supported_zero_flags); + + bdrv_ref(target); + state->target = bdrv_attach_child(top, target, "target", &child_of_bds, + BDRV_CHILD_DATA, errp); + if (!state->target) { + error_prepend(errp, "Cannot attach target child: "); + return -EINVAL; + } + + bdrv_ref(source); + top->file = bdrv_attach_child(top, source, "file", &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + errp); + if (!top->file) { + error_prepend(errp, "Cannot attach file child: "); + return -EINVAL; + } + + state->bcs = block_copy_state_new(top->file, state->target, false, compress, + errp); + if (!state->bcs) { + error_prepend(errp, "Cannot create block-copy-state: "); + return -EINVAL; + } + + return 0; +} + BlockDriver bdrv_cbw_filter = { .format_name = "copy-before-write", .instance_size = sizeof(BDRVCopyBeforeWriteState), @@ -181,36 +220,10 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, error_prepend(errp, "Cannot open driver: "); return NULL; } - state = top->opaque; - top->total_sectors = source->total_sectors; - top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | - (BDRV_REQ_FUA & source->supported_write_flags); - top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | - ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & - source->supported_zero_flags); - - bdrv_ref(target); - state->target = bdrv_attach_child(top, target, "target", &child_of_bds, - BDRV_CHILD_DATA, errp); - if (!state->target) { - error_prepend(errp, "Cannot attach target child: "); - goto fail; - } - bdrv_ref(source); - top->file = bdrv_attach_child(top, source, "file", &child_of_bds, - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, - errp); - if (!top->file) { - error_prepend(errp, "Cannot attach file child: "); - goto fail; - } - - state->bcs = block_copy_state_new(top->file, state->target, false, compress, - errp); - if (!state->bcs) { - error_prepend(errp, "Cannot create block-copy-state: "); + ret = cbw_init(top, source, target, compress, errp); + if (ret < 0) { goto fail; } From patchwork Wed Sep 1 15:15:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 550D8C432BE for ; Wed, 1 Sep 2021 15:34:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0180560231 for ; Wed, 1 Sep 2021 15:34:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0180560231 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:60330 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSGG-0005JP-72 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:34:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50182) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzv-000676-2a for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34213) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzg-000430-AJ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509459; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kvjs2cH3H+R831As71xJf8kNF2eqAADj6/JWclFFQf4=; b=f5httK7qeemqBKu806a4ez81ocECWtYUVw97aSz1aRP9CxHosD+ACwH7YBPT3LyaDzpw4R Ou4smGBJHTsNO3twReOx/8ELZNzQWPfQFNNUe6inKzWAzz5uhT1slXHm7ocXAgF9ztbal/ 67uWWz5Lo0J2OlKCq/OylHU5cCfFxXA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-438-Chv9xjPQPqqPj4t5aY7Eqg-1; Wed, 01 Sep 2021 11:17:38 -0400 X-MC-Unique: Chv9xjPQPqqPj4t5aY7Eqg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5F7971B2C980; Wed, 1 Sep 2021 15:17:37 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F382B60864; Wed, 1 Sep 2021 15:17:36 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 35/56] block/copy-before-write: cbw_init(): rename variables Date: Wed, 1 Sep 2021 17:15:58 +0200 Message-Id: <20210901151619.689075-36-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -24 X-Spam_score: -2.5 X-Spam_bar: -- X-Spam_report: (-2.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy One more step closer to real .bdrv_open() handler: use more usual names for bs being initialized and its state. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-16-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index a4fee645fd..d7f1833efa 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -144,38 +144,37 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init(BlockDriverState *top, BlockDriverState *source, +static int cbw_init(BlockDriverState *bs, BlockDriverState *source, BlockDriverState *target, bool compress, Error **errp) { - BDRVCopyBeforeWriteState *state = top->opaque; + BDRVCopyBeforeWriteState *s = bs->opaque; - top->total_sectors = source->total_sectors; - top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | + bs->total_sectors = source->total_sectors; + bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | (BDRV_REQ_FUA & source->supported_write_flags); - top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | + bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & source->supported_zero_flags); bdrv_ref(target); - state->target = bdrv_attach_child(top, target, "target", &child_of_bds, - BDRV_CHILD_DATA, errp); - if (!state->target) { + s->target = bdrv_attach_child(bs, target, "target", &child_of_bds, + BDRV_CHILD_DATA, errp); + if (!s->target) { error_prepend(errp, "Cannot attach target child: "); return -EINVAL; } bdrv_ref(source); - top->file = bdrv_attach_child(top, source, "file", &child_of_bds, - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, - errp); - if (!top->file) { + bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + errp); + if (!bs->file) { error_prepend(errp, "Cannot attach file child: "); return -EINVAL; } - state->bcs = block_copy_state_new(top->file, state->target, false, compress, - errp); - if (!state->bcs) { + s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp); + if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL; } From patchwork Wed Sep 1 15:15:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 282FFC4320A for ; Wed, 1 Sep 2021 15:41:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DEAC660F3A for ; Wed, 1 Sep 2021 15:41:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DEAC660F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:52292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSMT-0002ZJ-S3 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:41:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50074) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzp-0005p1-IX for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:55120) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzj-00045D-FB for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=82PouVtLFsB8o+9sE1/moPvstgwn3nbqIqyeZsCCMPI=; b=MFgBWNcT7NW1rzGbWgzRngTyPvjJrRVw2aFPWSoN66AI0eStMNjFqrQ67ltktZE5Gx9Qmg ug8S26rh1PENCemMtJQJlwMCjCOmYeuFFjqGLQqp1uDQMWJ85DQk/ze1/4YqIsVviKge3n KLi3NYp6t/QHfZx5M255B7gZYdAW9y8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-351-uu3qUgqoNBKSSCadxzyeXw-1; Wed, 01 Sep 2021 11:17:40 -0400 X-MC-Unique: uu3qUgqoNBKSSCadxzyeXw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7396B107ACF2; Wed, 1 Sep 2021 15:17:39 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1072560C9F; Wed, 1 Sep 2021 15:17:38 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 36/56] block/copy-before-write: cbw_init(): use file child after attaching Date: Wed, 1 Sep 2021 17:15:59 +0200 Message-Id: <20210901151619.689075-37-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy In the next commit we'll get rid of source argument of cbw_init(). Prepare to it now, to make next commit simpler: move the code block that uses source below attaching the child and use bs->file->bs instead of source variable. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-17-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index d7f1833efa..4858dcf8ff 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -149,13 +149,6 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source, { BDRVCopyBeforeWriteState *s = bs->opaque; - bs->total_sectors = source->total_sectors; - bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | - (BDRV_REQ_FUA & source->supported_write_flags); - bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | - ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & - source->supported_zero_flags); - bdrv_ref(target); s->target = bdrv_attach_child(bs, target, "target", &child_of_bds, BDRV_CHILD_DATA, errp); @@ -173,6 +166,13 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source, return -EINVAL; } + bs->total_sectors = bs->file->bs->total_sectors; + bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); + bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & + bs->file->bs->supported_zero_flags); + s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); From patchwork Wed Sep 1 15:16:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B78F5C432BE for ; Wed, 1 Sep 2021 15:48:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D31C60F3A for ; Wed, 1 Sep 2021 15:48:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3D31C60F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:46184 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSTt-0000zm-UG for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:48:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50106) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzr-0005wd-LU for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21209) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzl-000478-Eq for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509464; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4xUhVOmN1WapwPPD0b8Wq27caqMEL7MxhMWhZmtJDrU=; b=EkyCr4i0yLCFX+56ITYRbpm4w9J3OxljwTWJqseGOL24/HyW6/I9L4sIyyllk3m1ulrA0r RdZqgiDhl8s70XxmsUA3HlMQtLo9cXXwEQOrYpRJO66Viivsras8pgmyrwNdfLJMA5z1Dm bUB/Ad6RfnM5HTkawgmWEMb98eSDvXM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-199-2jGA6PNaN6O_pp5wuVFpQw-1; Wed, 01 Sep 2021 11:17:42 -0400 X-MC-Unique: 2jGA6PNaN6O_pp5wuVFpQw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6E8901B2C980; Wed, 1 Sep 2021 15:17:41 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 12EA92B3A5; Wed, 1 Sep 2021 15:17:40 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 37/56] block/copy-before-write: bdrv_cbw_append(): drop unused compress arg Date: Wed, 1 Sep 2021 17:16:00 +0200 Message-Id: <20210901151619.689075-38-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Reitz Message-Id: <20210824083856.17408-18-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.h | 1 - block/backup.c | 2 +- block/copy-before-write.c | 7 +++---- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/block/copy-before-write.h b/block/copy-before-write.h index b386fd8f01..51847e711a 100644 --- a/block/copy-before-write.h +++ b/block/copy-before-write.h @@ -32,7 +32,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, - bool compress, BlockCopyState **bcs, Error **errp); void bdrv_cbw_drop(BlockDriverState *bs); diff --git a/block/backup.c b/block/backup.c index 83516297cb..4869f1e5da 100644 --- a/block/backup.c +++ b/block/backup.c @@ -452,7 +452,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } - cbw = bdrv_cbw_append(bs, target, filter_node_name, false, &bcs, errp); + cbw = bdrv_cbw_append(bs, target, filter_node_name, &bcs, errp); if (!cbw) { goto error; } diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 4858dcf8ff..1e7180760a 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -145,7 +145,7 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } static int cbw_init(BlockDriverState *bs, BlockDriverState *source, - BlockDriverState *target, bool compress, Error **errp) + BlockDriverState *target, Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; @@ -173,7 +173,7 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); - s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp); + s->bcs = block_copy_state_new(bs->file, s->target, false, false, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL; @@ -202,7 +202,6 @@ BlockDriver bdrv_cbw_filter = { BlockDriverState *bdrv_cbw_append(BlockDriverState *source, BlockDriverState *target, const char *filter_node_name, - bool compress, BlockCopyState **bcs, Error **errp) { @@ -221,7 +220,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } state = top->opaque; - ret = cbw_init(top, source, target, compress, errp); + ret = cbw_init(top, source, target, errp); if (ret < 0) { goto fail; } From patchwork Wed Sep 1 15:16:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BB0BC432BE for ; Wed, 1 Sep 2021 15:46:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0E8E6056C for ; Wed, 1 Sep 2021 15:46:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E0E8E6056C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:40370 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSR8-0005L1-0H for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:46:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50130) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzs-00060Y-NY for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30402) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzn-00048p-BJ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509466; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6CrWSLc3t0yRvV/kkN1IYFmg4j1DGx5L52rMlwfjCjE=; b=CJ995Qc2jRdWLuhKyiHlzSg5QJbHTp7TmtrnZU5V+V1Fb0CUijn1uqoaD45lobSzhsQBGf 7OtnvAcP3hk9TKPbB3EC+RgO5ExEB2nMwpX0AMq8CH78p1gma/mUTIkgynI1BeCcppeyWU 4rH2OzXV19B85oCDvpAJxyJYDpPQX04= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-484-oj3Bkk8jO5W5CdLnoVod5w-1; Wed, 01 Sep 2021 11:17:44 -0400 X-MC-Unique: oj3Bkk8jO5W5CdLnoVod5w-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 86C24801B3C; Wed, 1 Sep 2021 15:17:43 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 20204380; Wed, 1 Sep 2021 15:17:42 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 38/56] block/copy-before-write: cbw_init(): use options Date: Wed, 1 Sep 2021 17:16:01 +0200 Message-Id: <20210901151619.689075-39-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Reitz Message-Id: <20210824083856.17408-19-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 1e7180760a..1cefcade78 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -144,25 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init(BlockDriverState *bs, BlockDriverState *source, - BlockDriverState *target, Error **errp) +static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; - bdrv_ref(target); - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds, - BDRV_CHILD_DATA, errp); - if (!s->target) { - error_prepend(errp, "Cannot attach target child: "); + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + false, errp); + if (!bs->file) { return -EINVAL; } - bdrv_ref(source); - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds, - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, - errp); - if (!bs->file) { - error_prepend(errp, "Cannot attach file child: "); + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds, + BDRV_CHILD_DATA, false, errp); + if (!s->target) { return -EINVAL; } @@ -209,6 +204,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, int ret; BDRVCopyBeforeWriteState *state; BlockDriverState *top; + QDict *opts; assert(source->total_sectors == target->total_sectors); @@ -220,7 +216,12 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } state = top->opaque; - ret = cbw_init(top, source, target, errp); + opts = qdict_new(); + qdict_put_str(opts, "file", bdrv_get_node_name(source)); + qdict_put_str(opts, "target", bdrv_get_node_name(target)); + + ret = cbw_init(top, opts, errp); + qobject_unref(opts); if (ret < 0) { goto fail; } From patchwork Wed Sep 1 15:16:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8173C432BE for ; Wed, 1 Sep 2021 15:56:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 892A360F12 for ; Wed, 1 Sep 2021 15:56:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 892A360F12 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:38558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSbB-0006YX-JS for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:56:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50144) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzt-000635-Gy for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28456) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzo-0004AU-RX for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bkexaVk2956fcAmUkUKfGiG6Hcnzi2lUU9Ilis/wakY=; b=cYmlwJje13rMSch+SVY6Cb275aYxcOYyrkOLfE5gcf/iao7p25wuVUFVQPIQyhfl18Xjp/ DJ+KSPfnTYeFeZQvGrE0yU1ERSamy7/bwh2SAZsuv6rb5X+abd5WKP8A83vQ9WYXx8j7sv 1Bh3omxFSbF0m1eksVJ46P/dMzEU1/0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-113-ycmAyOFON5iAMh7H8ShN9A-1; Wed, 01 Sep 2021 11:17:46 -0400 X-MC-Unique: ycmAyOFON5iAMh7H8ShN9A-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A258180124F; Wed, 1 Sep 2021 15:17:45 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4A43560864; Wed, 1 Sep 2021 15:17:45 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 39/56] block/copy-before-write: initialize block-copy bitmap Date: Wed, 1 Sep 2021 17:16:02 +0200 Message-Id: <20210901151619.689075-40-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to publish copy-before-write filter to be used in separate of backup. Future step would support bitmap for the filter. But let's start from full set bitmap. We have to modify backup, as bitmap is first initialized by copy-before-write filter, and then backup modifies it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-20-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/backup.c | 16 +++++++--------- block/copy-before-write.c | 4 ++++ 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/block/backup.c b/block/backup.c index 4869f1e5da..687d2882bc 100644 --- a/block/backup.c +++ b/block/backup.c @@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job) BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs); if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) { + bdrv_clear_dirty_bitmap(bcs_bitmap, NULL); ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap, NULL, true); assert(ret); - } else { - if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { - /* - * We can't hog the coroutine to initialize this thoroughly. - * Set a flag and resume work when we are able to yield safely. - */ - block_copy_set_skip_unallocated(job->bcs, true); - } - bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len); + } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { + /* + * We can't hog the coroutine to initialize this thoroughly. + * Set a flag and resume work when we are able to yield safely. + */ + block_copy_set_skip_unallocated(job->bcs, true); } estimate = bdrv_get_dirty_count(bcs_bitmap); diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 1cefcade78..2efe098aae 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -147,6 +147,7 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; + BdrvDirtyBitmap *copy_bitmap; bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds, BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, @@ -174,6 +175,9 @@ static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp) return -EINVAL; } + copy_bitmap = block_copy_dirty_bitmap(s->bcs); + bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap)); + return 0; } From patchwork Wed Sep 1 15:16:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FE6AC432BE for ; Wed, 1 Sep 2021 15:49:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 009DA60F3A for ; Wed, 1 Sep 2021 15:49:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 009DA60F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:48842 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSUg-0002lC-3d for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:49:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50178) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzu-00066a-TY for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48353) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzq-0004C2-IP for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509469; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EERqLCIKKoDFUSgawoBKW6rAOlWWwGjHgy6Cp617msI=; b=XdXcFtdKrv9XWDjOQIzfKhIlLUvEZtupWrEjAPesO5dblWtKIZ7+owI/lTmcT8rC9uoRW7 MwWdBwuiVr1BchkcxAvwBZbTtQHJp3tZxlceD+95Mq7EuYIzXPjb6iiowFxeMH0tyeNQRC 7TUmfO4vpK87UYlgL4nC+gCZfB0lcio= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-46-47xVUa4RNBu6qvz0v43pwA-1; Wed, 01 Sep 2021 11:17:48 -0400 X-MC-Unique: 47xVUa4RNBu6qvz0v43pwA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AAEED108AD9F; Wed, 1 Sep 2021 15:17:47 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5303F60864; Wed, 1 Sep 2021 15:17:47 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 40/56] block/block-copy: make setting progress optional Date: Wed, 1 Sep 2021 17:16:03 +0200 Message-Id: <20210901151619.689075-41-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Now block-copy will crash if user don't set progress meter by block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-21-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/block-copy.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.c index 5d0076ac93..443261e4e4 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -292,9 +292,11 @@ static void coroutine_fn block_copy_task_end(BlockCopyTask *task, int ret) bdrv_set_dirty_bitmap(task->s->copy_bitmap, task->offset, task->bytes); } QLIST_REMOVE(task, list); - progress_set_remaining(task->s->progress, - bdrv_get_dirty_count(task->s->copy_bitmap) + - task->s->in_flight_bytes); + if (task->s->progress) { + progress_set_remaining(task->s->progress, + bdrv_get_dirty_count(task->s->copy_bitmap) + + task->s->in_flight_bytes); + } qemu_co_queue_restart_all(&task->wait_queue); } @@ -594,7 +596,7 @@ static coroutine_fn int block_copy_task_entry(AioTask *task) t->call_state->ret = ret; t->call_state->error_is_read = error_is_read; } - } else { + } else if (s->progress) { progress_work_done(s->progress, t->bytes); } } @@ -700,9 +702,11 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, if (!ret) { qemu_co_mutex_lock(&s->lock); bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes); - progress_set_remaining(s->progress, - bdrv_get_dirty_count(s->copy_bitmap) + - s->in_flight_bytes); + if (s->progress) { + progress_set_remaining(s->progress, + bdrv_get_dirty_count(s->copy_bitmap) + + s->in_flight_bytes); + } qemu_co_mutex_unlock(&s->lock); } From patchwork Wed Sep 1 15:16:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F14B6C432BE for ; Wed, 1 Sep 2021 15:53:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89CDB61027 for ; Wed, 1 Sep 2021 15:53:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 89CDB61027 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:57792 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSY1-0000Qk-NI for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:53:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50196) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzv-00069x-OY for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44416) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzs-0004DA-KF for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509472; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OL65wIE4BxrNWLRnooY3DoPoTZOak05Ktdp9P00olTQ=; b=eyfIi0CmrpcHCVn+y4mnkdscNxatYMlymoDNX8JFo0IESprnSaZbHGEsoBU/ss1fmeqBdK spGeHQQmxKvTKVISQjf0NRajuMDM8dbTuRA2vvawGtvQeNNtUupkPiJY7YlFLjcUVCGlsN eEONBLfvVH+d7aG5y6yJg0OiYEAXpAg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-558-dyA6XehTOq-TCWbNvMpowQ-1; Wed, 01 Sep 2021 11:17:50 -0400 X-MC-Unique: dyA6XehTOq-TCWbNvMpowQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C91C9343A2; Wed, 1 Sep 2021 15:17:49 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5418A12D4A; Wed, 1 Sep 2021 15:17:49 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 41/56] block/copy-before-write: make public block driver Date: Wed, 1 Sep 2021 17:16:04 +0200 Message-Id: <20210901151619.689075-42-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Finally, copy-before-write gets own .bdrv_open and .bdrv_close handlers, block_init() call and becomes available through bdrv_open(). To achieve this: - cbw_init gets unused flags argument and becomes cbw_open - block_copy_state_free() call moved to new cbw_close() - in bdrv_cbw_append: - options are completed with driver and node-name, and we can simply use bdrv_insert_node() to do both open and drained replacing - in bdrv_cbw_drop: - cbw_close() is now responsible for freeing s->bcs, so don't do it here Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-22-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- block/copy-before-write.c | 60 ++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 32 deletions(-) diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 2efe098aae..2cd68b480a 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -144,7 +144,8 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp) +static int cbw_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; BdrvDirtyBitmap *copy_bitmap; @@ -181,10 +182,21 @@ static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp) return 0; } +static void cbw_close(BlockDriverState *bs) +{ + BDRVCopyBeforeWriteState *s = bs->opaque; + + block_copy_state_free(s->bcs); + s->bcs = NULL; +} + BlockDriver bdrv_cbw_filter = { .format_name = "copy-before-write", .instance_size = sizeof(BDRVCopyBeforeWriteState), + .bdrv_open = cbw_open, + .bdrv_close = cbw_close, + .bdrv_co_preadv = cbw_co_preadv, .bdrv_co_pwritev = cbw_co_pwritev, .bdrv_co_pwrite_zeroes = cbw_co_pwrite_zeroes, @@ -205,56 +217,40 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, Error **errp) { ERRP_GUARD(); - int ret; BDRVCopyBeforeWriteState *state; BlockDriverState *top; QDict *opts; assert(source->total_sectors == target->total_sectors); - top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name, - BDRV_O_RDWR, errp); - if (!top) { - error_prepend(errp, "Cannot open driver: "); - return NULL; - } - state = top->opaque; - opts = qdict_new(); + qdict_put_str(opts, "driver", "copy-before-write"); + if (filter_node_name) { + qdict_put_str(opts, "node-name", filter_node_name); + } qdict_put_str(opts, "file", bdrv_get_node_name(source)); qdict_put_str(opts, "target", bdrv_get_node_name(target)); - ret = cbw_init(top, opts, errp); - qobject_unref(opts); - if (ret < 0) { - goto fail; - } - - bdrv_drained_begin(source); - ret = bdrv_replace_node(source, top, errp); - bdrv_drained_end(source); - if (ret < 0) { - error_prepend(errp, "Cannot append copy-before-write filter: "); - goto fail; + top = bdrv_insert_node(source, opts, BDRV_O_RDWR, errp); + if (!top) { + return NULL; } + state = top->opaque; *bcs = state->bcs; return top; - -fail: - block_copy_state_free(state->bcs); - bdrv_unref(top); - return NULL; } void bdrv_cbw_drop(BlockDriverState *bs) { - BDRVCopyBeforeWriteState *s = bs->opaque; - bdrv_drop_filter(bs, &error_abort); - - block_copy_state_free(s->bcs); - bdrv_unref(bs); } + +static void cbw_init(void) +{ + bdrv_register(&bdrv_cbw_filter); +} + +block_init(cbw_init); From patchwork Wed Sep 1 15:16:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2425C432BE for ; Wed, 1 Sep 2021 16:01:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3CD156069E for ; Wed, 1 Sep 2021 16:01:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3CD156069E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:45974 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSgD-0003Ia-5b for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:01:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzy-0006L1-H0 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59713) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzw-0004H7-Rd for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pVIUEin1oy4lg/OxgUEgf04e8yc+mNsPOpmpD/MVnEE=; b=SoFqcAb4VwEovGuxKdZW0txBHI0rwqi9RwAD9QzavA1VvXAhwqw+AnM6xJ2PAOifMA8TrF +N31H2/hjwTmUCmgVzbeVmjPwvyDgQMuokMbTBEViivkrH+iLwVeovwkxChiVdYFlyjq6y CgBRowZxlmSuGwV7y9fgN6MVR7QQiCQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-428-Du4rDAe1PPK0EhlfgY1vGQ-1; Wed, 01 Sep 2021 11:17:53 -0400 X-MC-Unique: Du4rDAe1PPK0EhlfgY1vGQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E8A9710054F6; Wed, 1 Sep 2021 15:17:51 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 94C176A05B; Wed, 1 Sep 2021 15:17:51 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 42/56] qapi: publish copy-before-write filter Date: Wed, 1 Sep 2021 17:16:05 +0200 Message-Id: <20210901151619.689075-43-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Acked-by: Markus Armbruster Message-Id: <20210824083856.17408-23-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- qapi/block-core.json | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 06674c25c9..c8ce1d9d5d 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2825,13 +2825,14 @@ # @blklogwrites: Since 3.0 # @blkreplay: Since 4.2 # @compress: Since 5.0 +# @copy-before-write: Since 6.2 # # Since: 2.9 ## { 'enum': 'BlockdevDriver', 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs', - 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', - 'gluster', + 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg', + 'file', 'ftp', 'ftps', 'gluster', {'name': 'host_cdrom', 'if': 'HAVE_HOST_BLOCK_DEVICE' }, {'name': 'host_device', 'if': 'HAVE_HOST_BLOCK_DEVICE' }, 'http', 'https', 'iscsi', @@ -4049,6 +4050,25 @@ 'base': 'BlockdevOptionsGenericFormat', 'data': { '*bottom': 'str' } } +## +# @BlockdevOptionsCbw: +# +# Driver specific block device options for the copy-before-write driver, +# which does so called copy-before-write operations: when data is +# written to the filter, the filter first reads corresponding blocks +# from its file child and copies them to @target child. After successfully +# copying, the write request is propagated to file child. If copying +# fails, the original write request is failed too and no data is written +# to file child. +# +# @target: The target for copy-before-write operations. +# +# Since: 6.2 +## +{ 'struct': 'BlockdevOptionsCbw', + 'base': 'BlockdevOptionsGenericFormat', + 'data': { 'target': 'BlockdevRef' } } + ## # @BlockdevOptions: # @@ -4101,6 +4121,7 @@ 'bochs': 'BlockdevOptionsGenericFormat', 'cloop': 'BlockdevOptionsGenericFormat', 'compress': 'BlockdevOptionsGenericFormat', + 'copy-before-write':'BlockdevOptionsCbw', 'copy-on-read':'BlockdevOptionsCor', 'dmg': 'BlockdevOptionsGenericFormat', 'file': 'BlockdevOptionsFile', From patchwork Wed Sep 1 15:16:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A082C4320E for ; Wed, 1 Sep 2021 15:55:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A74EF61075 for ; Wed, 1 Sep 2021 15:55:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A74EF61075 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:37350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSag-0005eA-Sl for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:55:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50232) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzy-0006Lf-LT for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49759) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzw-0004HC-Un for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:17:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yobDSfgySgqd29KpvN63xhV3w5UZjvHfE5LSvjQhW38=; b=Rvr+/Vj0nlo4ioefxB6cXlJR2zt0xT4Jn6f94P9CTUU7sRQVvuWYH3UvNBQ1LAOu01wnkD QnXNwICM57NLTveyCCdX+noAGFdZlB6+7RCZJeaUxP+kKFAei2LwnfP85kxo3UFG9SLulW LOl8Z8gzkp2+/jGRBnCcdNn9n+Qw9u4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-176-_8cFQrbgOFm0PmIORdr1Xg-1; Wed, 01 Sep 2021 11:17:54 -0400 X-MC-Unique: _8cFQrbgOFm0PmIORdr1Xg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EF92F1B2C980; Wed, 1 Sep 2021 15:17:53 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9714F6A8FA; Wed, 1 Sep 2021 15:17:53 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 43/56] python/qemu/machine.py: refactor _qemu_args() Date: Wed, 1 Sep 2021 17:16:06 +0200 Message-Id: <20210901151619.689075-44-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy - use shorter construction - don't create new dict if not needed - drop extra unpacking key-val arguments - drop extra default values Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow Message-Id: <20210824083856.17408-24-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- python/qemu/machine/machine.py | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 8b935813e9..3fde73cf10 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -570,14 +570,12 @@ def _qmp(self) -> QEMUMonitorProtocol: return self._qmp_connection @classmethod - def _qmp_args(cls, _conv_keys: bool = True, **args: Any) -> Dict[str, Any]: - qmp_args = dict() - for key, value in args.items(): - if _conv_keys: - qmp_args[key.replace('_', '-')] = value - else: - qmp_args[key] = value - return qmp_args + def _qmp_args(cls, conv_keys: bool, + args: Dict[str, Any]) -> Dict[str, object]: + if conv_keys: + return {k.replace('_', '-'): v for k, v in args.items()} + + return args def qmp(self, cmd: str, conv_keys: bool = True, @@ -585,7 +583,7 @@ def qmp(self, cmd: str, """ Invoke a QMP command and return the response dict """ - qmp_args = self._qmp_args(conv_keys, **args) + qmp_args = self._qmp_args(conv_keys, args) return self._qmp.cmd(cmd, args=qmp_args) def command(self, cmd: str, @@ -596,7 +594,7 @@ def command(self, cmd: str, On success return the response dict. On failure raise an exception. """ - qmp_args = self._qmp_args(conv_keys, **args) + qmp_args = self._qmp_args(conv_keys, args) return self._qmp.command(cmd, **qmp_args) def get_qmp_event(self, wait: bool = False) -> Optional[QMPMessage]: From patchwork Wed Sep 1 15:16:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 182F5C432BE for ; Wed, 1 Sep 2021 15:52:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D32D860F3A for ; Wed, 1 Sep 2021 15:52:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D32D860F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56228 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSXb-0007lF-11 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:52:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS00-0006Tn-HU for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:38687) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLRzy-0004J8-QL for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P6+t8EfAiYegrjRViao4/M9Jw3O8LEUbUwt2eQQTVEk=; b=fVxn/SFhOi4IUZGqqNDLoouYxZW8PAdGL6ZzEZfjbSu4rHBhmTit5YDPKvDM0NiaSmSzko BrAXZ8Ptbinsw82Zp4Z1vcA3z8YczHG2cFunxPuBxzA/EbRs0LYmn1BOgzN7pD6BJjHgKc wBnmnqpPf1PRMrzAXgng0owM1z4e8IY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-187-oymNV2WZMrunM1NsD22Hsw-1; Wed, 01 Sep 2021 11:17:57 -0400 X-MC-Unique: oymNV2WZMrunM1NsD22Hsw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 02F301B2C981; Wed, 1 Sep 2021 15:17:56 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9C9D95D9DC; Wed, 1 Sep 2021 15:17:55 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 44/56] python/qemu/machine: QEMUMachine: improve qmp() method Date: Wed, 1 Sep 2021 17:16:07 +0200 Message-Id: <20210901151619.689075-45-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We often call qmp() with unpacking dict, like qmp('foo', **{...}). mypy don't really like it, it thinks that passed unpacked dict is a positional argument and complains that it type should be bool (because second argument of qmp() is conv_keys: bool). Allow passing dict directly, simplifying interface, and giving a way to satisfy mypy. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow Message-Id: <20210824083856.17408-25-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- python/qemu/machine/machine.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 3fde73cf10..6ec18570d9 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -578,11 +578,21 @@ def _qmp_args(cls, conv_keys: bool, return args def qmp(self, cmd: str, - conv_keys: bool = True, + args_dict: Optional[Dict[str, object]] = None, + conv_keys: Optional[bool] = None, **args: Any) -> QMPMessage: """ Invoke a QMP command and return the response dict """ + if args_dict is not None: + assert not args + assert conv_keys is None + args = args_dict + conv_keys = False + + if conv_keys is None: + conv_keys = True + qmp_args = self._qmp_args(conv_keys, args) return self._qmp.cmd(cmd, args=qmp_args) From patchwork Wed Sep 1 15:16:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48493C432BE for ; Wed, 1 Sep 2021 16:02:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D1C5860F3A for ; Wed, 1 Sep 2021 16:02:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D1C5860F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:48082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSgy-0004tS-Km for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:02:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50288) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS02-0006bS-HM for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52323) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS01-0004LO-1C for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509480; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ppq+QSYAlsWtGcVREKWkFatbEiFq9gyIv6KuAJXoB0U=; b=bO4yGgoMPBfblQ/RBirp1MEV2lu80qkgXFoIf1ztkI2Cu+i+PE4E+ku9eps+hKNndNwRop S8TLvvt02sM4w8BvvwxKyJaIe8BEDdMKreZtAMVXaXZ3KXYPbscegiiRMteAIYxYpERY1/ vd2XbrxqEWoxHTf7/FRkynVHGwVgO/E= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-244-pIUH_VsyMNiLtC9g5d67Fw-1; Wed, 01 Sep 2021 11:17:59 -0400 X-MC-Unique: pIUH_VsyMNiLtC9g5d67Fw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 30E9E107ACCA; Wed, 1 Sep 2021 15:17:58 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CBB2740AC; Wed, 1 Sep 2021 15:17:57 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 45/56] python:QEMUMachine: template typing for self returning methods Date: Wed, 1 Sep 2021 17:16:08 +0200 Message-Id: <20210901151619.689075-46-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy mypy thinks that return value of these methods in subclusses is QEMUMachine, which is wrong. So, make typing smarter. Suggested-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20210824083856.17408-26-vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz Signed-off-by: Hanna Reitz --- python/qemu/machine/machine.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 6ec18570d9..a7081b1845 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -36,6 +36,7 @@ Sequence, Tuple, Type, + TypeVar, ) from qemu.qmp import ( # pylint: disable=import-error @@ -73,6 +74,9 @@ class AbnormalShutdown(QEMUMachineError): """ +_T = TypeVar('_T', bound='QEMUMachine') + + class QEMUMachine: """ A QEMU VM. @@ -169,7 +173,7 @@ def __init__(self, self._remove_files: List[str] = [] self._user_killed = False - def __enter__(self) -> 'QEMUMachine': + def __enter__(self: _T) -> _T: return self def __exit__(self, @@ -185,8 +189,8 @@ def add_monitor_null(self) -> None: self._args.append('-monitor') self._args.append('null') - def add_fd(self, fd: int, fdset: int, - opaque: str, opts: str = '') -> 'QEMUMachine': + def add_fd(self: _T, fd: int, fdset: int, + opaque: str, opts: str = '') -> _T: """ Pass a file descriptor to the VM """ From patchwork Wed Sep 1 15:16:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D59D1C432BE for ; Wed, 1 Sep 2021 16:06:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6AE8F6069E for ; Wed, 1 Sep 2021 16:06:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6AE8F6069E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56636 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSkm-0002Ph-H8 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:06:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50312) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS05-0006kb-89 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27640) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS03-0004OI-AO for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509482; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8+4OrylBmgq5RYcdjsEuwiEWE2ENenPR08WEnjxxi0E=; b=ex/YgjlRTlebi5mW9LG0z3wgWjRMyjaAufmuuIrBvA9ykPuanDYH1UXor2ETYUy7P9meNq an+Rv2QJ2AtF1iF+J1DkNVke8HDgu9uIjN7hLBeQlMESQrG4K79bWQyPb9ie2YWgn8Hiy8 h1LohbUp9toV1nCO9ltdi/IJJYnn1LA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-295-4f6X3wQZNjSj1wqMkQdKHg-1; Wed, 01 Sep 2021 11:18:01 -0400 X-MC-Unique: 4f6X3wQZNjSj1wqMkQdKHg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2730110054F6; Wed, 1 Sep 2021 15:18:00 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C33885C25A; Wed, 1 Sep 2021 15:17:59 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 46/56] iotests/222: fix pylint and mypy complains Date: Wed, 1 Sep 2021 17:16:09 +0200 Message-Id: <20210901151619.689075-47-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Here: - long line - move to new interface of vm.qmp() (direct passing dict), to avoid mypy false-positive, as it thinks that unpacked dict is a positional argument. - extra parenthesis - handle event_wait possible None value Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-27-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/222 | 20 +++++++++++--------- tests/qemu-iotests/297 | 2 +- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index b48afe623e..5e2556f8df 100755 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -50,7 +50,8 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \ + iotests.FilePath('nbd.sock', + base_dir=iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: log('--- Setting up images ---') @@ -81,7 +82,7 @@ with iotests.FilePath('base.img') as base_img_path, \ tgt_node = "fleeceNode" # create tgt_node backed by src_node - log(vm.qmp("blockdev-add", **{ + log(vm.qmp("blockdev-add", { "driver": "qcow2", "node-name": tgt_node, "file": { @@ -103,8 +104,8 @@ with iotests.FilePath('base.img') as base_img_path, \ nbd_uri = 'nbd+unix:///%s?socket=%s' % (tgt_node, nbd_sock_path) log(vm.qmp("nbd-server-start", - **{"addr": { "type": "unix", - "data": { "path": nbd_sock_path } } })) + {"addr": { "type": "unix", + "data": { "path": nbd_sock_path } } })) log(vm.qmp("nbd-server-add", device=tgt_node)) @@ -112,7 +113,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Sanity Check ---') log('') - for p in (patterns + zeroes): + for p in patterns + zeroes: cmd = "read -P%s %s %s" % p log(cmd) assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0 @@ -130,7 +131,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Verifying Data ---') log('') - for p in (patterns + zeroes): + for p in patterns + zeroes: cmd = "read -P%s %s %s" % p log(cmd) assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0 @@ -140,8 +141,9 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') log(vm.qmp('block-job-cancel', device=src_node)) - log(vm.event_wait('BLOCK_JOB_CANCELLED'), - filters=[iotests.filter_qmp_event]) + e = vm.event_wait('BLOCK_JOB_CANCELLED') + assert e is not None + log(e, filters=[iotests.filter_qmp_event]) log(vm.qmp('nbd-server-stop')) log(vm.qmp('blockdev-del', node_name=tgt_node)) vm.shutdown() @@ -150,7 +152,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Confirming writes ---') log('') - for p in (overwrite + remainder): + for p in overwrite + remainder: cmd = "read -P%s %s %s" % p log(cmd) assert qemu_io_silent(base_img_path, '-c', cmd) == 0 diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 433b732336..345b617b34 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -31,7 +31,7 @@ SKIP_FILES = ( '096', '118', '124', '132', '136', '139', '147', '148', '149', '151', '152', '155', '163', '165', '169', '194', '196', '199', '202', '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', - '218', '219', '222', '224', '228', '234', '235', '236', '237', '238', + '218', '219', '224', '228', '234', '235', '236', '237', '238', '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', '299', '302', '303', '304', '307', From patchwork Wed Sep 1 15:16:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC5FEC432BE for ; Wed, 1 Sep 2021 16:08:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 586E961075 for ; Wed, 1 Sep 2021 16:08:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 586E961075 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:37058 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSnK-00005L-Fg for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:08:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50336) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS08-0006sO-Iu for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57457) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS05-0004Px-Af for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509484; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EpXF3XggS1tC1/gh/OURymTrPQsBGWJ9VNmyaS+8iN8=; b=cv4gRfpL8vZ1bXWeqQx3I35mKVQF4UG0eXByIrVGSSr9fANlDoqbkENPnebBo7UyTXgzgn CXWYfRf5sK0wmFfL5FnlV0nSxDO5WvRUpFvOS2dsfYSb5IMtDSOT8AJUzfaCR9sScJjJno LooN6YhvNIsJXwmY5LeToCBl0c/udFg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-331-t7MalZOtN3-im-OaJej2xA-1; Wed, 01 Sep 2021 11:18:03 -0400 X-MC-Unique: t7MalZOtN3-im-OaJej2xA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3B060107ACC7; Wed, 1 Sep 2021 15:18:02 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D4EB71F4; Wed, 1 Sep 2021 15:18:01 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 47/56] iotests/222: constantly use single quotes for strings Date: Wed, 1 Sep 2021 17:16:10 +0200 Message-Id: <20210901151619.689075-48-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy The file use both single and double quotes for strings. Let's be consistent. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-28-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/222 | 68 +++++++++++++++++++++--------------------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index 5e2556f8df..799369e290 100755 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -30,23 +30,23 @@ iotests.script_initialize( supported_platforms=['linux'], ) -patterns = [("0x5d", "0", "64k"), - ("0xd5", "1M", "64k"), - ("0xdc", "32M", "64k"), - ("0xcd", "0x3ff0000", "64k")] # 64M - 64K +patterns = [('0x5d', '0', '64k'), + ('0xd5', '1M', '64k'), + ('0xdc', '32M', '64k'), + ('0xcd', '0x3ff0000', '64k')] # 64M - 64K -overwrite = [("0xab", "0", "64k"), # Full overwrite - ("0xad", "0x00f8000", "64k"), # Partial-left (1M-32K) - ("0x1d", "0x2008000", "64k"), # Partial-right (32M+32K) - ("0xea", "0x3fe0000", "64k")] # Adjacent-left (64M - 128K) +overwrite = [('0xab', '0', '64k'), # Full overwrite + ('0xad', '0x00f8000', '64k'), # Partial-left (1M-32K) + ('0x1d', '0x2008000', '64k'), # Partial-right (32M+32K) + ('0xea', '0x3fe0000', '64k')] # Adjacent-left (64M - 128K) -zeroes = [("0", "0x00f8000", "32k"), # Left-end of partial-left (1M-32K) - ("0", "0x2010000", "32k"), # Right-end of partial-right (32M+64K) - ("0", "0x3fe0000", "64k")] # overwrite[3] +zeroes = [('0', '0x00f8000', '32k'), # Left-end of partial-left (1M-32K) + ('0', '0x2010000', '32k'), # Right-end of partial-right (32M+64K) + ('0', '0x3fe0000', '64k')] # overwrite[3] -remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] - ("0xdc", "32M", "32k"), # Left-end of partial-right [2] - ("0xcd", "0x3ff0000", "64k")] # patterns[3] +remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1] + ('0xdc', '32M', '32k'), # Left-end of partial-right [2] + ('0xcd', '0x3ff0000', '64k')] # patterns[3] with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('fleece.img') as fleece_img_path, \ @@ -58,7 +58,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 - assert qemu_img('create', '-f', "qcow2", fleece_img_path, '64M') == 0 + assert qemu_img('create', '-f', 'qcow2', fleece_img_path, '64M') == 0 for p in patterns: qemu_io('-f', iotests.imgfmt, @@ -78,43 +78,43 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Setting up Fleecing Graph ---') log('') - src_node = "drive0" - tgt_node = "fleeceNode" + src_node = 'drive0' + tgt_node = 'fleeceNode' # create tgt_node backed by src_node - log(vm.qmp("blockdev-add", { - "driver": "qcow2", - "node-name": tgt_node, - "file": { - "driver": "file", - "filename": fleece_img_path, + log(vm.qmp('blockdev-add', { + 'driver': 'qcow2', + 'node-name': tgt_node, + 'file': { + 'driver': 'file', + 'filename': fleece_img_path, }, - "backing": src_node, + 'backing': src_node, })) # Establish COW from source to fleecing node - log(vm.qmp("blockdev-backup", + log(vm.qmp('blockdev-backup', device=src_node, target=tgt_node, - sync="none")) + sync='none')) log('') log('--- Setting up NBD Export ---') log('') nbd_uri = 'nbd+unix:///%s?socket=%s' % (tgt_node, nbd_sock_path) - log(vm.qmp("nbd-server-start", - {"addr": { "type": "unix", - "data": { "path": nbd_sock_path } } })) + log(vm.qmp('nbd-server-start', + {'addr': { 'type': 'unix', + 'data': { 'path': nbd_sock_path } } })) - log(vm.qmp("nbd-server-add", device=tgt_node)) + log(vm.qmp('nbd-server-add', device=tgt_node)) log('') log('--- Sanity Check ---') log('') for p in patterns + zeroes: - cmd = "read -P%s %s %s" % p + cmd = 'read -P%s %s %s' % p log(cmd) assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0 @@ -123,7 +123,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') for p in overwrite: - cmd = "write -P%s %s %s" % p + cmd = 'write -P%s %s %s' % p log(cmd) log(vm.hmp_qemu_io(src_node, cmd)) @@ -132,7 +132,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') for p in patterns + zeroes: - cmd = "read -P%s %s %s" % p + cmd = 'read -P%s %s %s' % p log(cmd) assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0 @@ -153,7 +153,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') for p in overwrite + remainder: - cmd = "read -P%s %s %s" % p + cmd = 'read -P%s %s %s' % p log(cmd) assert qemu_io_silent(base_img_path, '-c', cmd) == 0 From patchwork Wed Sep 1 15:16:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A241C432BE for ; Wed, 1 Sep 2021 16:12:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2520860BD3 for ; Wed, 1 Sep 2021 16:12:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2520860BD3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:45294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSqq-0006Ay-A9 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:12:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50360) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0A-00070M-Ff for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:30208) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS07-0004RH-9Y for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509486; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uD19X4KylGOOEHcT5gxt0dZFmQ8ovL2twM2Pj9f3dsY=; b=fKntldfCQfddi3CnPLc/nO6aiGZQcqpZwvx5B59jSHMvRweWinkX/kM9CWLrYrEnbGM0dm UQ/MZXUrM5fhRMCpsBe6LENrBmFlKrjMd90u6sote/9rg1BRdgAkG5wjrJB+LEjghFlCPr BZrWdEbGPul4NkZco4v0/cdW4DGvUv8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-344-R1tGuWhRMKG0X6LvZXOW1g-1; Wed, 01 Sep 2021 11:18:05 -0400 X-MC-Unique: R1tGuWhRMKG0X6LvZXOW1g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5B953107ACC7; Wed, 1 Sep 2021 15:18:04 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0540960C05; Wed, 1 Sep 2021 15:18:03 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 48/56] iotests: move 222 to tests/image-fleecing Date: Wed, 1 Sep 2021 17:16:11 +0200 Message-Id: <20210901151619.689075-49-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Give a good name to test file. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-29-vsementsov@virtuozzo.com> [hreitz: Adjust .gitlab-ci.d/buildtest.yml] Signed-off-by: Hanna Reitz --- .gitlab-ci.d/buildtest.yml | 6 +++--- tests/qemu-iotests/{222 => tests/image-fleecing} | 0 tests/qemu-iotests/{222.out => tests/image-fleecing.out} | 0 3 files changed, 3 insertions(+), 3 deletions(-) rename tests/qemu-iotests/{222 => tests/image-fleecing} (100%) rename tests/qemu-iotests/{222.out => tests/image-fleecing.out} (100%) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 903ee65f32..e74998efb9 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -305,11 +305,11 @@ build-tcg-disabled: - cd tests/qemu-iotests/ - ./check -raw 001 002 003 004 005 008 009 010 011 012 021 025 032 033 048 052 063 077 086 101 104 106 113 148 150 151 152 157 159 160 163 - 170 171 183 184 192 194 208 221 222 226 227 236 253 277 + 170 171 183 184 192 194 208 221 226 227 236 253 277 image-fleecing - ./check -qcow2 028 051 056 057 058 065 068 082 085 091 095 096 102 122 124 132 139 142 144 145 151 152 155 157 165 194 196 200 202 - 208 209 216 218 222 227 234 246 247 248 250 254 255 257 258 - 260 261 262 263 264 270 272 273 277 279 + 208 209 216 218 227 234 246 247 248 250 254 255 257 258 + 260 261 262 263 264 270 272 273 277 279 image-fleecing build-user: extends: .native_build_job_template diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/tests/image-fleecing similarity index 100% rename from tests/qemu-iotests/222 rename to tests/qemu-iotests/tests/image-fleecing diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/tests/image-fleecing.out similarity index 100% rename from tests/qemu-iotests/222.out rename to tests/qemu-iotests/tests/image-fleecing.out From patchwork Wed Sep 1 15:16:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4CA4C432BE for ; Wed, 1 Sep 2021 16:06:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 711B260F3A for ; Wed, 1 Sep 2021 16:06:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 711B260F3A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:55042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSkR-0001MD-Dy for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:05:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0F-0007C1-3f for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35390) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0B-0004Us-TQ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tAnsFZWRMAsIZoWLP5dJAnhZ6g5UVkqbvyapYPlkz4w=; b=Q9v8hkedyq2Vi6Z13gl3tIpr0fJyQgljKBWifCNuAi2c840qPZ8lmWUGN0H7AbXc67dnQk 8khljwR7dN9jsHe3S7FN3xCgEQFk5rCUoSKYX9oSeb4YVnnM9bDfs6fF+HjHjQPG5ynBZt 8dh3UeFvD4A4MUZH6QpET362BtmOSsA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-221-QeDoLJuMMVOWIa98NqgHKQ-1; Wed, 01 Sep 2021 11:18:07 -0400 X-MC-Unique: QeDoLJuMMVOWIa98NqgHKQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74D4D1B2C984; Wed, 1 Sep 2021 15:18:06 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1D6F060BD8; Wed, 1 Sep 2021 15:18:05 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 49/56] iotests.py: hmp_qemu_io: support qdev Date: Wed, 1 Sep 2021 17:16:12 +0200 Message-Id: <20210901151619.689075-50-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow Message-Id: <20210824083856.17408-30-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/iotests.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 4c8971d946..11276f380a 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -696,9 +696,10 @@ def resume_drive(self, drive: str) -> None: self.hmp(f'qemu-io {drive} "remove_break bp_{drive}"') def hmp_qemu_io(self, drive: str, cmd: str, - use_log: bool = False) -> QMPMessage: + use_log: bool = False, qdev: bool = False) -> QMPMessage: """Write to a given drive using an HMP command""" - return self.hmp(f'qemu-io {drive} "{cmd}"', use_log=use_log) + d = '-d ' if qdev else '' + return self.hmp(f'qemu-io {d}{drive} "{cmd}"', use_log=use_log) def flatten_qmp_object(self, obj, output=None, basestr=''): if output is None: From patchwork Wed Sep 1 15:16:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 676DCC432BE for ; Wed, 1 Sep 2021 16:15:53 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DC26C610A8 for ; Wed, 1 Sep 2021 16:15:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DC26C610A8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:53782 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLStz-0003Y2-VX for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:15:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50442) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0E-0007Ai-Hg for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:54352) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0B-0004Ul-Jf for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=94U9NO1T3G8r6PomIk4a38GYu901Bx0GJ6jPkD7re7U=; b=Uy/j5ptlXEuhRiMQ7UFwhLgSIfyMOWdzf6BwxOqZsoa+bhOeHu5JsiBQXwkooxZDuA3tYT M2pPfvTCRn/AoXirmwKDwIuRaD0NXQ+QfbCOVgbrLwYzXSJI0jRGw2ePP51ny28V1575QY Z5aDEq98WR/93Vfq9RMNZnAfHctT0PY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-141-ZpXtBhZMM7CLARVbjxiVBQ-1; Wed, 01 Sep 2021 11:18:09 -0400 X-MC-Unique: ZpXtBhZMM7CLARVbjxiVBQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7ADDF87D542; Wed, 1 Sep 2021 15:18:08 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 22AE31F4; Wed, 1 Sep 2021 15:18:07 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 50/56] iotests/image-fleecing: proper source device Date: Wed, 1 Sep 2021 17:16:13 +0200 Message-Id: <20210901151619.689075-51-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Define scsi device to operate with it by qom-set in further patch. Give a new node-name to source block node, to not look like device name. Job now don't want to work without giving explicit id, so, let's call it "fleecing". Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-31-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/tests/image-fleecing | 12 ++++++++---- tests/qemu-iotests/tests/image-fleecing.out | 2 +- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing index 799369e290..961941bb27 100755 --- a/tests/qemu-iotests/tests/image-fleecing +++ b/tests/qemu-iotests/tests/image-fleecing @@ -70,7 +70,11 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Launching VM ---') log('') - vm.add_drive(base_img_path) + src_node = 'source' + vm.add_blockdev(f'driver={iotests.imgfmt},file.driver=file,' + f'file.filename={base_img_path},node-name={src_node}') + vm.add_device('virtio-scsi') + vm.add_device(f'scsi-hd,id=sda,drive={src_node}') vm.launch() log('Done') @@ -78,7 +82,6 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Setting up Fleecing Graph ---') log('') - src_node = 'drive0' tgt_node = 'fleeceNode' # create tgt_node backed by src_node @@ -94,6 +97,7 @@ with iotests.FilePath('base.img') as base_img_path, \ # Establish COW from source to fleecing node log(vm.qmp('blockdev-backup', + job_id='fleecing', device=src_node, target=tgt_node, sync='none')) @@ -125,7 +129,7 @@ with iotests.FilePath('base.img') as base_img_path, \ for p in overwrite: cmd = 'write -P%s %s %s' % p log(cmd) - log(vm.hmp_qemu_io(src_node, cmd)) + log(vm.hmp_qemu_io('/machine/peripheral/sda', cmd, qdev=True)) log('') log('--- Verifying Data ---') @@ -140,7 +144,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Cleanup ---') log('') - log(vm.qmp('block-job-cancel', device=src_node)) + log(vm.qmp('block-job-cancel', device='fleecing')) e = vm.event_wait('BLOCK_JOB_CANCELLED') assert e is not None log(e, filters=[iotests.filter_qmp_event]) diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out index 16643dde30..314a1b54e9 100644 --- a/tests/qemu-iotests/tests/image-fleecing.out +++ b/tests/qemu-iotests/tests/image-fleecing.out @@ -50,7 +50,7 @@ read -P0 0x3fe0000 64k --- Cleanup --- {"return": {}} -{"data": {"device": "drive0", "len": 67108864, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} +{"data": {"device": "fleecing", "len": 67108864, "offset": 393216, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}} {"return": {}} {"return": {}} From patchwork Wed Sep 1 15:16:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 290DDC432BE for ; Wed, 1 Sep 2021 16:19:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D02A561027 for ; Wed, 1 Sep 2021 16:19:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D02A561027 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:34364 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSxT-000126-TT for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:19:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0H-0007L4-Q8 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46766) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0F-0004Xv-F0 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509494; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hKAQ2j/+1lndmY7Z6U1t40uhJbDa9wD295BbC116zjQ=; b=LRPplTDFQNpZ4kO5kCVST0uDGpB/xHe81OOzg9JTatAsl1HOX6Bt7r4Fu2MAHPeK73twCk 9YmqEdM/hNuIl1W3PWtwmqnjt1FvxyGm2yxKx9QGZqAnbfK8p07B/L02Tsj0SCinZeQW+W lUuKUxnsEENpQGuepF4ddHodD4iQa3o= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-367-rgakWHQ5NnOeayMd0gA2_A-1; Wed, 01 Sep 2021 11:18:11 -0400 X-MC-Unique: rgakWHQ5NnOeayMd0gA2_A-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ABFFF501E0; Wed, 1 Sep 2021 15:18:10 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 455065D9DC; Wed, 1 Sep 2021 15:18:10 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 51/56] iotests/image-fleecing: rename tgt_node Date: Wed, 1 Sep 2021 17:16:14 +0200 Message-Id: <20210901151619.689075-52-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy Actually target of backup(sync=None) is not a final backup target: image fleecing is intended to be used with external tool, which will copy data from fleecing node to some real backup target. Also, we are going to add a test case for "push backup with fleecing", where instead of exporting fleecing node by NBD, we'll start a backup job from fleecing node to real backup target. To avoid confusion, let's rename temporary fleecing node now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-32-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/tests/image-fleecing | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing index 961941bb27..ec4ef5f3f6 100755 --- a/tests/qemu-iotests/tests/image-fleecing +++ b/tests/qemu-iotests/tests/image-fleecing @@ -71,6 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') src_node = 'source' + tmp_node = 'temp' vm.add_blockdev(f'driver={iotests.imgfmt},file.driver=file,' f'file.filename={base_img_path},node-name={src_node}') vm.add_device('virtio-scsi') @@ -82,12 +83,11 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Setting up Fleecing Graph ---') log('') - tgt_node = 'fleeceNode' - # create tgt_node backed by src_node + # create tmp_node backed by src_node log(vm.qmp('blockdev-add', { 'driver': 'qcow2', - 'node-name': tgt_node, + 'node-name': tmp_node, 'file': { 'driver': 'file', 'filename': fleece_img_path, @@ -99,19 +99,19 @@ with iotests.FilePath('base.img') as base_img_path, \ log(vm.qmp('blockdev-backup', job_id='fleecing', device=src_node, - target=tgt_node, + target=tmp_node, sync='none')) log('') log('--- Setting up NBD Export ---') log('') - nbd_uri = 'nbd+unix:///%s?socket=%s' % (tgt_node, nbd_sock_path) + nbd_uri = 'nbd+unix:///%s?socket=%s' % (tmp_node, nbd_sock_path) log(vm.qmp('nbd-server-start', {'addr': { 'type': 'unix', 'data': { 'path': nbd_sock_path } } })) - log(vm.qmp('nbd-server-add', device=tgt_node)) + log(vm.qmp('nbd-server-add', device=tmp_node)) log('') log('--- Sanity Check ---') @@ -149,7 +149,7 @@ with iotests.FilePath('base.img') as base_img_path, \ assert e is not None log(e, filters=[iotests.filter_qmp_event]) log(vm.qmp('nbd-server-stop')) - log(vm.qmp('blockdev-del', node_name=tgt_node)) + log(vm.qmp('blockdev-del', node_name=tmp_node)) vm.shutdown() log('') From patchwork Wed Sep 1 15:16:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EC5EC432BE for ; Wed, 1 Sep 2021 16:23:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E517361027 for ; Wed, 1 Sep 2021 16:23:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E517361027 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:42572 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLT11-0006cE-36 for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:23:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50496) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0I-0007NF-Cm for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53298) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0G-0004Yk-5I for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509495; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/pWcqpGSYiynjnqRWF70p5Ee6R9rPklv9VK+ANUNNeY=; b=TQQYnOSvZlFoeBh8TyQGEUnznn4Qxk46+zPyw3fkhcMIJIDFHHPz3R1K31EdrS5ozWH003 EBgatYmwdSGUoMDC4e56Yhxe+UtJ0lhNopJ0Cvy3qQEDUGwCyZn/UPFJYkhfIZm2QSKvcc O+AtPkXZ2ENk7eRpZaujYZFvyUWlOig= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-528-P3KoXrcwMPO9pBm9RyzQpA-1; Wed, 01 Sep 2021 11:18:13 -0400 X-MC-Unique: P3KoXrcwMPO9pBm9RyzQpA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 04457107ACC7; Wed, 1 Sep 2021 15:18:13 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 88F3D60864; Wed, 1 Sep 2021 15:18:12 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 52/56] iotests/image-fleecing: prepare for adding new test-case Date: Wed, 1 Sep 2021 17:16:15 +0200 Message-Id: <20210901151619.689075-53-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We are going to add a test-case with some behavior modifications. So, let's prepare a function to be reused. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-33-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/tests/image-fleecing | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing index ec4ef5f3f6..e210c00d28 100755 --- a/tests/qemu-iotests/tests/image-fleecing +++ b/tests/qemu-iotests/tests/image-fleecing @@ -48,12 +48,7 @@ remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1] ('0xdc', '32M', '32k'), # Left-end of partial-right [2] ('0xcd', '0x3ff0000', '64k')] # patterns[3] -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock', - base_dir=iotests.sock_dir) as nbd_sock_path, \ - iotests.VM() as vm: - +def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): log('--- Setting up images ---') log('') @@ -163,3 +158,15 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') log('Done') + + +def test(): + with iotests.FilePath('base.img') as base_img_path, \ + iotests.FilePath('fleece.img') as fleece_img_path, \ + iotests.FilePath('nbd.sock', + base_dir=iotests.sock_dir) as nbd_sock_path, \ + iotests.VM() as vm: + do_test(base_img_path, fleece_img_path, nbd_sock_path, vm) + + +test() From patchwork Wed Sep 1 15:16:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E924C432BE for ; Wed, 1 Sep 2021 15:55:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A9ACD61090 for ; Wed, 1 Sep 2021 15:55:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A9ACD61090 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:37212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSaf-0005YS-Jl for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 11:55:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50522) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0K-0007TQ-Fy for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31131) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0I-0004ao-IC for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509497; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jf1370es0DwsH0KjOGR5fulQcHrtKm7HwKVYC8v4B0U=; b=QB8DjfRfYuAnf7o1+epezjFJxFgjo2F4dAA0e7w0Q8J4L2bcwBYY0mROsqbc6uL80HHyU0 x1uXn2ptR5osOB35tVDohPL9tdqs2X4+8iJsRAR08X5U2635hyvT80nU00gcSVwnRNyYbD rqoZMBpYDlqMYL6n66JwX7Kcw2gJa/g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-46-PSnU6LBqPL6EGiPmFCOhaA-1; Wed, 01 Sep 2021 11:18:16 -0400 X-MC-Unique: PSnU6LBqPL6EGiPmFCOhaA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 88D08107ACE3; Wed, 1 Sep 2021 15:18:15 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E156660C9F; Wed, 1 Sep 2021 15:18:14 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 53/56] iotests/image-fleecing: add test-case for copy-before-write filter Date: Wed, 1 Sep 2021 17:16:16 +0200 Message-Id: <20210901151619.689075-54-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy New fleecing method becomes available: copy-before-write filter. Actually we don't need backup job to setup image fleecing. Add test for new recommended way of image fleecing. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20210824083856.17408-34-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz --- tests/qemu-iotests/tests/image-fleecing | 50 +++++++++----- tests/qemu-iotests/tests/image-fleecing.out | 72 +++++++++++++++++++++ 2 files changed, 107 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing index e210c00d28..f6318492c6 100755 --- a/tests/qemu-iotests/tests/image-fleecing +++ b/tests/qemu-iotests/tests/image-fleecing @@ -48,7 +48,7 @@ remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1] ('0xdc', '32M', '32k'), # Left-end of partial-right [2] ('0xcd', '0x3ff0000', '64k')] # patterns[3] -def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm): log('--- Setting up images ---') log('') @@ -67,6 +67,7 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): src_node = 'source' tmp_node = 'temp' + qom_path = '/machine/peripheral/sda' vm.add_blockdev(f'driver={iotests.imgfmt},file.driver=file,' f'file.filename={base_img_path},node-name={src_node}') vm.add_device('virtio-scsi') @@ -90,12 +91,22 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): 'backing': src_node, })) - # Establish COW from source to fleecing node - log(vm.qmp('blockdev-backup', - job_id='fleecing', - device=src_node, - target=tmp_node, - sync='none')) + # Establish CBW from source to fleecing node + if use_cbw: + log(vm.qmp('blockdev-add', { + 'driver': 'copy-before-write', + 'node-name': 'fl-cbw', + 'file': src_node, + 'target': tmp_node + })) + + log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw')) + else: + log(vm.qmp('blockdev-backup', + job_id='fleecing', + device=src_node, + target=tmp_node, + sync='none')) log('') log('--- Setting up NBD Export ---') @@ -124,7 +135,7 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): for p in overwrite: cmd = 'write -P%s %s %s' % p log(cmd) - log(vm.hmp_qemu_io('/machine/peripheral/sda', cmd, qdev=True)) + log(vm.hmp_qemu_io(qom_path, cmd, qdev=True)) log('') log('--- Verifying Data ---') @@ -139,10 +150,15 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): log('--- Cleanup ---') log('') - log(vm.qmp('block-job-cancel', device='fleecing')) - e = vm.event_wait('BLOCK_JOB_CANCELLED') - assert e is not None - log(e, filters=[iotests.filter_qmp_event]) + if use_cbw: + log(vm.qmp('qom-set', path=qom_path, property='drive', value=src_node)) + log(vm.qmp('blockdev-del', node_name='fl-cbw')) + else: + log(vm.qmp('block-job-cancel', device='fleecing')) + e = vm.event_wait('BLOCK_JOB_CANCELLED') + assert e is not None + log(e, filters=[iotests.filter_qmp_event]) + log(vm.qmp('nbd-server-stop')) log(vm.qmp('blockdev-del', node_name=tmp_node)) vm.shutdown() @@ -160,13 +176,17 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): log('Done') -def test(): +def test(use_cbw): with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('fleece.img') as fleece_img_path, \ iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: - do_test(base_img_path, fleece_img_path, nbd_sock_path, vm) + do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm) + +log('=== Test backup(sync=none) based fleecing ===\n') +test(False) -test() +log('=== Test filter based fleecing ===\n') +test(True) diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out index 314a1b54e9..e96d122a8b 100644 --- a/tests/qemu-iotests/tests/image-fleecing.out +++ b/tests/qemu-iotests/tests/image-fleecing.out @@ -1,3 +1,5 @@ +=== Test backup(sync=none) based fleecing === + --- Setting up images --- Done @@ -65,3 +67,73 @@ read -P0xdc 32M 32k read -P0xcd 0x3ff0000 64k Done +=== Test filter based fleecing === + +--- Setting up images --- + +Done + +--- Launching VM --- + +Done + +--- Setting up Fleecing Graph --- + +{"return": {}} +{"return": {}} +{"return": {}} + +--- Setting up NBD Export --- + +{"return": {}} +{"return": {}} + +--- Sanity Check --- + +read -P0x5d 0 64k +read -P0xd5 1M 64k +read -P0xdc 32M 64k +read -P0xcd 0x3ff0000 64k +read -P0 0x00f8000 32k +read -P0 0x2010000 32k +read -P0 0x3fe0000 64k + +--- Testing COW --- + +write -P0xab 0 64k +{"return": ""} +write -P0xad 0x00f8000 64k +{"return": ""} +write -P0x1d 0x2008000 64k +{"return": ""} +write -P0xea 0x3fe0000 64k +{"return": ""} + +--- Verifying Data --- + +read -P0x5d 0 64k +read -P0xd5 1M 64k +read -P0xdc 32M 64k +read -P0xcd 0x3ff0000 64k +read -P0 0x00f8000 32k +read -P0 0x2010000 32k +read -P0 0x3fe0000 64k + +--- Cleanup --- + +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} + +--- Confirming writes --- + +read -P0xab 0 64k +read -P0xad 0x00f8000 64k +read -P0x1d 0x2008000 64k +read -P0xea 0x3fe0000 64k +read -P0xd5 0x108000 32k +read -P0xdc 32M 32k +read -P0xcd 0x3ff0000 64k + +Done From patchwork Wed Sep 1 15:16:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0015C432BE for ; Wed, 1 Sep 2021 16:00:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6ECAA61075 for ; Wed, 1 Sep 2021 16:00:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6ECAA61075 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:44052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSel-0001qZ-IB for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:00:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50548) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0N-0007X5-TI for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:60846) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0K-0004cQ-NJ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509500; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B16UsAWl2nw5NvsxXldwqQNB7VovFGadB7OCkw4K1tQ=; b=b1aQOe9jlrpXR+sDzUBgFzOFAHv47zTzMaNVlx3o85N56FMELcdCo1oAiq2+ZaKdjsgSNK CxqiudKKSmu2pOwNcti9PYcO6oab9+wayCF1BFAhH7/Wmhkr9rzsPk4YeCdqUHW3U9kr++ 8nB8FCxHGsSwo+33Rdw85YfbLX1TOao= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-499-Ng1jZuQiOsmjHTQkxgYOrw-1; Wed, 01 Sep 2021 11:18:18 -0400 X-MC-Unique: Ng1jZuQiOsmjHTQkxgYOrw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B57C8801A93; Wed, 1 Sep 2021 15:18:17 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4C83C5C1BB; Wed, 1 Sep 2021 15:18:17 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 54/56] block/block-copy: block_copy_state_new(): drop extra arguments Date: Wed, 1 Sep 2021 17:16:17 +0200 Message-Id: <20210901151619.689075-55-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy The only caller pass copy_range and compress both false. Let's just drop these arguments. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20210824083856.17408-35-vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz Signed-off-by: Hanna Reitz --- include/block/block-copy.h | 1 - block/block-copy.c | 5 ++--- block/copy-before-write.c | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index b8a2d63545..99370fa38b 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -25,7 +25,6 @@ typedef struct BlockCopyState BlockCopyState; typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, - bool use_copy_range, bool compress, Error **errp); /* Function should be called prior any actual copy request */ diff --git a/block/block-copy.c b/block/block-copy.c index 443261e4e4..ce116318b5 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -384,8 +384,7 @@ static int64_t block_copy_calculate_cluster_size(BlockDriverState *target, } BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, - bool use_copy_range, - bool compress, Error **errp) + Error **errp) { BlockCopyState *s; int64_t cluster_size; @@ -434,7 +433,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, cluster_size), }; - block_copy_set_copy_opts(s, use_copy_range, compress); + block_copy_set_copy_opts(s, false, false); ratelimit_init(&s->rate_limit); qemu_co_mutex_init(&s->lock); diff --git a/block/copy-before-write.c b/block/copy-before-write.c index 2cd68b480a..2a5e57deca 100644 --- a/block/copy-before-write.c +++ b/block/copy-before-write.c @@ -170,7 +170,7 @@ static int cbw_open(BlockDriverState *bs, QDict *options, int flags, ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags); - s->bcs = block_copy_state_new(bs->file, s->target, false, false, errp); + s->bcs = block_copy_state_new(bs->file, s->target, errp); if (!s->bcs) { error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL; From patchwork Wed Sep 1 15:16:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65FF7C432BE for ; Wed, 1 Sep 2021 16:26:19 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0CE8360E98 for ; Wed, 1 Sep 2021 16:26:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0CE8360E98 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:52882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLT46-00053r-7B for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:26:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50574) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0R-0007ap-Aa for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39531) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0P-0004gW-IZ for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tspisZhwzbcfzPPQAd6r/tRYq6tD7ae8dAkawdhz9Qs=; b=dDjsT8s8BZiIqtjRgK6XlS390C52gBaYC7EwmsT8nMfsLqKRGfjMcG9Ddk2plEYj+A2DBD QVYhvT3w7n3DCwfK3AosoWnVywpHGEmfzTFGB5lKeBl5mu8KiWsbZf4FYzaEKUzn9FtjXT /5kybb/aPBtFhpFHKUN/mwCl15epv/c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-225-fDkZJgGlNxW2RIPmvbKrNg-1; Wed, 01 Sep 2021 11:18:21 -0400 X-MC-Unique: fDkZJgGlNxW2RIPmvbKrNg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 06D1F87D545; Wed, 1 Sep 2021 15:18:20 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9DB485C25A; Wed, 1 Sep 2021 15:18:19 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 55/56] block/export/fuse.c: fix fuse-lseek on uclibc or musl Date: Wed, 1 Sep 2021 17:16:18 +0200 Message-Id: <20210901151619.689075-56-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Fabrice Fontaine Include linux/fs.h to avoid the following build failure on uclibc or musl raised since version 6.0.0: ../block/export/fuse.c: In function 'fuse_lseek': ../block/export/fuse.c:641:19: error: 'SEEK_HOLE' undeclared (first use in this function) 641 | if (whence != SEEK_HOLE && whence != SEEK_DATA) { | ^~~~~~~~~ ../block/export/fuse.c:641:19: note: each undeclared identifier is reported only once for each function it appears in ../block/export/fuse.c:641:42: error: 'SEEK_DATA' undeclared (first use in this function); did you mean 'SEEK_SET'? 641 | if (whence != SEEK_HOLE && whence != SEEK_DATA) { | ^~~~~~~~~ | SEEK_SET Fixes: - http://autobuild.buildroot.org/results/33c90ebf04997f4d3557cfa66abc9cf9a3076137 Signed-off-by: Fabrice Fontaine Message-Id: <20210827220301.272887-1-fontaine.fabrice@gmail.com> Signed-off-by: Hanna Reitz --- block/export/fuse.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/export/fuse.c b/block/export/fuse.c index fc7b07d2b5..2e3bf8270b 100644 --- a/block/export/fuse.c +++ b/block/export/fuse.c @@ -31,6 +31,9 @@ #include #include +#ifdef __linux__ +#include +#endif /* Prevent overly long bounce buffer allocations */ #define FUSE_MAX_BOUNCE_BYTES (MIN(BDRV_REQUEST_MAX_BYTES, 64 * 1024 * 1024)) From patchwork Wed Sep 1 15:16:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hanna Czenczek X-Patchwork-Id: 12469717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89A59C432BE for ; Wed, 1 Sep 2021 16:05:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 147266069E for ; Wed, 1 Sep 2021 16:05:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 147266069E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:53144 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mLSjv-0008RW-5E for qemu-devel@archiver.kernel.org; Wed, 01 Sep 2021 12:05:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0Z-0007b7-07 for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51305) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mLS0P-0004gO-Iw for qemu-devel@nongnu.org; Wed, 01 Sep 2021 11:18:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1630509504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0ZDwfy+YkSefr08uFFIvrtW555UMNkICUqmBhvKDyrU=; b=AFtYKNZQ8wXDZVBnJMZiAXAV42QEMAhjAOjNmyoGiJZhv42INFWFWg/fFB9lqOfQBsbfyX U7Nx8zo+JtgjdB8a+x7NJ0HTvKjH61ilTsXj15HPClpRxuP8zxEtr5/p99Fbi1e51Jeqm9 oJLEpmR1SqiDhVr3ozrzyuZ6SGUcOJI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-322-94-PgcMEP5ub_uLbT6nZhQ-1; Wed, 01 Sep 2021 11:18:23 -0400 X-MC-Unique: 94-PgcMEP5ub_uLbT6nZhQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3AFD387D548; Wed, 1 Sep 2021 15:18:22 +0000 (UTC) Received: from localhost (unknown [10.39.193.250]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D50136060F; Wed, 1 Sep 2021 15:18:21 +0000 (UTC) From: Hanna Reitz To: qemu-block@nongnu.org Subject: [PULL 56/56] block/file-win32: add reopen handlers Date: Wed, 1 Sep 2021 17:16:19 +0200 Message-Id: <20210901151619.689075-57-hreitz@redhat.com> In-Reply-To: <20210901151619.689075-1-hreitz@redhat.com> References: <20210901151619.689075-1-hreitz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=hreitz@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=hreitz@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Peter Maydell , Hanna Reitz , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Viktor Prutyanov Make 'qemu-img commit' work on Windows. Command 'commit' requires reopening backing file in RW mode. So, add reopen prepare/commit/abort handlers and change dwShareMode for CreateFile call in order to allow further read/write reopening. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/418 Suggested-by: Hanna Reitz Signed-off-by: Viktor Prutyanov Tested-by: Helge Konetzka Message-Id: <20210825173625.19415-1-viktor.prutyanov@phystech.edu> Signed-off-by: Hanna Reitz --- block/file-win32.c | 101 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 100 insertions(+), 1 deletion(-) diff --git a/block/file-win32.c b/block/file-win32.c index 2642088bd6..b97c58d642 100644 --- a/block/file-win32.c +++ b/block/file-win32.c @@ -58,6 +58,10 @@ typedef struct BDRVRawState { QEMUWin32AIOState *aio; } BDRVRawState; +typedef struct BDRVRawReopenState { + HANDLE hfile; +} BDRVRawReopenState; + /* * Read/writes the data to/from a given linear buffer. * @@ -392,7 +396,7 @@ static int raw_open(BlockDriverState *bs, QDict *options, int flags, } s->hfile = CreateFile(filename, access_flags, - FILE_SHARE_READ, NULL, + FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, overlapped, NULL); if (s->hfile == INVALID_HANDLE_VALUE) { int err = GetLastError(); @@ -634,6 +638,97 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv, return raw_co_create(&options, errp); } +static int raw_reopen_prepare(BDRVReopenState *state, + BlockReopenQueue *queue, Error **errp) +{ + BDRVRawState *s = state->bs->opaque; + BDRVRawReopenState *rs; + int access_flags; + DWORD overlapped; + int ret = 0; + + if (s->type != FTYPE_FILE) { + error_setg(errp, "Can only reopen files"); + return -EINVAL; + } + + rs = g_new0(BDRVRawReopenState, 1); + + /* + * We do not support changing any options (only flags). By leaving + * all options in state->options, we tell the generic reopen code + * that we do not support changing any of them, so it will verify + * that their values did not change. + */ + + raw_parse_flags(state->flags, s->aio != NULL, &access_flags, &overlapped); + rs->hfile = CreateFile(state->bs->filename, access_flags, + FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, + OPEN_EXISTING, overlapped, NULL); + + if (rs->hfile == INVALID_HANDLE_VALUE) { + int err = GetLastError(); + + error_setg_win32(errp, err, "Could not reopen '%s'", + state->bs->filename); + if (err == ERROR_ACCESS_DENIED) { + ret = -EACCES; + } else { + ret = -EINVAL; + } + goto fail; + } + + if (s->aio) { + ret = win32_aio_attach(s->aio, rs->hfile); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not enable AIO"); + CloseHandle(rs->hfile); + goto fail; + } + } + + state->opaque = rs; + + return 0; + +fail: + g_free(rs); + state->opaque = NULL; + + return ret; +} + +static void raw_reopen_commit(BDRVReopenState *state) +{ + BDRVRawState *s = state->bs->opaque; + BDRVRawReopenState *rs = state->opaque; + + assert(rs != NULL); + + CloseHandle(s->hfile); + s->hfile = rs->hfile; + + g_free(rs); + state->opaque = NULL; +} + +static void raw_reopen_abort(BDRVReopenState *state) +{ + BDRVRawReopenState *rs = state->opaque; + + if (!rs) { + return; + } + + if (rs->hfile != INVALID_HANDLE_VALUE) { + CloseHandle(rs->hfile); + } + + g_free(rs); + state->opaque = NULL; +} + static QemuOptsList raw_create_opts = { .name = "raw-create-opts", .head = QTAILQ_HEAD_INITIALIZER(raw_create_opts.head), @@ -659,6 +754,10 @@ BlockDriver bdrv_file = { .bdrv_co_create_opts = raw_co_create_opts, .bdrv_has_zero_init = bdrv_has_zero_init_1, + .bdrv_reopen_prepare = raw_reopen_prepare, + .bdrv_reopen_commit = raw_reopen_commit, + .bdrv_reopen_abort = raw_reopen_abort, + .bdrv_aio_preadv = raw_aio_preadv, .bdrv_aio_pwritev = raw_aio_pwritev, .bdrv_aio_flush = raw_aio_flush,