From patchwork Fri Sep 17 05:40:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04C56C433EF for ; Fri, 17 Sep 2021 05:43:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0B336113E for ; Fri, 17 Sep 2021 05:43:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C0B336113E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:35826 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6fF-0008BT-LM for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:43:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47988) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cW-0004pZ-Mo for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:31175) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cT-0003Gl-MT for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s0md+7N5AD9Cg66Y5L4q8vzW0c2Eo8VrxhtRrNaJQGI=; b=PXnkSxFwR+qXDsnRKIxfJABG0rRBsODO/PWhvocd/eH/UnYE4lNwNFxo3G0a5D/+YY+QOE 6ZINuNJyVkIAuhZUdFUVPpNXGs82sm/3ylRiuSNtnyoJoJHpbAQ3uTuTuGEJn3oh4QHh17 GhKUHP9PZjxA1PiucbT6/qX4cbWRY7g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-445-1h0PLvnrNryj7XohHa4fDw-1; Fri, 17 Sep 2021 01:41:02 -0400 X-MC-Unique: 1h0PLvnrNryj7XohHa4fDw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E2119824FA8; Fri, 17 Sep 2021 05:41:01 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id A66D660843; Fri, 17 Sep 2021 05:41:00 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 01/15] python/aqmp: add greeting property to QMPClient Date: Fri, 17 Sep 2021 01:40:33 -0400 Message-Id: <20210917054047.2042843-2-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Expose the greeting as a read-only property of QMPClient so it can be retrieved at-will. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/aqmp/qmp_client.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py index 82e9dab124..d2ad7459f9 100644 --- a/python/qemu/aqmp/qmp_client.py +++ b/python/qemu/aqmp/qmp_client.py @@ -224,6 +224,11 @@ def __init__(self, name: Optional[str] = None) -> None: 'asyncio.Queue[QMPClient._PendingT]' ] = {} + @property + def greeting(self) -> Optional[Greeting]: + """The `Greeting` from the QMP server, if any.""" + return self._greeting + @upper_half async def _establish_session(self) -> None: """ From patchwork Fri Sep 17 05:40:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D42BC433F5 for ; Fri, 17 Sep 2021 05:43:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 092CE6113E for ; Fri, 17 Sep 2021 05:43:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 092CE6113E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:35286 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6f4-0007nH-LZ for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:43:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48016) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cX-0004ri-2j for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44057) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cV-0003JB-9D for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zjmi5j8qmOqbIPm0NjLaGSEyB4VQoNK46I9Y0EwGoZo=; b=eK4GvtT1Fe6OLXyr//3vyMiMm6x1OJnbhX9W3hzboBTK4gXH31cvgVsW09foe/BLrER+jx angd4kRC+88eXG9d6rxvO+uA8zDtCt88qLZcqkldYZwe0PvGle83/fgK8wbDfiDnriJHEf TOARGJ1rpfAiQ0dmOe0Y9QEFGAT92CQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-309-FAhzq5WAMY-YcXhdmZkzrw-1; Fri, 17 Sep 2021 01:41:05 -0400 X-MC-Unique: FAhzq5WAMY-YcXhdmZkzrw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74B5E824FA9; Fri, 17 Sep 2021 05:41:04 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 397C960843; Fri, 17 Sep 2021 05:41:02 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 02/15] python/aqmp: add .empty() method to EventListener Date: Fri, 17 Sep 2021 01:40:34 -0400 Message-Id: <20210917054047.2042843-3-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Synchronous clients may want to know if they're about to block waiting for an event or not. A method such as this is necessary to implement a compatible interface for the old QEMUMonitorProtocol using the new async internals. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/aqmp/events.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/python/qemu/aqmp/events.py b/python/qemu/aqmp/events.py index fb81d21610..271899f6b8 100644 --- a/python/qemu/aqmp/events.py +++ b/python/qemu/aqmp/events.py @@ -556,6 +556,12 @@ async def get(self) -> Message: """ return await self._queue.get() + def empty(self) -> bool: + """ + Return `True` if there are no pending events. + """ + return self._queue.empty() + def clear(self) -> None: """ Clear this listener of all pending events. From patchwork Fri Sep 17 05:40:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5855C433F5 for ; Fri, 17 Sep 2021 05:48:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 68BA061130 for ; Fri, 17 Sep 2021 05:48:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 68BA061130 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:43668 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6jh-0005AC-2N for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:48:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cb-0004vv-Ki for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36134) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cY-0003Kt-CL for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9rQTYxM12O0gGxp/LBmoCwHBzY/dzx3Uu+G1x1JDjrI=; b=isiWP+HMqy391OqV8oXWkawruKx7cUye7LzK/PzDc8TwCoKXdSV80+GOjGLxcucrKYMq/q EnErTN1I4g0K1idOqEd+kiL5P0BnmKd8tlCdWpAFuzSbLrUwPz38Kk6OM//aIhDyBlAnxh sERnLYHCYiBhjkjKuPnfqGmEswsnmAE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-164-EpER2eMrPISZCBhGor2Zyw-1; Fri, 17 Sep 2021 01:41:06 -0400 X-MC-Unique: EpER2eMrPISZCBhGor2Zyw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D8D061084681; Fri, 17 Sep 2021 05:41:05 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id C24DC60843; Fri, 17 Sep 2021 05:41:04 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 03/15] python/aqmp: Return cleared events from EventListener.clear() Date: Fri, 17 Sep 2021 01:40:35 -0400 Message-Id: <20210917054047.2042843-4-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This serves two purposes: (1) It is now possible to discern whether or not clear() removed any event(s) from the queue with absolute certainty, and (2) It is now very easy to get a List of all pending events in one chunk, which is useful for the sync bridge. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/aqmp/events.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/python/qemu/aqmp/events.py b/python/qemu/aqmp/events.py index 271899f6b8..5f7150c78d 100644 --- a/python/qemu/aqmp/events.py +++ b/python/qemu/aqmp/events.py @@ -562,7 +562,7 @@ def empty(self) -> bool: """ return self._queue.empty() - def clear(self) -> None: + def clear(self) -> List[Message]: """ Clear this listener of all pending events. @@ -570,17 +570,22 @@ def clear(self) -> None: pending FIFO queue synchronously. It can be also be used to manually clear any pending events, if desired. + :return: The cleared events, if any. + .. warning:: Take care when discarding events. Cleared events will be silently tossed on the floor. All events that were ever accepted by this listener are visible in `history()`. """ + events = [] while True: try: - self._queue.get_nowait() + events.append(self._queue.get_nowait()) except asyncio.QueueEmpty: break + return events + def __aiter__(self) -> AsyncIterator[Message]: return self From patchwork Fri Sep 17 05:40:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06F93C433EF for ; Fri, 17 Sep 2021 05:44:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BE56F61130 for ; Fri, 17 Sep 2021 05:44:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BE56F61130 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:36070 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6fH-0008LY-Mr for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:43:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48086) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cb-0004vx-L1 for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29278) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cY-0003MD-KB for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857270; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GBw5mtpr7i7V2RinC1R2gjVHzSYtc5auLIfH0TrTmBc=; b=ijGGKyOytX29N370oDD68HdEbo2bYI+4U8t/Qg8AbmZJPbJyBq4ZHLvagPllp5H3+rcgUP 86bB6vPiY0graZp9t816NKt26laQU1fyYtxlP6/qlWE8M2wRfHWSV6MEPiuzDsaa/GUeZ9 BVqrPiEl/XWzAd3s9SbuKLNAiaFCmiQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-230-70RLgkoBNXS8RUyMq_qfFQ-1; Fri, 17 Sep 2021 01:41:08 -0400 X-MC-Unique: 70RLgkoBNXS8RUyMq_qfFQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 69EA2824FA6; Fri, 17 Sep 2021 05:41:07 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1E23360843; Fri, 17 Sep 2021 05:41:06 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 04/15] python/qmp: clear events on get_events() call Date: Fri, 17 Sep 2021 01:40:36 -0400 Message-Id: <20210917054047.2042843-5-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" All callers in the tree *already* clear the events after a call to get_events(). Do it automatically instead and update callsites to remove the manual clear call. These semantics are quite a bit easier to emulate with async QMP, and nobody appears to be abusing some emergent properties of what happens if you decide not to clear them, so let's dial down to the dumber, simpler thing. Specifically: callers of clear() right after a call to get_events() are more likely expressing their desire to not see any events they just retrieved, whereas callers of clear_events() not in relation to a recent call to pull_event/get_events are likely expressing their desire to simply drop *all* pending events straight onto the floor. In the sync world, this is safe enough; in the async world it's nearly impossible to promise that nothing happens between getting and clearing the events. Making the retrieval also clear the queue is vastly simpler. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 1 - python/qemu/qmp/__init__.py | 4 +++- python/qemu/qmp/qmp_shell.py | 1 - 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 34131884a5..ae945ca3c9 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -631,7 +631,6 @@ def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: events = self._qmp.get_events(wait=wait) events.extend(self._events) del self._events[:] - self._qmp.clear_events() return events @staticmethod diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py index 269516a79b..ba15668c25 100644 --- a/python/qemu/qmp/__init__.py +++ b/python/qemu/qmp/__init__.py @@ -374,7 +374,9 @@ def get_events(self, wait: bool = False) -> List[QMPMessage]: @return The list of available QMP events. """ self.__get_events(wait) - return self.__events + events = self.__events + self.__events = [] + return events def clear_events(self) -> None: """ diff --git a/python/qemu/qmp/qmp_shell.py b/python/qemu/qmp/qmp_shell.py index 337acfce2d..e7d7eb18f1 100644 --- a/python/qemu/qmp/qmp_shell.py +++ b/python/qemu/qmp/qmp_shell.py @@ -381,7 +381,6 @@ def read_exec_command(self) -> bool: if cmdline == '': for event in self.get_events(): print(event) - self.clear_events() return True return self._execute_cmd(cmdline) From patchwork Fri Sep 17 05:40:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F3E4C433F5 for ; Fri, 17 Sep 2021 05:50:27 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BF71A61164 for ; Fri, 17 Sep 2021 05:50:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BF71A61164 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:51852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6lV-00029e-Ud for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:50:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48118) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cd-0004xZ-W5 for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56836) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cc-0003Pa-6O for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ot8HoQOgqwT7KWP3fKFHnIUaea26CVj45BSZLAmhnCs=; b=ioO/ZoJv+UP3+VvtapbMiA1WMBjZKwOe+Oz5OJND1LqNGBffVyUcUqyVuNizqbkJvlfvdx 1y5Kn8xSQGKWj/v9/RPQhgXJ41d0La1V1hNtQO8XtbUZtUe4xs/FieoJriTwL5paw+N42a TpoiygZPZm2SURyfoaAMqmbfdf1CyCY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-258-LbgdBv3jOPKmfBZoIXIGtw-1; Fri, 17 Sep 2021 01:41:10 -0400 X-MC-Unique: LbgdBv3jOPKmfBZoIXIGtw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 74E8F824FAB; Fri, 17 Sep 2021 05:41:09 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id A102E60843; Fri, 17 Sep 2021 05:41:07 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 05/15] python/qmp: add send_fd_scm directly to QEMUMonitorProtocol Date: Fri, 17 Sep 2021 01:40:37 -0400 Message-Id: <20210917054047.2042843-6-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" It turns out you can do this directly from Python ... and because of this, you don't need to worry about setting the inheritability of the fds or spawning another process. Doing this is helpful because it allows QEMUMonitorProtocol to keep its file descriptor and socket object as private implementation details. *that* is helpful in turn because it allows me to write a compatible, alternative implementation. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/qemu/machine/machine.py | 44 +++++++--------------------------- python/qemu/qmp/__init__.py | 21 +++++++--------- 2 files changed, 18 insertions(+), 47 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index ae945ca3c9..1c6532a3d6 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -213,48 +213,22 @@ def add_fd(self: _T, fd: int, fdset: int, def send_fd_scm(self, fd: Optional[int] = None, file_path: Optional[str] = None) -> int: """ - Send an fd or file_path to socket_scm_helper. + Send an fd or file_path to the remote via SCM_RIGHTS. - Exactly one of fd and file_path must be given. - If it is file_path, the helper will open that file and pass its own fd. + Exactly one of fd and file_path must be given. If it is + file_path, the file will be opened read-only and the new file + descriptor will be sent to the remote. """ - # In iotest.py, the qmp should always use unix socket. - assert self._qmp.is_scm_available() - if self._socket_scm_helper is None: - raise QEMUMachineError("No path to socket_scm_helper set") - if not os.path.exists(self._socket_scm_helper): - raise QEMUMachineError("%s does not exist" % - self._socket_scm_helper) - - # This did not exist before 3.4, but since then it is - # mandatory for our purpose - if hasattr(os, 'set_inheritable'): - os.set_inheritable(self._qmp.get_sock_fd(), True) - if fd is not None: - os.set_inheritable(fd, True) - - fd_param = ["%s" % self._socket_scm_helper, - "%d" % self._qmp.get_sock_fd()] - if file_path is not None: assert fd is None - fd_param.append(file_path) + with open(file_path, "rb") as passfile: + fd = passfile.fileno() + self._qmp.send_fd_scm(fd) else: assert fd is not None - fd_param.append(str(fd)) + self._qmp.send_fd_scm(fd) - proc = subprocess.run( - fd_param, - stdin=subprocess.DEVNULL, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, - check=False, - close_fds=False, - ) - if proc.stdout: - LOG.debug(proc.stdout) - - return proc.returncode + return 0 @staticmethod def _remove_if_exists(path: str) -> None: diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py index ba15668c25..c3b8a70ec9 100644 --- a/python/qemu/qmp/__init__.py +++ b/python/qemu/qmp/__init__.py @@ -21,6 +21,7 @@ import json import logging import socket +import struct from types import TracebackType from typing import ( Any, @@ -408,18 +409,14 @@ def settimeout(self, timeout: Optional[float]) -> None: raise ValueError(msg) self.__sock.settimeout(timeout) - def get_sock_fd(self) -> int: + def send_fd_scm(self, fd: int) -> None: """ - Get the socket file descriptor. - - @return The file descriptor number. - """ - return self.__sock.fileno() - - def is_scm_available(self) -> bool: + Send a file descriptor to the remote via SCM_RIGHTS. """ - Check if the socket allows for SCM_RIGHTS. + if self.__sock.family != socket.AF_UNIX: + raise RuntimeError("Can't use SCM_RIGHTS on non-AF_UNIX socket.") - @return True if SCM_RIGHTS is available, otherwise False. - """ - return self.__sock.family == socket.AF_UNIX + self.__sock.sendmsg( + [b' '], + [(socket.SOL_SOCKET, socket.SCM_RIGHTS, struct.pack('@i', fd))] + ) From patchwork Fri Sep 17 05:40:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E701EC433F5 for ; Fri, 17 Sep 2021 05:52:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7621D60EE2 for ; Fri, 17 Sep 2021 05:52:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7621D60EE2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6nr-00056W-Hv for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:52:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48182) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6ch-0004ye-9g for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25619) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cd-0003Qo-MJ for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r/4XULo2VvV2zFNIqij42bgS4IFwGIkNWp3LdaokFPk=; b=MXpo4EmNc8vtn0ZtJcB5ug76fqTR6mfsZa+6MYTQay2SFJP9de6M8yKKwdr5fn8vOcq/oC E16hegtbZDEgYu8eJdEDwrz6MjUv/DQwrog1SlD8++C5cRX/WWzyQGYKmN0v0wCbtYCfHu qtg+68+OkNYScup0HUzONHLhfVUtDAI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-298-AUoSljTGNP-H8qK1sto4TQ-1; Fri, 17 Sep 2021 01:41:11 -0400 X-MC-Unique: AUoSljTGNP-H8qK1sto4TQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C074A824FA6; Fri, 17 Sep 2021 05:41:10 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A64A60843; Fri, 17 Sep 2021 05:41:09 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 06/15] python, iotests: remove socket_scm_helper Date: Fri, 17 Sep 2021 01:40:38 -0400 Message-Id: <20210917054047.2042843-7-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" It's not used anymore, now. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/socket_scm_helper.c | 136 ------------------------- python/qemu/machine/machine.py | 3 - python/qemu/machine/qtest.py | 2 - tests/Makefile.include | 1 - tests/meson.build | 4 - tests/qemu-iotests/iotests.py | 3 - tests/qemu-iotests/meson.build | 5 - tests/qemu-iotests/testenv.py | 8 +- 8 files changed, 1 insertion(+), 161 deletions(-) delete mode 100644 tests/qemu-iotests/socket_scm_helper.c delete mode 100644 tests/qemu-iotests/meson.build diff --git a/tests/qemu-iotests/socket_scm_helper.c b/tests/qemu-iotests/socket_scm_helper.c deleted file mode 100644 index eb76d31aa9..0000000000 --- a/tests/qemu-iotests/socket_scm_helper.c +++ /dev/null @@ -1,136 +0,0 @@ -/* - * SCM_RIGHTS with unix socket help program for test - * - * Copyright IBM, Inc. 2013 - * - * Authors: - * Wenchao Xia - * - * This work is licensed under the terms of the GNU LGPL, version 2 or later. - * See the COPYING.LIB file in the top-level directory. - */ - -#include "qemu/osdep.h" -#include -#include - -/* #define SOCKET_SCM_DEBUG */ - -/* - * @fd and @fd_to_send will not be checked for validation in this function, - * a blank will be sent as iov data to notify qemu. - */ -static int send_fd(int fd, int fd_to_send) -{ - struct msghdr msg; - struct iovec iov[1]; - int ret; - char control[CMSG_SPACE(sizeof(int))]; - struct cmsghdr *cmsg; - - memset(&msg, 0, sizeof(msg)); - memset(control, 0, sizeof(control)); - - /* Send a blank to notify qemu */ - iov[0].iov_base = (void *)" "; - iov[0].iov_len = 1; - - msg.msg_iov = iov; - msg.msg_iovlen = 1; - - msg.msg_control = control; - msg.msg_controllen = sizeof(control); - - cmsg = CMSG_FIRSTHDR(&msg); - - cmsg->cmsg_len = CMSG_LEN(sizeof(int)); - cmsg->cmsg_level = SOL_SOCKET; - cmsg->cmsg_type = SCM_RIGHTS; - memcpy(CMSG_DATA(cmsg), &fd_to_send, sizeof(int)); - - do { - ret = sendmsg(fd, &msg, 0); - } while (ret < 0 && errno == EINTR); - - if (ret < 0) { - fprintf(stderr, "Failed to send msg, reason: %s\n", strerror(errno)); - } - - return ret; -} - -/* Convert string to fd number. */ -static int get_fd_num(const char *fd_str, bool silent) -{ - int sock; - char *err; - - errno = 0; - sock = strtol(fd_str, &err, 10); - if (errno) { - if (!silent) { - fprintf(stderr, "Failed in strtol for socket fd, reason: %s\n", - strerror(errno)); - } - return -1; - } - if (!*fd_str || *err || sock < 0) { - if (!silent) { - fprintf(stderr, "bad numerical value for socket fd '%s'\n", fd_str); - } - return -1; - } - - return sock; -} - -/* - * To make things simple, the caller needs to specify: - * 1. socket fd. - * 2. path of the file to be sent. - */ -int main(int argc, char **argv, char **envp) -{ - int sock, fd, ret; - -#ifdef SOCKET_SCM_DEBUG - int i; - for (i = 0; i < argc; i++) { - fprintf(stderr, "Parameter %d: %s\n", i, argv[i]); - } -#endif - - if (argc != 3) { - fprintf(stderr, - "Usage: %s < socket-fd > < file-path >\n", - argv[0]); - return EXIT_FAILURE; - } - - - sock = get_fd_num(argv[1], false); - if (sock < 0) { - return EXIT_FAILURE; - } - - fd = get_fd_num(argv[2], true); - if (fd < 0) { - /* Now only open a file in readonly mode for test purpose. If more - precise control is needed, use python script in file operation, which - is supposed to fork and exec this program. */ - fd = open(argv[2], O_RDONLY); - if (fd < 0) { - fprintf(stderr, "Failed to open file '%s'\n", argv[2]); - return EXIT_FAILURE; - } - } - - ret = send_fd(sock, fd); - if (ret < 0) { - close(fd); - return EXIT_FAILURE; - } - - close(fd); - return EXIT_SUCCESS; -} diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 1c6532a3d6..056d340e35 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -98,7 +98,6 @@ def __init__(self, name: Optional[str] = None, base_temp_dir: str = "/var/tmp", monitor_address: Optional[SocketAddrT] = None, - socket_scm_helper: Optional[str] = None, sock_dir: Optional[str] = None, drain_console: bool = False, console_log: Optional[str] = None, @@ -113,7 +112,6 @@ def __init__(self, @param name: prefix for socket and log file names (default: qemu-PID) @param base_temp_dir: default location where temp files are created @param monitor_address: address for QMP monitor - @param socket_scm_helper: helper program, required for send_fd_scm() @param sock_dir: where to create socket (defaults to base_temp_dir) @param drain_console: (optional) True to drain console socket to buffer @param console_log: (optional) path to console log file @@ -134,7 +132,6 @@ def __init__(self, self._base_temp_dir = base_temp_dir self._sock_dir = sock_dir or self._base_temp_dir self._log_dir = log_dir - self._socket_scm_helper = socket_scm_helper if monitor_address is not None: self._monitor_address = monitor_address diff --git a/python/qemu/machine/qtest.py b/python/qemu/machine/qtest.py index 395cc8fbfe..f2f9aaa5e5 100644 --- a/python/qemu/machine/qtest.py +++ b/python/qemu/machine/qtest.py @@ -115,7 +115,6 @@ def __init__(self, wrapper: Sequence[str] = (), name: Optional[str] = None, base_temp_dir: str = "/var/tmp", - socket_scm_helper: Optional[str] = None, sock_dir: Optional[str] = None, qmp_timer: Optional[float] = None): # pylint: disable=too-many-arguments @@ -126,7 +125,6 @@ def __init__(self, sock_dir = base_temp_dir super().__init__(binary, args, wrapper=wrapper, name=name, base_temp_dir=base_temp_dir, - socket_scm_helper=socket_scm_helper, sock_dir=sock_dir, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") diff --git a/tests/Makefile.include b/tests/Makefile.include index 6e16c05f10..5bd487a403 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -139,7 +139,6 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images check: ifeq ($(CONFIG_TOOLS)$(CONFIG_POSIX),yy) -QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = tests/qemu-iotests/socket_scm_helper$(EXESUF) check: check-block export PYTHON check-block: $(SRC_PATH)/tests/check-block.sh qemu-img$(EXESUF) \ diff --git a/tests/meson.build b/tests/meson.build index 55a7b08275..3f3882748a 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -67,10 +67,6 @@ if have_tools and 'CONFIG_VHOST_USER' in config_host and 'CONFIG_LINUX' in confi dependencies: [qemuutil, vhost_user]) endif -if have_system and 'CONFIG_POSIX' in config_host - subdir('qemu-iotests') -endif - test('decodetree', sh, args: [ files('decode/check.sh'), config_host['PYTHON'], files('../scripts/decodetree.py') ], workdir: meson.current_source_dir() / 'decode', diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 11276f380a..273d2777ae 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -109,8 +109,6 @@ qemu_valgrind = ['valgrind', valgrind_logfile, '--error-exitcode=99'] -socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper') - luks_default_secret_object = 'secret,id=keysec0,data=' + \ os.environ.get('IMGKEYSECRET', '') luks_default_key_secret_opt = 'key-secret=keysec0' @@ -600,7 +598,6 @@ def __init__(self, path_suffix=''): super().__init__(qemu_prog, qemu_opts, wrapper=wrapper, name=name, base_temp_dir=test_dir, - socket_scm_helper=socket_scm_helper, sock_dir=sock_dir, qmp_timer=timer) self._num_drives = 0 diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build deleted file mode 100644 index 67aed1e492..0000000000 --- a/tests/qemu-iotests/meson.build +++ /dev/null @@ -1,5 +0,0 @@ -if 'CONFIG_LINUX' in config_host - socket_scm_helper = executable('socket_scm_helper', 'socket_scm_helper.c') -else - socket_scm_helper = [] -endif diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 70da0d60c8..207bafb649 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -68,7 +68,7 @@ class TestEnv(ContextManager['TestEnv']): env_variables = ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR', 'OUTPUT_DIR', 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG', 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG', - 'SOCKET_SCM_HELPER', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS', + 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS', 'QEMU_IO_OPTIONS', 'QEMU_IO_OPTIONS_NO_FMT', 'QEMU_NBD_OPTIONS', 'IMGOPTS', 'IMGFMT', 'IMGPROTO', 'AIOMODE', 'CACHEMODE', 'VALGRIND_QEMU', @@ -137,7 +137,6 @@ def init_binaries(self) -> None: """Init binary path variables: PYTHON (for bash tests) QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PROG - SOCKET_SCM_HELPER """ self.python = sys.executable @@ -171,10 +170,6 @@ def root(*names: str) -> str: if not isxfile(b): sys.exit('Not executable: ' + b) - helper_path = os.path.join(self.build_iotests, 'socket_scm_helper') - if isxfile(helper_path): - self.socket_scm_helper = helper_path # SOCKET_SCM_HELPER - def __init__(self, imgfmt: str, imgproto: str, aiomode: str, cachemode: Optional[str] = None, imgopts: Optional[str] = None, @@ -300,7 +295,6 @@ def print_env(self) -> None: PLATFORM -- {platform} TEST_DIR -- {TEST_DIR} SOCK_DIR -- {SOCK_DIR} -SOCKET_SCM_HELPER -- {SOCKET_SCM_HELPER} GDB_OPTIONS -- {GDB_OPTIONS} VALGRIND_QEMU -- {VALGRIND_QEMU} PRINT_QEMU_OUTPUT -- {PRINT_QEMU} From patchwork Fri Sep 17 05:40:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7E9DC433F5 for ; Fri, 17 Sep 2021 05:49:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 960A361164 for ; Fri, 17 Sep 2021 05:49:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 960A361164 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:45876 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6kH-0006cz-OT for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:49:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48174) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cg-0004xh-Rp for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49446) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cd-0003QN-DX for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yEsV1VUq4HOVguJg2jE61C1nimpfJvQ/LTC/oELGyzQ=; b=hPfnTgEi8c4hKSCzL0dw2mlzCWWrqZYsmBMM38S45kfVlO0lbTsqF6cGXdj9K3GiFlPb8L AiP1zcpV1U5nyfZTzSP5tJQbf9qguk4R8cyF6dUZlnc3EiIHNXNnhG4GtHrbHRsAMdjdfC FyAa7cP5hSepGXniunDsDRshJkhwsIU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-333-IJtIOiNpOzeF7_uhNx5B0Q-1; Fri, 17 Sep 2021 01:41:13 -0400 X-MC-Unique: IJtIOiNpOzeF7_uhNx5B0Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5FB7F1800D41; Fri, 17 Sep 2021 05:41:12 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED69960C25; Fri, 17 Sep 2021 05:41:10 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 07/15] python/aqmp: add send_fd_scm Date: Fri, 17 Sep 2021 01:40:39 -0400 Message-Id: <20210917054047.2042843-8-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The single space is indeed required to successfully transmit the file descriptor to QEMU. Signed-off-by: John Snow --- python/qemu/aqmp/qmp_client.py | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py index d2ad7459f9..58f85990bc 100644 --- a/python/qemu/aqmp/qmp_client.py +++ b/python/qemu/aqmp/qmp_client.py @@ -9,6 +9,8 @@ import asyncio import logging +import socket +import struct from typing import ( Dict, List, @@ -624,3 +626,18 @@ async def execute(self, cmd: str, """ msg = self.make_execute_msg(cmd, arguments, oob=oob) return await self.execute_msg(msg) + + @upper_half + @require(Runstate.RUNNING) + def send_fd_scm(self, fd: int) -> None: + """ + Send a file descriptor to the remote via SCM_RIGHTS. + """ + assert self._writer is not None + sock = self._writer.transport.get_extra_info('socket') + + # Python 3.9+ adds socket.send_fds(...) + sock.sendmsg( + [b' '], + [(socket.SOL_SOCKET, socket.SCM_RIGHTS, struct.pack('@i', fd))] + ) From patchwork Fri Sep 17 05:40:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 963A5C433F5 for ; Fri, 17 Sep 2021 05:49:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2BBB161130 for ; Fri, 17 Sep 2021 05:49:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2BBB161130 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:47904 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6kg-0007xQ-Bs for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:49:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48248) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cj-00052o-W9 for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39527) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6ch-0003Xc-Op for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nytUlpDt/0b7IeqzHQ7da7PtUsm6ynnJqoV7SyXO5g8=; b=WLthcK0tXQQjLJQ3RwN8pZgFXs/r6PtE7o3q32Q5S8hZwLgZ28o/e/UGPoFNNJ3I+7H1vN BGCz2+i6r1FEcJN7ONumnrrHrpsdYtohnq2y4rCRHAduOapdeAxHM9HEBVywOwAVuAx5ok rLrJ5fqqPywpzXnz2RJiqbAaZ0JyijA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-HuevqPZ3NEib5KRQNLT8zA-1; Fri, 17 Sep 2021 01:41:15 -0400 X-MC-Unique: HuevqPZ3NEib5KRQNLT8zA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70DA71800D41; Fri, 17 Sep 2021 05:41:14 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8DBD060843; Fri, 17 Sep 2021 05:41:12 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 08/15] python/aqmp: Create MessageModel and StandaloneModel classes Date: Fri, 17 Sep 2021 01:40:40 -0400 Message-Id: <20210917054047.2042843-9-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This allows 'Greeting' to be subclass of 'Message'. We need the adapter classes to avoid some typing problems that occur if we try to put too much into the 'Model' class itself; the exact details of why are left as an exercise to the reader. Why bother? This makes 'Greeting' ⊆ 'Message', which is taxonomically true; but the real motivation is to be able to inherit and abuse all of the Mapping dunders so that we can call dict(greeting) or bytes(greeting), for example. Signed-off-by: John Snow --- python/qemu/aqmp/models.py | 67 ++++++++++++++++++++++++++++---------- 1 file changed, 50 insertions(+), 17 deletions(-) diff --git a/python/qemu/aqmp/models.py b/python/qemu/aqmp/models.py index 24c94123ac..eaeeebc25c 100644 --- a/python/qemu/aqmp/models.py +++ b/python/qemu/aqmp/models.py @@ -15,23 +15,22 @@ Sequence, ) +from .message import Message + class Model: """ - Abstract data model, representing some QMP object of some kind. - - :param raw: The raw object to be validated. - :raise KeyError: If any required fields are absent. - :raise TypeError: If any required fields have the wrong type. + Abstract data model, representing a QMP object of some kind. """ - def __init__(self, raw: Mapping[str, Any]): - self._raw = raw + @property + def _raw(self) -> Mapping[str, Any]: + raise NotImplementedError def _check_key(self, key: str) -> None: if key not in self._raw: raise KeyError(f"'{self._name}' object requires '{key}' member") - def _check_value(self, key: str, type_: type, typestr: str) -> None: + def _check_type(self, key: str, type_: type, typestr: str) -> None: assert key in self._raw if not isinstance(self._raw[key], type_): raise TypeError( @@ -40,7 +39,7 @@ def _check_value(self, key: str, type_: type, typestr: str) -> None: def _check_member(self, key: str, type_: type, typestr: str) -> None: self._check_key(key) - self._check_value(key, type_, typestr) + self._check_type(key, type_, typestr) @property def _name(self) -> str: @@ -50,7 +49,37 @@ def __repr__(self) -> str: return f"{self._name}({self._raw!r})" -class Greeting(Model): +class MessageModel(Message, Model): + """ + Abstract data model, representing a QMP Message of some sort. + + Adapter class that glues together `Model` and `Message`. + """ + def __init__(self, raw: Mapping[str, object]): + super().__init__(raw) + + @property + def _raw(self) -> Mapping[str, Any]: + return self._object + + def __setitem__(self, key: str, value: object) -> None: + # This is not good OO, but this turns off mutability here. + raise RuntimeError("Cannot mutate MessageModel") + + +class StandaloneModel(Model): + """ + Abstract data model, representing a (non-Message) QMP object of some sort. + """ + def __init__(self, raw: Mapping[str, object]): + self._raw_mapping = raw + + @property + def _raw(self) -> Mapping[str, Any]: + return self._raw_mapping + + +class Greeting(MessageModel): """ Defined in qmp-spec.txt, section 2.2, "Server Greeting". @@ -58,8 +87,9 @@ class Greeting(Model): :raise KeyError: If any required fields are absent. :raise TypeError: If any required fields have the wrong type. """ - def __init__(self, raw: Mapping[str, Any]): + def __init__(self, raw: Mapping[str, object]): super().__init__(raw) + #: 'QMP' member self.QMP: QMPGreeting # pylint: disable=invalid-name @@ -67,7 +97,7 @@ def __init__(self, raw: Mapping[str, Any]): self.QMP = QMPGreeting(self._raw['QMP']) -class QMPGreeting(Model): +class QMPGreeting(StandaloneModel): """ Defined in qmp-spec.txt, section 2.2, "Server Greeting". @@ -75,8 +105,9 @@ class QMPGreeting(Model): :raise KeyError: If any required fields are absent. :raise TypeError: If any required fields have the wrong type. """ - def __init__(self, raw: Mapping[str, Any]): + def __init__(self, raw: Mapping[str, object]): super().__init__(raw) + #: 'version' member self.version: Mapping[str, object] #: 'capabilities' member @@ -89,7 +120,7 @@ def __init__(self, raw: Mapping[str, Any]): self.capabilities = self._raw['capabilities'] -class ErrorResponse(Model): +class ErrorResponse(MessageModel): """ Defined in qmp-spec.txt, section 2.4.2, "error". @@ -97,8 +128,9 @@ class ErrorResponse(Model): :raise KeyError: If any required fields are absent. :raise TypeError: If any required fields have the wrong type. """ - def __init__(self, raw: Mapping[str, Any]): + def __init__(self, raw: Mapping[str, object]): super().__init__(raw) + #: 'error' member self.error: ErrorInfo #: 'id' member @@ -111,7 +143,7 @@ def __init__(self, raw: Mapping[str, Any]): self.id = raw['id'] -class ErrorInfo(Model): +class ErrorInfo(StandaloneModel): """ Defined in qmp-spec.txt, section 2.4.2, "error". @@ -119,8 +151,9 @@ class ErrorInfo(Model): :raise KeyError: If any required fields are absent. :raise TypeError: If any required fields have the wrong type. """ - def __init__(self, raw: Mapping[str, Any]): + def __init__(self, raw: Mapping[str, object]): super().__init__(raw) + #: 'class' member, with an underscore to avoid conflicts in Python. self.class_: str #: 'desc' member From patchwork Fri Sep 17 05:40:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE834C433EF for ; Fri, 17 Sep 2021 05:54:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 462C261130 for ; Fri, 17 Sep 2021 05:54:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 462C261130 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:32942 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6pt-0008Sb-Ff for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:54:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cq-00057U-5Q for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24630) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cj-0003Zb-4E for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p2L3ZicIcF9xvspUhH+jzt4GDVDyUjXAbCEzLhzGaTo=; b=FzOTw6VzU8l7+pfjoIi5KBtFJA6kf5fHZyM7CDXb5krEH4btYQZPNZ9FYGiWWDG5WUf4sz SPOxPEoQB7MJLPN+gHkH+V2e0Rgp+le7/OlT6R4zey80hh6Y3vPEVL8+IHkDntTZtvD2I0 XcYesshsSNlmm1WS5/t//JlDwVubjDk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-YRpc1DcTMrGqwzrf0nP13w-1; Fri, 17 Sep 2021 01:41:17 -0400 X-MC-Unique: YRpc1DcTMrGqwzrf0nP13w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E496F180830B; Fri, 17 Sep 2021 05:41:15 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9920E60843; Fri, 17 Sep 2021 05:41:14 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 09/15] python/machine: remove has_quit argument Date: Fri, 17 Sep 2021 01:40:41 -0400 Message-Id: <20210917054047.2042843-10-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If we spy on the QMP commands instead, we don't need callers to remember to pass it. Seems like a fair trade-off. The one slightly weird bit is overloading this instance variable for wait(), where we use it to mean "don't issue the qmp 'quit' command". This means that wait() will "fail" if the QEMU process does not terminate of its own accord. In most cases, we probably did already actually issue quit -- some iotests do this -- but in some others, we may be waiting for QEMU to terminate for some other reason. Signed-off-by: John Snow --- python/qemu/machine/machine.py | 35 +++++++++++++++++++--------------- tests/qemu-iotests/040 | 7 +------ tests/qemu-iotests/218 | 2 +- tests/qemu-iotests/255 | 2 +- 4 files changed, 23 insertions(+), 23 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 056d340e35..6e58d2f951 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -170,6 +170,7 @@ def __init__(self, self._console_socket: Optional[socket.socket] = None self._remove_files: List[str] = [] self._user_killed = False + self._has_quit = False def __enter__(self: _T) -> _T: return self @@ -368,6 +369,7 @@ def _post_shutdown(self) -> None: command = '' LOG.warning(msg, -int(exitcode), command) + self._has_quit = False self._user_killed = False self._launched = False @@ -443,15 +445,13 @@ def _hard_shutdown(self) -> None: self._subp.kill() self._subp.wait(timeout=60) - def _soft_shutdown(self, timeout: Optional[int], - has_quit: bool = False) -> None: + def _soft_shutdown(self, timeout: Optional[int]) -> None: """ Perform early cleanup, attempt to gracefully shut down the VM, and wait for it to terminate. :param timeout: Timeout in seconds for graceful shutdown. A value of None is an infinite wait. - :param has_quit: When True, don't attempt to issue 'quit' QMP command :raise ConnectionReset: On QMP communication errors :raise subprocess.TimeoutExpired: When timeout is exceeded waiting for @@ -460,21 +460,19 @@ def _soft_shutdown(self, timeout: Optional[int], self._early_cleanup() if self._qmp_connection: - if not has_quit: + if not self._has_quit: # Might raise ConnectionReset - self._qmp.cmd('quit') + self.qmp('quit') # May raise subprocess.TimeoutExpired self._subp.wait(timeout=timeout) - def _do_shutdown(self, timeout: Optional[int], - has_quit: bool = False) -> None: + def _do_shutdown(self, timeout: Optional[int]) -> None: """ Attempt to shutdown the VM gracefully; fallback to a hard shutdown. :param timeout: Timeout in seconds for graceful shutdown. A value of None is an infinite wait. - :param has_quit: When True, don't attempt to issue 'quit' QMP command :raise AbnormalShutdown: When the VM could not be shut down gracefully. The inner exception will likely be ConnectionReset or @@ -482,13 +480,13 @@ def _do_shutdown(self, timeout: Optional[int], may result in its own exceptions, likely subprocess.TimeoutExpired. """ try: - self._soft_shutdown(timeout, has_quit) + self._soft_shutdown(timeout) except Exception as exc: self._hard_shutdown() raise AbnormalShutdown("Could not perform graceful shutdown") \ from exc - def shutdown(self, has_quit: bool = False, + def shutdown(self, hard: bool = False, timeout: Optional[int] = 30) -> None: """ @@ -498,7 +496,6 @@ def shutdown(self, has_quit: bool = False, If the VM has not yet been launched, or shutdown(), wait(), or kill() have already been called, this method does nothing. - :param has_quit: When true, do not attempt to issue 'quit' QMP command. :param hard: When true, do not attempt graceful shutdown, and suppress the SIGKILL warning log message. :param timeout: Optional timeout in seconds for graceful shutdown. @@ -512,7 +509,7 @@ def shutdown(self, has_quit: bool = False, self._user_killed = True self._hard_shutdown() else: - self._do_shutdown(timeout, has_quit) + self._do_shutdown(timeout) finally: self._post_shutdown() @@ -529,7 +526,9 @@ def wait(self, timeout: Optional[int] = 30) -> None: :param timeout: Optional timeout in seconds. Default 30 seconds. A value of `None` is an infinite wait. """ - self.shutdown(has_quit=True, timeout=timeout) + # In case QEMU was stopped without issuing 'quit': + self._has_quit = True + self.shutdown(timeout=timeout) def set_qmp_monitor(self, enabled: bool = True) -> None: """ @@ -574,7 +573,10 @@ def qmp(self, cmd: str, conv_keys = True qmp_args = self._qmp_args(conv_keys, args) - return self._qmp.cmd(cmd, args=qmp_args) + ret = self._qmp.cmd(cmd, args=qmp_args) + if cmd == 'quit' and 'error' not in ret and 'return' in ret: + self._has_quit = True + return ret def command(self, cmd: str, conv_keys: bool = True, @@ -585,7 +587,10 @@ def command(self, cmd: str, On failure raise an exception. """ qmp_args = self._qmp_args(conv_keys, args) - return self._qmp.command(cmd, **qmp_args) + ret = self._qmp.command(cmd, **qmp_args) + if cmd == 'quit': + self._has_quit = True + return ret def get_qmp_event(self, wait: bool = False) -> Optional[QMPMessage]: """ diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 index f3677de9df..6af5ab9e76 100755 --- a/tests/qemu-iotests/040 +++ b/tests/qemu-iotests/040 @@ -92,10 +92,9 @@ class TestSingleDrive(ImageCommitTestCase): self.vm.add_device('virtio-scsi') self.vm.add_device("scsi-hd,id=scsi0,drive=drive0") self.vm.launch() - self.has_quit = False def tearDown(self): - self.vm.shutdown(has_quit=self.has_quit) + self.vm.shutdown() os.remove(test_img) os.remove(mid_img) os.remove(backing_img) @@ -127,8 +126,6 @@ class TestSingleDrive(ImageCommitTestCase): result = self.vm.qmp('quit') self.assert_qmp(result, 'return', {}) - self.has_quit = True - # Same as above, but this time we add the filter after starting the job @iotests.skip_if_unsupported(['throttle']) def test_commit_plus_filter_and_quit(self): @@ -147,8 +144,6 @@ class TestSingleDrive(ImageCommitTestCase): result = self.vm.qmp('quit') self.assert_qmp(result, 'return', {}) - self.has_quit = True - def test_device_not_found(self): result = self.vm.qmp('block-commit', device='nonexistent', top='%s' % mid_img) self.assert_qmp(result, 'error/class', 'DeviceNotFound') diff --git a/tests/qemu-iotests/218 b/tests/qemu-iotests/218 index 325d8244fb..4922b4d3b6 100755 --- a/tests/qemu-iotests/218 +++ b/tests/qemu-iotests/218 @@ -187,4 +187,4 @@ with iotests.VM() as vm, \ log(vm.qmp('quit')) with iotests.Timeout(5, 'Timeout waiting for VM to quit'): - vm.shutdown(has_quit=True) + vm.shutdown() diff --git a/tests/qemu-iotests/255 b/tests/qemu-iotests/255 index c43aa9c67a..3d6d0e80cb 100755 --- a/tests/qemu-iotests/255 +++ b/tests/qemu-iotests/255 @@ -123,4 +123,4 @@ with iotests.FilePath('src.qcow2') as src_path, \ vm.qmp_log('block-job-cancel', device='job0') vm.qmp_log('quit') - vm.shutdown(has_quit=True) + vm.shutdown() From patchwork Fri Sep 17 05:40:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E48EC433EF for ; Fri, 17 Sep 2021 05:50:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74B5661130 for ; Fri, 17 Sep 2021 05:50:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 74B5661130 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:53056 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6lr-0002yP-Gq for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:50:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cr-00058M-NP for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:35034) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6ci-0003Yf-FV for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zH5F3uen7nd7ziiWf9KhG96id2plwtBUemPAKgK2NTg=; b=eLt/8p/k6bUa87AeiO9lXEOpoPywkdRnWL4lVjOUgA4GX/vsV1N8Rcy+Ymb+EPoplbRITv LQCIKza5OATJSnYTOwuLpmbavFelKAMAb+QIzk6BmwIDNlCzVyfFuUvkNOdunxqmiQXjC4 LdpEwrWLCA3H8tYs+U+iRzQy1TNMASA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-582-0UeBdukINGCIweDYjEFo0Q-1; Fri, 17 Sep 2021 01:41:18 -0400 X-MC-Unique: 0UeBdukINGCIweDYjEFo0Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 90E18802936; Fri, 17 Sep 2021 05:41:17 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2369B7A8CE; Fri, 17 Sep 2021 05:41:16 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 10/15] python/machine: Add support for AQMP backend Date: Fri, 17 Sep 2021 01:40:42 -0400 Message-Id: <20210917054047.2042843-11-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" To use the AQMP backend, Machine just needs to be a little more diligent about what happens when closing a QMP connection. The operation is no longer a freebie in the async world. Because async QMP continues to check for messages asynchronously, it's almost certainly likely that the loop will have exited due to EOF after issuing the last 'quit' command. That error will ultimately be bubbled up when attempting to close the QMP connection. The manager class here then is free to discard it if it was expected. Signed-off-by: John Snow --- Yes, I regret that this class has become quite a dumping ground for complexity around the exit path. It's in need of a refactor to help separate out the exception handling and cleanup mechanisms from the VM-related stuff, but it's not a priority to do that just yet -- but it's on the list. --- python/qemu/machine/machine.py | 51 ++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 5 deletions(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 6e58d2f951..8f5a6649e5 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -342,9 +342,15 @@ def _post_shutdown(self) -> None: # Comprehensive reset for the failed launch case: self._early_cleanup() - if self._qmp_connection: - self._qmp.close() - self._qmp_connection = None + try: + self._close_qmp_connection() + except Exception as err: # pylint: disable=broad-except + LOG.warning( + "Exception closing QMP connection: %s", + str(err) if str(err) else type(err).__name__ + ) + finally: + assert self._qmp_connection is None self._close_qemu_log_file() @@ -420,6 +426,31 @@ def _launch(self) -> None: close_fds=False) self._post_launch() + def _close_qmp_connection(self) -> None: + """ + Close the underlying QMP connection, if any. + + Dutifully report errors that occurred while closing, but assume + that any error encountered indicates an abnormal termination + process and not a failure to close. + """ + if self._qmp_connection is None: + return + + try: + self._qmp.close() + except EOFError: + # EOF can occur as an Exception here when using the Async + # QMP backend. It indicates that the server closed the + # stream. If we successfully issued 'quit' at any point, + # then this was expected. If the remote went away without + # our permission, it's worth reporting that as an abnormal + # shutdown case. + if not self._has_quit: + raise + finally: + self._qmp_connection = None + def _early_cleanup(self) -> None: """ Perform any cleanup that needs to happen before the VM exits. @@ -461,8 +492,18 @@ def _soft_shutdown(self, timeout: Optional[int]) -> None: if self._qmp_connection: if not self._has_quit: - # Might raise ConnectionReset - self.qmp('quit') + try: + # May raise ExecInterruptedError or StateError if the + # connection dies or has already died. + self.qmp('quit') + finally: + # If 'quit' fails, we'll still want to call close(), + # which will raise any final errors that may have + # occurred while trying to send 'quit'. + self._close_qmp_connection() + else: + # Regardless, we want to tidy up the socket. + self._close_qmp_connection() # May raise subprocess.TimeoutExpired self._subp.wait(timeout=timeout) From patchwork Fri Sep 17 05:40:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8171BC433F5 for ; Fri, 17 Sep 2021 05:57:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1324361019 for ; Fri, 17 Sep 2021 05:57:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1324361019 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:37326 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6rs-00031u-AY for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:57:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48342) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cw-0005JX-LL for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:38993) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cp-0003cy-F0 for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857282; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vuzZfn/XMSo92JCqVP8WeIskaIB6XTwn/YGc4lWALmI=; b=VaA+bU3IkeXuiDd5fS3Lg3QNb8xBrJM8DydGkR5wFyyIJ1E9f9QV2r2EvJsaN45vK7REnI 46KRQ/uMM+jFZOCSLasrMQ1eXP8jaH3j0zmYcyuhvJJEPkLnRzT30VZcVI94AUolW5wOp+ lwQ8Vxp9YbZtkirKMqKoUjXqz1wUjM4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-203-uZXK43_ENTOUtzZNspef2Q-1; Fri, 17 Sep 2021 01:41:21 -0400 X-MC-Unique: uZXK43_ENTOUtzZNspef2Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 90BEB1808308; Fri, 17 Sep 2021 05:41:20 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id E54047A8CA; Fri, 17 Sep 2021 05:41:17 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 11/15] python/aqmp: Create sync QMP wrapper for iotests Date: Fri, 17 Sep 2021 01:40:43 -0400 Message-Id: <20210917054047.2042843-12-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz --- python/qemu/aqmp/legacy.py | 131 +++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 python/qemu/aqmp/legacy.py diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py new file mode 100644 index 0000000000..b8544083ad --- /dev/null +++ b/python/qemu/aqmp/legacy.py @@ -0,0 +1,131 @@ +""" +Sync QMP Wrapper + +This class pretends to be qemu.qmp.QEMUMonitorProtocol. +""" + +import asyncio +from typing import ( + Awaitable, + List, + Optional, + TypeVar, + Union, +) + +import qemu.qmp +from qemu.qmp import QMPMessage, QMPReturnValue, SocketAddrT + +from .qmp_client import QMPClient + + +# pylint: disable=missing-docstring + + +class QEMUMonitorProtocol(qemu.qmp.QEMUMonitorProtocol): + def __init__(self, address: SocketAddrT, + server: bool = False, + nickname: Optional[str] = None): + + # pylint: disable=super-init-not-called + self._aqmp = QMPClient(nickname) + self._aloop = asyncio.get_event_loop() + self._address = address + self._timeout: Optional[float] = None + + _T = TypeVar('_T') + + def _sync( + self, future: Awaitable[_T], timeout: Optional[float] = None + ) -> _T: + return self._aloop.run_until_complete( + asyncio.wait_for(future, timeout=timeout) + ) + + # __enter__ and __exit__ need no changes + # parse_address needs no changes + + def connect(self, negotiate: bool = True) -> Optional[QMPMessage]: + self._aqmp.await_greeting = negotiate + self._aqmp.negotiate = negotiate + + self._sync( + self._aqmp.connect(self._address) + ) + + if self._aqmp.greeting is not None: + return dict(self._aqmp.greeting) + return None + + def accept(self, timeout: Optional[float] = 15.0) -> QMPMessage: + self._aqmp.await_greeting = True + self._aqmp.negotiate = True + + self._sync( + self._aqmp.accept(self._address), + timeout + ) + + assert self._aqmp.greeting is not None + return dict(self._aqmp.greeting) + + def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: + return dict( + self._sync( + # pylint: disable=protected-access + + # _raw() isn't a public API, because turning off + # automatic ID assignment is discouraged. For + # compatibility with iotests *only*, do it anyway. + self._aqmp._raw(qmp_cmd, assign_id=False), + self._timeout + ) + ) + + # Default impl of cmd() delegates to cmd_obj + + def command(self, cmd: str, **kwds: object) -> QMPReturnValue: + return self._sync( + self._aqmp.execute(cmd, kwds), + self._timeout + ) + + def pull_event(self, + wait: Union[bool, float] = False) -> Optional[QMPMessage]: + if wait is False: + # Return None if there's no event ready to go + if self._aqmp.events.empty(): + return None + + timeout = None + if isinstance(wait, float): + timeout = wait + + return dict( + self._sync( + self._aqmp.events.get(), + timeout + ) + ) + + def get_events(self, wait: Union[bool, float] = False) -> List[QMPMessage]: + events = [dict(x) for x in self._aqmp.events.clear()] + if events: + return events + + event = self.pull_event(wait) + return [event] if event is not None else [] + + def clear_events(self) -> None: + self._aqmp.events.clear() + + def close(self) -> None: + self._sync( + self._aqmp.disconnect() + ) + + def settimeout(self, timeout: Optional[float]) -> None: + self._timeout = timeout + + def send_fd_scm(self, fd: int) -> None: + self._aqmp.send_fd_scm(fd) From patchwork Fri Sep 17 05:40:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F3FDC433F5 for ; Fri, 17 Sep 2021 05:48:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B353861130 for ; Fri, 17 Sep 2021 05:48:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B353861130 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:43696 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6jg-0005Ay-O4 for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:48:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cv-0005Go-OE for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:40732) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cp-0003dS-Fe for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qkwvaqgYVXKJ52bAFsA3LmVzXWBREOn5KcDadOq0Zmo=; b=QkO8ey3hBw9zHuDXw9Xue/RZjiJ9UbbQIag1WG7HxZw0clYPX/jenCjxbbGmBmj/EzAlTC fXKVs6CPQMUOmSuhCFLqxPNP0etgwGQDIuIgCBAsLK+lerZdd40RMaeXbjfNLXLb3aFXhT Ru6wNCi4UXKf8bHm1cTSimen3Xo+xO0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-91-WuHlwuG3MiWDP8NnrBkYIg-1; Fri, 17 Sep 2021 01:41:23 -0400 X-MC-Unique: WuHlwuG3MiWDP8NnrBkYIg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5A2271084681; Fri, 17 Sep 2021 05:41:22 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id C365F77F3C; Fri, 17 Sep 2021 05:41:20 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 12/15] iotests: Disable AQMP logging under non-debug modes Date: Fri, 17 Sep 2021 01:40:44 -0400 Message-Id: <20210917054047.2042843-13-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Disable the aqmp logger, which likes to (at the moment) print out intermediate warnings and errors that cause session termination; disable them so they don't interfere with the job output. Leave any "CRITICAL" warnings enabled though, those are ones that we should never see, no matter what. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 273d2777ae..47e5f9738b 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -1383,6 +1383,8 @@ def execute_setup_common(supported_fmts: Sequence[str] = (), if debug: sys.argv.remove('-d') logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN)) + if not debug: + logging.getLogger("qemu.aqmp.qmp_client").setLevel(logging.CRITICAL) _verify_image_format(supported_fmts, unsupported_fmts) _verify_protocol(supported_protocols, unsupported_protocols) From patchwork Fri Sep 17 05:40:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C978CC433F5 for ; Fri, 17 Sep 2021 05:54:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6475761019 for ; Fri, 17 Sep 2021 05:54:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6475761019 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:59180 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6pE-00077A-Fa for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:54:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48412) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cy-0005PO-QT for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22306) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cq-0003fk-En for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m6EUxYxhOFzLU6kuss4XhdFLrG7URHqgFVAK7WCGG9Y=; b=TgQZbI62fy8jXGtWQH0TJy3803cOJMmKzKn5svKPJ4Z42dTZ9cUH52G1kU35DNusdI5bJy A67APIXnfoY3icwkqzNGAcmgm4Tl8lKLt57b2FQaKj1XEZ8SPlde6ZzkfmzLAIoT9SzriI 01P9iabZ5qn5SIFJZfK4InCAh4V3QS4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-tjj2HYjnMTmgFgyvL74yXg-1; Fri, 17 Sep 2021 01:41:25 -0400 X-MC-Unique: tjj2HYjnMTmgFgyvL74yXg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 53AC0824FA6; Fri, 17 Sep 2021 05:41:24 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B56877F3C; Fri, 17 Sep 2021 05:41:22 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 13/15] iotests: Accommodate async QMP Exception classes Date: Fri, 17 Sep 2021 01:40:45 -0400 Message-Id: <20210917054047.2042843-14-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" (But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow --- scripts/simplebench/bench_block_job.py | 3 ++- tests/qemu-iotests/tests/mirror-top-perms | 6 +++++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/scripts/simplebench/bench_block_job.py b/scripts/simplebench/bench_block_job.py index 4f03c12169..a403c35b08 100755 --- a/scripts/simplebench/bench_block_job.py +++ b/scripts/simplebench/bench_block_job.py @@ -28,6 +28,7 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu.machine import QEMUMachine from qemu.qmp import QMPConnectError +from qemu.aqmp import ConnectError def bench_block_job(cmd, cmd_args, qemu_args): @@ -49,7 +50,7 @@ def bench_block_job(cmd, cmd_args, qemu_args): vm.launch() except OSError as e: return {'error': 'popen failed: ' + str(e)} - except (QMPConnectError, socket.timeout): + except (QMPConnectError, ConnectError, socket.timeout): return {'error': 'qemu failed: ' + str(vm.get_log())} try: diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests/tests/mirror-top-perms index 451a0666f8..7d448f4d23 100755 --- a/tests/qemu-iotests/tests/mirror-top-perms +++ b/tests/qemu-iotests/tests/mirror-top-perms @@ -103,7 +103,11 @@ class TestMirrorTopPerms(iotests.QMPTestCase): print('ERROR: VM B launched successfully, this should not have ' 'happened') except qemu.qmp.QMPConnectError: - assert 'Is another process using the image' in self.vm_b.get_log() + pass + except qemu.aqmp.ConnectError: + pass + + assert 'Is another process using the image' in self.vm_b.get_log() result = self.vm.qmp('block-job-cancel', device='mirror') From patchwork Fri Sep 17 05:40:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B076FC433F5 for ; Fri, 17 Sep 2021 05:55:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5AE5061019 for ; Fri, 17 Sep 2021 05:55:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5AE5061019 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:35498 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6qk-0001jf-Gw for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:55:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cz-0005RN-Gh for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27566) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cr-0003hY-Pb for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857288; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WWrmhH1ip3ipDC/lhH/iOOYCmIZtvP0xaTNg4tVJiwM=; b=bFeNv18zSB6OsHspppGscjIyDQLbAZK18Vzjnec1/1ymvbKKI6srdNNfob9NYOsNO1/8Ft T3+5/x/ByQxnrD44x7bCR8fO3LERJDXHYfoDRm2mOk4rtwuE4RBd4QO5b5i8cak+tfEmPe RdSm7fMhVtRvAnDpz5iZCF5gvWvof80= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-371-vlYmUwLSOGOF7c1Fml5r2Q-1; Fri, 17 Sep 2021 01:41:27 -0400 X-MC-Unique: vlYmUwLSOGOF7c1Fml5r2Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 50E70824FA6; Fri, 17 Sep 2021 05:41:26 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id EED1160843; Fri, 17 Sep 2021 05:41:24 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 14/15] python/aqmp: Remove scary message Date: Fri, 17 Sep 2021 01:40:46 -0400 Message-Id: <20210917054047.2042843-15-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow --- python/qemu/aqmp/__init__.py | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py index ab1782999c..4b7df53e00 100644 --- a/python/qemu/aqmp/__init__.py +++ b/python/qemu/aqmp/__init__.py @@ -21,8 +21,6 @@ # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -import warnings - from .error import AQMPError from .events import EventListener from .message import Message @@ -30,18 +28,6 @@ from .qmp_client import ExecInterruptedError, ExecuteError, QMPClient -_WMSG = """ - -The Asynchronous QMP library is currently in development and its API -should be considered highly fluid and subject to change. It should -not be used by any other scripts checked into the QEMU tree. - -Proceed with caution! -""" - -warnings.warn(_WMSG, FutureWarning) - - # The order of these fields impact the Sphinx documentation order. __all__ = ( # Classes, most to least important From patchwork Fri Sep 17 05:40:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 12501089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 967B6C433F5 for ; Fri, 17 Sep 2021 05:59:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4D29860F43 for ; Fri, 17 Sep 2021 05:59:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4D29860F43 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:41910 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mR6uH-00064k-HG for qemu-devel@archiver.kernel.org; Fri, 17 Sep 2021 01:59:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6d1-0005Tz-2d for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:20566) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mR6cu-0003kW-RH for qemu-devel@nongnu.org; Fri, 17 Sep 2021 01:41:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631857292; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9jCyR/xCKfjQQeBsjS5y4/tYPLeBcVRFK1FtN6PZxro=; b=OMOino5yQxqSyG8RUmuO1pq3EhsEhJqGD8aleXuhXFJvtst6k4oyJt/AeV4HrbFm3ykXsj sKmre0G/mEouWH9jc5HPBZfwTL0EILOoSgggIMJlbajDfv3WCqMhT7Ln6DrDAq4TQnXQZQ +pH0K3G7jkjdxXhj6nSlF2i1Mpw3WQk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-39-zAShy2RFO9aQ4a0pUXVj2Q-1; Fri, 17 Sep 2021 01:41:29 -0400 X-MC-Unique: zAShy2RFO9aQ4a0pUXVj2Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EB8731006AA7; Fri, 17 Sep 2021 05:41:27 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.11.132]) by smtp.corp.redhat.com (Postfix) with ESMTP id 923D360843; Fri, 17 Sep 2021 05:41:26 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 15/15] python, iotests: replace qmp with aqmp Date: Fri, 17 Sep 2021 01:40:47 -0400 Message-Id: <20210917054047.2042843-16-jsnow@redhat.com> In-Reply-To: <20210917054047.2042843-1-jsnow@redhat.com> References: <20210917054047.2042843-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync wrapper from qemu.aqmp instead. Add an escape hatch in the form of the environment variable QEMU_PYTHON_LEGACY_QMP which allows you to cajole QEMUMachine into using the old interface, proving that both implementations work concurrently. Signed-off-by: John Snow Reviewed-by: Hanna Reitz Tested-by: Hanna Reitz --- python/qemu/machine/machine.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index 8f5a6649e5..6b005dd5d1 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -41,7 +41,6 @@ ) from qemu.qmp import ( # pylint: disable=import-error - QEMUMonitorProtocol, QMPMessage, QMPReturnValue, SocketAddrT, @@ -50,6 +49,12 @@ from . import console_socket +if os.environ.get('QEMU_PYTHON_LEGACY_QMP'): + from qemu.qmp import QEMUMonitorProtocol +else: + from qemu.aqmp.legacy import QEMUMonitorProtocol + + LOG = logging.getLogger(__name__)