From patchwork Mon Oct 26 10:06:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856259 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B135C1130 for ; Mon, 26 Oct 2020 10:07:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4EFA1222EC for ; Mon, 26 Oct 2020 10:07:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EpIR0FiC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4EFA1222EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:60874 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzPq-00045l-3P for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:07:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOn-0002P8-DT for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:20865) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOl-0006ZR-2u for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=U/8TwMow7JFz41G6l7+MQj+1KDG2MIh2y51jl6IvirY=; b=EpIR0FiCSEaBA+mAv9OJRIidiFHLkKasEA3TWpqV3yiz+Prsz9UfMlENqI5Yf2lV2ogqjl Iavvd5ssF1kMX3MqE5K2HFrfb2nlcBHz2c5h2LutDjmqnIMbR+thKJucHFcK9lVS01I4/A eDtNJ4Yk5QiWHoe+u5lNcTFBqtLu1rM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-332-TA7i9RzbMiWUjGe8AM3IIw-1; Mon, 26 Oct 2020 06:06:38 -0400 X-MC-Unique: TA7i9RzbMiWUjGe8AM3IIw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6FB031882FA7; Mon, 26 Oct 2020 10:06:37 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5E28E8B849; Mon, 26 Oct 2020 10:06:36 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 01/31] tests/migration: fix memleak in wait_command/wait_command_fd Date: Mon, 26 Oct 2020 11:06:02 +0100 Message-Id: <20201026100632.212530-2-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Chen Qun Properly free each command resp to avoid memory leak. ASAN shows memory leak stack: Indirect leak of 2352520 byte(s) in 571 object(s) allocated from: #0 0x7f6ca3308d4e in __interceptor_calloc (/lib64/libasan.so.5+0x112d4e) #1 0x7f6ca3127a50 in g_malloc0 (/lib64/libglib-2.0.so.0+0x55a50) #2 0x557bf3c71d2b in qdict_new ../qobject/qdict.c:29 #3 0x557bf3c9caba in parse_object ../qobject/json-parser.c:318 #4 0x557bf3c9ce75 in json_parser_parse ../qobject/json-parser.c:580 #5 0x557bf3c8c8cf in json_message_process_token ../qobject/json-streamer.c:92 #6 0x557bf3c9ea59 in json_lexer_feed_char ../qobject/json-lexer.c:313 #7 0x557bf3c9eeb5 in json_lexer_feed ../qobject/json-lexer.c:350 #8 0x557bf3c4793a in qmp_fd_receive ../tests/qtest/libqtest.c:608 #9 0x557bf3c47b58 in qtest_qmp_receive ../tests/qtest/libqtest.c:618 #10 0x557bf3c44245 in wait_command ../tests/qtest/migration-helpers.c:59 #11 0x557bf3c445cb in migrate_query_status ../tests/qtest/migration-helpers.c:108 #12 0x557bf3c44642 in check_migration_status ../tests/qtest/migration-helpers.c:124 #13 0x557bf3c447e7 in wait_for_migration_status ../tests/qtest/migration-helpers.c:148 #14 0x557bf3c43b8f in test_migrate_auto_converge ../tests/qtest/migration-test.c:1243 ...... Fix: 5e34005571af5 Reported-by: Euler Robot Signed-off-by: Chen Qun Message-Id: <20201023061218.2080844-2-kuhn.chenqun@huawei.com> Signed-off-by: Thomas Huth --- tests/qtest/migration-helpers.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c index b799dbafb7..4ee26014b7 100644 --- a/tests/qtest/migration-helpers.c +++ b/tests/qtest/migration-helpers.c @@ -32,7 +32,7 @@ static void check_stop_event(QTestState *who) QDict *wait_command_fd(QTestState *who, int fd, const char *command, ...) { va_list ap; - QDict *resp; + QDict *resp, *ret; va_start(ap, command); qtest_qmp_vsend_fds(who, &fd, 1, command, ap); @@ -44,7 +44,11 @@ QDict *wait_command_fd(QTestState *who, int fd, const char *command, ...) g_assert(!qdict_haskey(resp, "error")); g_assert(qdict_haskey(resp, "return")); - return qdict_get_qdict(resp, "return"); + ret = qdict_get_qdict(resp, "return"); + qobject_ref(ret); + qobject_unref(resp); + + return ret; } /* @@ -53,7 +57,7 @@ QDict *wait_command_fd(QTestState *who, int fd, const char *command, ...) QDict *wait_command(QTestState *who, const char *command, ...) { va_list ap; - QDict *resp; + QDict *resp, *ret; va_start(ap, command); resp = qtest_vqmp(who, command, ap); @@ -64,7 +68,11 @@ QDict *wait_command(QTestState *who, const char *command, ...) g_assert(!qdict_haskey(resp, "error")); g_assert(qdict_haskey(resp, "return")); - return qdict_get_qdict(resp, "return"); + ret = qdict_get_qdict(resp, "return"); + qobject_ref(ret); + qobject_unref(resp); + + return ret; } /* From patchwork Mon Oct 26 10:06:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E855714C0 for ; Mon, 26 Oct 2020 10:09:53 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A28C2223AC for ; Mon, 26 Oct 2020 10:09:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Cj0yF21U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A28C2223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:41096 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzRo-0007Wt-Lz for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:09:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38670) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOp-0002QL-Qo for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:26280) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOl-0006aN-DS for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=+LT4RxKVcAVhYRwncs0L+BQ1wRj6aHo9PM7a1P0TdeU=; b=Cj0yF21UHXYtFH8YURMQELE7Oc/ni9R7LZHzsgAG/T68tzTtNw1xOMHCYVc7QoSt2jTHJy YU6ijikwSMdnzZW0dUZASSau+5Xau9R1wvoBeYvb385SVDPsRMkHoH2Ibf1VlAMY0JQaqN Kw45SOHjPCCOTLVR2YnZeaL7QSBSK/U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-390-IPZHlRcPPEa5doNrH5yJmw-1; Mon, 26 Oct 2020 06:06:40 -0400 X-MC-Unique: IPZHlRcPPEa5doNrH5yJmw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DBE295A09A; Mon, 26 Oct 2020 10:06:38 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE0A38B843; Mon, 26 Oct 2020 10:06:37 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 02/31] tests/qtest/libqtest: Fix detection of architecture for binaries without path Date: Mon, 26 Oct 2020 11:06:03 +0100 Message-Id: <20201026100632.212530-3-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The qtests can be run directly by specifying the QEMU binary with the QTEST_QEMU_BINARY environment variable, for example: $ QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 tests/qtest/test-hmp However, if you specify a binary without a path, for example with QTEST_QEMU_BINARY=qemu-system-x86_64 if the QEMU binary is in your $PATH, then the test currently simply crashes. Let's try a little bit smarter here by looking for the final '-' instead of the slash. Message-Id: <20201012114816.43546-1-thuth@redhat.com> Signed-off-by: Thomas Huth --- tests/qtest/libqtest.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index 08929f5ff6..b9ff29055b 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -870,9 +870,14 @@ char *qtest_hmp(QTestState *s, const char *fmt, ...) const char *qtest_get_arch(void) { const char *qemu = qtest_qemu_binary(); - const char *end = strrchr(qemu, '/'); + const char *end = strrchr(qemu, '-'); - return end + strlen("/qemu-system-"); + if (!end) { + fprintf(stderr, "Can't determine architecture from binary name.\n"); + abort(); + } + + return end + 1; } bool qtest_get_irq(QTestState *s, int num) From patchwork Mon Oct 26 10:06:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856257 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 803056A2 for ; Mon, 26 Oct 2020 10:07:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 28A2622250 for ; Mon, 26 Oct 2020 10:07:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="K0lWrdP+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28A2622250 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:60978 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzPp-00048F-Tr for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:07:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38652) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOo-0002PL-AN for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:54783) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOm-0006an-Im for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=V0PRhaPjLl3QP5oLM98HuwMXBLJa28Xqbdj0nuobKHE=; b=K0lWrdP+cGJJzBmF4BK/AvXlBiJyfio9ui532fa/gm/4Gi0AB701tVWZx4pg1ER4H2ZYci iyCidmJptr8WyAIhNq5Nycx72dXg+h9LANUoamRWFuSimXeLvd00C74P2XBXx38OR7rhjm qrluHM1MpXsrOw1odlGMAUv9dp6ypmA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-77-uKTaoNz9NzC6lHhY_ahv3Q-1; Mon, 26 Oct 2020 06:06:41 -0400 X-MC-Unique: uKTaoNz9NzC6lHhY_ahv3Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4F7976123B; Mon, 26 Oct 2020 10:06:40 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43B6F8B841; Mon, 26 Oct 2020 10:06:39 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 03/31] accel: Remove _WIN32 ifdef from qtest-cpus.c Date: Mon, 26 Oct 2020 11:06:04 +0100 Message-Id: <20201026100632.212530-4-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Jason Andryuk dummy-cpus.c is only compiled with CONFIG_POSIX, so the _WIN32 condition will never evaluate true. Remove it. Signed-off-by: Jason Andryuk Message-Id: <20201013140511.5681-2-jandryuk@gmail.com> Acked-by: Paolo Bonzini Reviewed-by: Claudio Fontana Reviewed-by: Thomas Huth Signed-off-by: Thomas Huth --- accel/qtest/qtest-cpus.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/accel/qtest/qtest-cpus.c b/accel/qtest/qtest-cpus.c index 7c5399ed9d..db094201c1 100644 --- a/accel/qtest/qtest-cpus.c +++ b/accel/qtest/qtest-cpus.c @@ -29,10 +29,6 @@ static void *qtest_cpu_thread_fn(void *arg) { -#ifdef _WIN32 - error_report("qtest is not supported under Windows"); - exit(1); -#else CPUState *cpu = arg; sigset_t waitset; int r; @@ -69,7 +65,6 @@ static void *qtest_cpu_thread_fn(void *arg) qemu_mutex_unlock_iothread(); rcu_unregister_thread(); return NULL; -#endif } static void qtest_start_vcpu_thread(CPUState *cpu) From patchwork Mon Oct 26 10:06:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856263 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BFE3C1130 for ; Mon, 26 Oct 2020 10:07:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5994822250 for ; Mon, 26 Oct 2020 10:07:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="adU1+k15" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5994822250 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:33126 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzPu-0004HX-37 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:07:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38692) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOr-0002Rq-JS for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29314) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOn-0006bY-T5 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=TGGBqiGB6r8dKyjh5nHD5mqpIoJiTqKb6xBSBz5qwVQ=; b=adU1+k15TD42zyv0SxTzGCZ5DSTuv4KPkKTa6SgwqeyvfFbBiXFyj0EaEWqYkbp2oSGuJq fYDKemT5mZ6XcanlwErNvqnjA/vgWNqFgWusFE5QdEmvdsqL+K2WvUXl0Z1tM78JmaiIRE lO86VgveMbIRE7wXuK0LRLseFGUS8+g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-259-cjBst0nUOPyE8xXlSVs-pg-1; Mon, 26 Oct 2020 06:06:42 -0400 X-MC-Unique: cjBst0nUOPyE8xXlSVs-pg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BA04380B714; Mon, 26 Oct 2020 10:06:41 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADAA88B846; Mon, 26 Oct 2020 10:06:40 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 04/31] accel: move qtest CpusAccel functions to a common location Date: Mon, 26 Oct 2020 11:06:05 +0100 Message-Id: <20201026100632.212530-5-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Jason Andryuk Move and rename accel/qtest/qtest-cpus.c files to accel/dummy-cpus.c so it can be re-used by Xen. Signed-off-by: Jason Andryuk Message-Id: <20201013140511.5681-3-jandryuk@gmail.com> Reviewed-by: Claudio Fontana Acked-by: Paolo Bonzini Signed-off-by: Thomas Huth --- accel/{qtest/qtest-cpus.c => dummy-cpus.c} | 22 ++++------------------ accel/meson.build | 7 +++++++ accel/qtest/meson.build | 1 - accel/qtest/qtest-cpus.h | 17 ----------------- accel/qtest/qtest.c | 5 ++++- include/sysemu/cpus.h | 3 +++ 6 files changed, 18 insertions(+), 37 deletions(-) rename accel/{qtest/qtest-cpus.c => dummy-cpus.c} (75%) delete mode 100644 accel/qtest/qtest-cpus.h diff --git a/accel/qtest/qtest-cpus.c b/accel/dummy-cpus.c similarity index 75% rename from accel/qtest/qtest-cpus.c rename to accel/dummy-cpus.c index db094201c1..10429fdfb2 100644 --- a/accel/qtest/qtest-cpus.c +++ b/accel/dummy-cpus.c @@ -1,5 +1,5 @@ /* - * QTest accelerator code + * Dummy cpu thread code * * Copyright IBM, Corp. 2011 * @@ -13,21 +13,12 @@ #include "qemu/osdep.h" #include "qemu/rcu.h" -#include "qapi/error.h" -#include "qemu/module.h" -#include "qemu/option.h" -#include "qemu/config-file.h" -#include "sysemu/accel.h" -#include "sysemu/qtest.h" #include "sysemu/cpus.h" -#include "sysemu/cpu-timers.h" #include "qemu/guest-random.h" #include "qemu/main-loop.h" #include "hw/core/cpu.h" -#include "qtest-cpus.h" - -static void *qtest_cpu_thread_fn(void *arg) +static void *dummy_cpu_thread_fn(void *arg) { CPUState *cpu = arg; sigset_t waitset; @@ -67,7 +58,7 @@ static void *qtest_cpu_thread_fn(void *arg) return NULL; } -static void qtest_start_vcpu_thread(CPUState *cpu) +void dummy_start_vcpu_thread(CPUState *cpu) { char thread_name[VCPU_THREAD_NAME_SIZE]; @@ -76,11 +67,6 @@ static void qtest_start_vcpu_thread(CPUState *cpu) qemu_cond_init(cpu->halt_cond); snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/DUMMY", cpu->cpu_index); - qemu_thread_create(cpu->thread, thread_name, qtest_cpu_thread_fn, cpu, + qemu_thread_create(cpu->thread, thread_name, dummy_cpu_thread_fn, cpu, QEMU_THREAD_JOINABLE); } - -const CpusAccel qtest_cpus = { - .create_vcpu_thread = qtest_start_vcpu_thread, - .get_virtual_clock = qtest_get_virtual_clock, -}; diff --git a/accel/meson.build b/accel/meson.build index bb00d0fd13..9a417396bd 100644 --- a/accel/meson.build +++ b/accel/meson.build @@ -5,3 +5,10 @@ subdir('kvm') subdir('tcg') subdir('xen') subdir('stubs') + +dummy_ss = ss.source_set() +dummy_ss.add(files( + 'dummy-cpus.c', +)) + +specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: dummy_ss) diff --git a/accel/qtest/meson.build b/accel/qtest/meson.build index e477cb2ae2..a2f3276459 100644 --- a/accel/qtest/meson.build +++ b/accel/qtest/meson.build @@ -1,7 +1,6 @@ qtest_ss = ss.source_set() qtest_ss.add(files( 'qtest.c', - 'qtest-cpus.c', )) specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: qtest_ss) diff --git a/accel/qtest/qtest-cpus.h b/accel/qtest/qtest-cpus.h deleted file mode 100644 index 739519a472..0000000000 --- a/accel/qtest/qtest-cpus.h +++ /dev/null @@ -1,17 +0,0 @@ -/* - * Accelerator CPUS Interface - * - * Copyright 2020 SUSE LLC - * - * This work is licensed under the terms of the GNU GPL, version 2 or later. - * See the COPYING file in the top-level directory. - */ - -#ifndef QTEST_CPUS_H -#define QTEST_CPUS_H - -#include "sysemu/cpus.h" - -extern const CpusAccel qtest_cpus; - -#endif /* QTEST_CPUS_H */ diff --git a/accel/qtest/qtest.c b/accel/qtest/qtest.c index 537e8b449c..b282cea5cf 100644 --- a/accel/qtest/qtest.c +++ b/accel/qtest/qtest.c @@ -25,7 +25,10 @@ #include "qemu/main-loop.h" #include "hw/core/cpu.h" -#include "qtest-cpus.h" +const CpusAccel qtest_cpus = { + .create_vcpu_thread = dummy_start_vcpu_thread, + .get_virtual_clock = qtest_get_virtual_clock, +}; static int qtest_init_accel(MachineState *ms) { diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h index 231685955d..e8156728c6 100644 --- a/include/sysemu/cpus.h +++ b/include/sysemu/cpus.h @@ -25,6 +25,9 @@ typedef struct CpusAccel { /* register accel-specific cpus interface implementation */ void cpus_register_accel(const CpusAccel *i); +/* Create a dummy vcpu for CpusAccel->create_vcpu_thread */ +void dummy_start_vcpu_thread(CPUState *); + /* interface available for cpus accelerator threads */ /* For temporary buffers for forming a name */ From patchwork Mon Oct 26 10:06:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856261 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BDDA1130 for ; Mon, 26 Oct 2020 10:07:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D428422250 for ; Mon, 26 Oct 2020 10:07:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="T0FnZH95" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D428422250 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:32998 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzPs-0004ET-Nl for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:07:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38698) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOs-0002T6-7m for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34842) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOp-0006cB-9H for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=7py8/BqSRsB8PFotGN4dp1fcL9wxowisE7ZbOTCvJQs=; b=T0FnZH95nVSnQIVllOAZgHoIiNFMB2q9ORHqrMSoSn1tWlGwhiJguehWG3TXNOQowBM9gH QvBx9iRb+ykG7v5GJHYg0xeUTBoGUArL9vmwrLuugFyUKm48aSJdp9jnQc9c5AdIsITc7+ UZwDn5ZBghMoNWfuvTNtlEIZdWHEZTE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-353-MqbfGWHMPYqtKODVByZVCA-1; Mon, 26 Oct 2020 06:06:44 -0400 X-MC-Unique: MqbfGWHMPYqtKODVByZVCA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2E934800D53; Mon, 26 Oct 2020 10:06:43 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 238398B846; Mon, 26 Oct 2020 10:06:41 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 05/31] accel: Add xen CpusAccel using dummy-cpus Date: Mon, 26 Oct 2020 11:06:06 +0100 Message-Id: <20201026100632.212530-6-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Jason Andryuk Xen was broken by commit 1583a3898853 ("cpus: extract out qtest-specific code to accel/qtest"). Xen relied on qemu_init_vcpu() calling qemu_dummy_start_vcpu() in the default case, but that was replaced by g_assert_not_reached(). Add a minimal "CpusAccel" for Xen using the dummy-cpus implementation used by qtest. Signed-off-by: Jason Andryuk Message-Id: <20201013140511.5681-4-jandryuk@gmail.com> Acked-by: Paolo Bonzini Reviewed-by: Claudio Fontana Acked-by: Anthony PERARD Signed-off-by: Thomas Huth --- accel/meson.build | 1 + accel/xen/xen-all.c | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/accel/meson.build b/accel/meson.build index 9a417396bd..b26cca227a 100644 --- a/accel/meson.build +++ b/accel/meson.build @@ -12,3 +12,4 @@ dummy_ss.add(files( )) specific_ss.add_all(when: ['CONFIG_SOFTMMU', 'CONFIG_POSIX'], if_true: dummy_ss) +specific_ss.add_all(when: ['CONFIG_XEN'], if_true: dummy_ss) diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c index 60b971d0a8..878a4089d9 100644 --- a/accel/xen/xen-all.c +++ b/accel/xen/xen-all.c @@ -16,6 +16,7 @@ #include "hw/xen/xen_pt.h" #include "chardev/char.h" #include "sysemu/accel.h" +#include "sysemu/cpus.h" #include "sysemu/xen.h" #include "sysemu/runstate.h" #include "migration/misc.h" @@ -153,6 +154,10 @@ static void xen_setup_post(MachineState *ms, AccelState *accel) } } +const CpusAccel xen_cpus = { + .create_vcpu_thread = dummy_start_vcpu_thread, +}; + static int xen_init(MachineState *ms) { MachineClass *mc = MACHINE_GET_CLASS(ms); @@ -180,6 +185,9 @@ static int xen_init(MachineState *ms) * opt out of system RAM being allocated by generic code */ mc->default_ram_id = NULL; + + cpus_register_accel(&xen_cpus); + return 0; } From patchwork Mon Oct 26 10:06:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 879A1921 for ; Mon, 26 Oct 2020 10:12:57 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2FC5F223AC for ; Mon, 26 Oct 2020 10:12:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RVkzIAbY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2FC5F223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:49446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzUl-0002lL-ST for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:12:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38704) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOs-0002TL-C6 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50038) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOq-0006ch-N8 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=q3npmmAYQgwjsXjr7HvTSfti3EGmRvKx+HzB3bP0fMU=; b=RVkzIAbY+UBsSGiuXKjth56rRC5YLayos3ZUA6jreK5BpcToWa0WlMw3TgrWNKrN+UQUB/ 6DL9jTOSXFuFLygbdqJC9rcZcRASn4U9E4btTH/X9fpeDTAqPXHwdocfaLnIHhRqGMuWiI emYr38MRDtfpueIIjy2lCnHDzm8fN+c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-433-PffQBB1kPv-dYq66yNKTjQ-1; Mon, 26 Oct 2020 06:06:45 -0400 X-MC-Unique: PffQBB1kPv-dYq66yNKTjQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 992716123A; Mon, 26 Oct 2020 10:06:44 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E86E8B843; Mon, 26 Oct 2020 10:06:43 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 06/31] tests/qtest: Make npcm7xx_timer-test conditional on CONFIG_NPCM7XX Date: Mon, 26 Oct 2020 11:06:07 +0100 Message-Id: <20201026100632.212530-7-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Havard Skinnemoen This test won't work if qemu was compiled without CONFIG_NPCM7XX, as pointed out by Thomas Huth on a different patch. Signed-off-by: Havard Skinnemoen Message-Id: <20201023210637.351238-2-hskinnemoen@google.com> Signed-off-by: Thomas Huth --- tests/qtest/meson.build | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 28d4068718..7e0ecaa2c5 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -133,12 +133,13 @@ qtests_sparc64 = \ (config_all_devices.has_key('CONFIG_ISA_TESTDEV') ? ['endianness-test'] : []) + \ ['prom-env-test', 'boot-serial-test'] +qtests_npcm7xx = ['npcm7xx_timer-test'] qtests_arm = \ (config_all_devices.has_key('CONFIG_PFLASH_CFI02') ? ['pflash-cfi02-test'] : []) + \ + (config_all_devices.has_key('CONFIG_NPCM7XX') ? qtests_npcm7xx : []) + \ ['arm-cpu-features', 'microbit-test', 'm25p80-test', - 'npcm7xx_timer-test', 'test-arm-mptimer', 'boot-serial-test', 'hexloader-test'] From patchwork Mon Oct 26 10:06:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92D8C14B7 for ; Mon, 26 Oct 2020 10:12:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 10CAF223AC for ; Mon, 26 Oct 2020 10:12:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QZLnY7bx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10CAF223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:49522 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzUm-0002nA-T7 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:12:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38722) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOt-0002WF-Ro for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47909) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOs-0006dT-7u for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=6bUL0ui1TLn5ldupLAum8Uuq9onAywLDJjG0T2R42Qw=; b=QZLnY7bxMsm7v+ucVfocb5SWBhKnHoQSrqQtUPiFx988T/aZinWI95PQDAM5dOaZm93yja KZDtfKTw/qdhmsGudhYmPxcJv1HGyy7B4mJJl5uInHjqO9og5SFtVdrH9UY3Ga3cJsy5sq hYNy2dmlI8hTcLvBq9xUZn6GN62D+Tk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-497-rmNs3LsCM5yrPzFioO_a8Q-1; Mon, 26 Oct 2020 06:06:47 -0400 X-MC-Unique: rmNs3LsCM5yrPzFioO_a8Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 483A21882FB4; Mon, 26 Oct 2020 10:06:46 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0338C8B843; Mon, 26 Oct 2020 10:06:44 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 07/31] libqtest: fix the order of buffered events Date: Mon, 26 Oct 2020 11:06:08 +0100 Message-Id: <20201026100632.212530-8-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Maxim Levitsky By a mistake I added the pending events in a wrong order. Fix this by using g_list_append. Signed-off-by: Maxim Levitsky Message-Id: <20201019163702.471239-3-mlevitsk@redhat.com> Signed-off-by: Thomas Huth --- tests/qtest/libqtest.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index b9ff29055b..0e304bdbd1 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -621,7 +621,7 @@ QDict *qtest_qmp_receive(QTestState *s) return response; } /* Stash the event for a later consumption */ - s->pending_events = g_list_prepend(s->pending_events, response); + s->pending_events = g_list_append(s->pending_events, response); } } From patchwork Mon Oct 26 10:06:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DBF95921 for ; Mon, 26 Oct 2020 10:15:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6D9F4223AC for ; Mon, 26 Oct 2020 10:15:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HhpiVsqf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D9F4223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57704 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzWq-0006AL-Au for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:15:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38746) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzOv-0002Zl-Rk for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:47708) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOu-0006ds-1d for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=FxlSdQYmaeFEpuCEpAVY6Alu+5SgRounXmrcaa3zd1M=; b=HhpiVsqf2xSW5958xqGiX8QgesJlRVphE3zrYGDOQsOzwinLEnXnxiLpa/vWGEEf8W7hqu 5aJpxowelpNE6qMakjI7yvPQj+9ywIl6yN/7OVZzvkU/4rzMsk96EPtPy5k9jXdevvjg5+ T8egbY/2ZGfmRvlbM960BAXToVUESIk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-24-ChffQBYOPtihzrAhIV1AdQ-1; Mon, 26 Oct 2020 06:06:48 -0400 X-MC-Unique: ChffQBYOPtihzrAhIV1AdQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C8C7A875110; Mon, 26 Oct 2020 10:06:47 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF06162A0B; Mon, 26 Oct 2020 10:06:46 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 08/31] libqtest: fix memory leak in the qtest_qmp_event_ref Date: Mon, 26 Oct 2020 11:06:09 +0100 Message-Id: <20201026100632.212530-9-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Maxim Levitsky The g_list_remove_link doesn't free the link element, opposed to what I thought. Switch to g_list_delete_link that does free it. Also refactor the code a bit. Thanks for Max Reitz for helping me with this. Signed-off-by: Maxim Levitsky Message-Id: <20201019163702.471239-4-mlevitsk@redhat.com> Signed-off-by: Thomas Huth --- tests/qtest/libqtest.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c index 0e304bdbd1..99deff47ef 100644 --- a/tests/qtest/libqtest.c +++ b/tests/qtest/libqtest.c @@ -795,15 +795,12 @@ void qtest_qmp_send_raw(QTestState *s, const char *fmt, ...) QDict *qtest_qmp_event_ref(QTestState *s, const char *event) { - GList *next = NULL; - QDict *response; - - for (GList *it = s->pending_events; it != NULL; it = next) { + while (s->pending_events) { - next = it->next; - response = (QDict *)it->data; + GList *first = s->pending_events; + QDict *response = (QDict *)first->data; - s->pending_events = g_list_remove_link(s->pending_events, it); + s->pending_events = g_list_delete_link(s->pending_events, first); if (!strcmp(qdict_get_str(response, "event"), event)) { return response; From patchwork Mon Oct 26 10:06:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63A0761C for ; Mon, 26 Oct 2020 10:10:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 10F48223AC for ; Mon, 26 Oct 2020 10:10:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hP+Rno7y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10F48223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:41576 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzSF-0007k3-1s for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:10:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38790) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzP1-0002bE-Sl for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:46573) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOx-0006f2-2x for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:06:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706813; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=Q7utiPbMI+tm18kijyblHQxsmeyYZVDQzQn75XCpiJ8=; b=hP+Rno7y3GLZWok9uWektJI5cSGzUxulM/vcVCfCm5f+zOb//vE3WuOGTtnF5fQi0SGVFF GPAtuIAoQifGow9LlWggpcwECraZ+K0w2aOtRQpUbns6jzevUUr63aU7k7YJgg/xuuviNT e7wBbtKqxvfXCSnOZkHGK+zJ/1LJ5ho= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-282-U-ydJKnnO6Oz4SIS7qtBKw-1; Mon, 26 Oct 2020 06:06:50 -0400 X-MC-Unique: U-ydJKnnO6Oz4SIS7qtBKw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7A1CD10E2188; Mon, 26 Oct 2020 10:06:49 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 609EB8B849; Mon, 26 Oct 2020 10:06:48 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 09/31] memory: Add FlatView foreach function Date: Mon, 26 Oct 2020 11:06:10 +0100 Message-Id: <20201026100632.212530-10-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov Acked-by: Paolo Bonzini Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-2-alxndr@bu.edu> Signed-off-by: Thomas Huth --- include/exec/memory.h | 5 +++++ softmmu/memory.c | 13 +++++++++++++ 2 files changed, 18 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 622207bde1..042918dd16 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -719,6 +719,11 @@ static inline FlatView *address_space_to_flatview(AddressSpace *as) return qatomic_rcu_read(&as->current_map); } +typedef int (*flatview_cb)(Int128 start, + Int128 len, + const MemoryRegion*, void*); + +void flatview_for_each_range(FlatView *fv, flatview_cb cb , void *opaque); /** * struct MemoryRegionSection: describes a fragment of a #MemoryRegion diff --git a/softmmu/memory.c b/softmmu/memory.c index 403ff3abc9..a5d1641820 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -656,6 +656,19 @@ static void render_memory_region(FlatView *view, } } +void flatview_for_each_range(FlatView *fv, flatview_cb cb , void *opaque) +{ + FlatRange *fr; + + assert(fv); + assert(cb); + + FOR_EACH_FLAT_RANGE(fr, fv) { + if (cb(fr->addr.start, fr->addr.size, fr->mr, opaque)) + break; + } +} + static MemoryRegion *memory_region_get_flatview_root(MemoryRegion *mr) { while (mr->enabled) { From patchwork Mon Oct 26 10:06:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5EEC814B7 for ; Mon, 26 Oct 2020 10:13:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DD415223AC for ; Mon, 26 Oct 2020 10:13:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eO2pEMqq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD415223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:50116 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzUs-00031v-TA for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:13:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzP8-0002dL-Jc for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:20728) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzOz-0006fR-4e for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706814; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=zjs/LlfyorZZEN/gZtgJisjjzu6dVnGHDiWn2RTnAxo=; b=eO2pEMqqV6zcMNJitgPlEKSxN+YQDIP2NU7iK5BeXfWeTRULrzFAcvXuatFWZExWxfaI2n XPLwlAtwh264pQ+O5/j6QDPwEHyY43fpfNolhXIuFr5bH9eDyEaZBTFMP5rd2uB3AsZXvt 1YCKeMjmq8/EBo/Se25jRw1fJ35eJ30= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-248-SP1mbj35NHmBGLtyrJS9OA-1; Mon, 26 Oct 2020 06:06:52 -0400 X-MC-Unique: SP1mbj35NHmBGLtyrJS9OA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 13DE0835B91; Mon, 26 Oct 2020 10:06:51 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id C95988B843; Mon, 26 Oct 2020 10:06:49 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 10/31] fuzz: Add generic virtual-device fuzzer Date: Mon, 26 Oct 2020 11:06:11 +0100 Message-Id: <20201026100632.212530-11-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov This is a generic fuzzer designed to fuzz a virtual device's MemoryRegions, as long as they exist within the Memory or Port IO (if it exists) AddressSpaces. The fuzzer's input is interpreted into a sequence of qtest commands (outb, readw, etc). The interpreted commands are separated by a magic seaparator, which should be easy for the fuzzer to guess. Without ASan, the separator can be specified as a "dictionary value" using the -dict argument (see libFuzzer documentation). Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-3-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/generic_fuzz.c | 516 ++++++++++++++++++++++++++++++++ tests/qtest/fuzz/meson.build | 1 + 2 files changed, 517 insertions(+) create mode 100644 tests/qtest/fuzz/generic_fuzz.c diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c new file mode 100644 index 0000000000..6e3faf4e92 --- /dev/null +++ b/tests/qtest/fuzz/generic_fuzz.c @@ -0,0 +1,516 @@ +/* + * Generic Virtual-Device Fuzzing Target + * + * Copyright Red Hat Inc., 2020 + * + * Authors: + * Alexander Bulekov + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" + +#include + +#include "hw/core/cpu.h" +#include "tests/qtest/libqos/libqtest.h" +#include "fuzz.h" +#include "fork_fuzz.h" +#include "exec/address-spaces.h" +#include "string.h" +#include "exec/memory.h" +#include "exec/ramblock.h" +#include "exec/address-spaces.h" +#include "hw/qdev-core.h" + +/* + * SEPARATOR is used to separate "operations" in the fuzz input + */ +#define SEPARATOR "FUZZ" + +enum cmds { + OP_IN, + OP_OUT, + OP_READ, + OP_WRITE, + OP_CLOCK_STEP, +}; + +#define DEFAULT_TIMEOUT_US 100000 +#define USEC_IN_SEC 1000000000 + +typedef struct { + ram_addr_t addr; + ram_addr_t size; /* The number of bytes until the end of the I/O region */ +} address_range; + +static useconds_t timeout = DEFAULT_TIMEOUT_US; + +static bool qtest_log_enabled; + +/* + * List of memory regions that are children of QOM objects specified by the + * user for fuzzing. + */ +static GHashTable *fuzzable_memoryregions; + +struct get_io_cb_info { + int index; + int found; + address_range result; +}; + +static int get_io_address_cb(Int128 start, Int128 size, + const MemoryRegion *mr, void *opaque) { + struct get_io_cb_info *info = opaque; + if (g_hash_table_lookup(fuzzable_memoryregions, mr)) { + if (info->index == 0) { + info->result.addr = (ram_addr_t)start; + info->result.size = (ram_addr_t)size; + info->found = 1; + return 1; + } + info->index--; + } + return 0; +} + +/* + * Here we want to convert a fuzzer-provided [io-region-index, offset] to + * a physical address. To do this, we iterate over all of the matched + * MemoryRegions. Check whether each region exists within the particular io + * space. Return the absolute address of the offset within the index'th region + * that is a subregion of the io_space and the distance until the end of the + * memory region. + */ +static bool get_io_address(address_range *result, AddressSpace *as, + uint8_t index, + uint32_t offset) { + FlatView *view; + view = as->current_map; + g_assert(view); + struct get_io_cb_info cb_info = {}; + + cb_info.index = index; + + /* + * Loop around the FlatView until we match "index" number of + * fuzzable_memoryregions, or until we know that there are no matching + * memory_regions. + */ + do { + flatview_for_each_range(view, get_io_address_cb , &cb_info); + } while (cb_info.index != index && !cb_info.found); + + *result = cb_info.result; + return cb_info.found; +} + +static bool get_pio_address(address_range *result, + uint8_t index, uint16_t offset) +{ + /* + * PIO BARs can be set past the maximum port address (0xFFFF). Thus, result + * can contain an addr that extends past the PIO space. When we pass this + * address to qtest_in/qtest_out, it is cast to a uint16_t, so we might end + * up fuzzing a completely different MemoryRegion/Device. Therefore, check + * that the address here is within the PIO space limits. + */ + bool found = get_io_address(result, &address_space_io, index, offset); + return result->addr <= 0xFFFF ? found : false; +} + +static bool get_mmio_address(address_range *result, + uint8_t index, uint32_t offset) +{ + return get_io_address(result, &address_space_memory, index, offset); +} + +static void op_in(QTestState *s, const unsigned char * data, size_t len) +{ + enum Sizes {Byte, Word, Long, end_sizes}; + struct { + uint8_t size; + uint8_t base; + uint16_t offset; + } a; + address_range abs; + + if (len < sizeof(a)) { + return; + } + memcpy(&a, data, sizeof(a)); + if (get_pio_address(&abs, a.base, a.offset) == 0) { + return; + } + + switch (a.size %= end_sizes) { + case Byte: + qtest_inb(s, abs.addr); + break; + case Word: + if (abs.size >= 2) { + qtest_inw(s, abs.addr); + } + break; + case Long: + if (abs.size >= 4) { + qtest_inl(s, abs.addr); + } + break; + } +} + +static void op_out(QTestState *s, const unsigned char * data, size_t len) +{ + enum Sizes {Byte, Word, Long, end_sizes}; + struct { + uint8_t size; + uint8_t base; + uint16_t offset; + uint32_t value; + } a; + address_range abs; + + if (len < sizeof(a)) { + return; + } + memcpy(&a, data, sizeof(a)); + + if (get_pio_address(&abs, a.base, a.offset) == 0) { + return; + } + + switch (a.size %= end_sizes) { + case Byte: + qtest_outb(s, abs.addr, a.value & 0xFF); + break; + case Word: + if (abs.size >= 2) { + qtest_outw(s, abs.addr, a.value & 0xFFFF); + } + break; + case Long: + if (abs.size >= 4) { + qtest_outl(s, abs.addr, a.value); + } + break; + } +} + +static void op_read(QTestState *s, const unsigned char * data, size_t len) +{ + enum Sizes {Byte, Word, Long, Quad, end_sizes}; + struct { + uint8_t size; + uint8_t base; + uint32_t offset; + } a; + address_range abs; + + if (len < sizeof(a)) { + return; + } + memcpy(&a, data, sizeof(a)); + + if (get_mmio_address(&abs, a.base, a.offset) == 0) { + return; + } + + switch (a.size %= end_sizes) { + case Byte: + qtest_readb(s, abs.addr); + break; + case Word: + if (abs.size >= 2) { + qtest_readw(s, abs.addr); + } + break; + case Long: + if (abs.size >= 4) { + qtest_readl(s, abs.addr); + } + break; + case Quad: + if (abs.size >= 8) { + qtest_readq(s, abs.addr); + } + break; + } +} + +static void op_write(QTestState *s, const unsigned char * data, size_t len) +{ + enum Sizes {Byte, Word, Long, Quad, end_sizes}; + struct { + uint8_t size; + uint8_t base; + uint32_t offset; + uint64_t value; + } a; + address_range abs; + + if (len < sizeof(a)) { + return; + } + memcpy(&a, data, sizeof(a)); + + if (get_mmio_address(&abs, a.base, a.offset) == 0) { + return; + } + + switch (a.size %= end_sizes) { + case Byte: + qtest_writeb(s, abs.addr, a.value & 0xFF); + break; + case Word: + if (abs.size >= 2) { + qtest_writew(s, abs.addr, a.value & 0xFFFF); + } + break; + case Long: + if (abs.size >= 4) { + qtest_writel(s, abs.addr, a.value & 0xFFFFFFFF); + } + break; + case Quad: + if (abs.size >= 8) { + qtest_writeq(s, abs.addr, a.value); + } + break; + } +} + +static void op_clock_step(QTestState *s, const unsigned char *data, size_t len) +{ + qtest_clock_step_next(s); +} + +static void handle_timeout(int sig) +{ + if (qtest_log_enabled) { + fprintf(stderr, "[Timeout]\n"); + fflush(stderr); + } + _Exit(0); +} + +/* + * Here, we interpret random bytes from the fuzzer, as a sequence of commands. + * Some commands can be variable-width, so we use a separator, SEPARATOR, to + * specify the boundaries between commands. SEPARATOR is used to separate + * "operations" in the fuzz input. Why use a separator, instead of just using + * the operations' length to identify operation boundaries? + * 1. This is a simple way to support variable-length operations + * 2. This adds "stability" to the input. + * For example take the input "AbBcgDefg", where there is no separator and + * Opcodes are capitalized. + * Simply, by removing the first byte, we end up with a very different + * sequence: + * BbcGdefg... + * By adding a separator, we avoid this problem: + * Ab SEP Bcg SEP Defg -> B SEP Bcg SEP Defg + * Since B uses two additional bytes as operands, the first "B" will be + * ignored. The fuzzer actively tries to reduce inputs, so such unused + * bytes are likely to be pruned, eventually. + * + * SEPARATOR is trivial for the fuzzer to discover when using ASan. Optionally, + * SEPARATOR can be manually specified as a dictionary value (see libfuzzer's + * -dict), though this should not be necessary. + * + * As a result, the stream of bytes is converted into a sequence of commands. + * In a simplified example where SEPARATOR is 0xFF: + * 00 01 02 FF 03 04 05 06 FF 01 FF ... + * becomes this sequence of commands: + * 00 01 02 -> op00 (0102) -> in (0102, 2) + * 03 04 05 06 -> op03 (040506) -> write (040506, 3) + * 01 -> op01 (-,0) -> out (-,0) + * ... + * + * Note here that it is the job of the individual opcode functions to check + * that enough data was provided. I.e. in the last command out (,0), out needs + * to check that there is not enough data provided to select an address/value + * for the operation. + */ +static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) +{ + void (*ops[]) (QTestState *s, const unsigned char* , size_t) = { + [OP_IN] = op_in, + [OP_OUT] = op_out, + [OP_READ] = op_read, + [OP_WRITE] = op_write, + [OP_CLOCK_STEP] = op_clock_step, + }; + const unsigned char *cmd = Data; + const unsigned char *nextcmd; + size_t cmd_len; + uint8_t op; + + if (fork() == 0) { + /* + * Sometimes the fuzzer will find inputs that take quite a long time to + * process. Often times, these inputs do not result in new coverage. + * Even if these inputs might be interesting, they can slow down the + * fuzzer, overall. Set a timeout to avoid hurting performance, too much + */ + if (timeout) { + struct sigaction sact; + struct itimerval timer; + + sigemptyset(&sact.sa_mask); + sact.sa_flags = SA_NODEFER; + sact.sa_handler = handle_timeout; + sigaction(SIGALRM, &sact, NULL); + + memset(&timer, 0, sizeof(timer)); + timer.it_value.tv_sec = timeout / USEC_IN_SEC; + timer.it_value.tv_usec = timeout % USEC_IN_SEC; + setitimer(ITIMER_VIRTUAL, &timer, NULL); + } + + while (cmd && Size) { + /* Get the length until the next command or end of input */ + nextcmd = memmem(cmd, Size, SEPARATOR, strlen(SEPARATOR)); + cmd_len = nextcmd ? nextcmd - cmd : Size; + + if (cmd_len > 0) { + /* Interpret the first byte of the command as an opcode */ + op = *cmd % (sizeof(ops) / sizeof((ops)[0])); + ops[op](s, cmd + 1, cmd_len - 1); + + /* Run the main loop */ + flush_events(s); + } + /* Advance to the next command */ + cmd = nextcmd ? nextcmd + sizeof(SEPARATOR) - 1 : nextcmd; + Size = Size - (cmd_len + sizeof(SEPARATOR) - 1); + } + _Exit(0); + } else { + flush_events(s); + wait(0); + } +} + +static void usage(void) +{ + printf("Please specify the following environment variables:\n"); + printf("QEMU_FUZZ_ARGS= the command line arguments passed to qemu\n"); + printf("QEMU_FUZZ_OBJECTS= " + "a space separated list of QOM type names for objects to fuzz\n"); + printf("Optionally: QEMU_FUZZ_TIMEOUT= Specify a custom timeout (us). " + "0 to disable. %d by default\n", timeout); + exit(0); +} + +static int locate_fuzz_memory_regions(Object *child, void *opaque) +{ + const char *name; + MemoryRegion *mr; + if (object_dynamic_cast(child, TYPE_MEMORY_REGION)) { + mr = MEMORY_REGION(child); + if ((memory_region_is_ram(mr) || + memory_region_is_ram_device(mr) || + memory_region_is_rom(mr)) == false) { + name = object_get_canonical_path_component(child); + /* + * We don't want duplicate pointers to the same MemoryRegion, so + * try to remove copies of the pointer, before adding it. + */ + g_hash_table_insert(fuzzable_memoryregions, mr, (gpointer)true); + } + } + return 0; +} + +static int locate_fuzz_objects(Object *child, void *opaque) +{ + char *pattern = opaque; + if (g_pattern_match_simple(pattern, object_get_typename(child))) { + /* Find and save ptrs to any child MemoryRegions */ + object_child_foreach_recursive(child, locate_fuzz_memory_regions, NULL); + + } else if (object_dynamic_cast(OBJECT(child), TYPE_MEMORY_REGION)) { + if (g_pattern_match_simple(pattern, + object_get_canonical_path_component(child))) { + MemoryRegion *mr; + mr = MEMORY_REGION(child); + if ((memory_region_is_ram(mr) || + memory_region_is_ram_device(mr) || + memory_region_is_rom(mr)) == false) { + g_hash_table_insert(fuzzable_memoryregions, mr, (gpointer)true); + } + } + } + return 0; +} + +static void generic_pre_fuzz(QTestState *s) +{ + GHashTableIter iter; + MemoryRegion *mr; + char **result; + + if (!getenv("QEMU_FUZZ_OBJECTS")) { + usage(); + } + if (getenv("QTEST_LOG")) { + qtest_log_enabled = 1; + } + if (getenv("QEMU_FUZZ_TIMEOUT")) { + timeout = g_ascii_strtoll(getenv("QEMU_FUZZ_TIMEOUT"), NULL, 0); + } + + fuzzable_memoryregions = g_hash_table_new(NULL, NULL); + + result = g_strsplit(getenv("QEMU_FUZZ_OBJECTS"), " ", -1); + for (int i = 0; result[i] != NULL; i++) { + printf("Matching objects by name %s\n", result[i]); + object_child_foreach_recursive(qdev_get_machine(), + locate_fuzz_objects, + result[i]); + } + g_strfreev(result); + printf("This process will try to fuzz the following MemoryRegions:\n"); + + g_hash_table_iter_init(&iter, fuzzable_memoryregions); + while (g_hash_table_iter_next(&iter, (gpointer)&mr, NULL)) { + printf(" * %s (size %lx)\n", + object_get_canonical_path_component(&(mr->parent_obj)), + (uint64_t)mr->size); + } + + if (!g_hash_table_size(fuzzable_memoryregions)) { + printf("No fuzzable memory regions found...\n"); + exit(1); + } + + counter_shm_init(); +} + +static GString *generic_fuzz_cmdline(FuzzTarget *t) +{ + GString *cmd_line = g_string_new(TARGET_NAME); + if (!getenv("QEMU_FUZZ_ARGS")) { + usage(); + } + g_string_append_printf(cmd_line, " -display none \ + -machine accel=qtest, \ + -m 512M %s ", getenv("QEMU_FUZZ_ARGS")); + return cmd_line; +} + +static void register_generic_fuzz_targets(void) +{ + fuzz_add_target(&(FuzzTarget){ + .name = "generic-fuzz", + .description = "Fuzz based on any qemu command-line args. ", + .get_init_cmdline = generic_fuzz_cmdline, + .pre_fuzz = generic_pre_fuzz, + .fuzz = generic_fuzz, + }); +} + +fuzz_target_init(register_generic_fuzz_targets); diff --git a/tests/qtest/fuzz/meson.build b/tests/qtest/fuzz/meson.build index b31ace7d5a..5162321f30 100644 --- a/tests/qtest/fuzz/meson.build +++ b/tests/qtest/fuzz/meson.build @@ -5,6 +5,7 @@ specific_fuzz_ss.add(files('fuzz.c', 'fork_fuzz.c', 'qos_fuzz.c', specific_fuzz_ss.add(when: 'CONFIG_I440FX', if_true: files('i440fx_fuzz.c')) specific_fuzz_ss.add(when: 'CONFIG_VIRTIO_NET', if_true: files('virtio_net_fuzz.c')) specific_fuzz_ss.add(when: 'CONFIG_VIRTIO_SCSI', if_true: files('virtio_scsi_fuzz.c')) +specific_fuzz_ss.add(files('generic_fuzz.c')) fork_fuzz = declare_dependency( link_args: config_host['FUZZ_EXE_LDFLAGS'].split() + From patchwork Mon Oct 26 10:06:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE402921 for ; Mon, 26 Oct 2020 10:15:23 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 49F86223AC for ; Mon, 26 Oct 2020 10:15:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ar0IOJmJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49F86223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:58598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzX7-0006YP-MZ for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:15:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38848) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPA-0002e0-Lt for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:52681) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzP1-0006gH-Ks for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706817; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=tggwgkkb1+KGW5RlyNzlZ6ZtcwmGXHxKNFTOsB5GyGw=; b=Ar0IOJmJO2xzxGNOwsCSSx0ea5Wa1sGgd+NjvicSZ5tp1GMJe1xpDw1i792UDmWqe39Pp5 EX2TkEM5Ps7yWG4Fdud5hDc99tFFaxvR6+87KX6kx9LmuDjQeqSTsph6eL+8KRsJuztQRW kHrbrBia8hD71yWiIvXr49UoQiix2f8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-508-d6MVrigYPwi-dNMWjrxTDg-1; Mon, 26 Oct 2020 06:06:54 -0400 X-MC-Unique: d6MVrigYPwi-dNMWjrxTDg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A135D10E2186; Mon, 26 Oct 2020 10:06:53 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id C623555774; Mon, 26 Oct 2020 10:06:51 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 11/31] fuzz: Add PCI features to the generic fuzzer Date: Mon, 26 Oct 2020 11:06:12 +0100 Message-Id: <20201026100632.212530-12-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov This patch compares TYPE_PCI_DEVICE objects against the user-provided matching pattern. If there is a match, we use some hacks and leverage QOS to map each possible BAR for that device. Now fuzzed inputs might be converted to pci_read/write commands which target specific. This means that we can fuzz a particular device's PCI configuration space, Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-4-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/generic_fuzz.c | 81 +++++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c index 6e3faf4e92..483d41fb2c 100644 --- a/tests/qtest/fuzz/generic_fuzz.c +++ b/tests/qtest/fuzz/generic_fuzz.c @@ -24,6 +24,7 @@ #include "exec/ramblock.h" #include "exec/address-spaces.h" #include "hw/qdev-core.h" +#include "hw/pci/pci.h" /* * SEPARATOR is used to separate "operations" in the fuzz input @@ -35,12 +36,17 @@ enum cmds { OP_OUT, OP_READ, OP_WRITE, + OP_PCI_READ, + OP_PCI_WRITE, OP_CLOCK_STEP, }; #define DEFAULT_TIMEOUT_US 100000 #define USEC_IN_SEC 1000000000 +#define PCI_HOST_BRIDGE_CFG 0xcf8 +#define PCI_HOST_BRIDGE_DATA 0xcfc + typedef struct { ram_addr_t addr; ram_addr_t size; /* The number of bytes until the end of the I/O region */ @@ -55,6 +61,7 @@ static bool qtest_log_enabled; * user for fuzzing. */ static GHashTable *fuzzable_memoryregions; +static GPtrArray *fuzzable_pci_devices; struct get_io_cb_info { int index; @@ -283,6 +290,65 @@ static void op_write(QTestState *s, const unsigned char * data, size_t len) } } +static void op_pci_read(QTestState *s, const unsigned char * data, size_t len) +{ + enum Sizes {Byte, Word, Long, end_sizes}; + struct { + uint8_t size; + uint8_t base; + uint8_t offset; + } a; + if (len < sizeof(a) || fuzzable_pci_devices->len == 0) { + return; + } + memcpy(&a, data, sizeof(a)); + PCIDevice *dev = g_ptr_array_index(fuzzable_pci_devices, + a.base % fuzzable_pci_devices->len); + int devfn = dev->devfn; + qtest_outl(s, PCI_HOST_BRIDGE_CFG, (1U << 31) | (devfn << 8) | a.offset); + switch (a.size %= end_sizes) { + case Byte: + qtest_inb(s, PCI_HOST_BRIDGE_DATA); + break; + case Word: + qtest_inw(s, PCI_HOST_BRIDGE_DATA); + break; + case Long: + qtest_inl(s, PCI_HOST_BRIDGE_DATA); + break; + } +} + +static void op_pci_write(QTestState *s, const unsigned char * data, size_t len) +{ + enum Sizes {Byte, Word, Long, end_sizes}; + struct { + uint8_t size; + uint8_t base; + uint8_t offset; + uint32_t value; + } a; + if (len < sizeof(a) || fuzzable_pci_devices->len == 0) { + return; + } + memcpy(&a, data, sizeof(a)); + PCIDevice *dev = g_ptr_array_index(fuzzable_pci_devices, + a.base % fuzzable_pci_devices->len); + int devfn = dev->devfn; + qtest_outl(s, PCI_HOST_BRIDGE_CFG, (1U << 31) | (devfn << 8) | a.offset); + switch (a.size %= end_sizes) { + case Byte: + qtest_outb(s, PCI_HOST_BRIDGE_DATA, a.value & 0xFF); + break; + case Word: + qtest_outw(s, PCI_HOST_BRIDGE_DATA, a.value & 0xFFFF); + break; + case Long: + qtest_outl(s, PCI_HOST_BRIDGE_DATA, a.value & 0xFFFFFFFF); + break; + } +} + static void op_clock_step(QTestState *s, const unsigned char *data, size_t len) { qtest_clock_step_next(s); @@ -341,6 +407,8 @@ static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) [OP_OUT] = op_out, [OP_READ] = op_read, [OP_WRITE] = op_write, + [OP_PCI_READ] = op_pci_read, + [OP_PCI_WRITE] = op_pci_write, [OP_CLOCK_STEP] = op_clock_step, }; const unsigned char *cmd = Data; @@ -432,6 +500,18 @@ static int locate_fuzz_objects(Object *child, void *opaque) /* Find and save ptrs to any child MemoryRegions */ object_child_foreach_recursive(child, locate_fuzz_memory_regions, NULL); + /* + * We matched an object. If its a PCI device, store a pointer to it so + * we can map BARs and fuzz its config space. + */ + if (object_dynamic_cast(OBJECT(child), TYPE_PCI_DEVICE)) { + /* + * Don't want duplicate pointers to the same PCIDevice, so remove + * copies of the pointer, before adding it. + */ + g_ptr_array_remove_fast(fuzzable_pci_devices, PCI_DEVICE(child)); + g_ptr_array_add(fuzzable_pci_devices, PCI_DEVICE(child)); + } } else if (object_dynamic_cast(OBJECT(child), TYPE_MEMORY_REGION)) { if (g_pattern_match_simple(pattern, object_get_canonical_path_component(child))) { @@ -464,6 +544,7 @@ static void generic_pre_fuzz(QTestState *s) } fuzzable_memoryregions = g_hash_table_new(NULL, NULL); + fuzzable_pci_devices = g_ptr_array_new(); result = g_strsplit(getenv("QEMU_FUZZ_OBJECTS"), " ", -1); for (int i = 0; result[i] != NULL; i++) { From patchwork Mon Oct 26 10:06:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03C3414C0 for ; Mon, 26 Oct 2020 10:10:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FB69223EA for ; Mon, 26 Oct 2020 10:10:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N6WUtvLj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FB69223EA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:41404 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzS0-0007eX-1D for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:10:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38882) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPC-0002eU-P1 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:35071) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzP2-0006gO-RX for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=ost2cOMGRJVru/pzvO+4OND1a7zyzspU2BgErPSvgd8=; b=N6WUtvLjbFtwx1sNq4rRq/YK7WoDPBEUG+jELbllAKJopkA6G6A2TTnQtk7pvoM2BGG48r l2SG/aBNe/8Uo7jaZgz8KuhKeqvpYUZ1apbvGqEYYk/cInvjWTAwrecCcCA3RSdLS8LKFU 3/sTlQcdq3LOzt3e5OfmEbNmH44rUZs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-340-Z-cmIUCSP2q54TDkPC28bQ-1; Mon, 26 Oct 2020 06:06:56 -0400 X-MC-Unique: Z-cmIUCSP2q54TDkPC28bQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 13898107ACF7; Mon, 26 Oct 2020 10:06:55 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 05D908B84A; Mon, 26 Oct 2020 10:06:53 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 12/31] fuzz: Add DMA support to the generic-fuzzer Date: Mon, 26 Oct 2020 11:06:13 +0100 Message-Id: <20201026100632.212530-13-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov When a virtual-device tries to access some buffer in memory over DMA, we add call-backs into the fuzzer(next commit). The fuzzer checks verifies that the DMA request maps to a physical RAM address and fills the memory with fuzzer-provided data. The patterns that we use to fill this memory are specified using add_dma_pattern and clear_dma_patterns operations. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-5-alxndr@bu.edu> [thuth: Reformatted one comment according to the QEMU coding style] Signed-off-by: Thomas Huth --- include/exec/memory.h | 7 + tests/qtest/fuzz/generic_fuzz.c | 230 ++++++++++++++++++++++++++++++++ 2 files changed, 237 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 042918dd16..93d27bff26 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -42,6 +42,13 @@ typedef struct IOMMUMemoryRegionClass IOMMUMemoryRegionClass; DECLARE_OBJ_CHECKERS(IOMMUMemoryRegion, IOMMUMemoryRegionClass, IOMMU_MEMORY_REGION, TYPE_IOMMU_MEMORY_REGION) +#ifdef CONFIG_FUZZ +void fuzz_dma_read_cb(size_t addr, + size_t len, + MemoryRegion *mr, + bool is_write); +#endif + extern bool global_dirty_log; typedef struct MemoryRegionOps MemoryRegionOps; diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c index 483d41fb2c..62a94de9dd 100644 --- a/tests/qtest/fuzz/generic_fuzz.c +++ b/tests/qtest/fuzz/generic_fuzz.c @@ -25,6 +25,7 @@ #include "exec/address-spaces.h" #include "hw/qdev-core.h" #include "hw/pci/pci.h" +#include "hw/boards.h" /* * SEPARATOR is used to separate "operations" in the fuzz input @@ -38,12 +39,16 @@ enum cmds { OP_WRITE, OP_PCI_READ, OP_PCI_WRITE, + OP_ADD_DMA_PATTERN, + OP_CLEAR_DMA_PATTERNS, OP_CLOCK_STEP, }; #define DEFAULT_TIMEOUT_US 100000 #define USEC_IN_SEC 1000000000 +#define MAX_DMA_FILL_SIZE 0x10000 + #define PCI_HOST_BRIDGE_CFG 0xcf8 #define PCI_HOST_BRIDGE_DATA 0xcfc @@ -56,6 +61,24 @@ static useconds_t timeout = DEFAULT_TIMEOUT_US; static bool qtest_log_enabled; +/* + * A pattern used to populate a DMA region or perform a memwrite. This is + * useful for e.g. populating tables of unique addresses. + * Example {.index = 1; .stride = 2; .len = 3; .data = "\x00\x01\x02"} + * Renders as: 00 01 02 00 03 02 00 05 02 00 07 02 ... + */ +typedef struct { + uint8_t index; /* Index of a byte to increment by stride */ + uint8_t stride; /* Increment each index'th byte by this amount */ + size_t len; + const uint8_t *data; +} pattern; + +/* Avoid filling the same DMA region between MMIO/PIO commands ? */ +static bool avoid_double_fetches; + +static QTestState *qts_global; /* Need a global for the DMA callback */ + /* * List of memory regions that are children of QOM objects specified by the * user for fuzzing. @@ -84,6 +107,169 @@ static int get_io_address_cb(Int128 start, Int128 size, return 0; } +/* + * List of dma regions populated since the last fuzzing command. Used to ensure + * that we only write to each DMA address once, to avoid race conditions when + * building reproducers. + */ +static GArray *dma_regions; + +static GArray *dma_patterns; +static int dma_pattern_index; + +/* + * Allocate a block of memory and populate it with a pattern. + */ +static void *pattern_alloc(pattern p, size_t len) +{ + int i; + uint8_t *buf = g_malloc(len); + uint8_t sum = 0; + + for (i = 0; i < len; ++i) { + buf[i] = p.data[i % p.len]; + if ((i % p.len) == p.index) { + buf[i] += sum; + sum += p.stride; + } + } + return buf; +} + +static int memory_access_size(MemoryRegion *mr, unsigned l, hwaddr addr) +{ + unsigned access_size_max = mr->ops->valid.max_access_size; + + /* + * Regions are assumed to support 1-4 byte accesses unless + * otherwise specified. + */ + if (access_size_max == 0) { + access_size_max = 4; + } + + /* Bound the maximum access by the alignment of the address. */ + if (!mr->ops->impl.unaligned) { + unsigned align_size_max = addr & -addr; + if (align_size_max != 0 && align_size_max < access_size_max) { + access_size_max = align_size_max; + } + } + + /* Don't attempt accesses larger than the maximum. */ + if (l > access_size_max) { + l = access_size_max; + } + l = pow2floor(l); + + return l; +} + +/* + * Call-back for functions that perform DMA reads from guest memory. Confirm + * that the region has not already been populated since the last loop in + * generic_fuzz(), avoiding potential race-conditions, which we don't have + * a good way for reproducing right now. + */ +void fuzz_dma_read_cb(size_t addr, size_t len, MemoryRegion *mr, bool is_write) +{ + /* Are we in the generic-fuzzer or are we using another fuzz-target? */ + if (!qts_global) { + return; + } + + /* + * Return immediately if: + * - We have no DMA patterns defined + * - The length of the DMA read request is zero + * - The DMA read is hitting an MR other than the machine's main RAM + * - The DMA request is not a read (what happens for a address_space_map + * with is_write=True? Can the device use the same pointer to do reads?) + * - The DMA request hits past the bounds of our RAM + */ + if (dma_patterns->len == 0 + || len == 0 + /* || mr != MACHINE(qdev_get_machine())->ram */ + || is_write + || addr > current_machine->ram_size) { + return; + } + + /* + * If we overlap with any existing dma_regions, split the range and only + * populate the non-overlapping parts. + */ + address_range region; + bool double_fetch = false; + for (int i = 0; + i < dma_regions->len && (avoid_double_fetches || qtest_log_enabled); + ++i) { + region = g_array_index(dma_regions, address_range, i); + if (addr < region.addr + region.size && addr + len > region.addr) { + double_fetch = true; + if (addr < region.addr + && avoid_double_fetches) { + fuzz_dma_read_cb(addr, region.addr - addr, mr, is_write); + } + if (addr + len > region.addr + region.size + && avoid_double_fetches) { + fuzz_dma_read_cb(region.addr + region.size, + addr + len - (region.addr + region.size), mr, is_write); + } + return; + } + } + + /* Cap the length of the DMA access to something reasonable */ + len = MIN(len, MAX_DMA_FILL_SIZE); + + address_range ar = {addr, len}; + g_array_append_val(dma_regions, ar); + pattern p = g_array_index(dma_patterns, pattern, dma_pattern_index); + void *buf = pattern_alloc(p, ar.size); + hwaddr l, addr1; + MemoryRegion *mr1; + uint8_t *ram_ptr; + while (len > 0) { + l = len; + mr1 = address_space_translate(first_cpu->as, + addr, &addr1, &l, true, + MEMTXATTRS_UNSPECIFIED); + + if (!(memory_region_is_ram(mr1) || + memory_region_is_romd(mr1))) { + l = memory_access_size(mr1, l, addr1); + } else { + /* ROM/RAM case */ + ram_ptr = qemu_map_ram_ptr(mr1->ram_block, addr1); + memcpy(ram_ptr, buf, l); + break; + } + len -= l; + buf += l; + addr += l; + + } + if (qtest_log_enabled) { + /* + * With QTEST_LOG, use a normal, slow QTest memwrite. Prefix the log + * that will be written by qtest.c with a DMA tag, so we can reorder + * the resulting QTest trace so the DMA fills precede the last PIO/MMIO + * command. + */ + fprintf(stderr, "[DMA] "); + if (double_fetch) { + fprintf(stderr, "[DOUBLE-FETCH] "); + } + fflush(stderr); + } + qtest_memwrite(qts_global, ar.addr, buf, ar.size); + g_free(buf); + + /* Increment the index of the pattern for the next DMA access */ + dma_pattern_index = (dma_pattern_index + 1) % dma_patterns->len; +} + /* * Here we want to convert a fuzzer-provided [io-region-index, offset] to * a physical address. To do this, we iterate over all of the matched @@ -349,6 +535,35 @@ static void op_pci_write(QTestState *s, const unsigned char * data, size_t len) } } +static void op_add_dma_pattern(QTestState *s, + const unsigned char *data, size_t len) +{ + struct { + /* + * index and stride can be used to increment the index-th byte of the + * pattern by the value stride, for each loop of the pattern. + */ + uint8_t index; + uint8_t stride; + } a; + + if (len < sizeof(a) + 1) { + return; + } + memcpy(&a, data, sizeof(a)); + pattern p = {a.index, a.stride, len - sizeof(a), data + sizeof(a)}; + p.index = a.index % p.len; + g_array_append_val(dma_patterns, p); + return; +} + +static void op_clear_dma_patterns(QTestState *s, + const unsigned char *data, size_t len) +{ + g_array_set_size(dma_patterns, 0); + dma_pattern_index = 0; +} + static void op_clock_step(QTestState *s, const unsigned char *data, size_t len) { qtest_clock_step_next(s); @@ -409,6 +624,8 @@ static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) [OP_WRITE] = op_write, [OP_PCI_READ] = op_pci_read, [OP_PCI_WRITE] = op_pci_write, + [OP_ADD_DMA_PATTERN] = op_add_dma_pattern, + [OP_CLEAR_DMA_PATTERNS] = op_clear_dma_patterns, [OP_CLOCK_STEP] = op_clock_step, }; const unsigned char *cmd = Data; @@ -438,6 +655,8 @@ static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) setitimer(ITIMER_VIRTUAL, &timer, NULL); } + op_clear_dma_patterns(s, NULL, 0); + while (cmd && Size) { /* Get the length until the next command or end of input */ nextcmd = memmem(cmd, Size, SEPARATOR, strlen(SEPARATOR)); @@ -454,6 +673,7 @@ static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) /* Advance to the next command */ cmd = nextcmd ? nextcmd + sizeof(SEPARATOR) - 1 : nextcmd; Size = Size - (cmd_len + sizeof(SEPARATOR) - 1); + g_array_set_size(dma_regions, 0); } _Exit(0); } else { @@ -468,6 +688,9 @@ static void usage(void) printf("QEMU_FUZZ_ARGS= the command line arguments passed to qemu\n"); printf("QEMU_FUZZ_OBJECTS= " "a space separated list of QOM type names for objects to fuzz\n"); + printf("Optionally: QEMU_AVOID_DOUBLE_FETCH= " + "Try to avoid racy DMA double fetch bugs? %d by default\n", + avoid_double_fetches); printf("Optionally: QEMU_FUZZ_TIMEOUT= Specify a custom timeout (us). " "0 to disable. %d by default\n", timeout); exit(0); @@ -539,9 +762,16 @@ static void generic_pre_fuzz(QTestState *s) if (getenv("QTEST_LOG")) { qtest_log_enabled = 1; } + if (getenv("QEMU_AVOID_DOUBLE_FETCH")) { + avoid_double_fetches = 1; + } if (getenv("QEMU_FUZZ_TIMEOUT")) { timeout = g_ascii_strtoll(getenv("QEMU_FUZZ_TIMEOUT"), NULL, 0); } + qts_global = s; + + dma_regions = g_array_new(false, false, sizeof(address_range)); + dma_patterns = g_array_new(false, false, sizeof(pattern)); fuzzable_memoryregions = g_hash_table_new(NULL, NULL); fuzzable_pci_devices = g_ptr_array_new(); From patchwork Mon Oct 26 10:06:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856345 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 860DF139F for ; Mon, 26 Oct 2020 10:28:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 30C782076D for ; Mon, 26 Oct 2020 10:28:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NhoL5J5U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 30C782076D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:43940 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzk7-0007Mm-76 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:28:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39014) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPL-0002xI-D1 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:59867) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPA-0006hW-3l for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=YtpemJyXeENG1iyhpnYrZIyJX9D2vPnFL8PqSIH7ibQ=; b=NhoL5J5U25IV6rpGvFMaSJcCxkaThFq5gHjkBGoU6Ys0PgRc/CubKj0duXrxM0y5J4Xzes GBPE06uGvW9GzsOqkITPmCOKKau6QU98u4Fb9j9JfPSUK52Cjm6hsLMaQmVMIerSuOX0U8 ruPMgy5Upb+Wz2tHJDhPLIPEwHLE2O4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-218-XnQVwli7NduG9h-LypXLCA-1; Mon, 26 Oct 2020 06:06:57 -0400 X-MC-Unique: XnQVwli7NduG9h-LypXLCA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 978EC1009E3E; Mon, 26 Oct 2020 10:06:56 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8863E8B84F; Mon, 26 Oct 2020 10:06:55 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 13/31] fuzz: Declare DMA Read callback function Date: Mon, 26 Oct 2020 11:06:14 +0100 Message-Id: <20201026100632.212530-14-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov This patch declares the fuzz_dma_read_cb function and uses the preprocessor and linker(weak symbols) to handle these cases: When we build softmmu/all with --enable-fuzzing, there should be no strong symbol defined for fuzz_dma_read_cb, and we link against a weak stub function. When we build softmmu/fuzz with --enable-fuzzing, we link against the strong symbol in generic_fuzz.c When we build softmmu/all without --enable-fuzzing, fuzz_dma_read_cb is an empty, inlined function. As long as we don't call any other functions when building the arguments, there should be no overhead. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-6-alxndr@bu.edu> Signed-off-by: Thomas Huth --- include/exec/memory.h | 8 ++++++++ softmmu/memory.c | 13 +++++++++++++ 2 files changed, 21 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 93d27bff26..4aaf578ce6 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -47,6 +47,14 @@ void fuzz_dma_read_cb(size_t addr, size_t len, MemoryRegion *mr, bool is_write); +#else +static inline void fuzz_dma_read_cb(size_t addr, + size_t len, + MemoryRegion *mr, + bool is_write) +{ + /* Do Nothing */ +} #endif extern bool global_dirty_log; diff --git a/softmmu/memory.c b/softmmu/memory.c index a5d1641820..cec0e0f743 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -3246,6 +3246,19 @@ void memory_region_init_rom_device(MemoryRegion *mr, vmstate_register_ram(mr, owner_dev); } +/* + * Support softmmu builds with CONFIG_FUZZ using a weak symbol and a stub for + * the fuzz_dma_read_cb callback + */ +#ifdef CONFIG_FUZZ +void __attribute__((weak)) fuzz_dma_read_cb(size_t addr, + size_t len, + MemoryRegion *mr, + bool is_write) +{ +} +#endif + static const TypeInfo memory_region_info = { .parent = TYPE_OBJECT, .name = TYPE_MEMORY_REGION, From patchwork Mon Oct 26 10:06:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5EC8921 for ; Mon, 26 Oct 2020 10:13:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8354C223AC for ; Mon, 26 Oct 2020 10:13:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e0Cy3DD3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8354C223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:49934 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzUq-0002xP-HV for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:13:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38910) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPF-0002hc-N0 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43873) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPA-0006he-DF for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=J6Qccw+D6CaPwu6y3tcv+lXXBvRJfVxMGCmrZbTcrR4=; b=e0Cy3DD31UqsmQjE2lsQi5f/UZFrhUGI25MgMiTtgYPZ4I4oj8Gmq0X+5YOLxtmKgJyiu5 kWyhrESUERQzVcLnZlLSkJ0UyN7LxIOxn9Va+BhspdwiZbDbDmZU2dOcYWj45vpX+FCbGo +wPc/olfoAMZdX7cpZeRydKjcBRXWTw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-177-gzSjcyGBMQ2kstYPkX5sgw-1; Mon, 26 Oct 2020 06:06:59 -0400 X-MC-Unique: gzSjcyGBMQ2kstYPkX5sgw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0E5BE64ACA; Mon, 26 Oct 2020 10:06:58 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA4928B84A; Mon, 26 Oct 2020 10:06:56 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 14/31] fuzz: Add fuzzer callbacks to DMA-read functions Date: Mon, 26 Oct 2020 11:06:15 +0100 Message-Id: <20201026100632.212530-15-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov We should be careful to not call any functions besides fuzz_dma_read_cb. Without --enable-fuzzing, fuzz_dma_read_cb is an empty inlined function. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-7-alxndr@bu.edu> Signed-off-by: Thomas Huth --- include/exec/memory.h | 1 + include/exec/memory_ldst_cached.h.inc | 3 +++ memory_ldst.c.inc | 4 ++++ softmmu/memory.c | 1 + softmmu/physmem.c | 2 ++ 5 files changed, 11 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 4aaf578ce6..aff6ef7605 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -2462,6 +2462,7 @@ address_space_read_cached(MemoryRegionCache *cache, hwaddr addr, void *buf, hwaddr len) { assert(addr < cache->len && len <= cache->len - addr); + fuzz_dma_read_cb(cache->xlat + addr, len, cache->mrs.mr, false); if (likely(cache->ptr)) { memcpy(buf, cache->ptr + addr, len); return MEMTX_OK; diff --git a/include/exec/memory_ldst_cached.h.inc b/include/exec/memory_ldst_cached.h.inc index fd4bbb40e7..aff574039f 100644 --- a/include/exec/memory_ldst_cached.h.inc +++ b/include/exec/memory_ldst_cached.h.inc @@ -28,6 +28,7 @@ static inline uint32_t ADDRESS_SPACE_LD_CACHED(l)(MemoryRegionCache *cache, hwaddr addr, MemTxAttrs attrs, MemTxResult *result) { assert(addr < cache->len && 4 <= cache->len - addr); + fuzz_dma_read_cb(cache->xlat + addr, 4, cache->mrs.mr, false); if (likely(cache->ptr)) { return LD_P(l)(cache->ptr + addr); } else { @@ -39,6 +40,7 @@ static inline uint64_t ADDRESS_SPACE_LD_CACHED(q)(MemoryRegionCache *cache, hwaddr addr, MemTxAttrs attrs, MemTxResult *result) { assert(addr < cache->len && 8 <= cache->len - addr); + fuzz_dma_read_cb(cache->xlat + addr, 8, cache->mrs.mr, false); if (likely(cache->ptr)) { return LD_P(q)(cache->ptr + addr); } else { @@ -50,6 +52,7 @@ static inline uint32_t ADDRESS_SPACE_LD_CACHED(uw)(MemoryRegionCache *cache, hwaddr addr, MemTxAttrs attrs, MemTxResult *result) { assert(addr < cache->len && 2 <= cache->len - addr); + fuzz_dma_read_cb(cache->xlat + addr, 2, cache->mrs.mr, false); if (likely(cache->ptr)) { return LD_P(uw)(cache->ptr + addr); } else { diff --git a/memory_ldst.c.inc b/memory_ldst.c.inc index c54aee4a95..8d45d2eeff 100644 --- a/memory_ldst.c.inc +++ b/memory_ldst.c.inc @@ -42,6 +42,7 @@ static inline uint32_t glue(address_space_ldl_internal, SUFFIX)(ARG1_DECL, MO_32 | devend_memop(endian), attrs); } else { /* RAM case */ + fuzz_dma_read_cb(addr, 4, mr, false); ptr = qemu_map_ram_ptr(mr->ram_block, addr1); switch (endian) { case DEVICE_LITTLE_ENDIAN: @@ -110,6 +111,7 @@ static inline uint64_t glue(address_space_ldq_internal, SUFFIX)(ARG1_DECL, MO_64 | devend_memop(endian), attrs); } else { /* RAM case */ + fuzz_dma_read_cb(addr, 8, mr, false); ptr = qemu_map_ram_ptr(mr->ram_block, addr1); switch (endian) { case DEVICE_LITTLE_ENDIAN: @@ -175,6 +177,7 @@ uint32_t glue(address_space_ldub, SUFFIX)(ARG1_DECL, r = memory_region_dispatch_read(mr, addr1, &val, MO_8, attrs); } else { /* RAM case */ + fuzz_dma_read_cb(addr, 1, mr, false); ptr = qemu_map_ram_ptr(mr->ram_block, addr1); val = ldub_p(ptr); r = MEMTX_OK; @@ -212,6 +215,7 @@ static inline uint32_t glue(address_space_lduw_internal, SUFFIX)(ARG1_DECL, MO_16 | devend_memop(endian), attrs); } else { /* RAM case */ + fuzz_dma_read_cb(addr, 2, mr, false); ptr = qemu_map_ram_ptr(mr->ram_block, addr1); switch (endian) { case DEVICE_LITTLE_ENDIAN: diff --git a/softmmu/memory.c b/softmmu/memory.c index cec0e0f743..ee4a6bc168 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1433,6 +1433,7 @@ MemTxResult memory_region_dispatch_read(MemoryRegion *mr, unsigned size = memop_size(op); MemTxResult r; + fuzz_dma_read_cb(addr, size, mr, false); if (!memory_region_access_valid(mr, addr, size, false, attrs)) { *pval = unassigned_mem_read(mr, addr, size); return MEMTX_DECODE_ERROR; diff --git a/softmmu/physmem.c b/softmmu/physmem.c index e319fb2a1e..a9adedb9f8 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -2832,6 +2832,7 @@ MemTxResult flatview_read_continue(FlatView *fv, hwaddr addr, stn_he_p(buf, l, val); } else { /* RAM case */ + fuzz_dma_read_cb(addr, len, mr, false); ram_ptr = qemu_ram_ptr_length(mr->ram_block, addr1, &l, false); memcpy(buf, ram_ptr, l); } @@ -3192,6 +3193,7 @@ void *address_space_map(AddressSpace *as, memory_region_ref(mr); *plen = flatview_extend_translation(fv, addr, len, mr, xlat, l, is_write, attrs); + fuzz_dma_read_cb(addr, *plen, mr, is_write); ptr = qemu_ram_ptr_length(mr->ram_block, xlat, plen, true); return ptr; From patchwork Mon Oct 26 10:06:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB929921 for ; Mon, 26 Oct 2020 10:15:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 947C2223AC for ; Mon, 26 Oct 2020 10:15:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="caAeYjqb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 947C2223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57866 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzWs-0006Ey-HR for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:15:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38902) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPF-0002gc-96 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:60625) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzP7-0006ha-M3 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=QwqLwETTwMwtQBrfjojZlNqQn/DL9uFwH1MP8X5pkyA=; b=caAeYjqbaNDs25fzFs85gs/ZAfcEV05E2D1FNdy7QzZwjo1coXNr8C0E730Y2cWdNBK/m8 UsZhCr6V4POEYBzvQusmKMkaRvw+BpZEtxOoyWONRPWtD3NpWgQMtynZVt6vWHo3839gFL Z7/Fz+vRqRomkhvtl35X2e85Zndj4NE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-528-VIJoKiMQOKipfCdKTGInqg-1; Mon, 26 Oct 2020 06:07:00 -0400 X-MC-Unique: VIJoKiMQOKipfCdKTGInqg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7584764AEB; Mon, 26 Oct 2020 10:06:59 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5E7D28B84C; Mon, 26 Oct 2020 10:06:58 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 15/31] fuzz: Add support for custom crossover functions Date: Mon, 26 Oct 2020 11:06:16 +0100 Message-Id: <20201026100632.212530-16-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov libfuzzer supports a "custom crossover function". Libfuzzer often tries to blend two inputs to create a new interesting input. Sometimes, we have a better idea about how to blend inputs together. This change allows fuzzers to specify a custom function for blending two inputs together. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-8-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/fuzz.c | 13 +++++++++++++ tests/qtest/fuzz/fuzz.h | 27 +++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/tests/qtest/fuzz/fuzz.c b/tests/qtest/fuzz/fuzz.c index eb0070437f..7be7226bc0 100644 --- a/tests/qtest/fuzz/fuzz.c +++ b/tests/qtest/fuzz/fuzz.c @@ -118,6 +118,19 @@ static FuzzTarget *fuzz_get_target(char* name) } +/* Sometimes called by libfuzzer to mutate two inputs into one */ +size_t LLVMFuzzerCustomCrossOver(const uint8_t *data1, size_t size1, + const uint8_t *data2, size_t size2, + uint8_t *out, size_t max_out_size, + unsigned int seed) +{ + if (fuzz_target->crossover) { + return fuzz_target->crossover(data1, size1, data2, size2, out, + max_out_size, seed); + } + return 0; +} + /* Executed for each fuzzing-input */ int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size) { diff --git a/tests/qtest/fuzz/fuzz.h b/tests/qtest/fuzz/fuzz.h index 8eb765edc8..ed9ce17154 100644 --- a/tests/qtest/fuzz/fuzz.h +++ b/tests/qtest/fuzz/fuzz.h @@ -77,6 +77,29 @@ typedef struct FuzzTarget { */ void(*fuzz)(QTestState *, const unsigned char *, size_t); + /* + * The fuzzer can specify a "Custom Crossover" function for combining two + * inputs from the corpus. This function is sometimes called by libfuzzer + * when mutating inputs. + * + * data1: location of first input + * size1: length of first input + * data1: location of second input + * size1: length of second input + * out: where to place the resulting, mutated input + * max_out_size: the maximum length of the input that can be placed in out + * seed: the seed that should be used to make mutations deterministic, when + * needed + * + * See libfuzzer's LLVMFuzzerCustomCrossOver API for more info. + * + * Can be NULL + */ + size_t(*crossover)(const uint8_t *data1, size_t size1, + const uint8_t *data2, size_t size2, + uint8_t *out, size_t max_out_size, + unsigned int seed); + } FuzzTarget; void flush_events(QTestState *); @@ -91,6 +114,10 @@ void fuzz_qtest_set_serialize(bool option); */ void fuzz_add_target(const FuzzTarget *target); +size_t LLVMFuzzerCustomCrossOver(const uint8_t *data1, size_t size1, + const uint8_t *data2, size_t size2, + uint8_t *out, size_t max_out_size, + unsigned int seed); int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size); int LLVMFuzzerInitialize(int *argc, char ***argv, char ***envp); From patchwork Mon Oct 26 10:06:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CD63139F for ; Mon, 26 Oct 2020 10:21:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BA3AB22242 for ; Mon, 26 Oct 2020 10:21:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BAo0W96U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA3AB22242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47072 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzci-0005It-N4 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:21:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPG-0002l7-Ui for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:45801) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPA-0006hj-EL for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=WbjpSFd8Wy+vijEQr6h3rVLa4JiCyrzL8aZ0esPPjcU=; b=BAo0W96UtlHILF+SCJ6YTbXr2sTBtI///KvgNEjUDCr0C8tTppBLy4mcwnsAPdy0gINgFi RJzbsioRge0Rpg2o24Xy7eNR9wmunf6t8saFAvZTQEChNdeM/10jUeV0VYiG+FOctyWqvr NOlkdXpMf2bsb5mVjWtanvaBNDqrVcQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-584-C3BiNeECMhKfY0O13VytPg-1; Mon, 26 Oct 2020 06:07:02 -0400 X-MC-Unique: C3BiNeECMhKfY0O13VytPg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DCDE480B723; Mon, 26 Oct 2020 10:07:00 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id D1EF18B842; Mon, 26 Oct 2020 10:06:59 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 16/31] fuzz: add a DISABLE_PCI op to generic-fuzzer Date: Mon, 26 Oct 2020 11:06:17 +0100 Message-Id: <20201026100632.212530-17-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov This new operation is used in the next commit, which concatenates two fuzzer-generated inputs. With this operation, we can prevent the second input from clobbering the PCI configuration performed by the first. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-9-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/generic_fuzz.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c index 62a94de9dd..6ec20464bd 100644 --- a/tests/qtest/fuzz/generic_fuzz.c +++ b/tests/qtest/fuzz/generic_fuzz.c @@ -39,6 +39,7 @@ enum cmds { OP_WRITE, OP_PCI_READ, OP_PCI_WRITE, + OP_DISABLE_PCI, OP_ADD_DMA_PATTERN, OP_CLEAR_DMA_PATTERNS, OP_CLOCK_STEP, @@ -116,6 +117,7 @@ static GArray *dma_regions; static GArray *dma_patterns; static int dma_pattern_index; +static bool pci_disabled; /* * Allocate a block of memory and populate it with a pattern. @@ -484,7 +486,7 @@ static void op_pci_read(QTestState *s, const unsigned char * data, size_t len) uint8_t base; uint8_t offset; } a; - if (len < sizeof(a) || fuzzable_pci_devices->len == 0) { + if (len < sizeof(a) || fuzzable_pci_devices->len == 0 || pci_disabled) { return; } memcpy(&a, data, sizeof(a)); @@ -514,7 +516,7 @@ static void op_pci_write(QTestState *s, const unsigned char * data, size_t len) uint8_t offset; uint32_t value; } a; - if (len < sizeof(a) || fuzzable_pci_devices->len == 0) { + if (len < sizeof(a) || fuzzable_pci_devices->len == 0 || pci_disabled) { return; } memcpy(&a, data, sizeof(a)); @@ -569,6 +571,11 @@ static void op_clock_step(QTestState *s, const unsigned char *data, size_t len) qtest_clock_step_next(s); } +static void op_disable_pci(QTestState *s, const unsigned char *data, size_t len) +{ + pci_disabled = true; +} + static void handle_timeout(int sig) { if (qtest_log_enabled) { @@ -624,6 +631,7 @@ static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) [OP_WRITE] = op_write, [OP_PCI_READ] = op_pci_read, [OP_PCI_WRITE] = op_pci_write, + [OP_DISABLE_PCI] = op_disable_pci, [OP_ADD_DMA_PATTERN] = op_add_dma_pattern, [OP_CLEAR_DMA_PATTERNS] = op_clear_dma_patterns, [OP_CLOCK_STEP] = op_clock_step, @@ -656,6 +664,7 @@ static void generic_fuzz(QTestState *s, const unsigned char *Data, size_t Size) } op_clear_dma_patterns(s, NULL, 0); + pci_disabled = false; while (cmd && Size) { /* Get the length until the next command or end of input */ From patchwork Mon Oct 26 10:06:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52C97921 for ; Mon, 26 Oct 2020 10:18:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E88E422242 for ; Mon, 26 Oct 2020 10:18:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PFvmWAmL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E88E422242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38730 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzaW-0001lJ-SH for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:18:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38930) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPG-0002k8-HT for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:56337) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPA-0006jg-E3 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=gJZy3kz//wBAQSHK401ir7KhMFQGorLzyO/DPPZAjnI=; b=PFvmWAmLpwl3yVFpYePgLqQq4+UF7B0fIpQ0NH7pp5+EXkx2/rh0FHbrlBAl+8Jubnpm9p KPfiC2G1Uk9Lq3vkW6mkgkvULrrGciOe3XdsKIvfUKgM6hc2PSB1cmj/kKQbdFsjNwJ1kq 58xtjNriA3H0nS1nCTusG5rPy2r1uBg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-mOGKAc7ZMduH5s_mzQI80w-1; Mon, 26 Oct 2020 06:07:03 -0400 X-MC-Unique: mOGKAc7ZMduH5s_mzQI80w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4D9281009E3D; Mon, 26 Oct 2020 10:07:02 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A2328B842; Mon, 26 Oct 2020 10:07:01 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 17/31] fuzz: add a crossover function to generic-fuzzer Date: Mon, 26 Oct 2020 11:06:18 +0100 Message-Id: <20201026100632.212530-18-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-10-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/generic_fuzz.c | 86 +++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c index 6ec20464bd..592b78ffe2 100644 --- a/tests/qtest/fuzz/generic_fuzz.c +++ b/tests/qtest/fuzz/generic_fuzz.c @@ -810,6 +810,91 @@ static void generic_pre_fuzz(QTestState *s) counter_shm_init(); } +/* + * When libfuzzer gives us two inputs to combine, return a new input with the + * following structure: + * + * Input 1 (data1) + * SEPARATOR + * Clear out the DMA Patterns + * SEPARATOR + * Disable the pci_read/write instructions + * SEPARATOR + * Input 2 (data2) + * + * The idea is to collate the core behaviors of the two inputs. + * For example: + * Input 1: maps a device's BARs, sets up three DMA patterns, and triggers + * device functionality A + * Input 2: maps a device's BARs, sets up one DMA pattern, and triggers device + * functionality B + * + * This function attempts to produce an input that: + * Ouptut: maps a device's BARs, set up three DMA patterns, triggers + * functionality A device, replaces the DMA patterns with a single + * patten, and triggers device functionality B. + */ +static size_t generic_fuzz_crossover(const uint8_t *data1, size_t size1, const + uint8_t *data2, size_t size2, uint8_t *out, + size_t max_out_size, unsigned int seed) +{ + size_t copy_len = 0, size = 0; + + /* Check that we have enough space for data1 and at least part of data2 */ + if (max_out_size <= size1 + strlen(SEPARATOR) * 3 + 2) { + return 0; + } + + /* Copy_Len in the first input */ + copy_len = size1; + memcpy(out + size, data1, copy_len); + size += copy_len; + max_out_size -= copy_len; + + /* Append a separator */ + copy_len = strlen(SEPARATOR); + memcpy(out + size, SEPARATOR, copy_len); + size += copy_len; + max_out_size -= copy_len; + + /* Clear out the DMA Patterns */ + copy_len = 1; + if (copy_len) { + out[size] = OP_CLEAR_DMA_PATTERNS; + } + size += copy_len; + max_out_size -= copy_len; + + /* Append a separator */ + copy_len = strlen(SEPARATOR); + memcpy(out + size, SEPARATOR, copy_len); + size += copy_len; + max_out_size -= copy_len; + + /* Disable PCI ops. Assume data1 took care of setting up PCI */ + copy_len = 1; + if (copy_len) { + out[size] = OP_DISABLE_PCI; + } + size += copy_len; + max_out_size -= copy_len; + + /* Append a separator */ + copy_len = strlen(SEPARATOR); + memcpy(out + size, SEPARATOR, copy_len); + size += copy_len; + max_out_size -= copy_len; + + /* Copy_Len over the second input */ + copy_len = MIN(size2, max_out_size); + memcpy(out + size, data2, copy_len); + size += copy_len; + max_out_size -= copy_len; + + return size; +} + + static GString *generic_fuzz_cmdline(FuzzTarget *t) { GString *cmd_line = g_string_new(TARGET_NAME); @@ -830,6 +915,7 @@ static void register_generic_fuzz_targets(void) .get_init_cmdline = generic_fuzz_cmdline, .pre_fuzz = generic_pre_fuzz, .fuzz = generic_fuzz, + .crossover = generic_fuzz_crossover }); } From patchwork Mon Oct 26 10:06:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856341 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1673E921 for ; Mon, 26 Oct 2020 10:25:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AFA0F207E8 for ; Mon, 26 Oct 2020 10:25:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LakXWspR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFA0F207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:35634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzgV-0003gc-Tv for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:25:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38962) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPH-0002me-N1 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:54820) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPC-0006k7-ER for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706829; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=ijFbgbhA9U4wuK2uqFm4kvGlGVKw2BLSDoKtZ2VHRZs=; b=LakXWspRogGrS+WeAcB36qqy6cALcjAHJigJlHBsFdF9i0kTLyNe13KCArFZBK9ScYryH8 jyp+N/cHcWY2XbGdw8qNIIvW0oJh5Y24JSyIqoJkGL3zKxz4Uluh2aMHOqXx6InJoGmfUJ vrcxL1ACMfRFjAQ6BTGPQXka4IUHnmw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-88-4eucRKiEPqK2DVn2Tj9hLA-1; Mon, 26 Oct 2020 06:07:04 -0400 X-MC-Unique: 4eucRKiEPqK2DVn2Tj9hLA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A5F3585EE96; Mon, 26 Oct 2020 10:07:03 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA79C8B842; Mon, 26 Oct 2020 10:07:02 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 18/31] scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace Date: Mon, 26 Oct 2020 11:06:19 +0100 Message-Id: <20201026100632.212530-19-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov The generic-fuzzer uses hooks to fulfill DMA requests just-in-time. This means that if we try to use QTEST_LOG=1 to build a reproducer, the DMA writes will be logged _after_ the in/out/read/write that triggered the DMA read. To work work around this, the generic-fuzzer annotates these just-in time DMA fulfilments with a tag that we can use to discern them. This script simply iterates over a raw qtest trace (including log messages, errors, timestamps etc), filters it and re-orders it so that DMA fulfillments are placed directly _before_ the qtest command that will cause the DMA access. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-11-alxndr@bu.edu> Signed-off-by: Thomas Huth --- .../oss-fuzz/reorder_fuzzer_qtest_trace.py | 103 ++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py diff --git a/scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py b/scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py new file mode 100755 index 0000000000..890e1def85 --- /dev/null +++ b/scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py @@ -0,0 +1,103 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +""" +Use this to convert qtest log info from a generic fuzzer input into a qtest +trace that you can feed into a standard qemu-system process. Example usage: + +QEMU_FUZZ_ARGS="-machine q35,accel=qtest" QEMU_FUZZ_OBJECTS="*" \ + ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=generic-pci-fuzz +# .. Finds some crash +QTEST_LOG=1 FUZZ_SERIALIZE_QTEST=1 \ +QEMU_FUZZ_ARGS="-machine q35,accel=qtest" QEMU_FUZZ_OBJECTS="*" \ + ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=generic-pci-fuzz + /path/to/crash 2> qtest_log_output +scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py qtest_log_output > qtest_trace +./i386-softmmu/qemu-fuzz-i386 -machine q35,accel=qtest \ + -qtest stdin < qtest_trace + +### Details ### + +Some fuzzer make use of hooks that allow us to populate some memory range, just +before a DMA read from that range. This means that the fuzzer can produce +activity that looks like: + [start] read from mmio addr + [end] read from mmio addr + [start] write to pio addr + [start] fill a DMA buffer just in time + [end] fill a DMA buffer just in time + [start] fill a DMA buffer just in time + [end] fill a DMA buffer just in time + [end] write to pio addr + [start] read from mmio addr + [end] read from mmio addr + +We annotate these "nested" DMA writes, so with QTEST_LOG=1 the QTest trace +might look something like: +[R +0.028431] readw 0x10000 +[R +0.028434] outl 0xc000 0xbeef # Triggers a DMA read from 0xbeef and 0xbf00 +[DMA][R +0.034639] write 0xbeef 0x2 0xAAAA +[DMA][R +0.034639] write 0xbf00 0x2 0xBBBB +[R +0.028431] readw 0xfc000 + +This script would reorder the above trace so it becomes: +readw 0x10000 +write 0xbeef 0x2 0xAAAA +write 0xbf00 0x2 0xBBBB +outl 0xc000 0xbeef +readw 0xfc000 + +I.e. by the time, 0xc000 tries to read from DMA, those DMA buffers have already +been set up, removing the need for the DMA hooks. We can simply provide this +reordered trace via -qtest stdio to reproduce the input + +Note: this won't work for traces where the device tries to read from the same +DMA region twice in between MMIO/PIO commands. E.g: + [R +0.028434] outl 0xc000 0xbeef + [DMA][R +0.034639] write 0xbeef 0x2 0xAAAA + [DMA][R +0.034639] write 0xbeef 0x2 0xBBBB + +The fuzzer will annotate suspected double-fetches with [DOUBLE-FETCH]. This +script looks for these tags and warns the users that the resulting trace might +not reproduce the bug. +""" + +import sys + +__author__ = "Alexander Bulekov " +__copyright__ = "Copyright (C) 2020, Red Hat, Inc." +__license__ = "GPL version 2 or (at your option) any later version" + +__maintainer__ = "Alexander Bulekov" +__email__ = "alxndr@bu.edu" + + +def usage(): + sys.exit("Usage: {} /path/to/qtest_log_output".format((sys.argv[0]))) + + +def main(filename): + with open(filename, "r") as f: + trace = f.readlines() + + # Leave only lines that look like logged qtest commands + trace[:] = [x.strip() for x in trace if "[R +" in x + or "[S +" in x and "CLOSED" not in x] + + for i in range(len(trace)): + if i+1 < len(trace): + if "[DMA]" in trace[i+1]: + if "[DOUBLE-FETCH]" in trace[i+1]: + sys.stderr.write("Warning: Likely double fetch on line" + "{}.\n There will likely be problems " + "reproducing behavior with the " + "resulting qtest trace\n\n".format(i+1)) + trace[i], trace[i+1] = trace[i+1], trace[i] + for line in trace: + print(line.split("]")[-1].strip()) + + +if __name__ == '__main__': + if len(sys.argv) == 1: + usage() + main(sys.argv[1]) From patchwork Mon Oct 26 10:06:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F6E2921 for ; Mon, 26 Oct 2020 10:23:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0394B207E8 for ; Mon, 26 Oct 2020 10:23:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="G1650yNw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0394B207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:55520 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzf2-0000LI-0C for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:23:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38950) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPH-0002lc-9Q for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27022) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPC-0006k2-DH for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706828; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=+nM0kYcQajA1fWMWAAseVtGUdQwwcwGeyDGj1H0tRms=; b=G1650yNwErlMwmZV4FWbPAOo6/uEJP5Vo6w+QgL2ko3mZrwxEvv8WxyfkMpAavXSLT8wME f8SNA2XacUKUwMjx1yhHW6AphwazCNXnWy2Z8JwnSU6E2CmqVlJ4u4fA2egkHzUpGryo1t Eoetr1SuhAHO95OrwSeRmv4FZrcKPTI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-385-HH44Z7QFMj2iWGkANlxdnA-1; Mon, 26 Oct 2020 06:07:06 -0400 X-MC-Unique: HH44Z7QFMj2iWGkANlxdnA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2324110E2184; Mon, 26 Oct 2020 10:07:05 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E9528B842; Mon, 26 Oct 2020 10:07:03 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 19/31] scripts/oss-fuzz: Add crash trace minimization script Date: Mon, 26 Oct 2020 11:06:20 +0100 Message-Id: <20201026100632.212530-20-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov Once we find a crash, we can convert it into a QTest trace. Usually this trace will contain many operations that are unneeded to reproduce the crash. This script tries to minimize the crashing trace, by removing operations and trimming QTest bufwrite(write addr len data...) commands. Signed-off-by: Alexander Bulekov Reviewed-by: Darren Kenny Message-Id: <20201023150746.107063-12-alxndr@bu.edu> Signed-off-by: Thomas Huth --- scripts/oss-fuzz/minimize_qtest_trace.py | 157 +++++++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/minimize_qtest_trace.py new file mode 100755 index 0000000000..5e405a0d5f --- /dev/null +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +""" +This takes a crashing qtest trace and tries to remove superflous operations +""" + +import sys +import os +import subprocess +import time +import struct + +QEMU_ARGS = None +QEMU_PATH = None +TIMEOUT = 5 +CRASH_TOKEN = None + +write_suffix_lookup = {"b": (1, "B"), + "w": (2, "H"), + "l": (4, "L"), + "q": (8, "Q")} + +def usage(): + sys.exit("""\ +Usage: QEMU_PATH="/path/to/qemu" QEMU_ARGS="args" {} input_trace output_trace +By default, will try to use the second-to-last line in the output to identify +whether the crash occred. Optionally, manually set a string that idenitifes the +crash by setting CRASH_TOKEN= +""".format((sys.argv[0]))) + +def check_if_trace_crashes(trace, path): + global CRASH_TOKEN + with open(path, "w") as tracefile: + tracefile.write("".join(trace)) + + rc = subprocess.Popen("timeout -s 9 {timeout}s {qemu_path} {qemu_args} 2>&1\ + < {trace_path}".format(timeout=TIMEOUT, + qemu_path=QEMU_PATH, + qemu_args=QEMU_ARGS, + trace_path=path), + shell=True, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE) + stdo = rc.communicate()[0] + output = stdo.decode('unicode_escape') + if rc.returncode == 137: # Timed Out + return False + if len(output.splitlines()) < 2: + return False + + if CRASH_TOKEN is None: + CRASH_TOKEN = output.splitlines()[-2] + + return CRASH_TOKEN in output + + +def minimize_trace(inpath, outpath): + global TIMEOUT + with open(inpath) as f: + trace = f.readlines() + start = time.time() + if not check_if_trace_crashes(trace, outpath): + sys.exit("The input qtest trace didn't cause a crash...") + end = time.time() + print("Crashed in {} seconds".format(end-start)) + TIMEOUT = (end-start)*5 + print("Setting the timeout for {} seconds".format(TIMEOUT)) + print("Identifying Crashes by this string: {}".format(CRASH_TOKEN)) + + i = 0 + newtrace = trace[:] + # For each line + while i < len(newtrace): + # 1.) Try to remove it completely and reproduce the crash. If it works, + # we're done. + prior = newtrace[i] + print("Trying to remove {}".format(newtrace[i])) + # Try to remove the line completely + newtrace[i] = "" + if check_if_trace_crashes(newtrace, outpath): + i += 1 + continue + newtrace[i] = prior + + # 2.) Try to replace write{bwlq} commands with a write addr, len + # command. Since this can require swapping endianness, try both LE and + # BE options. We do this, so we can "trim" the writes in (3) + if (newtrace[i].startswith("write") and not + newtrace[i].startswith("write ")): + suffix = newtrace[i].split()[0][-1] + assert(suffix in write_suffix_lookup) + addr = int(newtrace[i].split()[1], 16) + value = int(newtrace[i].split()[2], 16) + for endianness in ['<', '>']: + data = struct.pack("{end}{size}".format(end=endianness, + size=write_suffix_lookup[suffix][1]), + value) + newtrace[i] = "write {addr} {size} 0x{data}\n".format( + addr=hex(addr), + size=hex(write_suffix_lookup[suffix][0]), + data=data.hex()) + if(check_if_trace_crashes(newtrace, outpath)): + break + else: + newtrace[i] = prior + + # 3.) If it is a qtest write command: write addr len data, try to split + # it into two separate write commands. If splitting the write down the + # middle does not work, try to move the pivot "left" and retry, until + # there is no space left. The idea is to prune unneccessary bytes from + # long writes, while accommodating arbitrary MemoryRegion access sizes + # and alignments. + if newtrace[i].startswith("write "): + addr = int(newtrace[i].split()[1], 16) + length = int(newtrace[i].split()[2], 16) + data = newtrace[i].split()[3][2:] + if length > 1: + leftlength = int(length/2) + rightlength = length - leftlength + newtrace.insert(i+1, "") + while leftlength > 0: + newtrace[i] = "write {addr} {size} 0x{data}\n".format( + addr=hex(addr), + size=hex(leftlength), + data=data[:leftlength*2]) + newtrace[i+1] = "write {addr} {size} 0x{data}\n".format( + addr=hex(addr+leftlength), + size=hex(rightlength), + data=data[leftlength*2:]) + if check_if_trace_crashes(newtrace, outpath): + break + else: + leftlength -= 1 + rightlength += 1 + if check_if_trace_crashes(newtrace, outpath): + i -= 1 + else: + newtrace[i] = prior + del newtrace[i+1] + i += 1 + check_if_trace_crashes(newtrace, outpath) + + +if __name__ == '__main__': + if len(sys.argv) < 3: + usage() + + QEMU_PATH = os.getenv("QEMU_PATH") + QEMU_ARGS = os.getenv("QEMU_ARGS") + if QEMU_PATH is None or QEMU_ARGS is None: + usage() + # if "accel" not in QEMU_ARGS: + # QEMU_ARGS += " -accel qtest" + CRASH_TOKEN = os.getenv("CRASH_TOKEN") + QEMU_ARGS += " -qtest stdio -monitor none -serial none " + minimize_trace(sys.argv[1], sys.argv[2]) From patchwork Mon Oct 26 10:06:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3111921 for ; Mon, 26 Oct 2020 10:18:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5BCFB22242 for ; Mon, 26 Oct 2020 10:18:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ijNmUxBB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5BCFB22242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzaQ-0001U0-27 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:18:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38972) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPI-0002o0-4A for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26364) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPD-0006kD-3Z for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706829; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=rPj1DFhuZ+EQW7IN7zl+uQqA9OVlS+YiYiVtbtyvbdg=; b=ijNmUxBBbgZ0H2T/tMGegC3Si7L9/IW0yoUCrayqrTZJO3xMWM9e1uUUNett3W+I1e5xFD 6xdYih4W78iWthTqzwhzS8YhnAqxMBvwnXN5MTrk4tu8m+NI7YSExFt8eQeX3hFVDlN/AT 9MLabP3/ZSrwJUtflcyYrtDtG4eM6tQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-99-qwwkHxQtPmOndVfO_KXOKw-1; Mon, 26 Oct 2020 06:07:07 -0400 X-MC-Unique: qwwkHxQtPmOndVfO_KXOKw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9C30F10E2185; Mon, 26 Oct 2020 10:07:06 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 922E88B842; Mon, 26 Oct 2020 10:07:05 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 20/31] fuzz: Add instructions for using generic-fuzz Date: Mon, 26 Oct 2020 11:06:21 +0100 Message-Id: <20201026100632.212530-21-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-13-alxndr@bu.edu> Signed-off-by: Thomas Huth --- docs/devel/fuzzing.txt | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/docs/devel/fuzzing.txt b/docs/devel/fuzzing.txt index 96d71c94d7..03585c1a9b 100644 --- a/docs/devel/fuzzing.txt +++ b/docs/devel/fuzzing.txt @@ -125,6 +125,45 @@ provided by libfuzzer. Libfuzzer passes a byte array and length. Commonly the fuzzer loops over the byte-array interpreting it as a list of qtest commands, addresses, or values. +== The Generic Fuzzer == +Writing a fuzz target can be a lot of effort (especially if a device driver has +not be built-out within libqos). Many devices can be fuzzed to some degree, +without any device-specific code, using the generic-fuzz target. + +The generic-fuzz target is capable of fuzzing devices over their PIO, MMIO, +and DMA input-spaces. To apply the generic-fuzz to a device, we need to define +two env-variables, at minimum: + +QEMU_FUZZ_ARGS= is the set of QEMU arguments used to configure a machine, with +the device attached. For example, if we want to fuzz the virtio-net device +attached to a pc-i440fx machine, we can specify: +QEMU_FUZZ_ARGS="-M pc -nodefaults -netdev user,id=user0 \ + -device virtio-net,netdev=user0" + +QEMU_FUZZ_OBJECTS= is a set of space-delimited strings used to identify the +MemoryRegions that will be fuzzed. These strings are compared against +MemoryRegion names and MemoryRegion owner names, to decide whether each +MemoryRegion should be fuzzed. These strings support globbing. For the +virtio-net example, we could use QEMU_FUZZ_OBJECTS= + * 'virtio-net' + * 'virtio*' + * 'virtio* pcspk' (Fuzz the virtio devices and the PC speaker...) + * '*' (Fuzz the whole machine) + +The "info mtree" and "info qom-tree" monitor commands can be especially useful +for identifying the MemoryRegion and Object names used for matching. + +As a generic rule-of-thumb, the more MemoryRegions/Devices we match, the greater +the input-space, and the smaller the probability of finding crashing inputs for +individual devices. As such, it is usually a good idea to limit the fuzzer to +only a few MemoryRegions. + +To ensure that these env variables have been configured correctly, we can use: + +./qemu-fuzz-i386 --fuzz-target=generic-fuzz -runs=0 + +The output should contain a complete list of matched MemoryRegions. + = Implementation Details = == The Fuzzer's Lifecycle == From patchwork Mon Oct 26 10:06:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C887921 for ; Mon, 26 Oct 2020 10:16:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 23118223AC for ; Mon, 26 Oct 2020 10:16:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JOJd5//G" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 23118223AC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:33190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzYb-0007rJ-8m for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:16:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38988) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPI-0002qC-Ms for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:23890) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPE-0006kL-CS for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=QaEttNymZfsk/mbC/1bX/ZMA/Ngr/4a0I45bERnDajg=; b=JOJd5//Gj+kr1NqQRtLkcl/h951J0vjEF+awPiJoUr7LMv/Jr9D2OV7C0D/95ICPTvLurt KTzgChHQA9TpYLEg2+m3CE8wC0uUQuQfllNnaZPAsotjheB+WjRF67ejlkRi9cmVfi3GPa YFCyNgV++Z4bw7i65VfwruxCaDk2C5k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-209-PtftKGHRPsqBSKmzxJQpnQ-1; Mon, 26 Oct 2020 06:07:09 -0400 X-MC-Unique: PtftKGHRPsqBSKmzxJQpnQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 134401009E32; Mon, 26 Oct 2020 10:07:08 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 074B08B845; Mon, 26 Oct 2020 10:07:06 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 21/31] fuzz: add an "opaque" to the FuzzTarget struct Date: Mon, 26 Oct 2020 11:06:22 +0100 Message-Id: <20201026100632.212530-22-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov It can be useful to register FuzzTargets that have nearly-identical initialization handlers (e.g. for using the same fuzzing code, with different configuration options). Add an opaque pointer to the FuzzTarget struct, so that FuzzTargets can hold some data, useful for storing target-specific configuration options, that can be read by the get_init_cmdline function. Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-14-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/fuzz.h | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qtest/fuzz/fuzz.h b/tests/qtest/fuzz/fuzz.h index ed9ce17154..08e9560a79 100644 --- a/tests/qtest/fuzz/fuzz.h +++ b/tests/qtest/fuzz/fuzz.h @@ -100,6 +100,7 @@ typedef struct FuzzTarget { uint8_t *out, size_t max_out_size, unsigned int seed); + void *opaque; } FuzzTarget; void flush_events(QTestState *); From patchwork Mon Oct 26 10:06:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E696139F for ; Mon, 26 Oct 2020 10:18:46 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ECCB522242 for ; Mon, 26 Oct 2020 10:18:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iD5BuVUg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ECCB522242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37972 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzaO-0001Sd-UN for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:18:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39010) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPK-0002un-Bl for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:60128) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPH-0006ku-Sw for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706834; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=aucPwhwUbR6TENYaHakJx2OXhB3AYMAGQa1dTFTAkjM=; b=iD5BuVUgnk4sKaATh17zDsszzFgeWhoNO1uJxOnNcfDLLdVJX9bbg+bGqI4+KgTD+3DLmW 1ePgAFHsEcVB+AvfRFpxIhTXMREzVq12GQ9MJvi7Nd1goVQMpvNyN53PlXZl0csSA8uga7 kH4Vs857MYy10gU3q61kB5KYHwQ7qts= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-448-ZU975xKaMJ6vbG4swPYuTw-1; Mon, 26 Oct 2020 06:07:10 -0400 X-MC-Unique: ZU975xKaMJ6vbG4swPYuTw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BB868835BAE; Mon, 26 Oct 2020 10:07:09 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 70BE48B848; Mon, 26 Oct 2020 10:07:08 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 22/31] fuzz: add generic-fuzz configs for oss-fuzz Date: Mon, 26 Oct 2020 11:06:23 +0100 Message-Id: <20201026100632.212530-23-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov Predefine some generic-fuzz configs. For each of these, we will create a separate FuzzTarget that can be selected through argv0 and, therefore, fuzzed on oss-fuzz. Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-15-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/generic_fuzz_configs.h | 121 ++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 tests/qtest/fuzz/generic_fuzz_configs.h diff --git a/tests/qtest/fuzz/generic_fuzz_configs.h b/tests/qtest/fuzz/generic_fuzz_configs.h new file mode 100644 index 0000000000..c4d925f9e6 --- /dev/null +++ b/tests/qtest/fuzz/generic_fuzz_configs.h @@ -0,0 +1,121 @@ +/* + * Generic Virtual-Device Fuzzing Target Configs + * + * Copyright Red Hat Inc., 2020 + * + * Authors: + * Alexander Bulekov + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef GENERIC_FUZZ_CONFIGS_H +#define GENERIC_FUZZ_CONFIGS_H + +#include "qemu/osdep.h" + +typedef struct generic_fuzz_config { + const char *name, *args, *objects; +} generic_fuzz_config; + +const generic_fuzz_config predefined_configs[] = { + { + .name = "virtio-net-pci-slirp", + .args = "-M q35 -nodefaults " + "-device virtio-net,netdev=net0 -netdev user,id=net0", + .objects = "virtio*", + },{ + .name = "virtio-blk", + .args = "-machine q35 -device virtio-blk,drive=disk0 " + "-drive file=null-co://,id=disk0,if=none,format=raw", + .objects = "virtio*", + },{ + .name = "virtio-scsi", + .args = "-machine q35 -device virtio-scsi,num_queues=8 " + "-device scsi-hd,drive=disk0 " + "-drive file=null-co://,id=disk0,if=none,format=raw", + .objects = "scsi* virtio*", + },{ + .name = "virtio-gpu", + .args = "-machine q35 -nodefaults -device virtio-gpu", + .objects = "virtio*", + },{ + .name = "virtio-vga", + .args = "-machine q35 -nodefaults -device virtio-vga", + .objects = "virtio*", + },{ + .name = "virtio-rng", + .args = "-machine q35 -nodefaults -device virtio-rng", + .objects = "virtio*", + },{ + .name = "virtio-balloon", + .args = "-machine q35 -nodefaults -device virtio-balloon", + .objects = "virtio*", + },{ + .name = "virtio-serial", + .args = "-machine q35 -nodefaults -device virtio-serial", + .objects = "virtio*", + },{ + .name = "virtio-mouse", + .args = "-machine q35 -nodefaults -device virtio-mouse", + .objects = "virtio*", + },{ + .name = "e1000", + .args = "-M q35 -nodefaults " + "-device e1000,netdev=net0 -netdev user,id=net0", + .objects = "e1000", + },{ + .name = "e1000e", + .args = "-M q35 -nodefaults " + "-device e1000e,netdev=net0 -netdev user,id=net0", + .objects = "e1000e", + },{ + .name = "cirrus-vga", + .args = "-machine q35 -nodefaults -device cirrus-vga", + .objects = "cirrus*", + },{ + .name = "bochs-display", + .args = "-machine q35 -nodefaults -device bochs-display", + .objects = "bochs*", + },{ + .name = "intel-hda", + .args = "-machine q35 -nodefaults -device intel-hda,id=hda0 " + "-device hda-output,bus=hda0.0 -device hda-micro,bus=hda0.0 " + "-device hda-duplex,bus=hda0.0", + .objects = "intel-hda", + },{ + .name = "ide-hd", + .args = "-machine q35 -nodefaults " + "-drive file=null-co://,if=none,format=raw,id=disk0 " + "-device ide-hd,drive=disk0", + .objects = "ahci*", + },{ + .name = "floppy", + .args = "-machine pc -nodefaults -device floppy,id=floppy0 " + "-drive id=disk0,file=null-co://,file.read-zeroes=on,if=none " + "-device floppy,drive=disk0,drive-type=288", + .objects = "fd* floppy*", + },{ + .name = "xhci", + .args = "-machine q35 -nodefaults " + "-drive file=null-co://,if=none,format=raw,id=disk0 " + "-device qemu-xhci,id=xhci -device usb-tablet,bus=xhci.0 " + "-device usb-bot -device usb-storage,drive=disk0 " + "-chardev null,id=cd0 -chardev null,id=cd1 " + "-device usb-braille,chardev=cd0 -device usb-ccid -device usb-ccid " + "-device usb-kbd -device usb-mouse -device usb-serial,chardev=cd1 " + "-device usb-tablet -device usb-wacom-tablet -device usb-audio", + .objects = "*usb* *uhci* *xhci*", + },{ + .name = "pc-i440fx", + .args = "-machine pc", + .objects = "*", + },{ + .name = "pc-q35", + .args = "-machine q35", + .objects = "*", + } +}; + +#endif From patchwork Mon Oct 26 10:06:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856355 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D787B14B7 for ; Mon, 26 Oct 2020 10:31:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 788552076D for ; Mon, 26 Oct 2020 10:31:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ASzSpnTd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 788552076D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:52294 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzn7-0002YG-9Z for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:31:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39018) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPL-0002ye-US for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:22452) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPH-0006kp-Df for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706834; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=jta5pCSiEwBdmPKVB9/zb0GLa6GuoN2cpd51+Aozuoc=; b=ASzSpnTd2oKeqa8+Ggq2Q38NlujmK2suuOY8pfIVMQx9FdsyJOiAXDFaVHjIhkx4OYfBDF OtWzB0sshC7rs5UXks5pizcyv36lt33mEl2yY/e8JE1PCx/Ag+2zJb6CJymtp8NrrnBt51 vlyqW3ZbIib6+ixluyTtXvsB+ZtG3qw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-343-fb_1nQOYN86EqjRZLrpYiA-1; Mon, 26 Oct 2020 06:07:12 -0400 X-MC-Unique: fb_1nQOYN86EqjRZLrpYiA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 44F8EA0C01; Mon, 26 Oct 2020 10:07:11 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31A798B842; Mon, 26 Oct 2020 10:07:09 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 23/31] fuzz: register predefined generic-fuzz configs Date: Mon, 26 Oct 2020 11:06:24 +0100 Message-Id: <20201026100632.212530-24-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov We call get_generic_fuzz_configs, which fills an array with predefined {name, args, objects} triples. For each of these, we add a new FuzzTarget, that uses a small wrapper to set QEMU_FUZZ_{ARGS,OBJECTS} to the corresponding predefined values. Reviewed-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-16-alxndr@bu.edu> Signed-off-by: Thomas Huth --- tests/qtest/fuzz/generic_fuzz.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tests/qtest/fuzz/generic_fuzz.c b/tests/qtest/fuzz/generic_fuzz.c index 592b78ffe2..a8f5864883 100644 --- a/tests/qtest/fuzz/generic_fuzz.c +++ b/tests/qtest/fuzz/generic_fuzz.c @@ -26,6 +26,7 @@ #include "hw/qdev-core.h" #include "hw/pci/pci.h" #include "hw/boards.h" +#include "generic_fuzz_configs.h" /* * SEPARATOR is used to separate "operations" in the fuzz input @@ -907,6 +908,17 @@ static GString *generic_fuzz_cmdline(FuzzTarget *t) return cmd_line; } +static GString *generic_fuzz_predefined_config_cmdline(FuzzTarget *t) +{ + const generic_fuzz_config *config; + g_assert(t->opaque); + + config = t->opaque; + setenv("QEMU_FUZZ_ARGS", config->args, 1); + setenv("QEMU_FUZZ_OBJECTS", config->objects, 1); + return generic_fuzz_cmdline(t); +} + static void register_generic_fuzz_targets(void) { fuzz_add_target(&(FuzzTarget){ @@ -917,6 +929,26 @@ static void register_generic_fuzz_targets(void) .fuzz = generic_fuzz, .crossover = generic_fuzz_crossover }); + + GString *name; + const generic_fuzz_config *config; + + for (int i = 0; + i < sizeof(predefined_configs) / sizeof(generic_fuzz_config); + i++) { + config = predefined_configs + i; + name = g_string_new("generic-fuzz"); + g_string_append_printf(name, "-%s", config->name); + fuzz_add_target(&(FuzzTarget){ + .name = name->str, + .description = "Predefined generic-fuzz config.", + .get_init_cmdline = generic_fuzz_predefined_config_cmdline, + .pre_fuzz = generic_pre_fuzz, + .fuzz = generic_fuzz, + .crossover = generic_fuzz_crossover, + .opaque = (void *)config + }); + } } fuzz_target_init(register_generic_fuzz_targets); From patchwork Mon Oct 26 10:06:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69CC9921 for ; Mon, 26 Oct 2020 10:19:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 18BA622242 for ; Mon, 26 Oct 2020 10:19:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OMrWllbB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 18BA622242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:41598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzbD-0002xU-1v for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:19:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39012) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPK-0002va-Mv for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21417) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPI-0006l9-Jb for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=1A88vDyeWzKZh8wpgNrYx3cs/bqR5vdM561dmuGFsGg=; b=OMrWllbBliUWT0J298sZlnGrbFQnUUMRMghx4aMNm0/THVJFDQZ9onxV1GrUHSPBPXVo2O Wq08E/AmlSaS5vHxIL1xE4lV/sMjjcM+SlWpd8EdBOKrw0c1lBqfaaxhCRELOnWsd1UF9s tSDwcP4UT/VtaIMcFzG05F2AN42IlUk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-421-Ot9GQ0sePtWSnAf-OpuQYQ-1; Mon, 26 Oct 2020 06:07:13 -0400 X-MC-Unique: Ot9GQ0sePtWSnAf-OpuQYQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A63C51009E27; Mon, 26 Oct 2020 10:07:12 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B84255774; Mon, 26 Oct 2020 10:07:11 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 24/31] scripts/oss-fuzz: use hardlinks instead of copying Date: Mon, 26 Oct 2020 11:06:25 +0100 Message-Id: <20201026100632.212530-25-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov Prior to this, fuzzers in the output oss-fuzz directory were exactly the same executable, with a different name to do argv[0]-based fuzz-target selection. This is a waste of space, especially since these binaries can weigh many MB. Instead of copying, use hard links, to cut down on wasted space. We need to place the primary copy of the executable into DEST_DIR, since this is a separate file-system on oss-fuzz. We should not place it directly into $DEST_DIR, since oss-fuzz will treat it as an independent fuzzer and try to run it for fuzzing. Instead, we create a DEST_DIR/bin directory to store the primary copy. Suggested-by: Darren Kenny Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-17-alxndr@bu.edu> Reviewed-by: Darren Kenny Signed-off-by: Thomas Huth --- scripts/oss-fuzz/build.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh index 0c3ca9e06f..0ce2867732 100755 --- a/scripts/oss-fuzz/build.sh +++ b/scripts/oss-fuzz/build.sh @@ -62,6 +62,9 @@ fi mkdir -p "$DEST_DIR/lib/" # Copy the shared libraries here +mkdir -p "$DEST_DIR/bin/" # Copy executables that shouldn't + # be treated as fuzzers by oss-fuzz here + # Build once to get the list of dynamic lib paths, and copy them over ../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \ --prefix="$DEST_DIR" --bindir="$DEST_DIR" --datadir="$DEST_DIR/data/" \ @@ -88,13 +91,16 @@ make "-j$(nproc)" qemu-fuzz-i386 V=1 # Copy over the datadir cp -r ../pc-bios/ "$DEST_DIR/pc-bios" +cp "./qemu-fuzz-i386" "$DEST_DIR/bin/" + # Run the fuzzer with no arguments, to print the help-string and get the list # of available fuzz-targets. Copy over the qemu-fuzz-i386, naming it according # to each available fuzz target (See 05509c8e6d fuzz: select fuzz target using # executable name) for target in $(./qemu-fuzz-i386 | awk '$1 ~ /\*/ {print $2}'); do - cp qemu-fuzz-i386 "$DEST_DIR/qemu-fuzz-i386-target-$target" + ln "$DEST_DIR/bin/qemu-fuzz-i386" \ + "$DEST_DIR/qemu-fuzz-i386-target-$target" done echo "Done. The fuzzers are located in $DEST_DIR" From patchwork Mon Oct 26 10:06:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC763921 for ; Mon, 26 Oct 2020 10:21:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5E1F422242 for ; Mon, 26 Oct 2020 10:21:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YlxLIwY9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E1F422242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:46406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzcb-00051L-Cw for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:21:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39064) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPO-00035H-M5 for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37246) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPM-0006nV-KD for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=3+2K7QseZ31G6O6eGWPUSYWr2X8ySATmc361BMEOoFg=; b=YlxLIwY90rVjCOsUikMc7GCS46g48W6HRjSjSzAJJKeWKYEA9zwVpmjiOu6CzeEABXvpyf df9mbSBDVRt425EsksUCUKbgUfD0eBqBBosQLrRVcCFIhrO/aAwMz1feAmS5ovqM+nqy28 FfD95OuhreQ1hjn1z2WJ7TZ1gO0nuB0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-369-YbN6D2GmMrmp-c6tnWNT7g-1; Mon, 26 Oct 2020 06:07:15 -0400 X-MC-Unique: YbN6D2GmMrmp-c6tnWNT7g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B9C310E2184; Mon, 26 Oct 2020 10:07:14 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02C0F8B840; Mon, 26 Oct 2020 10:07:12 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 25/31] scripts/oss-fuzz: ignore the generic-fuzz target Date: Mon, 26 Oct 2020 11:06:26 +0100 Message-Id: <20201026100632.212530-26-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/26 02:39:09 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Alexander Bulekov generic-fuzz is not a standalone fuzzer - it requires some env variables to be set. On oss-fuzz, we set these with some predefined generic-fuzz-{...} targets, that are thin wrappers around generic-fuzz. Do not make a link for the generic-fuzz from the oss-fuzz build, so oss-fuzz does not treat it as a standalone fuzzer. Signed-off-by: Alexander Bulekov Message-Id: <20201023150746.107063-18-alxndr@bu.edu> Reviewed-by: Darren Kenny [thuth: Reformatted one comment to stay within the 80 columns limit] Signed-off-by: Thomas Huth --- scripts/oss-fuzz/build.sh | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/scripts/oss-fuzz/build.sh b/scripts/oss-fuzz/build.sh index 0ce2867732..fcae4a0c26 100755 --- a/scripts/oss-fuzz/build.sh +++ b/scripts/oss-fuzz/build.sh @@ -99,8 +99,14 @@ cp "./qemu-fuzz-i386" "$DEST_DIR/bin/" # executable name) for target in $(./qemu-fuzz-i386 | awk '$1 ~ /\*/ {print $2}'); do - ln "$DEST_DIR/bin/qemu-fuzz-i386" \ - "$DEST_DIR/qemu-fuzz-i386-target-$target" + # Ignore the generic-fuzz target, as it requires some environment variables + # to be configured. We have some generic-fuzz-{pc-q35, floppy, ...} targets + # that are thin wrappers around this target that set the required + # environment variables according to predefined configs. + if [ "$target" != "generic-fuzz" ]; then + ln "$DEST_DIR/bin/qemu-fuzz-i386" \ + "$DEST_DIR/qemu-fuzz-i386-target-$target" + fi done echo "Done. The fuzzers are located in $DEST_DIR" From patchwork Mon Oct 26 10:06:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 65D55139F for ; Mon, 26 Oct 2020 10:23:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E3A2B207E8 for ; Mon, 26 Oct 2020 10:23:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GVn8lJdk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E3A2B207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:54730 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzet-0008Sr-TG for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:23:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39088) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPP-00038U-PE for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:50072) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPN-0006o1-Jb for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zKsN/iAC05uvxgDfrZ6Mxdox1qphBhcT6J32MC45aOg=; b=GVn8lJdkOpwnUOPRnIHXwk3iDEEJwi68juwfP1/YwHrrXB/Y6KSl/An1KqjwAQW+yRd+7r WLLTp7k7DbxWzzqyeo5AFO5XSGUbOt+MrQlneij1l5hv5i1IH9e31KhhKIyWb2pQ3Ct0BQ k7IwtuY7jGnzhewt2QdgssNWn4oSA8g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-232-_t7emAljPQ6gPS8A2Y_uCw-1; Mon, 26 Oct 2020 06:07:16 -0400 X-MC-Unique: _t7emAljPQ6gPS8A2Y_uCw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 860A31009E2C; Mon, 26 Oct 2020 10:07:15 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7A5ED8B847; Mon, 26 Oct 2020 10:07:14 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 26/31] tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive Date: Mon, 26 Oct 2020 11:06:27 +0100 Message-Id: <20201026100632.212530-27-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The current URL on cdn.netbsd.org is failing - using archive.netbsd.org instead seems to be fine. Message-Id: <20201023073351.251332-2-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/acceptance/ppc_prep_40p.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/acceptance/ppc_prep_40p.py b/tests/acceptance/ppc_prep_40p.py index 1515561249..e82755c9cf 100644 --- a/tests/acceptance/ppc_prep_40p.py +++ b/tests/acceptance/ppc_prep_40p.py @@ -35,7 +35,7 @@ class IbmPrep40pMachine(Test): '7020-40p/P12H0456.IMG') bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03' bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash) - drive_url = ('https://cdn.netbsd.org/pub/NetBSD/NetBSD-archive/' + drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/' 'NetBSD-4.0/prep/installation/floppy/generic_com0.fs') drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb' drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash) From patchwork Mon Oct 26 10:06:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D421B139F for ; Mon, 26 Oct 2020 10:22:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6498422242 for ; Mon, 26 Oct 2020 10:22:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VaRk4O9O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6498422242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:50028 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzdX-0006W8-Fu for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:21:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39070) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPP-00036S-2r for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:27052) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPN-0006nc-2u for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lEYlJQc8unISlHVYrQcVwNIoZboimZpudB3FA//629w=; b=VaRk4O9O5mIgAKEtbSjNatwJqCXr4+y5DKaZ/BKfkg7dL1b8N6Yl71uqNP0Z9YHnJ5zgpH w/3NNYy3XHmGuqKVkcqa5YDMI5WdxPOKElM5UqoJ4FcTr3X+zeVP6XQZS04FjexayOpo56 32Oe4SPL9FFJMpKvLJ3rddsTc04HEVA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-386-Ze2GltQrPFKWkSzIN6G_2w-1; Mon, 26 Oct 2020 06:07:18 -0400 X-MC-Unique: Ze2GltQrPFKWkSzIN6G_2w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F1014A0C07; Mon, 26 Oct 2020 10:07:16 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4C108B840; Mon, 26 Oct 2020 10:07:15 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 27/31] test/acceptance: Remove the CONTINUOUS_INTEGRATION tags Date: Mon, 26 Oct 2020 11:06:28 +0100 Message-Id: <20201026100632.212530-28-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We are not running the acceptance tests on Travis anymore, so these checks can be removed now. Message-Id: <20201023073351.251332-3-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/acceptance/ppc_prep_40p.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/tests/acceptance/ppc_prep_40p.py b/tests/acceptance/ppc_prep_40p.py index e82755c9cf..96ba13b894 100644 --- a/tests/acceptance/ppc_prep_40p.py +++ b/tests/acceptance/ppc_prep_40p.py @@ -22,7 +22,6 @@ class IbmPrep40pMachine(Test): # All rights reserved. # U.S. Government Users Restricted Rights - Use, duplication or disclosure # restricted by GSA ADP Schedule Contract with IBM Corp. - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code') def test_factory_firmware_and_netbsd(self): """ @@ -61,7 +60,6 @@ class IbmPrep40pMachine(Test): wait_for_console_pattern(self, '>> Memory: 192M') wait_for_console_pattern(self, '>> CPU type PowerPC,604') - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') def test_openbios_and_netbsd(self): """ :avocado: tags=arch:ppc From patchwork Mon Oct 26 10:06:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856335 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60057139F for ; Mon, 26 Oct 2020 10:24:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14BE7222EA for ; Mon, 26 Oct 2020 10:24:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q9TpFtmm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14BE7222EA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:58418 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzfb-0001Uz-1X for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:24:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39098) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPQ-0003AZ-Gm for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:51729) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPO-0006od-OU for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YMtRQrH9IZNsT/G1dPS6fD9Yz8Zn3wXQQYMwfK7Lefk=; b=Q9TpFtmms3eqJ5OwQ6jeKFtuyQgTf55vRG3XdCkSqRrTMlYUjtbO0D14vpj9lBwGAP9z0Y D4Z7sViSb4NInQnzL6dBuIlj9hSqtqSUZMZFei0XbJ93Evuwwsmudv6eX9BeJXXq//vt+9 BKT+N8oh6IUgffzVoEr4irQNBoDvktU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-486-SXlFJhkPPDGWl4jvCloZjA-1; Mon, 26 Oct 2020 06:07:19 -0400 X-MC-Unique: SXlFJhkPPDGWl4jvCloZjA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6862D1882FB3; Mon, 26 Oct 2020 10:07:18 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 59F1E8B840; Mon, 26 Oct 2020 10:07:17 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 28/31] tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI Date: Mon, 26 Oct 2020 11:06:29 +0100 Message-Id: <20201026100632.212530-29-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The tests are running in containers here, so it should be OK to run with AVOCADO_ALLOW_UNTRUSTED_CODE enabled in this case. Message-Id: <20201023073351.251332-4-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 66ad7aa5c2..5d6773efd2 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -66,6 +66,7 @@ include: - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then du -chs ${CI_PROJECT_DIR}/avocado-cache ; fi + - export AVOCADO_ALLOW_UNTRUSTED_CODE=1 after_script: - cd build - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP", "CANCEL")]' | xargs cat From patchwork Mon Oct 26 10:06:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856343 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88EAA921 for ; Mon, 26 Oct 2020 10:26:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 21EDF207E8 for ; Mon, 26 Oct 2020 10:26:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gQ+8+tni" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21EDF207E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWziM-00057M-2z for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:26:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39126) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPS-0003EG-2M for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:38994) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPP-0006ou-UD for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eIcd7zNIDLSZPyeH0LlHqqlH7IKik6kk7NxlKYOMVI4=; b=gQ+8+tnikTzSqvSNHAj/7yu0KEGvh2+r3wEqOhcs8gi8yf7UkAm4JeROP/lTMx8YmwYxFh mz57xTx74s/ycqg5PgHW4+UKVFxYdPLiRfjsxcBUFZfafRrv5PRipRMC6tkXpjCp6I0xM2 6cf9bVnRXDi5UEzKYPln2fJLPnaPMxk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-464-9LW9HBsDMZqDKO2VetVllg-1; Mon, 26 Oct 2020 06:07:20 -0400 X-MC-Unique: 9LW9HBsDMZqDKO2VetVllg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CF7FAA0C00; Mon, 26 Oct 2020 10:07:19 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3F358B847; Mon, 26 Oct 2020 10:07:18 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 29/31] test/docker/dockerfiles: Add missing packages for acceptance tests Date: Mon, 26 Oct 2020 11:06:30 +0100 Message-Id: <20201026100632.212530-30-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Some of the "check-acceptance" tests are still skipped in the CI since the docker images do not provide the necessary packages, e.g. the netcat binary. Add them to get more test coverage. Message-Id: <20201023073351.251332-5-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 3 +++ tests/docker/dockerfiles/fedora.docker | 1 + tests/docker/dockerfiles/ubuntu2004.docker | 1 + 4 files changed, 6 insertions(+) diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker index 585dfad9be..a589142114 100644 --- a/tests/docker/dockerfiles/centos8.docker +++ b/tests/docker/dockerfiles/centos8.docker @@ -18,6 +18,7 @@ ENV PACKAGES \ lzo-devel \ make \ mesa-libEGL-devel \ + nmap-ncat \ nettle-devel \ ninja-build \ perl-Test-Harness \ diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker index 314c6bae83..55075d9fce 100644 --- a/tests/docker/dockerfiles/debian-amd64.docker +++ b/tests/docker/dockerfiles/debian-amd64.docker @@ -23,6 +23,9 @@ RUN apt update && \ libsnappy-dev \ libvte-dev \ netcat-openbsd \ + openssh-client \ + python3-numpy \ + python3-opencv \ python3-venv # virgl diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index ac79d95418..0b5053f2d0 100644 --- a/tests/docker/dockerfiles/fedora.docker +++ b/tests/docker/dockerfiles/fedora.docker @@ -73,6 +73,7 @@ ENV PACKAGES \ mingw64-pixman \ mingw64-pkg-config \ mingw64-SDL2 \ + nmap-ncat \ ncurses-devel \ nettle-devel \ ninja-build \ diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker index 17b37cda38..355bbb3c63 100644 --- a/tests/docker/dockerfiles/ubuntu2004.docker +++ b/tests/docker/dockerfiles/ubuntu2004.docker @@ -47,6 +47,7 @@ ENV PACKAGES flex bison \ libxen-dev \ libzstd-dev \ make \ + netcat-openbsd \ ninja-build \ python3-numpy \ python3-opencv \ From patchwork Mon Oct 26 10:06:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856361 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E9EC14B7 for ; Mon, 26 Oct 2020 10:35:13 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EE14820770 for ; Mon, 26 Oct 2020 10:35:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Veraqw3f" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE14820770 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:60766 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzqJ-0006Ag-Or for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:35:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPT-0003HZ-CQ for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:35086) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPR-0006pv-HP for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706844; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtGyUxgpbnAJC2pUaJOU8eGTDOM6Sv/l+7ShBPHTAog=; b=Veraqw3f1Bleaxsr7qQNNVJAIAbZqGZFRswMIxA8JJfSgd7+F9OuppKR8IL+emYo08W8g2 3rDNJ1WyJNvarGKDfoDVmYdiBCmTQZ3i2h78hBKZyK5F7aRrHNpIsPRag3mtZ/AXzGWjmb qAUN5ONgDNJTCaKsHknVvvmk5HS00iE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-478-HC5Op537Nqa0PK6xSbO8Ng-1; Mon, 26 Oct 2020 06:07:22 -0400 X-MC-Unique: HC5Op537Nqa0PK6xSbO8Ng-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 46B801009E44; Mon, 26 Oct 2020 10:07:21 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3B4158B840; Mon, 26 Oct 2020 10:07:20 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 30/31] tests/acceptance: Remove unused import Date: Mon, 26 Oct 2020 11:06:31 +0100 Message-Id: <20201026100632.212530-31-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Philippe Mathieu-Daudé Signed-off-by: Philippe Mathieu-Daudé Message-Id: <20201021105035.2477784-2-f4bug@amsat.org> Tested-by: Thomas Huth Signed-off-by: Thomas Huth --- tests/acceptance/machine_m68k_nextcube.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/acceptance/machine_m68k_nextcube.py b/tests/acceptance/machine_m68k_nextcube.py index 32cf571f94..9d289f2fab 100644 --- a/tests/acceptance/machine_m68k_nextcube.py +++ b/tests/acceptance/machine_m68k_nextcube.py @@ -9,7 +9,6 @@ import os import re import time import logging -import distutils.spawn from avocado_qemu import Test from avocado import skipUnless From patchwork Mon Oct 26 10:06:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 11856369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAB71139F for ; Mon, 26 Oct 2020 10:38:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5A84A20791 for ; Mon, 26 Oct 2020 10:38:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QoiMBOjh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A84A20791 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:40864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWzt4-0001MP-8U for patchwork-qemu-devel@patchwork.kernel.org; Mon, 26 Oct 2020 06:38:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWzPU-0003LX-QP for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:35412) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kWzPS-0006qG-UD for qemu-devel@nongnu.org; Mon, 26 Oct 2020 06:07:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603706845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eYu3EUAAaV9viSHuaicHSt5ZTnVbdYj7opK/cMyTMLQ=; b=QoiMBOjhizDOGbk1fokwVXpdhywk8w5s3W0QgNtOAxuNvE5R/LUk/ILyvW3vTsCd8UDoCF 6u1kEo/4BcM9S0QVEVPV5LNcx7ZX+eQZiJiknRlNlCRQQHkj/BNFIEDzzGhRJnNoo4flm4 dcR1ngVdZIIMB9PwLZSoD33FnD24BWc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-150-MUsGFFKqP0ahGQ9ucqvyGw-1; Mon, 26 Oct 2020 06:07:23 -0400 X-MC-Unique: MUsGFFKqP0ahGQ9ucqvyGw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B271910E2185; Mon, 26 Oct 2020 10:07:22 +0000 (UTC) Received: from thuth.com (ovpn-112-104.ams2.redhat.com [10.36.112.104]) by smtp.corp.redhat.com (Postfix) with ESMTP id A5FF78B840; Mon, 26 Oct 2020 10:07:21 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 31/31] tests/acceptance: Use .ppm extention for Portable PixMap files Date: Mon, 26 Oct 2020 11:06:32 +0100 Message-Id: <20201026100632.212530-32-thuth@redhat.com> In-Reply-To: <20201026100632.212530-1-thuth@redhat.com> References: <20201026100632.212530-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/25 21:03:19 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Philippe Mathieu-Daudé The HMP 'screendump' command generates Portable PixMap files. Make it obvious by using the .ppm file extention. Signed-off-by: Philippe Mathieu-Daudé Message-Id: <20201021105035.2477784-3-f4bug@amsat.org> Reviewed-by: Thomas Huth Signed-off-by: Thomas Huth --- tests/acceptance/machine_m68k_nextcube.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/acceptance/machine_m68k_nextcube.py b/tests/acceptance/machine_m68k_nextcube.py index 9d289f2fab..2baba5fdc2 100644 --- a/tests/acceptance/machine_m68k_nextcube.py +++ b/tests/acceptance/machine_m68k_nextcube.py @@ -69,7 +69,7 @@ class NextCubeMachine(Test): @skipUnless(PIL_AVAILABLE, 'Python PIL not installed') def test_bootrom_framebuffer_size(self): - screenshot_path = os.path.join(self.workdir, "dump.png") + screenshot_path = os.path.join(self.workdir, "dump.ppm") self.check_bootrom_framebuffer(screenshot_path) width, height = Image.open(screenshot_path).size @@ -78,7 +78,7 @@ class NextCubeMachine(Test): @skipUnless(tesseract_available(3), 'tesseract v3 OCR tool not available') def test_bootrom_framebuffer_ocr_with_tesseract_v3(self): - screenshot_path = os.path.join(self.workdir, "dump.png") + screenshot_path = os.path.join(self.workdir, "dump.ppm") self.check_bootrom_framebuffer(screenshot_path) console_logger = logging.getLogger('console') @@ -94,7 +94,7 @@ class NextCubeMachine(Test): # that it is still alpha-level software. @skipUnless(tesseract_available(4), 'tesseract v4 OCR tool not available') def test_bootrom_framebuffer_ocr_with_tesseract_v4(self): - screenshot_path = os.path.join(self.workdir, "dump.png") + screenshot_path = os.path.join(self.workdir, "dump.ppm") self.check_bootrom_framebuffer(screenshot_path) console_logger = logging.getLogger('console')