From patchwork Thu Jun 1 16:31:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 13264256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70C07C7EE23 for ; Thu, 1 Jun 2023 16:33:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q4lD7-0002Ww-Iy; Thu, 01 Jun 2023 12:31:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD6-0002WW-9S for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD4-0007Fm-RX for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685637094; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v9Oi51bhCX5qIbsmMbnbS5ABJuc4m/obv+3BSa/y2jo=; b=MYgNv2jb6hYcQVESn63Ckf9DjTWTRIzgSzS/V+zl16GwCEdjaZF380V1wOcMsbXlXQNw8P 08op/l+LmCnQju9ZfPnMkWZVUwJUAyUqP7tb5/DJ80Z97NhGtvh/KLpzUZhARMWgKHX43U rgOQCahuM4Fx0SEnjU+uDS70lirO80Y= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-212-v6TiPR27PP6y1hN62cAMeg-1; Thu, 01 Jun 2023 12:31:29 -0400 X-MC-Unique: v6TiPR27PP6y1hN62cAMeg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 77A6D1C0693A; Thu, 1 Jun 2023 16:31:27 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.28.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5995E2166B25; Thu, 1 Jun 2023 16:31:26 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Paolo Bonzini , John Snow , Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= Subject: [PATCH 1/6] qtest: bump min meson timeout to 60 seconds Date: Thu, 1 Jun 2023 17:31:18 +0100 Message-Id: <20230601163123.1805282-2-berrange@redhat.com> In-Reply-To: <20230601163123.1805282-1-berrange@redhat.com> References: <20230601163123.1805282-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.166, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Even some of the relatively fast qtests can sometimes hit the 30 second timeout in GitLab CI under high parallelism/load conditions. Bump the min to 60 seconds to give a higher margin for reliability. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qtest/meson.build | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 087f2dc9d7..18d046b53c 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -1,12 +1,7 @@ slow_qtests = { - 'ahci-test' : 60, 'bios-tables-test' : 120, - 'boot-serial-test' : 60, 'migration-test' : 150, 'npcm7xx_pwm-test': 150, - 'prom-env-test' : 60, - 'pxe-test' : 60, - 'qos-test' : 60, 'qom-test' : 300, 'test-hmp' : 120, } @@ -373,8 +368,8 @@ foreach dir : target_dirs env: qtest_env, args: ['--tap', '-k'], protocol: 'tap', - timeout: slow_qtests.get(test, 30), - priority: slow_qtests.get(test, 30), + timeout: slow_qtests.get(test, 60), + priority: slow_qtests.get(test, 60), suite: ['qtest', 'qtest-' + target_base]) endforeach endforeach From patchwork Thu Jun 1 16:31:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 13264255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98A66C7EE23 for ; Thu, 1 Jun 2023 16:33:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q4lD7-0002Wv-96; Thu, 01 Jun 2023 12:31:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD5-0002WO-FX for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD3-0007Fh-RQ for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685637092; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bZJ+h6yEVR3M3K3/wAYWtQF1Ne5IREviDXco9o9kwFY=; b=YAtgWRiW3QLoHNgfM+EncAT6IaXhwA2/xK0gJorfithufHi7pDq2Z3cMZbk0Z9QU02jXMJ vsujUAMo5jR9F7f3+v02+VnzbMVNlNsoAwn82ickx9+PvlUg/dNvnyCMtS43zdKgaWufny JfQEJyYI2bNlwTjx3DYOM1q+ncbjBUg= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-275-BfO_61ROObm5tceHdM6U6g-1; Thu, 01 Jun 2023 12:31:29 -0400 X-MC-Unique: BfO_61ROObm5tceHdM6U6g-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 749C728135A1; Thu, 1 Jun 2023 16:31:29 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.28.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id BFCB92166B27; Thu, 1 Jun 2023 16:31:27 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Paolo Bonzini , John Snow , Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= Subject: [PATCH 2/6] qtest: bump migration-test timeout to 5 minutes Date: Thu, 1 Jun 2023 17:31:19 +0100 Message-Id: <20230601163123.1805282-3-berrange@redhat.com> In-Reply-To: <20230601163123.1805282-1-berrange@redhat.com> References: <20230601163123.1805282-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.166, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The migration test should take between 1 min 30 and 2 mins on reasonably modern hardware. The test is not especially compute bound, rather its running time is dominated by the guest RAM size relative to the bandwidth cap, which forces each iteration to take at least 30 seconds. None the less under high load conditions with multiple QEMU processes spawned and competing with other parallel tests, the worst case running time might be somewhat extended. Bumping the timeout to 5 minutes gives us good headroom, while still catching stuck tests relatively quickly. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qtest/meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 18d046b53c..6684591fcf 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -1,6 +1,6 @@ slow_qtests = { 'bios-tables-test' : 120, - 'migration-test' : 150, + 'migration-test' : 300, 'npcm7xx_pwm-test': 150, 'qom-test' : 300, 'test-hmp' : 120, From patchwork Thu Jun 1 16:31:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 13264252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AAA6DC7EE2A for ; Thu, 1 Jun 2023 16:32:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q4lD9-0002XZ-Bq; Thu, 01 Jun 2023 12:31:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD7-0002Wx-Sh for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD6-0007Gz-BV for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685637095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cciXctEGV7NmbEtjS1ATqI6wDg6uREgu96kdcuyUnBE=; b=WEHS5yUtkSIqTpmLqYiRbq/+zveHF/3EAIa/oQ3Rgjxh3RN8laAGh81JSN8orIjwJDEK2y ddLxloxoFybzQD5BeeCb1GrwhT4s0zIVm2LLEcMcckr1zc6aWo3+TwO1CqNZ6jjPozYIOh d3kDH65JstEQmlZwTo2mcpdn1yPB7Y8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-637-jRMPCpVRN5mFUGiFoY0n6Q-1; Thu, 01 Jun 2023 12:31:34 -0400 X-MC-Unique: jRMPCpVRN5mFUGiFoY0n6Q-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 855B1811E9F; Thu, 1 Jun 2023 16:31:31 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.28.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id AACB72166B25; Thu, 1 Jun 2023 16:31:29 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Paolo Bonzini , John Snow , Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= Subject: [PATCH 3/6] qtest: bump qom-test timeout to 7 minutes Date: Thu, 1 Jun 2023 17:31:20 +0100 Message-Id: <20230601163123.1805282-4-berrange@redhat.com> In-Reply-To: <20230601163123.1805282-1-berrange@redhat.com> References: <20230601163123.1805282-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.166, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The qom-test is periodically hitting the 5 minute timeout when running on the aarch64 emulator under GitLab CI. Add another 2 minutes headroom to the timeout to improve reliability. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qtest/meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 6684591fcf..6943bbfdd5 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -2,7 +2,7 @@ slow_qtests = { 'bios-tables-test' : 120, 'migration-test' : 300, 'npcm7xx_pwm-test': 150, - 'qom-test' : 300, + 'qom-test' : 420, 'test-hmp' : 120, } From patchwork Thu Jun 1 16:31:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 13264250 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21C9AC7EE2E for ; Thu, 1 Jun 2023 16:31:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q4lDA-0002Xp-3Z; Thu, 01 Jun 2023 12:31:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD8-0002XQ-Ol for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD7-0007HI-BK for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685637096; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GR0QirnJMiyI0U3XkAzHWV/YuEOtMc541qkVrRBGMTs=; b=ENbSTVrylguMKRs8JuYDVBDDSr4EZjq0K92952XaNgXuZgO4O25DyQQeyCqDE3N5T0fCS6 qX5hDcGmT5da6Pc2HYVV3ZYplexmAxr7JkSZR9MrdDySazffSz8/4m4lRdr7ugMoA+g6IS iWk9IcgVt438dc7ekF2KGA6rK56ndm4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-526-utOH3l5bO0GYSHwiP_ehIg-1; Thu, 01 Jun 2023 12:31:35 -0400 X-MC-Unique: utOH3l5bO0GYSHwiP_ehIg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 15081857F86; Thu, 1 Jun 2023 16:31:33 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.28.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id C64952166B27; Thu, 1 Jun 2023 16:31:31 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Paolo Bonzini , John Snow , Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= Subject: [PATCH 4/6] qtest: bump aspeed_smc-test timeout to 2 minutes Date: Thu, 1 Jun 2023 17:31:21 +0100 Message-Id: <20230601163123.1805282-5-berrange@redhat.com> In-Reply-To: <20230601163123.1805282-1-berrange@redhat.com> References: <20230601163123.1805282-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.166, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org On a reasonably modern laptop this test takes 40 seconds with the arm emulator. Raising the timeout to 2 minutes gives greater headroom for slowdown under GitLab CI. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qtest/meson.build | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 6943bbfdd5..d9fa30edbc 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -1,4 +1,5 @@ slow_qtests = { + 'aspeed_smc-test': 120, 'bios-tables-test' : 120, 'migration-test' : 300, 'npcm7xx_pwm-test': 150, From patchwork Thu Jun 1 16:31:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 13264251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D0A6C77B7A for ; Thu, 1 Jun 2023 16:32:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q4lDF-0002Z3-9A; Thu, 01 Jun 2023 12:31:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lDB-0002YD-RZ for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lDA-0007IJ-Em for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685637098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nx5fsVXColqcBMBqsULmVdZVV9Jw60x5WpkS8vJyYeI=; b=cTIdTOhjsZSjn94cOk2gp4ncJwK1qopdw7nLPhlsCg8Gt6JT4GMpm9ilRhw4O5T8BRMAGk hMMvdDb1MgDMGDNYqPEEjAPZNDEom0hp2WGmbbiS0WqalxFx7PEdh4S2gQtsQvZhF+CeUG AY+GmFLbGHhYVPDf7mtBrd6zUYofdnI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-331-6dbAL3BoNYiXQ5iS8jOaVA-1; Thu, 01 Jun 2023 12:31:35 -0400 X-MC-Unique: 6dbAL3BoNYiXQ5iS8jOaVA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 765AA384953E; Thu, 1 Jun 2023 16:31:34 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.28.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40A5F2166B27; Thu, 1 Jun 2023 16:31:33 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Paolo Bonzini , John Snow , Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= Subject: [PATCH 5/6] qtest: bump bios-table-test timeout to 6 minutes Date: Thu, 1 Jun 2023 17:31:22 +0100 Message-Id: <20230601163123.1805282-6-berrange@redhat.com> In-Reply-To: <20230601163123.1805282-1-berrange@redhat.com> References: <20230601163123.1805282-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.166, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is reliably hitting the current 2 minute timeout in GitLab CI for the TCI job, and even hits a 4 minute timeout. At 6 minutes it looks sufficiently reliable. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qtest/meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index d9fa30edbc..4f45369421 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -1,6 +1,6 @@ slow_qtests = { 'aspeed_smc-test': 120, - 'bios-tables-test' : 120, + 'bios-tables-test' : 360, 'migration-test' : 300, 'npcm7xx_pwm-test': 150, 'qom-test' : 420, From patchwork Thu Jun 1 16:31:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 13264253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B992C7EE2E for ; Thu, 1 Jun 2023 16:32:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q4lDE-0002Yf-1L; Thu, 01 Jun 2023 12:31:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lDB-0002YA-72 for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q4lD9-0007IS-Lm for qemu-devel@nongnu.org; Thu, 01 Jun 2023 12:31:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685637098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=23vcARQxIY7gh1NYVRtuGbogmRD03bZ7QlD5l2q1H+U=; b=D7pP4orsVSKHuL6D6jHuKvvENNjRShUs6PHqJxFHqU2Wmx/JcQXzhh8ORAbvk9n2Zjk/Kz rG6wwSvHHbLVyZ+ebreg/sBSAWPz8BqJjp2uW6eA0PfTt/oY/aBB1YVtzN6t3pBDGPI5x5 T05nfj6+iIFUR7XOGb1lPOmh0a2RLeM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-388-rK08IfCONKKFfKcn6WNWMQ-1; Thu, 01 Jun 2023 12:31:37 -0400 X-MC-Unique: rK08IfCONKKFfKcn6WNWMQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B50688582BE; Thu, 1 Jun 2023 16:31:35 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.42.28.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id A452D2166B28; Thu, 1 Jun 2023 16:31:34 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , Paolo Bonzini , John Snow , Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= Subject: [PATCH 6/6] mtest2make: stop disabling meson test timeouts Date: Thu, 1 Jun 2023 17:31:23 +0100 Message-Id: <20230601163123.1805282-7-berrange@redhat.com> In-Reply-To: <20230601163123.1805282-1-berrange@redhat.com> References: <20230601163123.1805282-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -22 X-Spam_score: -2.3 X-Spam_bar: -- X-Spam_report: (-2.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.166, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The mtest2make.py script passes the arg '-t 0' to 'meson test' which disables all test timeouts. This is a major source of pain when running in GitLab CI and a test gets stuck. It will stall until GitLab kills the CI job. This leaves us with little easily consumable information about the stalled test. The TAP format doesn't show the test name until it is completed, and TAP output from multiple tests it interleaved. So we have to analyse the log to figure out what tests had un-finished TAP output present and thus infer which test case caused the hang. This is very time consuming and error prone. By allowing meson to kill stalled tests, we get a direct display of what test program got stuck, which lets us more directly focus in on what specific test case within the test program hung. The other issue with disabling meson test timeouts by default is that it makes it more likely that maintainers inadvertantly introduce slowdowns. For example the recent-ish change that accidentally made migrate-test take 15-20 minutes instead of around 1 minute. The main risk of this change is that the individual test timeouts might be too short to allow completion in high load scenarios. Thus, there is likely to be some short term pain where we have to bump the timeouts for certain tests to make them reliable enough. The preceeding few patches raised the timeouts for all failures that were immediately apparent in GitLab CI. Even with the possible short term instability, this should still be a net win for debuggability of failed CI pipelines over the long term. Signed-off-by: Daniel P. Berrangé --- scripts/mtest2make.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py index 179dd54871..eb01a05ddb 100644 --- a/scripts/mtest2make.py +++ b/scripts/mtest2make.py @@ -27,7 +27,8 @@ def names(self, base): .speed.slow = $(foreach s,$(sort $(filter-out %-thorough, $1)), --suite $s) .speed.thorough = $(foreach s,$(sort $1), --suite $s) -.mtestargs = --no-rebuild -t 0 +TIMEOUT_MULTIPLIER = 1 +.mtestargs = --no-rebuild -t $(TIMEOUT_MULTIPLIER) ifneq ($(SPEED), quick) .mtestargs += --setup $(SPEED) endif