From patchwork Fri Sep 4 16:42:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758211 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4DC55618 for ; Fri, 4 Sep 2020 16:43:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 27CB82064E for ; Fri, 4 Sep 2020 16:43:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27CB82064E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:48036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEEoc-00081p-9F for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:43:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEnz-0006iI-DI for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:15 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:22287 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEnv-0000Ao-VS for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-482-opbDeT5uOLOd14OCXWwJ_g-1; Fri, 04 Sep 2020 12:43:06 -0400 X-MC-Unique: opbDeT5uOLOd14OCXWwJ_g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E3B521074660; Fri, 4 Sep 2020 16:43:04 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C952719C59; Fri, 4 Sep 2020 16:43:03 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 1/7] scripts/ci/gitlab-pipeline-status: make branch name configurable Date: Fri, 4 Sep 2020 12:42:52 -0400 Message-Id: <20200904164258.240278-2-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=crosa@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 03:58:24 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" With the utility function `get_local_staging_branch_commit()`, the name of the branch is hard coded (including in the function name). For extensibility reasons, let's make that configurable. Signed-off-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé --- scripts/ci/gitlab-pipeline-status | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index 348a49b6a4..194dd4d0bb 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -23,20 +23,20 @@ import time import sys -def get_local_staging_branch_commit(): +def get_local_branch_commit(branch='staging'): """ Returns the commit sha1 for the *local* branch named "staging" """ - result = subprocess.run(['git', 'rev-parse', 'staging'], + result = subprocess.run(['git', 'rev-parse', branch], stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=os.path.dirname(__file__), universal_newlines=True).stdout.strip() - if result == 'staging': - raise ValueError("There's no local branch named 'staging'") + if result == branch: + raise ValueError("There's no local branch named '%s'" % branch) if len(result) != 40: - raise ValueError("Branch staging HEAD doesn't look like a sha1") + raise ValueError("Branch '%s' HEAD doesn't look like a sha1" % branch) return result @@ -110,7 +110,7 @@ def main(): 'for https://gitlab.com/qemu-project/qemu, that ' 'is, "%(default)s"')) try: - default_commit = get_local_staging_branch_commit() + default_commit = get_local_branch_commit() commit_required = False except ValueError: default_commit = '' From patchwork Fri Sep 4 16:42:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758229 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85F8514EB for ; Fri, 4 Sep 2020 16:45:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58A882064E for ; Fri, 4 Sep 2020 16:45:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JCKtF9T9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58A882064E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:53202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEEpo-0001mG-Hi for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:45:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41264) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEnv-0006g7-On for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34632) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEnt-0000AH-Pk for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599237788; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=afv68zSSKhrJyHAblKUIOTaNZ9p55vBWA5wqKbMlDZA=; b=JCKtF9T9CpQgZNJ9IDzFolfwP+KSj24uSkhjfD4Cy9x6OPy01OggBUGb3XfF3vrFZnpV9X jP6slLDgXKdPcFlcKHRWLXY4LCitQlbHnJzjz3dVOPoY70EaUuzDVF0HfdtSLSaMPhPcUA Q0RWFgVO1Wf8Jms5vqEaxKjiGjeXUvY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-39-kVNvbccAMOe9hCLr8C2MxA-1; Fri, 04 Sep 2020 12:43:07 -0400 X-MC-Unique: kVNvbccAMOe9hCLr8C2MxA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 32EB780BCA1; Fri, 4 Sep 2020 16:43:06 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1B49519C59; Fri, 4 Sep 2020 16:43:05 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 2/7] scripts/ci/gitlab-pipeline-status: improve message regarding timeout Date: Fri, 4 Sep 2020 12:42:53 -0400 Message-Id: <20200904164258.240278-3-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 11:45:25 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The script has its own timeout, which is about how long the script will wait (when called with --wait) for the pipeline to complete, and not necessarily for the pipeline to complete. Hopefully this will new wording will be clearer. Signed-off-by: Cleber Rosa --- scripts/ci/gitlab-pipeline-status | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index 194dd4d0bb..2a36f74696 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -69,7 +69,10 @@ def wait_on_pipeline_success(timeout, interval, start = time.time() while True: if time.time() >= (start + timeout): - print("Waiting on the pipeline timed out") + msg = ("Timeout (-t/--timeout) of %i seconds reached, " + "won't wait any longer for the pipeline to complete") + msg %= timeout + print(msg) return False status = get_pipeline_status(project_id, commit_sha) From patchwork Fri Sep 4 16:42:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758233 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19B2F14EB for ; Fri, 4 Sep 2020 16:46:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EABA32073B for ; Fri, 4 Sep 2020 16:46:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EABA32073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:55376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEErC-0002uy-5F for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:46:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41296) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEnx-0006gp-Mt for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:13 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:21461 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEnu-0000AT-Hj for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:13 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-183-kiPXpiAaOo6UKGwVRCqigg-1; Fri, 04 Sep 2020 12:43:08 -0400 X-MC-Unique: kiPXpiAaOo6UKGwVRCqigg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 76CD91074650; Fri, 4 Sep 2020 16:43:07 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5D86019C59; Fri, 4 Sep 2020 16:43:06 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 3/7] scripts/ci/gitlab-pipeline-status: give early feedback on running pipelines Date: Fri, 4 Sep 2020 12:42:54 -0400 Message-Id: <20200904164258.240278-4-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 03:57:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" When waiting for a pipeline to run and finish, it's better to give early feedback, and then sleep and wait, than the other wait around. Specially for the first iteration, it's frustrating to see nothing while the script is sleeping. Signed-off-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé --- scripts/ci/gitlab-pipeline-status | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index 2a36f74696..18609553be 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -77,8 +77,8 @@ def wait_on_pipeline_success(timeout, interval, status = get_pipeline_status(project_id, commit_sha) if status['status'] == 'running': - time.sleep(interval) print('running...') + time.sleep(interval) continue if status['status'] == 'success': From patchwork Fri Sep 4 16:42:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758209 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 798A3618 for ; Fri, 4 Sep 2020 16:43:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 55D052064E for ; Fri, 4 Sep 2020 16:43:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 55D052064E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47998 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEEob-00080w-Dj for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:43:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41310) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEnz-0006is-Ml for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:16 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:46523 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEnw-0000An-1W for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-9LTyus_2OFqPuv1_T9XX3Q-1; Fri, 04 Sep 2020 12:43:09 -0400 X-MC-Unique: 9LTyus_2OFqPuv1_T9XX3Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BB361425E1; Fri, 4 Sep 2020 16:43:08 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A23AB19D7D; Fri, 4 Sep 2020 16:43:07 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 4/7] scripts/ci/gitlab-pipeline-status: refactor parser creation Date: Fri, 4 Sep 2020 12:42:55 -0400 Message-Id: <20200904164258.240278-5-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=crosa@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 01:57:12 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Out of the main function. Signed-off-by: Cleber Rosa --- scripts/ci/gitlab-pipeline-status | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index 18609553be..8355b6a427 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -89,10 +89,7 @@ def wait_on_pipeline_success(timeout, interval, return False -def main(): - """ - Script entry point - """ +def create_parser(): parser = argparse.ArgumentParser( prog='pipeline-status', description='check or wait on a pipeline status') @@ -127,7 +124,13 @@ def main(): parser.add_argument('--verbose', action='store_true', default=False, help=('A minimal verbosity level that prints the ' 'overall result of the check/wait')) + return parser +def main(): + """ + Script entry point + """ + parser = create_parser() args = parser.parse_args() try: From patchwork Fri Sep 4 16:42:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758239 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6316E14EB for ; Fri, 4 Sep 2020 16:49:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 400972064E for ; Fri, 4 Sep 2020 16:49:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 400972064E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:33558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEEu5-0005lc-Ei for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:49:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41392) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEoA-0006xV-U1 for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:26 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:26377 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEo9-0000Dd-96 for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-487-Nq7w6g46MMeMHxK51sw-5w-1; Fri, 04 Sep 2020 12:43:11 -0400 X-MC-Unique: Nq7w6g46MMeMHxK51sw-5w-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 08C5D8015FA; Fri, 4 Sep 2020 16:43:10 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E4EFB19C59; Fri, 4 Sep 2020 16:43:08 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 5/7] scripts/ci/gitlab-pipeline-status: handle keyboard interrupts Date: Fri, 4 Sep 2020 12:42:56 -0400 Message-Id: <20200904164258.240278-6-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=crosa@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 03:58:24 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" So that exits based on user requests are handled more gracefully. Signed-off-by: Cleber Rosa --- scripts/ci/gitlab-pipeline-status | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index 8355b6a427..ced488f27c 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -132,7 +132,7 @@ def main(): """ parser = create_parser() args = parser.parse_args() - + success = False try: if args.wait: success = wait_on_pipeline_success( @@ -145,9 +145,11 @@ def main(): args.commit) success = status['status'] == 'success' except Exception as error: # pylint: disable=W0703 - success = False if args.verbose: print("ERROR: %s" % error.args[0]) + except KeyboardInterrupt: + if args.verbose: + print("Exiting on user's request") if success: if args.verbose: From patchwork Fri Sep 4 16:42:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90C4F91F for ; Fri, 4 Sep 2020 16:47:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6E9D52064E for ; Fri, 4 Sep 2020 16:47:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E9D52064E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57570 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEEs9-0003wN-DT for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:47:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41326) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEo1-0006jJ-6O for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:17 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:39996 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEnz-0000BN-2m for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:16 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-547-pZuFRBWLMYObZAG_DiMNcA-1; Fri, 04 Sep 2020 12:43:12 -0400 X-MC-Unique: pZuFRBWLMYObZAG_DiMNcA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4DBF81074662; Fri, 4 Sep 2020 16:43:11 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3312519C59; Fri, 4 Sep 2020 16:43:10 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 6/7] scripts/ci/gitlab-pipeline-status: use more descriptive exceptions Date: Fri, 4 Sep 2020 12:42:57 -0400 Message-Id: <20200904164258.240278-7-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=crosa@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 03:58:24 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" For two very different error conditions. Signed-off-by: Cleber Rosa --- scripts/ci/gitlab-pipeline-status | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index ced488f27c..628150ce0b 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -23,6 +23,14 @@ import time import sys +class CommunicationFailure(Exception): + """Failed to communicate to gitlab.com APIs.""" + + +class NoPipelineFound(Exception): + """Communication is successfull but pipeline is not found.""" + + def get_local_branch_commit(branch='staging'): """ Returns the commit sha1 for the *local* branch named "staging" @@ -50,14 +58,14 @@ def get_pipeline_status(project_id, commit_sha1): connection.request('GET', url=url) response = connection.getresponse() if response.code != http.HTTPStatus.OK: - raise ValueError("Failed to receive a successful response") + raise CommunicationFailure("Failed to receive a successful response") json_response = json.loads(response.read()) # As far as I can tell, there should be only one pipeline for the same # project + commit. If this assumption is false, we can add further # filters to the url, such as username, and order_by. if not json_response: - raise ValueError("No pipeline found") + raise NoPipelineFound("No pipeline found") return json_response[0] From patchwork Fri Sep 4 16:42:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cleber Rosa X-Patchwork-Id: 11758237 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8E7414EB for ; Fri, 4 Sep 2020 16:48:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB0B12064E for ; Fri, 4 Sep 2020 16:48:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iJaT1YQ1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB0B12064E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:59736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kEEtI-0004tS-S6 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 04 Sep 2020 12:48:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41328) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kEEo1-0006jp-I0 for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:19 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:35825 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kEEnz-0000BZ-Rr for qemu-devel@nongnu.org; Fri, 04 Sep 2020 12:43:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599237795; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rHHf3/LDAtbo5xhGvAYZ98cUty3t4ZOTgFXkNR2xX2g=; b=iJaT1YQ1LS/Wc4cSuKEVAeZrXpbXcM7f5KfOevR+KwA3d9zuitqVj881wDani+hxPqgIfM +8qJepdXi+Hwm2LktQZx2pmzhuv8zvPehNcCdIMowa81p3kqPfL6vmVO9i4N/JPcfNohO4 QS5wCgQ9l190KGobyPyIrISc59NQJMU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-92-aIOyD_UsNhS50HbIP2RRaQ-1; Fri, 04 Sep 2020 12:43:13 -0400 X-MC-Unique: aIOyD_UsNhS50HbIP2RRaQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 942B81074650; Fri, 4 Sep 2020 16:43:12 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-166.rdu2.redhat.com [10.10.120.166]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7A4F319C59; Fri, 4 Sep 2020 16:43:11 +0000 (UTC) From: Cleber Rosa To: qemu-devel@nongnu.org, Peter Maydell Subject: [PATCH 7/7] scripts/ci/gitlab-pipeline-status: wait for pipeline creation Date: Fri, 4 Sep 2020 12:42:58 -0400 Message-Id: <20200904164258.240278-8-crosa@redhat.com> In-Reply-To: <20200904164258.240278-1-crosa@redhat.com> References: <20200904164258.240278-1-crosa@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=crosa@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 01:57:12 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Beraldo Leal , Wainer dos Santos Moschetta , Willian Rampazzo , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Eduardo Habkost Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" When called in wait mode, this script will also wait for the pipeline to be get to a "running" state. Because many more status may be seen until a pipeline gets to "running", and those need to be handle too. Reference: https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines Signed-off-by: Cleber Rosa --- scripts/ci/gitlab-pipeline-status | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/scripts/ci/gitlab-pipeline-status b/scripts/ci/gitlab-pipeline-status index 628150ce0b..bac8233079 100755 --- a/scripts/ci/gitlab-pipeline-status +++ b/scripts/ci/gitlab-pipeline-status @@ -83,13 +83,22 @@ def wait_on_pipeline_success(timeout, interval, print(msg) return False - status = get_pipeline_status(project_id, commit_sha) - if status['status'] == 'running': - print('running...') + try: + status = get_pipeline_status(project_id, commit_sha) + except NoPipelineFound: + print('Pipeline has not been found, it may not have been created yet.') + time.sleep(1) + continue + + pipeline_status = status['status'] + status_to_wait = ('created', 'waiting_for_resource', 'preparing', + 'pending', 'running') + if pipeline_status in status_to_wait: + print('%s...' % pipeline_status) time.sleep(interval) continue - if status['status'] == 'success': + if pipeline_status == 'success': return True msg = "Pipeline failed, check: %s" % status['web_url']