From patchwork Fri May 3 16:59:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78D07912 for ; Fri, 3 May 2019 17:01:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B06E2874E for ; Fri, 3 May 2019 17:01:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 692FD2876D; Fri, 3 May 2019 17:01:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 203432874E for ; Fri, 3 May 2019 17:01:42 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXm-00089F-06; Fri, 03 May 2019 17:00:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXk-00086n-4c for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:16 +0000 X-Inumbo-ID: e414a79c-6dc4-11e9-b966-872648a61e13 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e414a79c-6dc4-11e9-b966-872648a61e13; Fri, 03 May 2019 17:00:02 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085329" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:47 +0100 Message-ID: <20190503165957.5960-2-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 01/11] mg-execute-flight: Save an mro in tmp/ X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This may be useful for some things. For example, it will be used in just a moment. Signed-off-by: Ian Jackson --- mg-execute-flight | 1 + 1 file changed, 1 insertion(+) diff --git a/mg-execute-flight b/mg-execute-flight index b3cdf431..391f4810 100755 --- a/mg-execute-flight +++ b/mg-execute-flight @@ -81,6 +81,7 @@ echo >&5 "flight $flight reporting" ./sg-report-flight $basisflightopt \ --html-dir=$OSSTEST_HTMLPUB_DIR/$flight/ \ + --machine-readable-output=tmp/$flight.mro \ $flight >tmp/$flight.report subject=`sed -n 1p X-Patchwork-Id: 10929031 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0CDE1390 for ; Fri, 3 May 2019 17:01:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B199A28763 for ; Fri, 3 May 2019 17:01:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF9DF28783; Fri, 3 May 2019 17:01:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5BAF828763 for ; Fri, 3 May 2019 17:01:53 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXZ-0007uW-94; Fri, 03 May 2019 17:00:05 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXY-0007qM-JC for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:04 +0000 X-Inumbo-ID: e4b2618c-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e4b2618c-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:03 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085330" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:48 +0100 Message-ID: <20190503165957.5960-3-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 02/11] cs-adjust-flight: Break out copy_jobs_* X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change. Signed-off-by: Ian Jackson --- cs-adjust-flight | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/cs-adjust-flight b/cs-adjust-flight index ee1d917c..badabeff 100755 --- a/cs-adjust-flight +++ b/cs-adjust-flight @@ -194,10 +194,8 @@ sub do_rm_job ($) { verbose "$dstflight.$job job deleted\n" if $count; } -sub copy_jobs ($$) { - my ($srcflight, $jobs) = @_; - prep_rm_jobs(); - my @job_copy_qs = map { $dbh_tests->prepare($_) } +sub copy_jobs_qs () { + map { $dbh_tests->prepare($_) } ("INSERT INTO jobs (flight, job, recipe, status)". " SELECT ?, job, recipe, 'queued'". " FROM jobs". @@ -206,12 +204,22 @@ sub copy_jobs ($$) { " SELECT ?, job, name, val, 'f'". " FROM runvars". " WHERE flight = ? AND job = ? AND NOT synth"); +} +sub copy_jobs_do ($$) { + my ($qs, $srcflight, $srcjob) = @_; + $_->execute($dstflight, $srcflight, $job) foreach @$qs; + verbose "$dstflight.$job job created from $srcflight.$job\n"; +} + +sub copy_jobs ($$) { + my ($srcflight, $jobs) = @_; + prep_rm_jobs(); + my @job_copy_qs = copy_jobs_qs; for_jobs($dstflight, $jobs, \&do_rm_job, 1); for_jobs($srcflight, $jobs, \&do_rm_job); for_jobs($srcflight, $jobs, sub { my ($job) = @_; - $_->execute($dstflight, $srcflight, $job) foreach @job_copy_qs; - verbose "$dstflight.$job job created from $srcflight.$job\n"; + copy_jobs_do(\@job_copy_qs, $srcflight, $job); }); } From patchwork Fri May 3 16:59:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929021 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 046871390 for ; Fri, 3 May 2019 17:01:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB2C92868D for ; Fri, 3 May 2019 17:01:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF7DC28735; Fri, 3 May 2019 17:01:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 78DC42855A for ; Fri, 3 May 2019 17:01:38 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXb-000815-HY; Fri, 03 May 2019 17:00:07 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXZ-0007xe-St for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:05 +0000 X-Inumbo-ID: e578f492-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e578f492-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:04 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085331" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:49 +0100 Message-ID: <20190503165957.5960-4-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 03/11] cs-adjust-flight: Use db_prepare and honour -D -D for sql dumps X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This makes debugging it easier. No functional change with zero or one occurrences of -D. Signed-off-by: Ian Jackson --- cs-adjust-flight | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/cs-adjust-flight b/cs-adjust-flight index badabeff..cc1684b4 100755 --- a/cs-adjust-flight +++ b/cs-adjust-flight @@ -75,6 +75,7 @@ csreadconfig(); our $debug = 0; open VERBOSE, ">/dev/null" or die $!; +open DEBUG, ">/dev/null" or die $!; our $verbose_buffer; while (@ARGV && $ARGV[0] =~ m/^\-/) { @@ -83,6 +84,7 @@ while (@ARGV && $ARGV[0] =~ m/^\-/) { if (m/^-v$/) { open VERBOSE, ">&STDERR" or die $!; } elsif (m/^-D$/) { + open DEBUG, ">&STDERR" if $debug; $debug = 1; } else { die "$_ ?"; @@ -138,9 +140,9 @@ sub for_things ($$$$$$$) { my ($spec, $fn, $ifnone, $table, $keycol, $basecond, $basecondvals) = @_; - my $thing_q = $dbh_tests->prepare + my $thing_q = db_prepare ("SELECT * FROM $table WHERE $basecond AND $keycol = ?"); - my $things_q = $dbh_tests->prepare + my $things_q = db_prepare ("SELECT * FROM $table WHERE $basecond"); my ($specfn,$exact) = notspec_exfn("FOR_THINGS $table.$keycol",$spec); @@ -182,7 +184,7 @@ our @job_rm_qs; sub prep_rm_jobs () { return if @job_rm_qs; - @job_rm_qs = map { $dbh_tests->prepare($_) } + @job_rm_qs = map { db_prepare($_) } ("DELETE FROM runvars WHERE flight = ? AND job = ?", "DELETE FROM jobs WHERE flight = ? AND job = ?"); } @@ -195,7 +197,7 @@ sub do_rm_job ($) { } sub copy_jobs_qs () { - map { $dbh_tests->prepare($_) } + map { db_prepare($_) } ("INSERT INTO jobs (flight, job, recipe, status)". " SELECT ?, job, recipe, 'queued'". " FROM jobs". @@ -205,8 +207,8 @@ sub copy_jobs_qs () { " FROM runvars". " WHERE flight = ? AND job = ? AND NOT synth"); } -sub copy_jobs_do ($$) { - my ($qs, $srcflight, $srcjob) = @_; +sub copy_jobs_do ($$$) { + my ($qs, $srcflight, $job) = @_; $_->execute($dstflight, $srcflight, $job) foreach @$qs; verbose "$dstflight.$job job created from $srcflight.$job\n"; } @@ -253,9 +255,9 @@ sub change__copy_jobs { copy_jobs($srcflight, $jobs); } -our $runvar_rm_q = $dbh_tests->prepare +our $runvar_rm_q = db_prepare ("DELETE FROM runvars WHERE flight = ? AND job = ? AND name = ?"); -our $runvar_insert_q = $dbh_tests->prepare +our $runvar_insert_q = db_prepare ("INSERT INTO runvars (flight, job, name, val, synth)". " VALUES (?, ?, ?, ?, 'f')"); @@ -430,7 +432,7 @@ sub change__job_status { die unless @changes >= 1; my $jobs = shift @changes; - my $q = $dbh_tests->prepare(<prepare(<prepare(<prepare("SELECT * FROM steps WHERE flight = ?"); + db_prepare("SELECT * FROM steps WHERE flight = ?"); $steps_q->execute($dstflight); my $step = $steps_q->fetchrow_hashref(); if ($step) { From patchwork Fri May 3 16:59:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929029 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD6061390 for ; Fri, 3 May 2019 17:01:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BDA4028741 for ; Fri, 3 May 2019 17:01:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BBF952874C; Fri, 3 May 2019 17:01:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6C8372874F for ; Fri, 3 May 2019 17:01:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXc-00081o-SZ; Fri, 03 May 2019 17:00:08 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXb-00080y-0X for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:07 +0000 X-Inumbo-ID: e67ad4ac-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e67ad4ac-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:06 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085334" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:50 +0100 Message-ID: <20190503165957.5960-5-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 04/11] mg-repro-setup: Improve a doc message slightly X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Ian Jackson --- mg-repro-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mg-repro-setup b/mg-repro-setup index 6ed4d85e..c8bcad33 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -33,7 +33,7 @@ usage () { cat < is \`host') OPTIONs - -t estimated duration (default = 28d) + -t estimated overall allocation lifetime (default = 28d) --rogue bypass queuing system and allocate now --capture do not suppress log capture -r= set runvar From patchwork Fri May 3 16:59:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0630912 for ; Fri, 3 May 2019 17:02:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A37632878E for ; Fri, 3 May 2019 17:02:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A149D2875F; Fri, 3 May 2019 17:02:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 584DE285A1 for ; Fri, 3 May 2019 17:02:07 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXl-00088R-JZ; Fri, 03 May 2019 17:00:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXk-00086o-4i for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:16 +0000 X-Inumbo-ID: e58115a2-6dc4-11e9-9293-077e3ee12d0c Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id e58115a2-6dc4-11e9-9293-077e3ee12d0c; Fri, 03 May 2019 17:00:06 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085336" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:51 +0100 Message-ID: <20190503165957.5960-6-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 05/11] mg-repro-setup: Break out compute_adjusts X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change. Signed-off-by: Ian Jackson --- mg-repro-setup | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/mg-repro-setup b/mg-repro-setup index c8bcad33..10ab65a8 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -54,7 +54,6 @@ set -e -o posix mgexecflags=() adjustsets=() -adjusts=() allocs=() logfile=tmp/mg-repro-setup.log duration=28d @@ -104,14 +103,17 @@ adjrunvar () { delrunvar "$1" adjusts+=(runvar-set "$job" "$1" "$2") } - -for arg in "${adjustsets[@]}"; do - case "$arg" in - !*|^*) delrunvar "${arg#?}" ;; - *=*) adjrunvar "${arg%%=*}" "${arg#*=}" ;; - *) bad-adjuistset-pattern ;; - esac -done +compute_adjusts () { + adjusts=() + for arg in "$@"; do + case "$arg" in + !*|^*) delrunvar "${arg#?}" ;; + *=*) adjrunvar "${arg%%=*}" "${arg#*=}" ;; + *) bad-adjuistset-pattern ;; + esac + done +} +compute_adjusts "${adjustsets[@]}" while [ $# -ne 0 ]; do arg=$1; shift From patchwork Fri May 3 16:59:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929017 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1823912 for ; Fri, 3 May 2019 17:01:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B253B28716 for ; Fri, 3 May 2019 17:01:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6ABC28704; Fri, 3 May 2019 17:01:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 287DD286E2 for ; Fri, 3 May 2019 17:01:35 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXd-00081w-6W; Fri, 03 May 2019 17:00:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXc-00081Z-8q for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:08 +0000 X-Inumbo-ID: e6fdcf77-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e6fdcf77-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:06 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085338" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:52 +0100 Message-ID: <20190503165957.5960-7-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 06/11] mg-repro-setup: Move logging setup to later X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change. Signed-off-by: Ian Jackson --- mg-repro-setup | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mg-repro-setup b/mg-repro-setup index 10ab65a8..b41bf478 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -115,6 +115,13 @@ compute_adjusts () { } compute_adjusts "${adjustsets[@]}" +progressf () { printf >&2 "$@"; } +progress () { progressf "%s\n" "$1"; } + +progress "logging to $logfile" +savelog "$logfile" +exec 3>"$logfile" + while [ $# -ne 0 ]; do arg=$1; shift @@ -160,13 +167,6 @@ while [ $# -ne 0 ]; do esac done -progressf () { printf >&2 "$@"; } -progress () { progressf "%s\n" "$1"; } - -progress "logging to $logfile" -savelog "$logfile" -exec 3>"$logfile" - OSSTEST_TASK=$(perl -e ' use Osstest; use Osstest::Executive; From patchwork Fri May 3 16:59:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE7A014DB for ; Fri, 3 May 2019 17:01:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0188285A1 for ; Fri, 3 May 2019 17:01:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9E44728741; Fri, 3 May 2019 17:01:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9045D283C5 for ; Fri, 3 May 2019 17:01:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXd-000826-GV; Fri, 03 May 2019 17:00:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXc-00081a-Ab for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:08 +0000 X-Inumbo-ID: e766dca4-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e766dca4-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:07 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085340" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:53 +0100 Message-ID: <20190503165957.5960-8-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 07/11] mg-repro-setup: Allow arguments to badusage X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change with existing call sites. Signed-off-by: Ian Jackson --- mg-repro-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mg-repro-setup b/mg-repro-setup index b41bf478..d63e29b6 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -46,7 +46,7 @@ END } -badusage () { echo >&2 "bad usage"; usage >&2; exit 126; } +badusage () { echo >&2 "bad usage$*"; usage >&2; exit 126; } set -e -o posix From patchwork Fri May 3 16:59:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929019 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 154471390 for ; Fri, 3 May 2019 17:01:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 082C52852E for ; Fri, 3 May 2019 17:01:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F0C9328717; Fri, 3 May 2019 17:01:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A587828716 for ; Fri, 3 May 2019 17:01:36 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXl-00087x-86; Fri, 03 May 2019 17:00:17 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXk-00086q-3H for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:16 +0000 X-Inumbo-ID: eb861d4b-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id eb861d4b-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:14 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085374" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:54 +0100 Message-ID: <20190503165957.5960-9-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 08/11] mg-repro-setup: Move flight creation up before task creation X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No significant functional change. Signed-off-by: Ian Jackson --- mg-repro-setup | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mg-repro-setup b/mg-repro-setup index d63e29b6..2e1d3b88 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -167,6 +167,9 @@ while [ $# -ne 0 ]; do esac done +flight=$(./cs-adjust-flight new:$blessing) +progress "new flight is $flight" + OSSTEST_TASK=$(perl -e ' use Osstest; use Osstest::Executive; @@ -176,9 +179,6 @@ OSSTEST_TASK=$(perl -e ' ') export OSSTEST_TASK -flight=$(./cs-adjust-flight new:$blessing) -progress "new flight is $flight" - if [ "${alloc_idents[*]}" ]; then progress "allocating ${alloc_idents[*]} ..." alloc_output=tmp/$flight.allocations From patchwork Fri May 3 16:59:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 433E7912 for ; Fri, 3 May 2019 17:01:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32FE528704 for ; Fri, 3 May 2019 17:01:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 314092868D; Fri, 3 May 2019 17:01:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8040228704 for ; Fri, 3 May 2019 17:01:43 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXf-00083Y-QP; Fri, 03 May 2019 17:00:11 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXe-000839-NS for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:10 +0000 X-Inumbo-ID: e81792a7-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e81792a7-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:09 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085342" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:55 +0100 Message-ID: <20190503165957.5960-10-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 09/11] mg-repro-flight: Provide --rebuild to make variant build jobs too X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This allows a single command to repro a particular job with a variety of different source code. The implementation technique is: - run the build job in a separate flight, so that it can run with a separate task which gives its host up after the build - do much of the heavy lifting of runvar fiddling etc. in a new helper routine in cs-adjust-flight Signed-off-by: Ian Jackson --- v2: Add a missing `continue' (without which everything goes quite wrong) --- cs-adjust-flight | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ mg-repro-setup | 85 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 193 insertions(+), 1 deletion(-) diff --git a/cs-adjust-flight b/cs-adjust-flight index cc1684b4..5be5af38 100755 --- a/cs-adjust-flight +++ b/cs-adjust-flight @@ -469,6 +469,115 @@ END $runvarq->execute($newjob, $dstflight, $oldjob); }) } + +sub change__repro_buildjobs { + # helper for mg-repro-setup + # arguments to this change are + # EXAMPLE-FLIGHT EXAMPLE-JOB THING... + # where THING is + # .BUILDJOB =BUILDJOBRUNVAR +TREE=URL#REVISION + # (we eat all until it doesn't start with . = +) + # stdout output is + # list of runvar settings for repro job + die unless @changes >= 2; + my $eflight = shift @changes; + my $ejob = shift @changes; + my @refspecs; + my @varspecs; + my %treespecs; + while (@changes && $changes[0] =~ m/^[.=+]/) { + local $_ = shift @changes; + if (m/^[.=]/) { + push @refspecs, $_; + } elsif (m/^\+(.*?)\=(.*)\#(.*)$/) { + die $1 if $treespecs{$1}; + $treespecs{$1} = [$2,$3]; + } elsif (m/^-r/) { + push @varspecs, $_; + } else { + die "bad .BUILDJOB / =BUILDJOBRUNVAR / +TREE=URL#REVISION): $_\n"; + } + } + my $testq = db_prepare(<execute($eflight, $ejob); + while (my ($refvar, $bjobref) = $testq->fetchrow_array()) { + my ($bflight, $bjob) = flight_otherjob($eflight,$bjobref); + if (@refspecs) { + my $y = 0; + foreach my $rs (@refspecs) { + if ($rs eq ".$bjob" || $rs eq "=$refvar") { + $y = 1; + last; + } + } + next unless $y; + } + $buildq->execute($bflight,$bjob,@buildq_xvars); + my %got; + while (my ($treevar) = $buildq->fetchrow_array()) { + $treevar =~ m/^tree_/ or die "$treevar ?"; + $got{$'} = 1; + next unless $treespecs{$'}; + $treespecs{$'}[2]++; + } + next unless %got || @refspecs; + + $bjobs{$bjob} //= { Template => $bflight }; + $bjobs{$bjob}{Template} eq $bflight or + die "inconsistent $bjob: $bjobs{$bjob}{Template} != $bflight"; + push @{ $bjobs{$bjob}{Refs} }, $refvar; + $bjobs{$bjob}{Trees}{$_} = 1 foreach keys %got; + } + + foreach my $tree (sort keys %treespecs) { + die "unused tree/revision adjustment $tree" + .(@refspecs + ? " (no tree_$tree var in any of " + .(join ' ', map { "$bjobs{$_}{Template}.$_" } + sort keys %bjobs).")" + : '') + unless $treespecs{$tree}[2]; + } + + my @copy_jobs_qs = copy_jobs_qs(); + foreach my $bjob (sort keys %bjobs) { + copy_jobs_do(\@copy_jobs_qs, $bjobs{$bjob}{Template}, $bjob); + foreach my $tree (sort keys %{ $bjobs{$bjob}{Trees} }) { + runvar_set($bjob, "tree_$tree", $treespecs{$tree}[0]); + runvar_set($bjob, "revision_$tree", $treespecs{$tree}[1]); + } + foreach (@varspecs) { + if (m/^-r!|^-r^/) { + $runvar_rm_q->execute($dstflight, $bjob, $'); + } elsif (m/^-r(.*?)=/) { + runvar_set($bjob, $1, $'); + } else { + die "bad -r $_ ?"; + } + } + foreach my $refvar (@{ $bjobs{$bjob}{Refs} }) { + die "$bjob $refvar $& ?" if + "$bjob.$refvar" =~ m{[^+=_./,:0-9a-z-]}i; + print "runvar-set . $refvar $dstflight.$bjob\n" or die $!; + } + } +} + sub changes () { debug("CHANGES...\n"); diff --git a/mg-repro-setup b/mg-repro-setup index 2e1d3b88..7f075f4e 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -19,7 +19,7 @@ usage () { cat <] [-r...] + [.BUILDJOB | =BUILDJOBRUNVAR...] + +TREE=URL#REVISION... + Also use a different build. Specifically, use URL and REVISION + for certain builds which mention TREE and which are referenced + (directly) in the job JOB in EXAMPLE-FLIGHT. The affected builds + are those referenced by any of the runvars BUILDJOBRUNVAR (in + which case only that job reference is edited) or any whose job + name is any of the BUILDJOB (in which case all references to that + job are adjusted). If no .BUILDJOB and no =BUILDJOBRUNVAR are + specified, then all jobs referred to from the example JOB + which mention TREE are affected. + + For example, + --rebuild =xenbuildjob \ + +xen=git://xenbits.xen.org/people/alice/xen.git#fixes + would look for xenbuild and (say) find that it referred to + build-amd64; it would then copy and use the build-amd64 job + that EXAMPLE-FLIGHT.JOB used, adjusting \`tree_xen' and + \`revision_xen' as specified, and use that for \`xenbuildjob' in + the repro (but not for \`buildjob') + + Whereas + --rebuild .build-amd64 \ + +linux=git://xenbits.xen.org/people/alice/linux.git#fixes + would replace all references to any job named build-amd64 + with a new build-amd64 job. + + And + --rebuild \ + +xen=git://xenbits.xen.org/people/alice/linux.git#fixes + would replace all builds mentioning xen, including for example + build-amd64 (for xenbuildjob) and build-i386 (for buildjob) + in an x86 32-on-64 flight. + + Host allocation for the build is done "normally" (ie, the host + is thrown away after the build is complete), with a default + blessing of \`adhoc'. + + --rebuild is affected by -E or -P but not general -B or -f. + END } @@ -122,10 +165,38 @@ progress "logging to $logfile" savelog "$logfile" exec 3>"$logfile" +rebuilds_flight='' + while [ $# -ne 0 ]; do arg=$1; shift case "$arg" in + + --rebuild) + rebuilds_blessing=adhoc + rebuild_specs=() + while true; do + case "$1" in + [.=+]*|-r) rebuild_specs+=("$1"); shift ;; + -B?*) rebuilds_blessing=${1#-B}; shift ;; + -*) badusage ': bad --rebuild option' ;; + *) break ;; + esac + done + if [ x$rebuilds_flight = x ]; then + rebuilds_flight=$( + ./cs-adjust-flight new:$rebuilds_blessing + ) + fi + + adjusts+=($( + ./cs-adjust-flight $rebuilds_flight \ + repro-buildjobs $example_flight $job \ + "${rebuild_specs[@]}" + )) + continue + ;; + none:) # provided so we can repro a job with no hosts ;; @@ -170,6 +241,18 @@ done flight=$(./cs-adjust-flight new:$blessing) progress "new flight is $flight" +if [ "x$rebuilds_flight" != x ]; then + progress "running build(s) $rebuilds_flight" + ./mg-execute-flight -B$rebuilds_blessing --progress-fd=2 \ + "${mgexecflags[@]}" $rebuilds_flight + + mro=tmp/$rebuilds_flight.mro + if ! egrep '^perfect' >/dev/null $mro; then + echo >&2 "build(s) failed (no 'perfect' in $mro)" + exit 1 + fi +fi + OSSTEST_TASK=$(perl -e ' use Osstest; use Osstest::Executive; From patchwork Fri May 3 16:59:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DFDD912 for ; Fri, 3 May 2019 17:02:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DEA5285CB for ; Fri, 3 May 2019 17:02:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4C22128741; Fri, 3 May 2019 17:02:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F05E428758 for ; Fri, 3 May 2019 17:02:00 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXh-00084L-2q; Fri, 03 May 2019 17:00:13 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXg-00083c-05 for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:12 +0000 X-Inumbo-ID: e977b22b-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e977b22b-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:11 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085345" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:56 +0100 Message-ID: <20190503165957.5960-11-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 10/11] mg-repro-setup: Detect options (-...) in wrong place X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Without this they tend to be interpreted as HOSTSPECs leading to lossage. Signed-off-by: Ian Jackson --- v2: New patch --- mg-repro-setup | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mg-repro-setup b/mg-repro-setup index 7f075f4e..dc6c5cbb 100755 --- a/mg-repro-setup +++ b/mg-repro-setup @@ -197,6 +197,10 @@ while [ $# -ne 0 ]; do continue ;; + -*) + badusage ': misplaced option, expected REBUILD or HOSTSPEC' + ;; + none:) # provided so we can repro a job with no hosts ;; From patchwork Fri May 3 16:59:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10929023 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37CF1912 for ; Fri, 3 May 2019 17:01:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2864C2873C for ; Fri, 3 May 2019 17:01:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C9D12869B; Fri, 3 May 2019 17:01:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9DE8D28717 for ; Fri, 3 May 2019 17:01:38 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXi-000858-Ca; Fri, 03 May 2019 17:00:14 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hMbXh-00084T-Aa for xen-devel@lists.xenproject.org; Fri, 03 May 2019 17:00:13 +0000 X-Inumbo-ID: e9f68e9c-6dc4-11e9-843c-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id e9f68e9c-6dc4-11e9-843c-bc764e045a96; Fri, 03 May 2019 17:00:11 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,426,1549929600"; d="scan'208";a="85085350" From: Ian Jackson To: Date: Fri, 3 May 2019 17:59:57 +0100 Message-ID: <20190503165957.5960-12-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190503165957.5960-1-ian.jackson@eu.citrix.com> References: <20190503165957.5960-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 11/11] mg-allocate: Don't try allocating the same resource for two specs X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is obviously wrong. In fact it does not work (we bomb out in the allocation). Signed-off-by: Ian Jackson --- v2: New patch --- mg-allocate | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mg-allocate b/mg-allocate index 087b14b0..b5dc185e 100755 --- a/mg-allocate +++ b/mg-allocate @@ -437,6 +437,16 @@ sub plan () { map { [ @$possreqs, $_ ] } @reqlist; } @possmatrix; } + @possmatrix = grep { + my $possreqs = $_; + my %got; + my $ok=1; + foreach (@$possreqs) { + next if !$got{$_->{Reso}}++; + $ok=0; last; + } + $ok; + } @possmatrix; my $planned; my @reqlist;