From patchwork Thu Apr 18 16:31:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907711 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6AA7922 for ; Thu, 18 Apr 2019 16:33:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F71328D55 for ; Thu, 18 Apr 2019 16:33:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93F8F28D58; Thu, 18 Apr 2019 16:33:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 481E928D55 for ; Thu, 18 Apr 2019 16:33:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xN-00026J-Vt; Thu, 18 Apr 2019 16:32:13 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xM-00023B-8j for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:12 +0000 X-Inumbo-ID: 83e60480-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 83e60480-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:11 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913785" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:38 +0100 Message-ID: <20190418163158.11408-2-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 01/21] ts-hosts-allocate-Executive: with -U, just append to the same logfile X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is much more convenient for ad-hoc use. Signed-off-by: Ian Jackson --- ts-hosts-allocate-Executive | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive index dcfc70f2..3da138b1 100755 --- a/ts-hosts-allocate-Executive +++ b/ts-hosts-allocate-Executive @@ -75,16 +75,18 @@ sub setup () { $taskid= findtask(); - my $logbase = "hosts-allocate.debug".($compressdebug?".gz":""); - my $logfh = open_unique_stashfile \$logbase; + my $logbase = "hosts-allocate.debug"; if ($compressdebug) { + $logbase .= ".gz"; + my $logfh = open_unique_stashfile \$logbase; my $logchild = open DEBUG, "|-"; defined $logchild or die $!; if (!$logchild) { open STDOUT, ">&", $logfh or die $!; exec "gzip" or die $!; } } else { - open DEBUG, ">&", $logfh or die $!; + my $logfh = "$stash/$logbase"; + open DEBUG, ">>", $logfh or die $!; } DEBUG->autoflush(1); logm("host allocation debug log in $logbase"); From patchwork Thu Apr 18 16:31:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907707 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 124F513B5 for ; Thu, 18 Apr 2019 16:33:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F07D328D55 for ; Thu, 18 Apr 2019 16:33:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E4EFE28D59; Thu, 18 Apr 2019 16:33:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A1DC228D55 for ; Thu, 18 Apr 2019 16:33:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xH-0001zF-Kg; Thu, 18 Apr 2019 16:32:07 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xG-0001z1-HH for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:06 +0000 X-Inumbo-ID: 802a5459-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 802a5459-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:05 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913760" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:39 +0100 Message-ID: <20190418163158.11408-3-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 02/21] selecthost: Honour new $none_ok optional parameter X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change with existing callers. Signed-off-by: Ian Jackson --- Osstest/TestSupport.pm | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index ec867e4f..38c17d60 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -1125,9 +1125,9 @@ sub power_state ($$;$) { #---------- host selection and properties ---------- -sub selecthost ($); -sub selecthost ($) { - my ($ident) = @_; +sub selecthost ($;$); +sub selecthost ($;$) { + my ($ident, $none_ok) = @_; # must be run outside transaction # $ident is @@ -1154,7 +1154,10 @@ sub selecthost ($) { $r{$ident}= $name; } else { $name= $r{$ident}; - die "no specified $ident" unless defined $name; + if (!defined $name) { + return undef if $none_ok; + die "no specified $ident"; + } } my $ho= { From patchwork Thu Apr 18 16:31:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 76AB313B5 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5FA0528D55 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5295E28D5A; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4DB5B28D55 for ; Thu, 18 Apr 2019 16:33:45 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xI-0001zf-Ua; Thu, 18 Apr 2019 16:32:08 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xG-0001z3-MF for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:06 +0000 X-Inumbo-ID: 80c5613d-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 80c5613d-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:05 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913761" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:40 +0100 Message-ID: <20190418163158.11408-4-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 03/21] ts-logs-capture: Do not try to capture logs of hosts not allocated X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Little significant change with existing non-broken flights etc. Signed-off-by: Ian Jackson --- ts-logs-capture | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/ts-logs-capture b/ts-logs-capture index a429bb76..83234f6f 100755 --- a/ts-logs-capture +++ b/ts-logs-capture @@ -32,7 +32,12 @@ if (!$mjobdb->jobdb_enable_log_capture) { our ($whhost) = @ARGV; $whhost ||= 'host'; -our $ho= selecthost($whhost); + +our $ho= selecthost($whhost, 1); +if (!$ho) { + logm("host $whhost not specified, not capturing logs"); + exit 0; +} our @guests; From patchwork Thu Apr 18 16:31:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A4F3922 for ; Thu, 18 Apr 2019 16:33:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B57D28D56 for ; Thu, 18 Apr 2019 16:33:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0FA4428D5B; Thu, 18 Apr 2019 16:33:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8193928D56 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xU-0002IZ-Kc; Thu, 18 Apr 2019 16:32:20 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xR-0002CN-JH for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:17 +0000 X-Inumbo-ID: 86bfa75d-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 86bfa75d-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:15 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913805" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:41 +0100 Message-ID: <20190418163158.11408-5-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 04/21] alloc_resources: Support special abandonment values X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This gives a way for the caller's $resourcecall to signal something interesting, back to its main loop. This is useful for calling broken, for example: that can't be done within $resourcecall because $resourcecall operates within the allocation db transaction (which ought to be rolled back...) Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index 0d8502b5..b27b8d25 100644 --- a/Osstest/Executive.pm +++ b/Osstest/Executive.pm @@ -737,7 +737,8 @@ sub alloc_resources { # # values of $ok # 0 rollback, wait and try again - # 1 commit, completed ok + # 1 commit, completed ok } return the + # >1 rollback and abandon } value of $ok # $bookinglist should be undef or a hash for making a booking # # $resourcecall should not look at tasks.live @@ -859,10 +860,14 @@ sub alloc_resources { # $resourcecall itself failed: bomb out (don't retry) $ok=-1; } - return db_retry_abort() unless $ok>0; + return db_retry_abort() unless $ok==1; }]); - if ($bookinglist && $ok!=-1) { + if ($ok<0 || $ok>1) { + return 1; + } + + if ($bookinglist) { my %prstart; foreach my $book (@{ $bookinglist->{Bookings} }) { my $pr = $book->{Reso}; @@ -890,8 +895,6 @@ sub alloc_resources { if ($ok==1) { print $qserv "thought-done\n" or die $!; - } elsif ($ok<0) { - return 1; } else { # 0 logm("resource allocation: deferring"); print $qserv "thought-wait\n" or die $!; @@ -935,8 +938,12 @@ END $ok= 0; } } - die unless $ok==1; - logm("resource allocation: successful."); + if ($ok==1) { + logm("resource allocation: successful."); + } else { + logm("resource allocation: resourcecall returned ($ok,...)"); + } + return $ok; } sub resource_check_allocated ($$) { From patchwork Thu Apr 18 16:31:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907701 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CDFD613B5 for ; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5A1628D55 for ; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8FDA28D58; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 666F928D56 for ; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xM-000239-5J; Thu, 18 Apr 2019 16:32:12 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xK-00021J-Bn for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:10 +0000 X-Inumbo-ID: 80d03be8-61f7-11e9-85af-93c07e66e988 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 80d03be8-61f7-11e9-85af-93c07e66e988; Thu, 18 Apr 2019 16:32:06 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913762" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:42 +0100 Message-ID: <20190418163158.11408-6-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 05/21] starvation: Teach sg-report-flight about starved step state X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP We are going to use this for situations where the resources to run the test weren't available. In general we are going to treat this as not a regression. Signed-off-by: Ian Jackson --- sg-report-flight | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sg-report-flight b/sg-report-flight index a1adf952..2677c479 100755 --- a/sg-report-flight +++ b/sg-report-flight @@ -805,9 +805,9 @@ END next; } - if ($st eq 'blocked') { - print MRO "blocked $s->{job} $s->{testid}\n"; - print DEBUG " blocked, call it justifiable\n"; + if ($st eq 'blocked' or $st eq 'starved') { + print MRO "$st $s->{job} $s->{testid}\n"; + print DEBUG " $st, call it justifiable\n"; $failv->{Summary}= 'n/a'; $failv->{Blocker}= ''; next; From patchwork Thu Apr 18 16:31:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907709 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C52213B5 for ; Thu, 18 Apr 2019 16:33:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 543E528D55 for ; Thu, 18 Apr 2019 16:33:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 488D328D58; Thu, 18 Apr 2019 16:33:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E830728D55 for ; Thu, 18 Apr 2019 16:33:53 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xP-000287-FH; Thu, 18 Apr 2019 16:32:15 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xN-00025i-O2 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:13 +0000 X-Inumbo-ID: 83ee2663-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 83ee2663-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:11 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913792" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:43 +0100 Message-ID: <20190418163158.11408-7-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 06/21] starvation: Teach archaeologists about starved job state X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP sg-report-flight is a bit awkward. It thinks mostly about step status, not job status. So, when justifying, if we don't find a step, and the job state is starved, we treat the step as starved. If there are only starved steps, then we don't have evidence that this is a regression, because the test wasn't run in the baseline. If there are other steps we look at those instead. Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 4 ++-- sg-report-flight | 11 +++++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index b27b8d25..a9f9ac78 100644 --- a/Osstest/Executive.pm +++ b/Osstest/Executive.pm @@ -362,8 +362,8 @@ END Content => (join " | ", @content), ColourAttr => "bgcolor=\"$failcolour\"", }; - } elsif ($status eq 'blocked') { - return $single->("blocked", $purple), + } elsif ($status eq 'blocked' or $status eq 'starved') { + return $single->($status, $purple), } elsif ($status eq 'queued') { return $single->("($status)", $grey_dark), } elsif ($status eq 'preparing') { diff --git a/sg-report-flight b/sg-report-flight index 2677c479..6c481f6f 100755 --- a/sg-report-flight +++ b/sg-report-flight @@ -160,6 +160,10 @@ sub findaflight ($$$$$) { return undef; } + my $jcheckq= db_prepare(<execute($tflight, $job, $testid); ($chkst) = $checkq->fetchrow_array(); + if (!defined $chkst) { + $jcheckq->execute($tflight, $job); + my ($jchkst) = $jcheckq->fetchrow_array(); + $chkst = $jchkst if $jchkst eq 'starved'; + } } $whynot= $sub->($tflight, $chkst) if !defined $whynot; if (!defined $whynot) { @@ -861,6 +870,8 @@ END my $tblocker = $st eq 'fail' ? 'regression' : $st; my $tpriority = $j->{job} =~ m/^build/ ? 600 : 100; $rtup= [ $tpriority, $tblocker, "REGR. vs.", $chkflight ]; + } elsif ($chkst eq 'starved') { # actually, the job was 'starved' + $rtup= [ 50, '', "starved in", $chkflight ]; } else { $rtup= [ 170, '', "$chkst in", $chkflight ]; } From patchwork Thu Apr 18 16:31:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E6F7D1932 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D25D628D56 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C742128D59; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 88E3B28D58 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xJ-0001zl-8m; Thu, 18 Apr 2019 16:32:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xH-0001zL-UF for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:07 +0000 X-Inumbo-ID: 81303893-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 81303893-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:06 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913766" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:44 +0100 Message-ID: <20190418163158.11408-8-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 07/21] starvation: Teach ms-flights-summary about job state starved X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This just affects sorting etc. in the summary display. Signed-off-by: Ian Jackson --- ms-flights-summary | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/ms-flights-summary b/ms-flights-summary index 9d15dd14..8293d4f6 100755 --- a/ms-flights-summary +++ b/ms-flights-summary @@ -145,10 +145,11 @@ sub sort_stats($) { preparing => 2, 'prep.alloc.' => 3, blocked => 4, - running => 5, - pass => 6, - fail => 7, - broken => 8, + starved => 5, + running => 6, + pass => 7, + fail => 8, + broken => 9, ); return sort { ($so{$a}//0) <=> ($so{$b}//0) } (keys %{$stats}); } From patchwork Thu Apr 18 16:31:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907689 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB5EC922 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 946F228D56 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 88EAA28D5A; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 207CF28D59 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xJ-00020S-IC; Thu, 18 Apr 2019 16:32:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xH-0001zM-Vp for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:08 +0000 X-Inumbo-ID: 8191b2ab-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 8191b2ab-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:07 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913767" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:45 +0100 Message-ID: <20190418163158.11408-9-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 08/21] starvation: Teach sg-execute-flight about job state starved X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP It needs to not mind if a job reports as `starved', even if sg-run-job exited non-zero. Signed-off-by: Ian Jackson --- sg-execute-flight | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sg-execute-flight b/sg-execute-flight index aed9823d..02f63316 100755 --- a/sg-execute-flight +++ b/sg-execute-flight @@ -189,7 +189,7 @@ proc main_iteration {} { if {!$found} { error "job not in database $flight $job" } switch -exact -- $jobinfo(status) { - pass - fail - broken - blocked - truncated { + pass - fail - broken - blocked - truncated - starved { } queued - preparing - running { jobdb::job-set-status-unlocked $flight $job broken From patchwork Thu Apr 18 16:31:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907683 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BD831922 for ; Thu, 18 Apr 2019 16:33:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A727328D55 for ; Thu, 18 Apr 2019 16:33:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B86328D58; Thu, 18 Apr 2019 16:33:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5280928D55 for ; Thu, 18 Apr 2019 16:33:44 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xL-00022m-T4; Thu, 18 Apr 2019 16:32:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xK-00021L-CH for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:10 +0000 X-Inumbo-ID: 819cb696-61f7-11e9-a51d-0bdd52c2ff68 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 819cb696-61f7-11e9-a51d-0bdd52c2ff68; Thu, 18 Apr 2019 16:32:07 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913769" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:46 +0100 Message-ID: <20190418163158.11408-10-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 09/21] step handling: Preserve step states set by ts-* scripts X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP sg-run-job would unconditionally set the step state to the value it calculated, which would usually be `pass' or `fail' or `broken' (according to the recipe). Relax this interface somewhat to allow a test script to set the step status itself: specifically, do not overwrite an existing status of aborted broken starved Signed-off-by: Ian Jackson --- tcl/JobDB-Executive.tcl | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl index 70728953..57ded025 100644 --- a/tcl/JobDB-Executive.tcl +++ b/tcl/JobDB-Executive.tcl @@ -316,11 +316,14 @@ proc spawn-step-commit {flight job stepno testid} { proc step-set-status {flight job stepno st} { transaction flights { - db-update-1 " + db-execute " UPDATE steps SET status='$st', finished=[clock seconds] WHERE flight=$flight AND job='$job' AND stepno=$stepno + AND status<>'aborted' + AND status<>'broken' + AND status<>'starved' " set pause 0 db-execute-array stopinfo " From patchwork Thu Apr 18 16:31:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66B14922 for ; Thu, 18 Apr 2019 16:33:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4FEE628D55 for ; Thu, 18 Apr 2019 16:33:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 39B4828D58; Thu, 18 Apr 2019 16:33:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E8A7C28D55 for ; Thu, 18 Apr 2019 16:33:32 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xQ-0002A8-HF; Thu, 18 Apr 2019 16:32:16 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xO-00026E-42 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:14 +0000 X-Inumbo-ID: 850054c1-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 850054c1-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:13 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913794" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:47 +0100 Message-ID: <20190418163158.11408-11-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 10/21] TestSupport: Make "broken" print the actual job state X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP No functional change except to a message. Signed-off-by: Ian Jackson --- Osstest/TestSupport.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index 38c17d60..e218ff35 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -242,7 +242,7 @@ sub broken ($;$) { AND (status='queued' OR status='running' OR status='preparing') END }); - die "BROKEN: $m; ". ($affected>0 ? "marked $flight.$job $newst" + die uc($newst).": $m; ". ($affected>0 ? "marked $flight.$job $newst" : "($flight.$job not marked $newst)"); } From patchwork Thu Apr 18 16:31:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907699 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6703C1850 for ; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50D1E28D55 for ; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 456EC28D58; Thu, 18 Apr 2019 16:33:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 000E328D56 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xJ-00020w-TW; Thu, 18 Apr 2019 16:32:09 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xI-0001zU-67 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:08 +0000 X-Inumbo-ID: 81dfbc42-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 81dfbc42-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:07 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913770" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:48 +0100 Message-ID: <20190418163158.11408-12-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 11/21] JobDB::Executive: step_*: fix log messages to talk about "steps" X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP These functions are about to be sometimes called for non-substep steps. Signed-off-by: Ian Jackson --- Osstest/JobDB/Executive.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Osstest/JobDB/Executive.pm b/Osstest/JobDB/Executive.pm index ea349052..3a680c9d 100644 --- a/Osstest/JobDB/Executive.pm +++ b/Osstest/JobDB/Executive.pm @@ -281,7 +281,7 @@ END $logfile // "$stepno.$script.log", $testid,time); }); - logm("---------- substep $stepno $testid running ----------"); + logm("---------- step $stepno $testid running ----------"); } sub step_finish ($$) { #method @@ -311,7 +311,7 @@ END unless $oldstatus eq 'running'; $setq->execute($stepstatus,time, $flight,$job,$testid, $stepno); }); - logm("---------- substep $stepno $testid $stepstatus ----------"); + logm("---------- step $stepno $testid $stepstatus ----------"); } sub host_check_allocated ($$) { #method From patchwork Thu Apr 18 16:31:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53FF91850 for ; Thu, 18 Apr 2019 16:33:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E44B28D55 for ; Thu, 18 Apr 2019 16:33:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3268028D56; Thu, 18 Apr 2019 16:33:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E583928D5A for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xK-00021F-7Z; Thu, 18 Apr 2019 16:32:10 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xJ-0001zq-FS for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:09 +0000 X-Inumbo-ID: 822aa6ce-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 822aa6ce-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:08 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913772" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:49 +0100 Message-ID: <20190418163158.11408-13-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 12/21] starvation: Permit step_finish to set the state `starved' X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Ian Jackson --- Osstest/JobDB/Executive.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Osstest/JobDB/Executive.pm b/Osstest/JobDB/Executive.pm index 3a680c9d..2e11c8f3 100644 --- a/Osstest/JobDB/Executive.pm +++ b/Osstest/JobDB/Executive.pm @@ -287,7 +287,7 @@ END sub step_finish ($$) { #method my ($jd,$testid,$stepstatus) = @_; die "$flight.$job $testid $stepstatus" unless grep { $stepstatus eq $_ } - qw(pass fail skip blocked broken); + qw(pass fail skip blocked broken starved); my $checkq = $dbh_tests->prepare(< X-Patchwork-Id: 10907687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 853261850 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F50528D56 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 63F2128D5F; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0606628D56 for ; Thu, 18 Apr 2019 16:33:46 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xK-00021e-Gt; Thu, 18 Apr 2019 16:32:10 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xJ-0001zr-Gk for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:09 +0000 X-Inumbo-ID: 828dba8c-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 828dba8c-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:08 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913773" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:50 +0100 Message-ID: <20190418163158.11408-14-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 13/21] TestSupport: Make "broken" set the step state too X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Ian Jackson --- Osstest/TestSupport.pm | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index e218ff35..8e20244d 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -235,6 +235,17 @@ sub broken ($;$) { # must be run outside transaction my $affected; $newst= 'broken' unless defined $newst; + eval { + # This is not always effective, but when we really care is + # situations where $newst is not `broken' (eg `starved') and + # we are running in Executive mode under sg-run-job, and then + # this works. If we are running under sg-run-job, sg-run-job + # will set the step status if we don't manage it. + my $testid = $ENV{'OSSTEST_TESTID'} + || die "OSSTEST_TESTID not set"; + $mjobdb->step_finish($testid, $newst); + }; + warn "failed to set step status $newst: $@" if $@; db_retry($flight,'running', $dbh_tests,[qw(flights)], sub { $affected= $dbh_tests->do(< X-Patchwork-Id: 10907695 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B948313B5 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0A4728D55 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 94C5628D59; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4DCDB28D55 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xM-00023s-HJ; Thu, 18 Apr 2019 16:32:12 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xK-00021p-O7 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:10 +0000 X-Inumbo-ID: 82e37e41-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 82e37e41-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:09 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913774" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:51 +0100 Message-ID: <20190418163158.11408-15-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 14/21] tcl/JobDB-Executive: Do not squash "starved" status X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP ts-hosts-allocate is going to set the job status to `starved' sometimes, and then die. `starved' needs to be added to the list of job statuses that sg-run-job leaves alone. Signed-off-by: Ian Jackson --- tcl/JobDB-Executive.tcl | 1 + 1 file changed, 1 insertion(+) diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl index 57ded025..4f3409a9 100644 --- a/tcl/JobDB-Executive.tcl +++ b/tcl/JobDB-Executive.tcl @@ -55,6 +55,7 @@ proc job-set-status-unlocked {flight job st} { UPDATE jobs SET status='$st' WHERE flight=$flight AND job='$job' AND status<>'aborted' AND status<>'broken' + AND status<>'starved' " } From patchwork Thu Apr 18 16:31:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907669 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 59CBC922 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42BB628D55 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 36CF328D56; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 797C728D59 for ; Thu, 18 Apr 2019 16:33:30 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xQ-0002Ay-U9; Thu, 18 Apr 2019 16:32:16 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xP-00027g-64 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:15 +0000 X-Inumbo-ID: 858c0478-61f7-11e9-8d3f-9f6ad06a8bc4 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 858c0478-61f7-11e9-8d3f-9f6ad06a8bc4; Thu, 18 Apr 2019 16:32:13 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913797" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:52 +0100 Message-ID: <20190418163158.11408-16-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 15/21] starvation: Propagate starved job status into dependent jobs X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP If a build job is starved, then the same status (No jobs are marked `starved' yet.) Signed-off-by: Ian Jackson --- Osstest/JobDB/Executive.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Osstest/JobDB/Executive.pm b/Osstest/JobDB/Executive.pm index 2e11c8f3..be5588fc 100644 --- a/Osstest/JobDB/Executive.pm +++ b/Osstest/JobDB/Executive.pm @@ -380,6 +380,8 @@ END } else { die "job $oflight.$ojob ($why) queued (we are $ourstatus)"; } + } elsif ($jstatus eq 'starved') { + broken("job $oflight.$ojob ($why) $jstatus", $jstatus); } else { broken("job $oflight.$ojob ($why) $jstatus", 'blocked'); } From patchwork Thu Apr 18 16:31:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907697 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E81C9922 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D266F28D56 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C744B28D59; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7FD1E28D56 for ; Thu, 18 Apr 2019 16:33:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xM-00024I-Sa; Thu, 18 Apr 2019 16:32:12 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xK-00022D-W2 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:11 +0000 X-Inumbo-ID: 8354c13e-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 8354c13e-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:10 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913784" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:53 +0100 Message-ID: <20190418163158.11408-17-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 16/21] ts-host-allocate-Executive: Break out $now and add a newline X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Prep work, no functional change. Signed-off-by: Ian Jackson --- ts-hosts-allocate-Executive | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive index 3da138b1..3425c8ce 100755 --- a/ts-hosts-allocate-Executive +++ b/ts-hosts-allocate-Executive @@ -756,12 +756,14 @@ sub attempt_allocation { logm("host allocation: successful, reporting to planner."); } else { logm("host allocation: planned start in $best->{Start} seconds."); + my $maxwait = $r{hostalloc_maxwait_max}; + my $now = time // die $!; if (defined $maxwait) { # We quit if: # * we have been waiting at least $maxwait/2 # * we estimate it will take at least $maxwait overall - my $wait_sofar = (time // die $!) - $alloc_start_time; + my $wait_sofar = $now - $alloc_start_time; die "timed out: $wait_sofar, $best->{Start}, $maxwait" if $wait_sofar > $maxwait/2 && $wait_sofar + $best->{Start} > $maxwait; From patchwork Thu Apr 18 16:31:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907667 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2636A13B5 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E8EB28D55 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0307B28D5D; Thu, 18 Apr 2019 16:33:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7393628D55 for ; Thu, 18 Apr 2019 16:33:30 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xR-0002BY-9b; Thu, 18 Apr 2019 16:32:17 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xP-00028E-Jx for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:15 +0000 X-Inumbo-ID: 85b12970-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 85b12970-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:14 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913798" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:54 +0100 Message-ID: <20190418163158.11408-18-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 17/21] starvation: Use "starved" for hostalloc_maxwait_max X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Previously this was "broken". We mustn't just call `broken' inside attempt_allocation because that runs in a db transaction. Instead, we arrange that attempt_allocation returns 2, which threads its way back out to the return value from alloc_resources, and then call broken there. Signed-off-by: Ian Jackson --- ts-hosts-allocate-Executive | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive index 3425c8ce..8c9ddaf7 100755 --- a/ts-hosts-allocate-Executive +++ b/ts-hosts-allocate-Executive @@ -683,12 +683,17 @@ sub alloc_hosts () { ? -10000 : -10 * @hids; - alloc_resources(WaitStart => + my $ok = alloc_resources(WaitStart => ($ENV{OSSTEST_RESOURCE_WAITSTART} || $fi->{started}), WaitStartAdjust => $waitstartadjust, DebugFh => \*DEBUG, \&attempt_allocation); + if ($ok==2) { + broken "starved or timed out", 'starved'; + } + die $ok unless $ok==1; + foreach my $hid (@hids) { my $sel= $hid->{Selected}; die $hid->{Ident} unless defined $sel->{resname}; @@ -764,9 +769,11 @@ sub attempt_allocation { # * we have been waiting at least $maxwait/2 # * we estimate it will take at least $maxwait overall my $wait_sofar = $now - $alloc_start_time; - die "timed out: $wait_sofar, $best->{Start}, $maxwait" - if $wait_sofar > $maxwait/2 - && $wait_sofar + $best->{Start} > $maxwait; + if ($wait_sofar > $maxwait/2 + && $wait_sofar + $best->{Start} > $maxwait) { + logm "timed out: $wait_sofar, $best->{Start}, $maxwait"; + return 2; + } } } From patchwork Thu Apr 18 16:31:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907673 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18AE513B5 for ; Thu, 18 Apr 2019 16:33:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0E2A28D55 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E577128D59; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 606C428D55 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xS-0002Dj-64; Thu, 18 Apr 2019 16:32:18 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xP-00029D-Um for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:15 +0000 X-Inumbo-ID: 8631863e-61f7-11e9-92d7-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 8631863e-61f7-11e9-92d7-bc764e045a96; Thu, 18 Apr 2019 16:32:15 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913801" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:55 +0100 Message-ID: <20190418163158.11408-19-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 18/21] starvation: Infrastructure for jobs which are delaying their flights X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Provide hostalloc_starvation_* in Osstest::Executive, and a comment saying what we are going to do. And provide a demo utility which prints the effect of some particular runvar value on a range of situations. Signed-off-by: Ian Jackson --- Osstest/Executive.pm | 70 ++++++++++++++++++++++++++++++++++++++++++++ mg-hostalloc-starvation-demo | 53 +++++++++++++++++++++++++++++++++ 2 files changed, 123 insertions(+) create mode 100755 mg-hostalloc-starvation-demo diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm index a9f9ac78..e741f529 100644 --- a/Osstest/Executive.pm +++ b/Osstest/Executive.pm @@ -22,6 +22,7 @@ use warnings; use Osstest; +use Carp; use POSIX; use IO::File; use File::Copy; @@ -55,6 +56,8 @@ BEGIN { alloc_resources alloc_resources_rollback_begin_work resource_check_allocated executive_resource_shared_mark_ready + hostalloc_starvation_parse_runvar + hostalloc_starvation_calculate_X duration_estimator db_pg_dsn opendb opendb_state db_schema_updates_applied db_schema_updates_intree @@ -1053,6 +1056,73 @@ END logm("$restype $resname shared $sharetype marked ready"); } +# hostalloc_maxwait_starvation +# +# $r{hostalloc_maxwait_starvation} is either "never" or +# comma-separated list of assignments: +# = +# where is C, A, L or Xh and is +# a floating point number. +# +# terms: +# calculated or observed values: +# X wait factor, ratio between amount of time +# to allow for slow jobs, compared to other jobs +# W number of jobs waiting - strictly, jobs in states +# preparing queued running +# D number of jobs done - strictly, other states +# tuning parameters: +# Xt X when D=9 W=1, ie cancel one job out of ten +# Xh X when D=1 W=1, ie cancel one job out of two +# I length of time this expected delay must have +# persisted before we abandon this job +# C,A tuning parameters relating X to W and D (see below) +# +# we calculate: +# X = C + L * W^A / D^B +# where C, A are specified +# and L, B are calculated from C, A, Xh, Xt +# +# and then we are willing to allow each job to take +# X * (time spent for done jobs) + I +# +# default values are +# C = 1.3 +# A = 1 +# Xt = 5 +# Xh = 200 +# I = 3600 (seconds) + +sub hostalloc_starvation_parse_runvar ($) { + my ($r_hostalloc_maxwait_starvation) = @_; + # returns \%p hashref to be fed to hostalloc_starvation_calculate_X + # caller may inspect elements of %p + # if was `never' then !%p + $r_hostalloc_maxwait_starvation //= ''; + return { } if $r_hostalloc_maxwait_starvation eq 'never'; + my %p = ( C => 1.3, A => 1, Xt => 5, Xh => 200, I => 3600 ); + foreach (split /,/, $r_hostalloc_maxwait_starvation) { + m/=/ or die "$r_hostalloc_maxwait_starvation ?"; + exists($p{$`}) or die "$r_hostalloc_maxwait_starvation $` ?"; + $p{$`} = $' + 0.0; + } + # Xh = C + L * 1^A / 1^B => L = Xh - C + # Xt = C + L * 1^A / 9^B + # Xt = C + (Xh - C) * 1^A / 9^B => B = log_9[ Xh - C) / (Xt - C) ] + $p{L} = $p{Xh} - $p{C}; + $p{B} = log(($p{Xh} - $p{C}) / ($p{Xt} - $p{C})) / log(9); + logm("hostalloc_maxwait_starvation: ". + join ' ', map { "$_=$p{$_}" } sort keys %p); + return \%p; +} + +sub hostalloc_starvation_calculate_X ($$$) { + my ($p, $W, $D) = @_; # => X, undef meaning "forever" + return undef unless %$p and $D; + confess unless $W > 0 && $D > 0; + return $p->{C} + $p->{L} * $W**$p->{A} / $D**$p->{B}; +} + #---------- duration estimator ---------- sub duration_estimator ($$;$$) { diff --git a/mg-hostalloc-starvation-demo b/mg-hostalloc-starvation-demo new file mode 100755 index 00000000..7944cc24 --- /dev/null +++ b/mg-hostalloc-starvation-demo @@ -0,0 +1,53 @@ +#!/usr/bin/perl -w + +use strict; +BEGIN { unshift @INC, qw(.); } + +use strict; +use POSIX; +use Osstest::Executive; +use Data::Dumper; + +die unless @ARGV>=1; + +our ($rv) = $ARGV[0]; # $r{hostalloc_starvation}; +shift @ARGV; + +my $p = hostalloc_starvation_parse_runvar($rv); + +my $l = 8; + +my @ts = qw(1 2 3 5 10 20 33 66 100 200 333); + +foreach my $w (qw(332 99 32 9 3 2 1)) { + printf "W=%3d |", $w; + foreach my $t (@ts) { + my $d = $t - $w; + print " | "; + if ($d <= 0) { + printf " %${l}s", ''; + } else { + my $x = hostalloc_starvation_calculate_X($p, $w, $d); + printf "%3d%% ", 100.0*$w/$t; + if (defined $x) { + printf "%${l}.2f", $x; + } else { + printf "%${l}s", 'never'; + } + } + } + print "\n"; +} + +print "\n"; +print " "; +foreach my $t (@ts) { + printf " | %${l}s", sprintf "T=%3d", $t; +} +print "\n"; + +foreach (@ARGV) { + m/,/ or die $!; + my $x = hostalloc_starvation_calculate_X($p, $', $`); + printf "D=%3d W=%3d X=%${l}.2f\n", $`, $', $x; +} From patchwork Thu Apr 18 16:31:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2D953922 for ; Thu, 18 Apr 2019 16:33:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12CBF28D56 for ; Thu, 18 Apr 2019 16:33:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 06C4728D55; Thu, 18 Apr 2019 16:33:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7A4CC28D55 for ; Thu, 18 Apr 2019 16:33:33 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xT-0002GS-Ji; Thu, 18 Apr 2019 16:32:19 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xR-0002C4-EH for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:17 +0000 X-Inumbo-ID: 866c7f9e-61f7-11e9-b3e8-bf25b6989550 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 866c7f9e-61f7-11e9-b3e8-bf25b6989550; Thu, 18 Apr 2019 16:32:15 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913803" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:56 +0100 Message-ID: <20190418163158.11408-20-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 19/21] starvation: Abandon jobs which are unreasonably delaying their flight X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Sometimes, due to a shortage of available resources, a flight might be delayed because a handful of jobs are waiting much longer than the rest. Add a heuristic which causes these jobs to be abandoned. We consider ourselves starving if we are starving now, based on the most optimistic start time seen in the last I. Signed-off-by: Ian Jackson --- ts-hosts-allocate-Executive | 105 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive index 8c9ddaf7..7ea3c4af 100755 --- a/ts-hosts-allocate-Executive +++ b/ts-hosts-allocate-Executive @@ -62,6 +62,8 @@ our %magictaskid; our $fi; our $jobinfo; our $harness_rev; +our $starvation_p; +our @abs_start_estimates; #---------- general utilities, setup, etc. ---------- @@ -114,12 +116,16 @@ END } $alloc_start_time = time // die $!; + + $starvation_p = + hostalloc_starvation_parse_runvar($r{hostalloc_maxwait_starvation}); } #---------- prepared sql statements ---------- # all users of these must ->finish them afterwards, to avoid db deadlock our ($flagscheckq, $equivflagscheckq, $duration_estimator, $resprop_q, + $starvation_q, $alloc_findres_q, $alloc_shared_q, $alloc_sharing_slot_q, $claim_share_reuse_q, $claim_maxshare_q, $claim_rmshares_q, $claim_noshares_q, $claim_rmshare_q, $claim_setres_q, @@ -146,6 +152,15 @@ END AND name = ? END + $starvation_q= $dbh_tests->prepare(<prepare(<{Start}. + # Returns the most optimistic absolute start time "in the last + # $period". Returns undef if we don't have good data yet. + + push @abs_start_estimates, { At => $now, Got => $best->{Start} + $now }; + + # Actually, what we do is prune all but the last entry from before + # $period, and we expect at least 4 estimates. That ensures that + # the answer involves at least one estimate at least $period ago. + # Ie what we actually return is + # Consider the most recent estimate which is at least $period + # ago (the "oldest relevant"), and all subsequent estimates. + # Answer is the most optimistic start time of all of those, + # provided there are at least 4 of them. + my $is_old = sub { return $_[0]{At} <= $now - $period; }; + my $need_estimates = 4; + while (@abs_start_estimates > $need_estimates && + $is_old->($abs_start_estimates[1])) { + # estimates[1] is at least $period ago and more recent + # than $estimates[0], so $estimates[0] cannot be the + # oldest relevant and is indeed older than the oldest + # relevant. + shift @abs_start_estimates; + } + + my $pr = sub { + my ($e) = @_; + printf(DEBUG ' %s (@%s)', + $e->{Got} - $now, + $e->{At} - $now); + }; + + print DEBUG "most_optimistic: all:"; + my $optimist; + foreach (@abs_start_estimates) { + $pr->($_); + $optimist = $_ if !$optimist || $_->{Got} < $optimist->{Got}; + } + print DEBUG "\n"; + printf(DEBUG "most_optimistic: (period=%s):", $period); + $pr->($optimist); + print DEBUG "\n"; + + return undef unless @abs_start_estimates >= $need_estimates; + + return $optimist->{Got}; +} + +sub starving ($) { + my ($best_start_abs) = @_; + return (0, 'runvar says never give up') unless %$starvation_p; + return (0, 'no estimate') unless defined $best_start_abs; + $starvation_q->execute($flight); + my $d=0; + my $w=0; + my $maxfin=0; + while (my ($j,$st,$fin) = $starvation_q->fetchrow_array()) { + if ($st eq 'preparing' || + $st eq 'queued' || + $st eq 'running') { + $w++; + } else { + $d++; + return (0, "job $j status $st but no step finished time!") + unless defined $fin; + $maxfin = $fin if $fin > $maxfin; + } + } + # we quit if the total time from the start of the flight + # to our expected finish is more than the total time so + # far (for the completed jobs) by the margin X and I + my $X = hostalloc_starvation_calculate_X($starvation_p, $w, $d); + return (0, 'X=inf') unless defined $X; + my $total_d = $maxfin - $fi->{started}; + my $projected_me = $best_start_abs - $fi->{started}; + my $m = "D=$d W=$w X=$X maxfin=$maxfin"; + my $bad = $projected_me > $X * $total_d + $starvation_p->{I}; + return ($bad, $m); +} + sub attempt_allocation { my $mayalloc; ($plan, $mayalloc) = @_; @@ -772,6 +869,14 @@ sub attempt_allocation { if ($wait_sofar > $maxwait/2 && $wait_sofar + $best->{Start} > $maxwait) { logm "timed out: $wait_sofar, $best->{Start}, $maxwait"; + } elsif (%$starvation_p) { + my $est_abs = most_optimistic($best, $now, $starvation_p->{I}); + my ($starving, $m) = starving($est_abs); + $starvation_q->finish(); + if (!$starving) { + print DEBUG "not starving: $m\n"; + } else { + logm "starving ($m)"; return 2; } } From patchwork Thu Apr 18 16:31:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 798E41850 for ; Thu, 18 Apr 2019 16:33:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62A6E28D55 for ; Thu, 18 Apr 2019 16:33:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5706628D58; Thu, 18 Apr 2019 16:33:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0AC4628D55 for ; Thu, 18 Apr 2019 16:33:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xP-00029C-Q1; Thu, 18 Apr 2019 16:32:15 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xN-000268-UX for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:13 +0000 X-Inumbo-ID: 843157fe-61f7-11e9-a093-d366d8b4972d Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 843157fe-61f7-11e9-a093-d366d8b4972d; Thu, 18 Apr 2019 16:32:11 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913789" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:57 +0100 Message-ID: <20190418163158.11408-21-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 20/21] hostalloc_maxwait_max: Use starvation most_optimistic X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Now that we have a more sophisticated estimate of our likely scheduled time, use it here too. Signed-off-by: Ian Jackson --- ts-hosts-allocate-Executive | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive index 7ea3c4af..b75927c1 100755 --- a/ts-hosts-allocate-Executive +++ b/ts-hosts-allocate-Executive @@ -865,10 +865,15 @@ sub attempt_allocation { # We quit if: # * we have been waiting at least $maxwait/2 # * we estimate it will take at least $maxwait overall - my $wait_sofar = $now - $alloc_start_time; - if ($wait_sofar > $maxwait/2 - && $wait_sofar + $best->{Start} > $maxwait) { - logm "timed out: $wait_sofar, $best->{Start}, $maxwait"; + my $est_abs = most_optimistic($best, $now, $maxwait/2); + if (defined $est_abs) { + my $wait_sofar = $now - $alloc_start_time; + my $wait_exp = $est_abs - $alloc_start_time; + if ($wait_exp > $maxwait) { + logm "timed out: $wait_exp > $maxwait ($wait_sofar)", + return 2; + } + } } elsif (%$starvation_p) { my $est_abs = most_optimistic($best, $now, $starvation_p->{I}); my ($starving, $m) = starving($est_abs); From patchwork Thu Apr 18 16:31:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Jackson X-Patchwork-Id: 10907677 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D943B1850 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD5AB28D58 for ; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B22D128D5D; Thu, 18 Apr 2019 16:33:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7469F28D58 for ; Thu, 18 Apr 2019 16:33:30 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xU-0002HC-2I; Thu, 18 Apr 2019 16:32:20 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hH9xR-0002C3-E8 for xen-devel@lists.xenproject.org; Thu, 18 Apr 2019 16:32:17 +0000 X-Inumbo-ID: 86a2232e-61f7-11e9-8634-4356937a26ca Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 86a2232e-61f7-11e9-8634-4356937a26ca; Thu, 18 Apr 2019 16:32:16 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,366,1549929600"; d="scan'208";a="83913807" From: Ian Jackson To: Date: Thu, 18 Apr 2019 17:31:58 +0100 Message-ID: <20190418163158.11408-22-ian.jackson@eu.citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190418163158.11408-1-ian.jackson@eu.citrix.com> References: <20190418163158.11408-1-ian.jackson@eu.citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [OSSTEST PATCH 21/21] starvation: Better logging/debugging output X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Ian Jackson --- ts-hosts-allocate-Executive | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive index b75927c1..64c4c9f9 100755 --- a/ts-hosts-allocate-Executive +++ b/ts-hosts-allocate-Executive @@ -777,8 +777,8 @@ sub most_optimistic ($$$) { return $optimist->{Got}; } -sub starving ($) { - my ($best_start_abs) = @_; +sub starving ($$) { + my ($best_start_abs, $now) = @_; return (0, 'runvar says never give up') unless %$starvation_p; return (0, 'no estimate') unless defined $best_start_abs; $starvation_q->execute($flight); @@ -801,11 +801,16 @@ sub starving ($) { # to our expected finish is more than the total time so # far (for the completed jobs) by the margin X and I my $X = hostalloc_starvation_calculate_X($starvation_p, $w, $d); - return (0, 'X=inf') unless defined $X; + return (0, "D=$d W=$w X=inf") unless defined $X; my $total_d = $maxfin - $fi->{started}; my $projected_me = $best_start_abs - $fi->{started}; - my $m = "D=$d W=$w X=$X maxfin=$maxfin"; - my $bad = $projected_me > $X * $total_d + $starvation_p->{I}; + my $lim = $X * $total_d + $starvation_p->{I}; + my $Xcmp = ($projected_me - $starvation_p->{I}) / $total_d; + my $m = sprintf + "D=%d W=%d X=%.3f t_D=%s t_me=%s t_lim=%.3f X'=%.4f (fi.s=%s)", + $d, $w, $X, $total_d, $projected_me, $lim, $Xcmp, + $fi->{started} - $now; + my $bad = $projected_me > $lim; return ($bad, $m); } @@ -876,7 +881,7 @@ sub attempt_allocation { } } elsif (%$starvation_p) { my $est_abs = most_optimistic($best, $now, $starvation_p->{I}); - my ($starving, $m) = starving($est_abs); + my ($starving, $m) = starving($est_abs, $now); $starvation_q->finish(); if (!$starving) { print DEBUG "not starving: $m\n";