| Message ID | 20170706144227.36580-12-roger.pau@citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Roger Pau Monne writes ("[PATCH v4 11/16] osstest: introduce a script to set the runtime hostflags runvar for FreeBSD jobs"):
> Due to the nature of the FreeBSD install media, which is
> self-generated from the ts-freebsd-build script, the hostflags runvar
> set to FreeBSD jobs are related to the current version under test.
This is good. However:
> +set_runtime_hostflag("host",
> + "share-build-freebsd-$arch-$hash,freebsd-$version");
If you agree with my comments on the previous patch, you might want to
change this to two calls to set_runtime_hostflag, and to pass $ho.
Ian.
Roger Pau Monne writes ("[PATCH v4 11/16] osstest: introduce a script to set the runtime hostflags runvar for FreeBSD jobs"):
> Due to the nature of the FreeBSD install media, which is
> self-generated from the ts-freebsd-build script, the hostflags runvar
> set to FreeBSD jobs are related to the current version under test.
...
> +set_runtime_hostflag("host",
> + "share-build-freebsd-$arch-$hash,freebsd-$version");
So, having thought about this:
You are currently unconditionally calling this script for build jobs,
allowing it to assume the host ident name.
You don't call it at all for test jobs. This is wrong because test
jobs need the freebsd-* hostflag too.
Currently there is no test job sharing, so they need not to have the
share-* flag.
I think this means this ts-freebsd-set-hostflags script needs to be
told somehow what flags to set. Since the sharetype is calculated
from the share-* hostflag by ts-hosts-allocate-Executive, but then
recalculated from the parameters to
$mjobdb->jobdb_resource_shared_mark_ready, the code and runvars need
to be coordinated so there is no problem making the hostflag be
approximately-fixed.
So I suggest adding a --share flag to ts-freebsd-set-hostflags.
Also: you have not noticed this because currently nothing call
$mjobdb->jobdb_resource_shared_mark_ready. Normally this is done by
ts-xen-build-prep which is run by prepare-build-host-linux. You
haven't noticed that you aren't calling this because currently you
aren't building Xen at all.
One way to deal with this would be to fix ts-xen-build-prep for
FreeBSD (at least to the point where it runs to completion, even if
not all the Xen build-dependencies are installed), and then call it in
prepare-build-host-freebsd. But that looks quite annoying because
ts-xen-build-prep reorganises the storage in rather linux-specific
ways.
Another would be to split out the call to
$mjobdb->jobdb_resource_shared_mark_ready
from ts-xen-build-prep into its own script ts-host-shared-ready.
That new script would have to gain a substitution system.
An ad-hoc method would be to call
$mjobdb->jobdb_resource_shared_mark_ready
from ts-freebsd-install. That seems to suggest FreeBSD installs not
needing different storage arrangements or something for build hosts
and test dom0s.
If you were feeling keen, you could sort out the anomaly that is a
test script calling straight into $mjobdb. Eg.
1. Rename resource_shared_mark_ready to
executive_resource_shared_mark_ready and change all callers.
2. Introduce a wrapper resource_shared_mark_ready
which calls $mjobdb->jobdb_resource_shared_mark_ready
and change the call in ts-xen-build-prep to use it.
I'll come to you in a moment to handwave at you about all this.
Ian.
diff --git a/ts-freebsd-set-hostflags b/ts-freebsd-set-hostflags new file mode 100755 index 00000000..1b0206a8 --- /dev/null +++ b/ts-freebsd-set-hostflags @@ -0,0 +1,64 @@ +#!/usr/bin/perl -w +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2017 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +# This script sets the host_hostflags for a FreeBSD job based on the runvars +# provided and the contents of the extra_hostflags runvar. +# +# If the freebsd_distpath runvar is set the installer image will be retrieved +# from "freebsd_distpath"/install.img, and the FreeBSD version from the +# freebsd_version runvar. Note that both those runvars should be set on the +# current job. +# +# If freebsd_distpath is not set, it is assumed that freebsdbuildjob runvar is +# set and the installer image will be retrieved from the path pointed to by +# "path_freebsdddist"/install.img, and the FreeBSD version will be obtained +# from the "freebsd_buildversion" runvar. Both of those runvars belong to the +# flight and job pointed to by freebsdbuildjob. +# +# As output upon successful completion this script will set the host_hostflags +# runvar for the current job. Note that this _must_ be done before running +# ts-host-allocate. +# + +use strict qw(vars); +use DBI; +use POSIX; + +unshift @INC, qw(.); +use Osstest; +use Osstest::TestSupport; + +tsreadconfig(); + +sub get_freebsd_image_hash() { + my $distpath = $r{"freebsd_distpath"} || + get_stashed("path_freebsddist", $r{"freebsdbuildjob"}); + + return sha256file("$distpath/install.img", 16); +} + +sub get_freebsd_version() { + return $r{"freebsd_version"} || + get_runvar("freebsd_buildversion", $r{"freebsdbuildjob"}); +} + +my $hash = get_freebsd_image_hash(); +my $version = get_freebsd_version(); +my $arch = $r{"arch"}; + +set_runtime_hostflag("host", + "share-build-freebsd-$arch-$hash,freebsd-$version");
Due to the nature of the FreeBSD install media, which is self-generated from the ts-freebsd-build script, the hostflags runvar set to FreeBSD jobs are related to the current version under test. The following hostflags might need to be fetched from the runvars of a previous build-$arch-freebsd job: - share-build-freebsd-$arch-$hash: the $hash used here is calculated from the checksum of the installer image used by this specific job. This allows osstest to share FreeBSD build hosts, and be sure the exact desired FreeBSD version is used. - freebsd-$version: $version contains the major FreeBSD version under test. This version is obtained from the FreeBSD source code used to create the install media. Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> --- Changes since v3: - Make use of the set_runtime_hostflag helper. - Make use of the sha256file helper. Changes since v2: - New in this version. --- ts-freebsd-set-hostflags | 64 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 ts-freebsd-set-hostflags