| Message ID | 20170523135148.77673-5-roger.pau@citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Roger Pau Monne writes ("[PATCH 4/7] osstest: add a FreeBSD host install recipe"):
> The installation is performed using the bsdinstall tool, which is
> part of the FreeBSD base system. The installer image is setup with
> the osstest ssh keys and sshd enabled by default, which allows the
> test harness to just ssh into the box, create the install config
> file and launch the scripted install.
Thanks. I've read this in some detail now. My comments follow.
> In order to support the FreeBSD installer a new method is added,
> that allows setting the pxe boot of a host using a memdisk. Also, a
> new tftp path is added in order to point to the place where the
> FreeBSD installed images should be stored.
Can you please add to the commit message a deployment note that the
tftp root needs to be provide with a copy of memdisk, eg
/home/tftp/memdisk -> /usr/lib/syslinux/memdisk
> diff --git a/Osstest.pm b/Osstest.pm
> index a78728cd..a0c0339c 100644
> --- a/Osstest.pm
> +++ b/Osstest.pm
> @@ -227,6 +227,7 @@ sub readglobalconfig () {
> $c{TftpTmpDir} ||= "$c{TftpPlayDir}tmp/";
>
> $c{TftpDiBase} ||= "$c{TftpPlayDir}debian-installer";
> + $c{TftpFreeBSDBase} ||= "$c{TftpPlayDir}freebsd-installer";
You need to document this in README.
> + foreach (@sets, "MANIFEST") {
I think it would be better to give this loop control variable a name.
When the scope of a use of $_ is too big, $_ has a tendency to get
trashed; this makes the code fragile in the face of future changes.
> + if (!defined($r{"freebsd_sets"})) {
> + # Get everything before the "." (ie: get base from base.txz)
> + my $stash_name = lc((split /\./)[0]);
IMO that's not very idomatic perl. I won't insist you change it, but
maybe this would be better
my $stash_name = $set;
$stash_name =~ s/[^.]+$//;
or
$set =~ m/^[^.]+/ or die "$set ?";
$stash_name = $&;
(Do you intend to split on the first `.' ?)
> +sub setup_netboot_installer () {
> + my $pxeimg = "$ho->{Tftp}{FreeBSDBase}/install-$ho->{Name}.img";
I think this would be better under TftpTmpDir. Cf ts-host-install's
placement of $ho--initrd.gz.
Maybe we want a function target_tftp_prefix or something which returns
$ho->{Tftp}{TmpDir}".hostnamepath($ho)
?
Also the sha names could be in {TmpDir}/freebsd-images. Does this, in
fact, need to be configurable at all ? I think probably not.
> + my $script = <<END;
> +set -ex
> +basedir=$ho->{Tftp}{Path}/$ho->{Tftp}{FreeBSDBase}/
> +hash=`sha256sum $image|head -c 64`
> +localpath="$r{arch}/\$hash/install.img";
> +mkdir -p \$basedir
> +(
> +flock -x -w 600 200
In osstest we generally use with-lock-ex from chiark-utils, rather
than flock from util-linux. (Bonus: it should be more portable for if
someone wants to run a controller on BSD...)
with-lock-ex needs to be used in the wrapper way.
(Also, calling flock on a directory is rather outre'.)
You've written all this in bash because you found it too hard or
annoying in Perl ? I don't much mind that, but the escaping is a bit
irritating. Maybe you want to pass the script its parameters as
arguments so that you can use <<'END' rather than <<END ?
However,
> + logm("Executing:\n$script");
> + my $ret = system("/bin/bash -c '$script'");
This is quite wrong. Your unparsing is unsound. Amazingly your
script doesn't currently contain ' so this doesn't matter right now,
but that could change.
You should use the multi-argument form of system(3perl). And, of
course, you should use Osstest::system_checked which does the error
handling for you.
You might want to build a command in an array @cmd, like copydir in
cr-publish-flight-logs does. (Not sure why that doesn't use
system_checked...)
> +sub install () {
> + my $authkeys = authorized_keys();
> + my $knownhosts = known_hosts();
> + my $sshd_keys_url = create_ssh_overlay();
> + my $disk_names = "ada0 da0 ad0";
This should probably be
my @disk_names = qw(ada0 da0 ad0);
in case anyone wants to manipulate it in perl. You can substitute
it straight into the here document; perl will put in the " " again.
> + my $installer_sets = join(" ", @sets);
> + my $target_sets = "/tmp/osstest_sets";
Hardcoded /tmp antipattern. Maybe this is technicallty OK because
it's an installer environment, but I think it sets a very bad
example. Is there some other path you could use ?
> + target_cmd_root($ho, 'chsh -s /bin/sh', 10);
!! What's the default ?
> +for disk in $disk_names; do
> + if [ -c "/dev/\$disk" ]; then
> + echo \$disk
> + exit 0
> + fi
> +done
> +exit 1
> +END
> + defined($disk) or die "Unable to find a valid disk";
> + logm("Using $disk as destination disk device");
I have found that on some hosts, when installing Debian GNU/Linux, I
have to expect a nonstandard disk name. Currently in the DB I can
only find this
hydrazine disk-device /dev/cciss/c0d0
(in Cambridge; referring to a gone-away machine; NB that the property
name is in the obsolete containing-spaces syntax and is equivalent to
DiskDevice.)
I think you may want to check a host property. It should probably be
called Freebsd_DiskDevice or something. (Weird capitalisation required
by the word splitting name transformation rules for host propertiess.)
> + logm("Trying to figure out primary nic device name");
> + $nic = target_cmd_output_root($ho, <<END, 30);
> +nics=`ifconfig -l`
> +for nic in \$nics; do
> + addr=`ifconfig \$nic inet|grep inet|awk {'print \$2'}`
> + if [ "\$addr" = "$ho->{Ip}" ]; then
> + echo \$nic
> + exit 0
> + fi
> +done
> +exit 1
I have quite a lot of this kind of thing:
gall-mite 'interface force' eth0
itch-mite 'interface force' eth0
grain-weevil 'interface force' eth1
rice-weevil 'interface force' eth1
arndale-bluewater Interface_Force eth0
arndale-lakeside Interface_Force eth0
arndale-metrocentre Interface_Force eth0
arndale-westfield Interface_Force eth0
This may not be needed for FreeBSD, but I thought I would mention it.
What would happen if you tried to run this setup on a host where
FreeBSD's idea of the first network interface is not the one which
osstest did the install on ?
IIRC there is a way for pxelinux to pass the NIC MAC address to the OS
it is running. Also, the host database contains the MAC address of
the host (and this is generally necessary for osstest to work even in
standalone mode I think), so you might use that rather than the IP
address ?
> + foreach (get_sets_path()) {
> + target_putfile_root($ho, 600, $_->{path}, "$target_sets/$_->{name}"
);
Needs linewrap.
> + logm("Creating the installer script");
> + target_cmd_root($ho, <<END, 60);
> + set -e
> + cat << ENDSCRIPT > installscript
OK, my brain is fully bent now. Can we not create installscript on
the controller and transfer it with target_putfilecontents_root_stash ?
That way the logs would contain a copy, too.
> +# Setup serial console
> +printf "%s" "-h -S$c{Baud}" >> /boot.config
> +cat << ENDBOOT >> /boot/loader.conf
> +boot_serial="YES"
> +comconsole_speed="$c{Baud}"
> +console="comconsole"
> +boot_verbose="YES"
> +beastie_disable="YES"
> +ENDBOOT
Where does the installer's output go ? Ie, does booting the installer
image produce serial log output ?
Ian.
Roger Pau Monne writes ("[PATCH 4/7] osstest: add a FreeBSD host install recipe"):
> The installation is performed using the bsdinstall tool, which is
> part of the FreeBSD base system. The installer image is setup with
> the osstest ssh keys and sshd enabled by default, which allows the
> test harness to just ssh into the box, create the install config
> file and launch the scripted install.
...
> +our $image = $r{"freebsd_image"} ||
> + get_stashed("path_freebsd-image", $r{"freebsd_buildjob"});
I've had a thought about this.
Maybe it would be worth mentioning near the top of the script the
runvars it consumes (and any it sets) ?
I realise I haven't documented the runvars very much so far but maybe
we should start. What do you think ?
Ian.
On Tue, May 23, 2017 at 05:04:10PM +0100, Ian Jackson wrote: > Roger Pau Monne writes ("[PATCH 4/7] osstest: add a FreeBSD host install recipe"): > > The installation is performed using the bsdinstall tool, which is > > part of the FreeBSD base system. The installer image is setup with > > the osstest ssh keys and sshd enabled by default, which allows the > > test harness to just ssh into the box, create the install config > > file and launch the scripted install. > > Thanks. I've read this in some detail now. My comments follow. > > > In order to support the FreeBSD installer a new method is added, > > that allows setting the pxe boot of a host using a memdisk. Also, a > > new tftp path is added in order to point to the place where the > > FreeBSD installed images should be stored. > > Can you please add to the commit message a deployment note that the > tftp root needs to be provide with a copy of memdisk, eg > /home/tftp/memdisk -> /usr/lib/syslinux/memdisk Done. > > diff --git a/Osstest.pm b/Osstest.pm > > index a78728cd..a0c0339c 100644 > > --- a/Osstest.pm > > +++ b/Osstest.pm > > @@ -227,6 +227,7 @@ sub readglobalconfig () { > > $c{TftpTmpDir} ||= "$c{TftpPlayDir}tmp/"; > > > > $c{TftpDiBase} ||= "$c{TftpPlayDir}debian-installer"; > > + $c{TftpFreeBSDBase} ||= "$c{TftpPlayDir}freebsd-installer"; > > You need to document this in README. (not needed anymore since we got rid of this). > > + foreach (@sets, "MANIFEST") { > > I think it would be better to give this loop control variable a name. > When the scope of a use of $_ is too big, $_ has a tendency to get > trashed; this makes the code fragile in the face of future changes. Done. > > > + if (!defined($r{"freebsd_sets"})) { > > + # Get everything before the "." (ie: get base from base.txz) > > + my $stash_name = lc((split /\./)[0]); > > IMO that's not very idomatic perl. I won't insist you change it, but > maybe this would be better > > my $stash_name = $set; > $stash_name =~ s/[^.]+$//; > > or > > $set =~ m/^[^.]+/ or die "$set ?"; > $stash_name = $&; > > (Do you intend to split on the first `.' ?) I don't expect this names to ever have more than one dot, this is not something that changes, so there should really be only one dot (or none in the MANIFEST case). I prefer the second one because it strips the ".", while the first one doesn't. (as you probably already guessed my perl/regexp is not very good) > > > +sub setup_netboot_installer () { > > + my $pxeimg = "$ho->{Tftp}{FreeBSDBase}/install-$ho->{Name}.img"; > > I think this would be better under TftpTmpDir. Cf ts-host-install's > placement of $ho--initrd.gz. > > Maybe we want a function target_tftp_prefix or something which returns > $ho->{Tftp}{TmpDir}".hostnamepath($ho) > ? Sounds fine, I guess I will add this as a pre-patch since there's already a consumer in ts-host-install. > Also the sha names could be in {TmpDir}/freebsd-images. Does this, in > fact, need to be configurable at all ? I think probably not. OK, so then I will just drop FreeBSDBase and just use $ho->{Tftp}{TmpDir}/freebsd-images. I don't think there's a lot of value in have it in a standard folder, since those are just random builds. I guess this makes more sense for Debian because they are actual releases, and thus can be used for other stuff also? > > + my $script = <<END; > > +set -ex > > +basedir=$ho->{Tftp}{Path}/$ho->{Tftp}{FreeBSDBase}/ > > +hash=`sha256sum $image|head -c 64` > > +localpath="$r{arch}/\$hash/install.img"; > > +mkdir -p \$basedir > > +( > > +flock -x -w 600 200 > > In osstest we generally use with-lock-ex from chiark-utils, rather > than flock from util-linux. (Bonus: it should be more portable for if > someone wants to run a controller on BSD...) > > with-lock-ex needs to be used in the wrapper way. > (Also, calling flock on a directory is rather outre'.) I don't have any problems using with-lock-ex, I just didn't know it existed :). > You've written all this in bash because you found it too hard or > annoying in Perl ? I don't much mind that, but the escaping is a bit > irritating. Maybe you want to pass the script its parameters as > arguments so that you can use <<'END' rather than <<END ? Hm, yes, I wasn't really confident of writing this in perl and being sure that the lock was always released. I'm more confident with shell. > However, > > > + logm("Executing:\n$script"); > > + my $ret = system("/bin/bash -c '$script'"); > > This is quite wrong. Your unparsing is unsound. Amazingly your > script doesn't currently contain ' so this doesn't matter right now, > but that could change. > > You should use the multi-argument form of system(3perl). And, of > course, you should use Osstest::system_checked which does the error > handling for you. > > You might want to build a command in an array @cmd, like copydir in > cr-publish-flight-logs does. (Not sure why that doesn't use > system_checked...) I've changed this to use an array with all the parameters and system_checked, it looks much better now IMHO, thanks. > > +sub install () { > > + my $authkeys = authorized_keys(); > > + my $knownhosts = known_hosts(); > > + my $sshd_keys_url = create_ssh_overlay(); > > + my $disk_names = "ada0 da0 ad0"; > > This should probably be > my @disk_names = qw(ada0 da0 ad0); > in case anyone wants to manipulate it in perl. You can substitute > it straight into the here document; perl will put in the " " again. > > > + my $installer_sets = join(" ", @sets); > > + my $target_sets = "/tmp/osstest_sets"; > > Hardcoded /tmp antipattern. Maybe this is technicallty OK because > it's an installer environment, but I think it sets a very bad > example. Is there some other path you could use ? I'm open to suggestions. We could also use ~/osstest_sets. I don't really have a preference. I've used /tmp because I though it would be less controversial. > > + target_cmd_root($ho, 'chsh -s /bin/sh', 10); > > !! What's the default ? csh. > > +for disk in $disk_names; do > > + if [ -c "/dev/\$disk" ]; then > > + echo \$disk > > + exit 0 > > + fi > > +done > > +exit 1 > > +END > > + defined($disk) or die "Unable to find a valid disk"; > > + logm("Using $disk as destination disk device"); > > I have found that on some hosts, when installing Debian GNU/Linux, I > have to expect a nonstandard disk name. Currently in the DB I can > only find this > hydrazine disk-device /dev/cciss/c0d0 > (in Cambridge; referring to a gone-away machine; NB that the property > name is in the obsolete containing-spaces syntax and is equivalent to > DiskDevice.) > > I think you may want to check a host property. It should probably be > called Freebsd_DiskDevice or something. (Weird capitalisation required > by the word splitting name transformation rules for host propertiess.) Yes, I can do that, but we would have to fill the DB manually. Would you be OK with leaving this as-is in this patch and me adding the property fetching later on? > > + logm("Trying to figure out primary nic device name"); > > + $nic = target_cmd_output_root($ho, <<END, 30); > > +nics=`ifconfig -l` > > +for nic in \$nics; do > > + addr=`ifconfig \$nic inet|grep inet|awk {'print \$2'}` > > + if [ "\$addr" = "$ho->{Ip}" ]; then > > + echo \$nic > > + exit 0 > > + fi > > +done > > +exit 1 > > I have quite a lot of this kind of thing: > > gall-mite 'interface force' eth0 > itch-mite 'interface force' eth0 > grain-weevil 'interface force' eth1 > rice-weevil 'interface force' eth1 > arndale-bluewater Interface_Force eth0 > arndale-lakeside Interface_Force eth0 > arndale-metrocentre Interface_Force eth0 > arndale-westfield Interface_Force eth0 > > This may not be needed for FreeBSD, but I thought I would mention it. > > What would happen if you tried to run this setup on a host where > FreeBSD's idea of the first network interface is not the one which > osstest did the install on ? FreeBSD doesn't really have an idea of the first network interface (unless you count the order in the output from ifconfig -l). Also, on FreeBSD each driver has it's own device, with different name, ie: there are em0, em1, bge0, bce0... interfaces. We could add a host property like Freebsd_NicDevice, but this fallback should stay in case the parameter is not defined? > IIRC there is a way for pxelinux to pass the NIC MAC address to the OS > it is running. Also, the host database contains the MAC address of > the host (and this is generally necessary for osstest to work even in > standalone mode I think), so you might use that rather than the IP > address ? This is not suitable in this case because from FreeBSD installer PoV it has been booted from a disk (because memdisk is used), so none of the PXE variables are propagated. > > + foreach (get_sets_path()) { > > + target_putfile_root($ho, 600, $_->{path}, "$target_sets/$_->{name}" > ); > > Needs linewrap. > > > + logm("Creating the installer script"); > > + target_cmd_root($ho, <<END, 60); > > + set -e > > + cat << ENDSCRIPT > installscript > > OK, my brain is fully bent now. Can we not create installscript on > the controller and transfer it with target_putfilecontents_root_stash ? > That way the logs would contain a copy, too. Yes, that's much better, thanks! > > +# Setup serial console > > +printf "%s" "-h -S$c{Baud}" >> /boot.config > > +cat << ENDBOOT >> /boot/loader.conf > > +boot_serial="YES" > > +comconsole_speed="$c{Baud}" > > +console="comconsole" > > +boot_verbose="YES" > > +beastie_disable="YES" > > +ENDBOOT > > Where does the installer's output go ? Ie, does booting the installer > image produce serial log output ? Yes, the installer image is built with serial output already. The output of the installer itself (bsdinstall) goes to the job log file. Thanks, Roger.
Roger Pau Monne writes ("Re: [PATCH 4/7] osstest: add a FreeBSD host install recipe"):
> OK, so then I will just drop FreeBSDBase and just use
> $ho->{Tftp}{TmpDir}/freebsd-images. I don't think there's a lot of
> value in have it in a standard folder, since those are just random
> builds. I guess this makes more sense for Debian because they are
> actual releases, and thus can be used for other stuff also?
Yes. Indeed they might not be managed by osstest at all.
> > > + my $installer_sets = join(" ", @sets);
> > > + my $target_sets = "/tmp/osstest_sets";
> >
> > Hardcoded /tmp antipattern. Maybe this is technicallty OK because
> > it's an installer environment, but I think it sets a very bad
> > example. Is there some other path you could use ?
>
> I'm open to suggestions. We could also use ~/osstest_sets. I don't
> really have a preference. I've used /tmp because I though it would be
> less controversial.
Hah :-).
I'm fine with almost any path other than /tmp/FIXED_STRING.
> > > + target_cmd_root($ho, 'chsh -s /bin/sh', 10);
> >
> > !! What's the default ?
>
> csh.
omg wtf bbq. Right. Fine.
> > I have found that on some hosts, when installing Debian GNU/Linux, I
> > have to expect a nonstandard disk name. Currently in the DB I can
> > only find this
> > hydrazine disk-device /dev/cciss/c0d0
> > (in Cambridge; referring to a gone-away machine; NB that the property
> > name is in the obsolete containing-spaces syntax and is equivalent to
> > DiskDevice.)
> >
> > I think you may want to check a host property. It should probably be
> > called Freebsd_DiskDevice or something. (Weird capitalisation required
> > by the word splitting name transformation rules for host propertiess.)
>
> Yes, I can do that, but we would have to fill the DB manually. Would
> you be OK with leaving this as-is in this patch and me adding the
> property fetching later on?
OK. Hopefully it won't come up.
> > What would happen if you tried to run this setup on a host where
> > FreeBSD's idea of the first network interface is not the one which
> > osstest did the install on ?
>
> FreeBSD doesn't really have an idea of the first network interface
> (unless you count the order in the output from ifconfig -l).
>
> Also, on FreeBSD each driver has it's own device, with different name,
> ie: there are em0, em1, bge0, bce0... interfaces. We could add
> a host property like Freebsd_NicDevice, but this fallback should stay
> in case the parameter is not defined?
I think this is too complicated and hypothetical. Let's leave it for
now, as you have it.
> > > + logm("Creating the installer script");
> > > + target_cmd_root($ho, <<END, 60);
> > > + set -e
> > > + cat << ENDSCRIPT > installscript
> >
> > OK, my brain is fully bent now. Can we not create installscript on
> > the controller and transfer it with target_putfilecontents_root_stash ?
> > That way the logs would contain a copy, too.
>
> Yes, that's much better, thanks!
:-)
> > > +# Setup serial console
> > > +printf "%s" "-h -S$c{Baud}" >> /boot.config
> > > +cat << ENDBOOT >> /boot/loader.conf
> > > +boot_serial="YES"
> > > +comconsole_speed="$c{Baud}"
> > > +console="comconsole"
> > > +boot_verbose="YES"
> > > +beastie_disable="YES"
> > > +ENDBOOT
> >
> > Where does the installer's output go ? Ie, does booting the installer
> > image produce serial log output ?
>
> Yes, the installer image is built with serial output already.
>
> The output of the installer itself (bsdinstall) goes to the job log
> file.
Great. Looking good :-).
Thanks,
Ian./
diff --git a/Osstest.pm b/Osstest.pm index a78728cd..a0c0339c 100644 --- a/Osstest.pm +++ b/Osstest.pm @@ -227,6 +227,7 @@ sub readglobalconfig () { $c{TftpTmpDir} ||= "$c{TftpPlayDir}tmp/"; $c{TftpDiBase} ||= "$c{TftpPlayDir}debian-installer"; + $c{TftpFreeBSDBase} ||= "$c{TftpPlayDir}freebsd-installer"; $c{TftpDiVersion} ||= $c{ "TftpDiVersion_$c{DebianSuite}" } // 'current'; $c{TftpGrubBase} ||= "$c{TftpPlayDir}grub"; diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index 8c7078c5..4418f024 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -118,7 +118,7 @@ BEGIN { await_webspace_fetch_byleaf create_webfile file_link_contents get_timeout setup_netboot_di setup_netboot_local host_netboot_file - subst_netboot_template + subst_netboot_template setup_netboot_memdisk ether_prefix @@ -1032,7 +1032,7 @@ sub selecthost ($) { $ho->{Tftp} = { }; $ho->{Tftp}{$_} = $c{"Tftp${_}_${tftpscope}"} || $c{"Tftp${_}"} foreach qw(Path TmpDir PxeDir NetbootGroup PxeTemplates PxeTemplatesReal - DiBase GrubBase + DiBase GrubBase FreeBSDBase NetGrubDir NetGrubTemplates NetGrubTemplatesReal PxeGroup); $ho->{TftpNetbootGroup} //= $ho->{TftpPxeGroup}; # compatibility @@ -2553,6 +2553,20 @@ default local END } +sub setup_netboot_memdisk ($$) { + my ($ho, $img) = @_; + setup_netboot_bootcfg($ho, <<END); +serial 0 $c{Baud} +timeout 5 +label overwrite + menu label ^Overwrite + menu default + kernel memdisk + append initrd=$img +default overwrite +END +} + # uboot emulates pxelinux, so reuse BIOS stuff sub setup_netboot_di_uboot ($$$$$;%) { return &setup_netboot_di_bios; } sub setup_netboot_local_uboot ($) { return &setup_netboot_local_bios; } diff --git a/ts-freebsd-host-install b/ts-freebsd-host-install new file mode 100755 index 00000000..d59b66a4 --- /dev/null +++ b/ts-freebsd-host-install @@ -0,0 +1,259 @@ +#!/usr/bin/perl -w +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2017 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +use strict qw(vars); +use DBI; +use POSIX; + +unshift @INC, qw(.); +use Osstest; +use Osstest::TestSupport; + +tsreadconfig(); + +our %xopts; + +our ($whhost) = @ARGV; +$whhost ||= 'host'; +our $ho= selecthost($whhost); +exit 0 if $ho->{Flags}{'no-reinstall'}; +exit 0 if $ho->{SharedReady}; + +our %timeout= qw(Sshd 1000); + +our @sets = qw(base.txz kernel.txz); + +our $image = $r{"freebsd_image"} || + get_stashed("path_freebsd-image", $r{"freebsd_buildjob"}); + +sub get_sets_path () { + my @paths; + + foreach (@sets, "MANIFEST") { + my $path; + + if (!defined($r{"freebsd_sets"})) { + # Get everything before the "." (ie: get base from base.txz) + my $stash_name = lc((split /\./)[0]); + $path = get_stashed("path_freebsd-$stash_name", + $r{"freebsd_buildjob"}); + } else { + $path = $r{"freebsd_sets"} . "/$_"; + } + + push @paths, { name => "$_", path => "$path" }; + } + + return @paths; +} + +sub create_ssh_overlay () { + my $url = create_webfile($ho, "ssh.tar", sub { + my ($fh) = @_; + contents_make_cpio($fh, 'ustar', "$c{OverlayLocal}/etc/ssh/"); + }); + + return $url; +} + +sub setup_netboot_installer () { + my $pxeimg = "$ho->{Tftp}{FreeBSDBase}/install-$ho->{Name}.img"; + my $script = <<END; +set -ex +basedir=$ho->{Tftp}{Path}/$ho->{Tftp}{FreeBSDBase}/ +hash=`sha256sum $image|head -c 64` +localpath="$r{arch}/\$hash/install.img"; +mkdir -p \$basedir +( +flock -x -w 600 200 +cd \$basedir +if [ ! -f \$localpath ]; then + mkdir -p `dirname \$localpath` + cp $image \$localpath +fi +rm -f install-$ho->{Name}.img +ln \$localpath install-$ho->{Name}.img +for hash in `ls $r{arch}/`; do + count=`stat -c %h $r{arch}/\$hash/install.img` + if [ \$count -eq 1 ]; then + rm -rf $r{arch}/\$hash + fi +done +) 200<\$basedir +END + + logm("Executing:\n$script"); + my $ret = system("/bin/bash -c '$script'"); + $ret == 0 or die "Unable to setup netboot"; + + # Setup the pxelinux config file + logm("Booting from installer image at $pxeimg"); + setup_netboot_memdisk($ho, $pxeimg); +} + +sub install () { + my $authkeys = authorized_keys(); + my $knownhosts = known_hosts(); + my $sshd_keys_url = create_ssh_overlay(); + my $disk_names = "ada0 da0 ad0"; + my $installer_sets = join(" ", @sets); + my $target_sets = "/tmp/osstest_sets"; + my $disk; + my $nic; + + target_cmd_root($ho, 'chsh -s /bin/sh', 10); + + logm("Trying to find a disk to install to"); + $disk = target_cmd_output_root($ho, <<END, 30); +for disk in $disk_names; do + if [ -c "/dev/\$disk" ]; then + echo \$disk + exit 0 + fi +done +exit 1 +END + defined($disk) or die "Unable to find a valid disk"; + logm("Using $disk as destination disk device"); + + logm("Trying to figure out primary nic device name"); + $nic = target_cmd_output_root($ho, <<END, 30); +nics=`ifconfig -l` +for nic in \$nics; do + addr=`ifconfig \$nic inet|grep inet|awk {'print \$2'}` + if [ "\$addr" = "$ho->{Ip}" ]; then + echo \$nic + exit 0 + fi +done +exit 1 +END + defined($nic) or die "Unable to find primary network interface"; + logm("Using $nic as primary network interface"); + + logm("Uploading the install sets to the system"); + target_cmd_root($ho, <<END, 30); +mkdir -p $target_sets +mount -o size=1G -t tmpfs tmpfs $target_sets +END + + foreach (get_sets_path()) { + target_putfile_root($ho, 600, $_->{path}, "$target_sets/$_->{name}"); + } + + logm("Creating the installer script"); + target_cmd_root($ho, <<END, 60); + set -e + cat << ENDSCRIPT > installscript +set -a +BSDINSTALL_DISTDIR="$target_sets" +ZFSBOOT_DISKS="$disk" +DISTRIBUTIONS="$installer_sets" +nonInteractive=1 + +#!/bin/sh +set -ex + +# Setup nic and sshd +echo "ifconfig_$nic=DHCP" >> /etc/rc.conf +echo "sshd_enable=YES" >> /etc/rc.conf + +# Disable sendmail +echo "sendmail_enable=NONE" >> /etc/rc.conf + +# Set proxy for the pkg manager +mkdir -p /usr/local/etc/ +cat << ENDPKG >> /usr/local/etc/pkg.conf +pkg_env: { http_proxy = $c{HttpProxy} } +default_always_yes: true +assume_always_yes: true +ENDPKG + +# Bootstap the package manager +export HTTP_PROXY=$c{HttpProxy} +export ASSUME_ALWAYS_YES=yes +pkg update + +# Allow root user login and setup ssh keys +chsh -s /bin/sh root +echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config +mkdir -p /root/.ssh +cat << ENDKEYS > /root/.ssh/authorized_keys +$authkeys +ENDKEYS +cat << ENDHOSTS > /root/.ssh/known_hosts +$knownhosts +ENDHOSTS + +# Fetch host keys +fetch $sshd_keys_url -o - | tar -xf - -C /etc/ssh/ +# Set correct permissions +chown root:wheel /etc/ssh/ssh_host_*_key* +chmod 0600 /etc/ssh/ssh_host_*_key +chmod 0644 /etc/ssh/ssh_host_*_key.pub + +# Add a osstest user +pw useradd osstest -m +chsh -s /bin/sh osstest +mkdir -p /home/osstest/.ssh +cat << ENDKEYS > /home/osstest/.ssh/authorized_keys +$authkeys +ENDKEYS +cat << ENDHOSTS > /home/osstest/.ssh/known_hosts +$knownhosts +ENDHOSTS + +# Setup serial console +printf "%s" "-h -S$c{Baud}" >> /boot.config +cat << ENDBOOT >> /boot/loader.conf +boot_serial="YES" +comconsole_speed="$c{Baud}" +console="comconsole" +boot_verbose="YES" +beastie_disable="YES" +ENDBOOT + +ENDSCRIPT +END + + logm("Launch the installer"); + target_cmd_root($ho, 'bsdinstall script installscript', 1200); + + target_reboot($ho); + + logm("Waiting for the host to boot"); + await_tcp(get_timeout($ho,'reboot',$timeout{Sshd}), 14, $ho); + + logm("FreeBSD installed succesfully"); +} + +# Switch off, setup PXE and switch on to the installer +power_state($ho, 0); +setup_netboot_installer(); +power_cycle_sleep($ho); +power_state($ho, 1); + +# Wait for the host to finish booting +logm("Waiting for the installer to boot"); +await_tcp(get_timeout($ho,'reboot',$timeout{Sshd}), 14, $ho); + +# Next boot will be from local disk +setup_netboot_local($ho); + +# Proceed with the install +install(); +
The installation is performed using the bsdinstall tool, which is part of the FreeBSD base system. The installer image is setup with the osstest ssh keys and sshd enabled by default, which allows the test harness to just ssh into the box, create the install config file and launch the scripted install. Currently the installation is done with ZFS only, in stripe mode, and a single disk. In order to support the FreeBSD installer a new method is added, that allows setting the pxe boot of a host using a memdisk. Also, a new tftp path is added in order to point to the place where the FreeBSD installed images should be stored. The install script either picks the binary images from the output of a previous FreeBSD buildjob (yet to be introduced), or from the freebsd_image and freebsd_sets runvars, that should point to a FreeBSD installer image and to the folder that contain the install sets respectively. When relying on the output from a previous FreeBSD buildjob (freebsd_buildjob runvar is set), the install image is picked from the path_freebsd-image runvar, and the sets from path_freebsd-{base,kernel,manifest} of the previous job. Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> --- Osstest.pm | 1 + Osstest/TestSupport.pm | 18 +++- ts-freebsd-host-install | 259 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 276 insertions(+), 2 deletions(-) create mode 100755 ts-freebsd-host-install