[OSSTEST,v2,00/49] Switch to Debian buster (= Debian stable)
mbox series

Message ID 20200529111945.21394-1-ian.jackson@eu.citrix.com
Headers show
Series
  • Switch to Debian buster (= Debian stable)
Related show

Message

Ian Jackson May 29, 2020, 11:18 a.m. UTC
This series looks about as ready as it is going to be.  Unfortunately
there are still two issues, each of which cropped up once in my final
formal retest.  See below.

What are people's opinions?  Should I push this to osstest pretest
soon after the Xen codefreeze (eg, after we get the first push after
freeze) ?

The downside would be introducing new low-probability heisenbugs.  But
in my tests it seems that some existing heisenbugs may be abated.
It's hard to say for sure.  The upside is a lower risk that Debian
will break our stuff and also being able to commission a number of
machines which do not work with stretch (Debian oldstable).

The two issues:

1. In one test, libvirt segfaulted.  This is with osstest's own-built
   tested version of libvirt; osstest's own-build tested version of
   Linux; and osstest's own-built xen.git#master.

   http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-i386-libvirt/info.html
   http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-i386-libvirt/rimava1---var-log-kern.log.gz
   | May 28 21:39:11 rimava1 kernel: [ 1146.326593] libvirtd[16868]: segfault at 820 ip b72535b0 sp abce5090 error 4 in libc-2.28.so[b71f0000+14e000]

2. In one test, the Debian installer hung.  The kernel printed many
   soft lockup messages.  This is with osstest's own dom0 kernel and
   hypervisor, but the 4.19.67-2+deb10u1 kernel from Debian.

   http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-amd64-xl-qemut-debianhvm-i386-xsm/info.html
   http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-amd64-xl-qemut-debianhvm-i386-xsm/godello1---var-log-xen-qemu-dm-debianhvm.guest.osstest.log.gz

Ian Jackson (49):
  ts-logs-capture: Cope if xl shutdown leaves domain running for a bit
  ts-xen-build-prep: Install rsync
  lvcreate argments: pass --yes -Z y -W y
  TestSupport: allow more time for apt
  Booting: Use `--' rather than `---' to introduce host cmdline
  di_installcmdline_core: Pass locale on d-i command line
  setupboot_grub2: Drop $submenu variable
  ts-leak-check: Ignore buster's udevd too
  Bodge systemd random seed arrangements
  Debian guests made with xen-tools: Write systemd random seed file
  ts-debian-di-install: Provide guest with more RAM
  Debian: preseed: use priority= alias
  Debian: Specify `priority=critical' rather than locale
  Honour 'LinuxSerialConsole <suite>' host property
  buster: make-hosts-flight: Add to possible suites for hosts flight
  buster: Extend grub2 uefi no install workaround
  buster: ts-host-install: Extend net.ifnames workaround
  buster: Deinstall the "systemd" package
  buster: preseed partman-auto-lvm/guided_size
  buster: ts-host-install: NTP not honoured bug remains
  buster: Extend ARM clock workaround
  buster: Extend guest bootloader workaround
  Honour DebianImageFile_SUITE_ARCH
  buster: Specify DebianImageFile_SUITE_ARCH
  20_linux_xen: Copy Debian buster version into our initramfs area
  20_linux_xen: Adhoc template substitution
  20_linux_xen: Ignore xenpolicy and config files too
  20_linux_xen: Support Xen Security Modules (XSM/FLASK)
  mg-debian-installer-update: support overlay-intramfs-SUITE
  overlay-initrd-buster/sbin/reopen-console: Copy from Debian
  overlay-initrd-buster/sbin/reopen-console: Fix #932416
  buster: chiark-scripts: Install a new version on buster too
  buster: Provide TftpDiVersion
  buster: grub, arm64: extend chainloading workaround
  buster: setupboot_grub2: Note what files exist in /boot
  buster: setupboot_grub2: Handle missing policy file bug
  buster: Extend workaround for dhcpd EROFS bug
  ts-xen-install: Add $ho argument to some_extradebs
  ts-xen-install: Move some_extradebs to Debian.pm
  Debian.pm: Break out standard_extradebs
  Debian.pm: Move standard_extradebs to ts-host-install
  buster: Install own linux-libc-dev package (!)
  setupboot_grub2: Insist on space after directives
  setupboot_grub2: Print line number of entry we are using
  setupboot_grub2: Recognise --nounzip for initramfs
  setupboot_grub2: Copy hv command line from grub to xen.cfg
  setupboot_grub2: Do not boot with XSM policy etc. unless xsm=1
  buster 20_linux_xen: Only load policy in XSM-enabled builds
  buster: Switch to Debian buster as the default suite

 Osstest.pm                                    |   2 +-
 Osstest/Debian.pm                             | 128 +++++--
 Osstest/TestSupport.pm                        |  15 +-
 make-hosts-flight                             |   2 +-
 mfi-common                                    |   9 +
 mg-debian-installer-update                    |  20 ++
 overlay-buster/etc/grub.d/20_linux_xen        | 327 ++++++++++++++++++
 overlay-initrd-buster/sbin/reopen-console     | 126 +++++++
 .../override.conf                             |   3 +
 overlay/usr/local/bin/random-seed-add         |  33 ++
 production-config                             |   5 +
 ts-debian-di-install                          |   2 +-
 ts-debian-fixup                               |   6 +
 ts-host-install                               |   6 +-
 ts-leak-check                                 |   1 +
 ts-logs-capture                               |   1 +
 ts-xen-build-prep                             |   6 +-
 ts-xen-install                                |  41 +--
 18 files changed, 656 insertions(+), 77 deletions(-)
 create mode 100755 overlay-buster/etc/grub.d/20_linux_xen
 create mode 100755 overlay-initrd-buster/sbin/reopen-console
 create mode 100644 overlay/etc/systemd/system/systemd-random-seed.service.d/override.conf
 create mode 100755 overlay/usr/local/bin/random-seed-add

Comments

Paul Durrant May 29, 2020, 11:30 a.m. UTC | #1
> -----Original Message-----
> From: Ian Jackson <ian.jackson@eu.citrix.com>
> Sent: 29 May 2020 12:19
> To: xen-devel@lists.xenproject.org; Paul Durrant <paul@xen.org>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>; committers@xenproject.org
> Subject: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)
> 
> This series looks about as ready as it is going to be.  Unfortunately
> there are still two issues, each of which cropped up once in my final
> formal retest.  See below.
> 
> What are people's opinions?  Should I push this to osstest pretest
> soon after the Xen codefreeze (eg, after we get the first push after
> freeze) ?
> 

That sounds reasonable; I don't think we want to perturb osstest until after the first push.

> The downside would be introducing new low-probability heisenbugs.  But
> in my tests it seems that some existing heisenbugs may be abated.
> It's hard to say for sure.  The upside is a lower risk that Debian
> will break our stuff and also being able to commission a number of
> machines which do not work with stretch (Debian oldstable).
> 

I assume we can revert if things go badly wrong and being able to commission more machines would seem to be quite beneficial at this
stage.

  Paul

> The two issues:
> 
> 1. In one test, libvirt segfaulted.  This is with osstest's own-built
>    tested version of libvirt; osstest's own-build tested version of
>    Linux; and osstest's own-built xen.git#master.
> 
>    http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-i386-libvirt/info.html
>    http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-i386-libvirt/rimava1---var-log-
> kern.log.gz
>    | May 28 21:39:11 rimava1 kernel: [ 1146.326593] libvirtd[16868]: segfault at 820 ip b72535b0 sp
> abce5090 error 4 in libc-2.28.so[b71f0000+14e000]
> 
> 2. In one test, the Debian installer hung.  The kernel printed many
>    soft lockup messages.  This is with osstest's own dom0 kernel and
>    hypervisor, but the 4.19.67-2+deb10u1 kernel from Debian.
> 
>    http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-amd64-xl-qemut-debianhvm-i386-
> xsm/info.html
>    http://logs.test-lab.xenproject.org/osstest/logs/150456/test-amd64-amd64-xl-qemut-debianhvm-i386-
> xsm/godello1---var-log-xen-qemu-dm-debianhvm.guest.osstest.log.gz
> 
> Ian Jackson (49):
>   ts-logs-capture: Cope if xl shutdown leaves domain running for a bit
>   ts-xen-build-prep: Install rsync
>   lvcreate argments: pass --yes -Z y -W y
>   TestSupport: allow more time for apt
>   Booting: Use `--' rather than `---' to introduce host cmdline
>   di_installcmdline_core: Pass locale on d-i command line
>   setupboot_grub2: Drop $submenu variable
>   ts-leak-check: Ignore buster's udevd too
>   Bodge systemd random seed arrangements
>   Debian guests made with xen-tools: Write systemd random seed file
>   ts-debian-di-install: Provide guest with more RAM
>   Debian: preseed: use priority= alias
>   Debian: Specify `priority=critical' rather than locale
>   Honour 'LinuxSerialConsole <suite>' host property
>   buster: make-hosts-flight: Add to possible suites for hosts flight
>   buster: Extend grub2 uefi no install workaround
>   buster: ts-host-install: Extend net.ifnames workaround
>   buster: Deinstall the "systemd" package
>   buster: preseed partman-auto-lvm/guided_size
>   buster: ts-host-install: NTP not honoured bug remains
>   buster: Extend ARM clock workaround
>   buster: Extend guest bootloader workaround
>   Honour DebianImageFile_SUITE_ARCH
>   buster: Specify DebianImageFile_SUITE_ARCH
>   20_linux_xen: Copy Debian buster version into our initramfs area
>   20_linux_xen: Adhoc template substitution
>   20_linux_xen: Ignore xenpolicy and config files too
>   20_linux_xen: Support Xen Security Modules (XSM/FLASK)
>   mg-debian-installer-update: support overlay-intramfs-SUITE
>   overlay-initrd-buster/sbin/reopen-console: Copy from Debian
>   overlay-initrd-buster/sbin/reopen-console: Fix #932416
>   buster: chiark-scripts: Install a new version on buster too
>   buster: Provide TftpDiVersion
>   buster: grub, arm64: extend chainloading workaround
>   buster: setupboot_grub2: Note what files exist in /boot
>   buster: setupboot_grub2: Handle missing policy file bug
>   buster: Extend workaround for dhcpd EROFS bug
>   ts-xen-install: Add $ho argument to some_extradebs
>   ts-xen-install: Move some_extradebs to Debian.pm
>   Debian.pm: Break out standard_extradebs
>   Debian.pm: Move standard_extradebs to ts-host-install
>   buster: Install own linux-libc-dev package (!)
>   setupboot_grub2: Insist on space after directives
>   setupboot_grub2: Print line number of entry we are using
>   setupboot_grub2: Recognise --nounzip for initramfs
>   setupboot_grub2: Copy hv command line from grub to xen.cfg
>   setupboot_grub2: Do not boot with XSM policy etc. unless xsm=1
>   buster 20_linux_xen: Only load policy in XSM-enabled builds
>   buster: Switch to Debian buster as the default suite
> 
>  Osstest.pm                                    |   2 +-
>  Osstest/Debian.pm                             | 128 +++++--
>  Osstest/TestSupport.pm                        |  15 +-
>  make-hosts-flight                             |   2 +-
>  mfi-common                                    |   9 +
>  mg-debian-installer-update                    |  20 ++
>  overlay-buster/etc/grub.d/20_linux_xen        | 327 ++++++++++++++++++
>  overlay-initrd-buster/sbin/reopen-console     | 126 +++++++
>  .../override.conf                             |   3 +
>  overlay/usr/local/bin/random-seed-add         |  33 ++
>  production-config                             |   5 +
>  ts-debian-di-install                          |   2 +-
>  ts-debian-fixup                               |   6 +
>  ts-host-install                               |   6 +-
>  ts-leak-check                                 |   1 +
>  ts-logs-capture                               |   1 +
>  ts-xen-build-prep                             |   6 +-
>  ts-xen-install                                |  41 +--
>  18 files changed, 656 insertions(+), 77 deletions(-)
>  create mode 100755 overlay-buster/etc/grub.d/20_linux_xen
>  create mode 100755 overlay-initrd-buster/sbin/reopen-console
>  create mode 100644 overlay/etc/systemd/system/systemd-random-seed.service.d/override.conf
>  create mode 100755 overlay/usr/local/bin/random-seed-add
> 
> --
> 2.20.1
Ian Jackson May 29, 2020, 1:22 p.m. UTC | #2
Paul Durrant writes ("RE: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)"):
> I assume we can revert if things go badly wrong and being able to commission more machines would seem to be quite beneficial at this
> stage.

Thanks for your opinion.

It would be possible to revert the final switch, yes.  Most of the
other changes are supposed to work with stretch too.

I haven't done a test run with the new code, but the old version of
Debian.  I will do that.  That will give confidence that we could, in
fact, revert things.

Ian.
Ian Jackson June 8, 2020, 3:34 p.m. UTC | #3
Ian Jackson writes ("RE: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)"):
> Paul Durrant writes ("RE: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)"):
> > I assume we can revert if things go badly wrong and being able to commission more machines would seem to be quite beneficial at this
> > stage.
> 
> Thanks for your opinion.
> 
> It would be possible to revert the final switch, yes.  Most of the
> other changes are supposed to work with stretch too.
> 
> I haven't done a test run with the new code, but the old version of
> Debian.  I will do that.  That will give confidence that we could, in
> fact, revert things.

I have now done this and it resulted in one bugfix.

So if you are still content to go ahead I would like to push this
series to osstest's own self-push-gate now.

Ian.
Paul Durrant June 8, 2020, 3:36 p.m. UTC | #4
> -----Original Message-----
> From: Ian Jackson <ian.jackson@citrix.com>
> Sent: 08 June 2020 16:34
> To: paul@xen.org
> Cc: xen-devel@lists.xenproject.org; committers@xenproject.org
> Subject: RE: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)
> 
> Ian Jackson writes ("RE: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)"):
> > Paul Durrant writes ("RE: [OSSTEST PATCH v2 00/49] Switch to Debian buster (= Debian stable)"):
> > > I assume we can revert if things go badly wrong and being able to commission more machines would
> seem to be quite beneficial at this
> > > stage.
> >
> > Thanks for your opinion.
> >
> > It would be possible to revert the final switch, yes.  Most of the
> > other changes are supposed to work with stretch too.
> >
> > I haven't done a test run with the new code, but the old version of
> > Debian.  I will do that.  That will give confidence that we could, in
> > fact, revert things.
> 
> I have now done this and it resulted in one bugfix.
> 
> So if you are still content to go ahead I would like to push this
> series to osstest's own self-push-gate now.
> 

Yes, I am content to go ahead.

  Paul