mbox series

[0/5] tools/tests: More cleanup for automation improvements

Message ID 20210615161905.9831-1-andrew.cooper3@citrix.com (mailing list archive)
Headers show
Series tools/tests: More cleanup for automation improvements | expand

Message

Andrew Cooper June 15, 2021, 4:19 p.m. UTC
This series takes us one step closer towards the "autotests" plan for
simplifying the running of unit and low level system tests.

With this series in place, all the tests which need running in dom0 have
working install/uninstall targets, so they can be packaged suitably.

Some questions, concerning future changes.

Ian: Presuambly OSSTest is going to complain if I stick a test- prefix on
depriv-fd-checker?  I've left that test alone for now, as it was the only
preexisting test with working install runes.

Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC).  While they are unit
tests, we still potentially want to run them in dom0 rather than the build
environment - particularly for x86_emulator which is heavily CPUID based and
wants to run on a wide set of hardware.  Any issues moving them off $(HOSTCC)?

Roger: vhpet isn't even wired into the build system, and seems non-trivial to
run in the first place.  How should "success" be judged?

Andrew Cooper (5):
  tools/tests: Drop obsolete mce-test infrastructure
  tools/tests: Drop run runes
  tests/resource: Rework Makefile
  tests/cpu-policy: Rework Makefile
  tests/xenstore: Rework Makefile

 .gitignore                                         |   2 -
 tools/misc/.gitignore                              |   1 +
 tools/misc/Makefile                                |   4 +
 tools/{tests/mce-test/tools => misc}/xen-mceinj.c  |  32 +--
 tools/tests/Makefile                               |   1 -
 tools/tests/cpu-policy/Makefile                    |  33 +--
 tools/tests/mce-test/Makefile                      |  12 -
 tools/tests/mce-test/README                        |  75 ------
 tools/tests/mce-test/cases/srao_llc/dom0/cases.sh  |  73 ------
 tools/tests/mce-test/cases/srao_llc/guest/cases.sh |  94 --------
 tools/tests/mce-test/cases/srao_llc/xen/cases.sh   |  69 ------
 tools/tests/mce-test/cases/srao_mem/dom0/cases.sh  |  73 ------
 tools/tests/mce-test/cases/srao_mem/guest/cases.sh |  94 --------
 tools/tests/mce-test/cases/srao_mem/xen/cases.sh   |  69 ------
 tools/tests/mce-test/cases/ucna_llc/dom0/cases.sh  |  72 ------
 tools/tests/mce-test/cases/ucna_llc/guest/cases.sh |  92 --------
 tools/tests/mce-test/cases/ucna_llc/xen/cases.sh   |  68 ------
 tools/tests/mce-test/config/setup.conf             |  24 --
 tools/tests/mce-test/lib/xen-mceinj-tool.sh        | 260 ---------------------
 tools/tests/mce-test/tools/Makefile                |  24 --
 tools/tests/mce-test/tools/README                  |  24 --
 tools/tests/resource/Makefile                      |  11 +-
 tools/tests/vpci/Makefile                          |   4 -
 tools/tests/x86_emulator/Makefile                  |   4 -
 tools/tests/xenstore/.gitignore                    |   1 +
 tools/tests/xenstore/Makefile                      |  31 ++-
 .../tests/xenstore/{xs-test.c => test-xenstore.c}  |   0
 27 files changed, 71 insertions(+), 1176 deletions(-)
 rename tools/{tests/mce-test/tools => misc}/xen-mceinj.c (97%)
 delete mode 100644 tools/tests/mce-test/Makefile
 delete mode 100644 tools/tests/mce-test/README
 delete mode 100644 tools/tests/mce-test/cases/srao_llc/dom0/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/srao_llc/guest/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/srao_llc/xen/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/srao_mem/dom0/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/srao_mem/guest/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/srao_mem/xen/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/ucna_llc/dom0/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/ucna_llc/guest/cases.sh
 delete mode 100644 tools/tests/mce-test/cases/ucna_llc/xen/cases.sh
 delete mode 100644 tools/tests/mce-test/config/setup.conf
 delete mode 100644 tools/tests/mce-test/lib/xen-mceinj-tool.sh
 delete mode 100644 tools/tests/mce-test/tools/Makefile
 delete mode 100644 tools/tests/mce-test/tools/README
 create mode 100644 tools/tests/xenstore/.gitignore
 rename tools/tests/xenstore/{xs-test.c => test-xenstore.c} (100%)

Comments

Jan Beulich June 16, 2021, 6:38 a.m. UTC | #1
On 15.06.2021 18:19, Andrew Cooper wrote:
> Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC).  While they are unit
> tests, we still potentially want to run them in dom0 rather than the build
> environment - particularly for x86_emulator which is heavily CPUID based and
> wants to run on a wide set of hardware.  Any issues moving them off $(HOSTCC)?

Well, yes, I'm afraid: If anything, we may need to build two binaries,
or build the one binary two different ways: The "run" (and "run32" for
the emulator harness) target wants a binary built with HOSTCC. The
install target (which prior to your series does nothing) indeed wants
building with CC. So maybe we want something like

install: HOSTCC:=$(CC)

plus suitable detection of whether the opposite set of objects are
presently in the build area, requiring a full rebuild? (Of course this
will work only as long as HOSTCC isn't used for any build time helper
binaries. See "x86emul: test AMX insns" for when this starts not to be
the case anymore for the emulator harness. So we'd need yet another
variable to express this detail.)

Jan
Andrew Cooper June 16, 2021, 1:22 p.m. UTC | #2
On 16/06/2021 07:38, Jan Beulich wrote:
> On 15.06.2021 18:19, Andrew Cooper wrote:
>> Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC).  While they are unit
>> tests, we still potentially want to run them in dom0 rather than the build
>> environment - particularly for x86_emulator which is heavily CPUID based and
>> wants to run on a wide set of hardware.  Any issues moving them off $(HOSTCC)?
> Well, yes, I'm afraid: If anything, we may need to build two binaries,
> or build the one binary two different ways: The "run" (and "run32" for
> the emulator harness) target wants a binary built with HOSTCC. The
> install target (which prior to your series does nothing) indeed wants
> building with CC. So maybe we want something like
>
> install: HOSTCC:=$(CC)
>
> plus suitable detection of whether the opposite set of objects are
> presently in the build area, requiring a full rebuild? (Of course this
> will work only as long as HOSTCC isn't used for any build time helper
> binaries. See "x86emul: test AMX insns" for when this starts not to be
> the case anymore for the emulator harness. So we'd need yet another
> variable to express this detail.)

Having slept on the problem overnight, I'm going to argue that HOSTCC is
conceptually wrong to use here in the first place.

In an arm64 environment, cross-compiling x86_64, this will explode
everywhere, and the fault is with using HOSTCC rather than CC.

HOSTCC is specifically for compiling utilities executed as part of the
build.  Tests, and particularly arch-specific ones like x86_emulate, are
not in this category.  Whether you happen to be able to run
test_x86_emulator in the build environment is a property of whether
you're cross-compiling.

For a non-cross-compiled builds, HOSTCC and CC are largely
interchangeable, and won't impact the ability to run the binary in the
build environment.

~Andrew
Jan Beulich June 16, 2021, 2:14 p.m. UTC | #3
On 16.06.2021 15:22, Andrew Cooper wrote:
> On 16/06/2021 07:38, Jan Beulich wrote:
>> On 15.06.2021 18:19, Andrew Cooper wrote:
>>> Jan/Roger: x86_emulator and vpci use $(HOSTCC) not $(CC).  While they are unit
>>> tests, we still potentially want to run them in dom0 rather than the build
>>> environment - particularly for x86_emulator which is heavily CPUID based and
>>> wants to run on a wide set of hardware.  Any issues moving them off $(HOSTCC)?
>> Well, yes, I'm afraid: If anything, we may need to build two binaries,
>> or build the one binary two different ways: The "run" (and "run32" for
>> the emulator harness) target wants a binary built with HOSTCC. The
>> install target (which prior to your series does nothing) indeed wants
>> building with CC. So maybe we want something like
>>
>> install: HOSTCC:=$(CC)
>>
>> plus suitable detection of whether the opposite set of objects are
>> presently in the build area, requiring a full rebuild? (Of course this
>> will work only as long as HOSTCC isn't used for any build time helper
>> binaries. See "x86emul: test AMX insns" for when this starts not to be
>> the case anymore for the emulator harness. So we'd need yet another
>> variable to express this detail.)
> 
> Having slept on the problem overnight, I'm going to argue that HOSTCC is
> conceptually wrong to use here in the first place.
> 
> In an arm64 environment, cross-compiling x86_64, this will explode
> everywhere, and the fault is with using HOSTCC rather than CC.

In principle, if there wasn't the massive amount of inline assembly,
and if the emulator wasn't just re-executing the instructions it is
asked to emulate, building all of this on Arm ought to be possible.
But with the code we have this simply makes no sense.

> HOSTCC is specifically for compiling utilities executed as part of the
> build.  Tests, and particularly arch-specific ones like x86_emulate, are
> not in this category.

Hmm, right now they definitely are. Running them directly (which is
their only purpose right now, with the install targets doing nothing)
from the build tree puts them into this category. But they aren't
anymore as soon as you want to install them. Hence the need to have
two modes here.

>  Whether you happen to be able to run
> test_x86_emulator in the build environment is a property of whether
> you're cross-compiling.

In a way, yes. I'd consider the run32 target to also be cross-like,
yet that binary can then still be run from the build tree (if the
distro supports 32-bit binaries).

> For a non-cross-compiled builds, HOSTCC and CC are largely
> interchangeable, and won't impact the ability to run the binary in the
> build environment.

Not exactly, I think. If I override CC but not HOSTCC (which I do
normally), I still don't need to worry about that other CC finding
all the bits and pieces it needs for building a host binary. For the
C compiler this may not mean much, but the C++ compiler (if we used
it and hence would need to treat it similarly) wants e.g. its own
copies of the headers, which may not be readily available. I will
admit that my environment may be pretty non-standard, as I run non-
default compilers (and binutils) also directly from their build
trees. But things have been working fine this way for all sorts of
projects, so I expect the test harness here to not break in this
regard.

Jan