diff mbox series

[v2,6/6] selftests: arm64: Add build and documentation for FP tests

Message ID 20200819114837.51466-7-broonie@kernel.org
State New
Headers show
Series selftests: arm64: Add floating point selftests | expand

Commit Message

Mark Brown Aug. 19, 2020, 11:48 a.m. UTC
Integrate the FP tests with the build system and add some documentation
for the ones run outside the kselftest infrastructure.  The content in
the README was largely written by Dave Martin with edits by me.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 tools/testing/selftests/arm64/Makefile      |   2 +-
 tools/testing/selftests/arm64/fp/.gitignore |   5 +
 tools/testing/selftests/arm64/fp/Makefile   |  17 ++++
 tools/testing/selftests/arm64/fp/README     | 100 ++++++++++++++++++++
 4 files changed, 123 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/arm64/fp/.gitignore
 create mode 100644 tools/testing/selftests/arm64/fp/Makefile
 create mode 100644 tools/testing/selftests/arm64/fp/README

Comments

Dave Martin Sept. 1, 2020, 3:38 p.m. UTC | #1
On Wed, Aug 19, 2020 at 12:48:37PM +0100, Mark Brown wrote:
> Integrate the FP tests with the build system and add some documentation
> for the ones run outside the kselftest infrastructure.  The content in
> the README was largely written by Dave Martin with edits by me.

Apologies, I never got around to looking at all this, though it seems
reasonable.


I don't know whether this is worth following up with a TODO?

Some things I was aware of:

 * The sve-test/fpsimd-test programs contain a lot of common
   boilerplate and could probably be merged together.

 * A fair amount of the asm in sve-test/fpsimd-test could be converted
   to C, with -fgeneral-regs-only.  This would be helpful since the
   code is highly unmaintainable in its current form (I know, I've
   tried).  Calling library functions would still be a problem, but we
   might be able to lift a printf implementation and some basic syscall
   wrappers from elsewhere rather than reimplementing everything from
   scratch.

 * The sve-stress/fpsimd-stress scripts could likewise be merged.
   Also, doing the required process management from the shell seems a
   doomed enterprise and it never really worked 100% right.  Eventually
   it might be worth rewriting a common test driver for these in a real
   language.

 * While the tests confirm that basic aspects of the SVE support don't
   explode, there is not a lot of checking that the kernel does the
   _correct_ thing -- so there's scope for improvement here if somebody
   gets around to it.

Cheers
---Dave
Mark Brown Sept. 1, 2020, 3:47 p.m. UTC | #2
On Tue, Sep 01, 2020 at 04:38:42PM +0100, Dave Martin wrote:

> I don't know whether this is worth following up with a TODO?

> Some things I was aware of:

Well volunteered :P

>  * The sve-test/fpsimd-test programs contain a lot of common
>    boilerplate and could probably be merged together.

>  * A fair amount of the asm in sve-test/fpsimd-test could be converted
>    to C, with -fgeneral-regs-only.  This would be helpful since the
>    code is highly unmaintainable in its current form (I know, I've
>    tried).  Calling library functions would still be a problem, but we
>    might be able to lift a printf implementation and some basic syscall
>    wrappers from elsewhere rather than reimplementing everything from
>    scratch.

Or just keep the existing asm for the syscall/print wrappers.

>  * The sve-stress/fpsimd-stress scripts could likewise be merged.
>    Also, doing the required process management from the shell seems a
>    doomed enterprise and it never really worked 100% right.  Eventually
>    it might be worth rewriting a common test driver for these in a real
>    language.

>  * While the tests confirm that basic aspects of the SVE support don't
>    explode, there is not a lot of checking that the kernel does the
>    _correct_ thing -- so there's scope for improvement here if somebody
>    gets around to it.

Yeah, more errors get trapped by the kernel's own internal checking than
by the tests themselves.
Dave Martin Sept. 1, 2020, 4:06 p.m. UTC | #3
On Tue, Sep 01, 2020 at 04:47:02PM +0100, Mark Brown wrote:
> On Tue, Sep 01, 2020 at 04:38:42PM +0100, Dave Martin wrote:
> 
> > I don't know whether this is worth following up with a TODO?
> 
> > Some things I was aware of:
> 
> Well volunteered :P
> 
> >  * The sve-test/fpsimd-test programs contain a lot of common
> >    boilerplate and could probably be merged together.
> 
> >  * A fair amount of the asm in sve-test/fpsimd-test could be converted
> >    to C, with -fgeneral-regs-only.  This would be helpful since the
> >    code is highly unmaintainable in its current form (I know, I've
> >    tried).  Calling library functions would still be a problem, but we
> >    might be able to lift a printf implementation and some basic syscall
> >    wrappers from elsewhere rather than reimplementing everything from
> >    scratch.
> 
> Or just keep the existing asm for the syscall/print wrappers.
> 
> >  * The sve-stress/fpsimd-stress scripts could likewise be merged.
> >    Also, doing the required process management from the shell seems a
> >    doomed enterprise and it never really worked 100% right.  Eventually
> >    it might be worth rewriting a common test driver for these in a real
> >    language.
> 
> >  * While the tests confirm that basic aspects of the SVE support don't
> >    explode, there is not a lot of checking that the kernel does the
> >    _correct_ thing -- so there's scope for improvement here if somebody
> >    gets around to it.
> 
> Yeah, more errors get trapped by the kernel's own internal checking than
> by the tests themselves.


OK, I can follow up with a patch so long as these points sounds
reasonable to you.  Either way, it's not urgent.

Cheers
---Dave
diff mbox series

Patch

diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile
index 93b567d23c8b..a28d4e7c9948 100644
--- a/tools/testing/selftests/arm64/Makefile
+++ b/tools/testing/selftests/arm64/Makefile
@@ -4,7 +4,7 @@ 
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),aarch64 arm64))
-ARM64_SUBTARGETS ?= tags signal
+ARM64_SUBTARGETS ?= tags signal fp
 else
 ARM64_SUBTARGETS :=
 endif
diff --git a/tools/testing/selftests/arm64/fp/.gitignore b/tools/testing/selftests/arm64/fp/.gitignore
new file mode 100644
index 000000000000..d66f76d2a650
--- /dev/null
+++ b/tools/testing/selftests/arm64/fp/.gitignore
@@ -0,0 +1,5 @@ 
+fpsimd-test
+sve-probe-vls
+sve-ptrace
+sve-test
+vlset
diff --git a/tools/testing/selftests/arm64/fp/Makefile b/tools/testing/selftests/arm64/fp/Makefile
new file mode 100644
index 000000000000..a57009d3a0dc
--- /dev/null
+++ b/tools/testing/selftests/arm64/fp/Makefile
@@ -0,0 +1,17 @@ 
+# SPDX-License-Identifier: GPL-2.0
+
+CFLAGS += -I../../../../../usr/include/
+TEST_GEN_PROGS := sve-ptrace sve-probe-vls
+TEST_PROGS_EXTENDED := fpsimd-test fpsimd-stress sve-test sve-stress vlset
+
+all: $(TEST_GEN_PROGS) $(TEST_PROGS_EXTENDED)
+
+fpsimd-test: fpsimd-test.o
+	$(CC) -nostdlib $^ -o $@
+sve-ptrace: sve-ptrace.o sve-ptrace-asm.o
+sve-probe-vls: sve-probe-vls.o
+sve-test: sve-test.o
+	$(CC) -nostdlib $^ -o $@
+vlset: vlset.o
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/arm64/fp/README b/tools/testing/selftests/arm64/fp/README
new file mode 100644
index 000000000000..03e3dad865d8
--- /dev/null
+++ b/tools/testing/selftests/arm64/fp/README
@@ -0,0 +1,100 @@ 
+This directory contains a mix of tests integrated with kselftest and
+standalone stress tests.
+
+kselftest tests
+===============
+
+sve-probe-vls - Checks the SVE vector length enumeration interface
+sve-ptrace - Checks the SVE ptrace interface
+
+Running the non-kselftest tests
+===============================
+
+sve-stress performs an SVE context switch stress test, as described
+below.
+
+(The fpsimd-stress test works the same way; just substitute "fpsimd" for
+"sve" in the following commands.)
+
+
+The test runs until killed by the user.
+
+If no context switch error was detected, you will see output such as
+the following:
+
+$ ./sve-stress
+(wait for some time)
+^C
+Vector length:        512 bits
+PID:    1573
+Terminated by signal 15, no error, iterations=9467, signals=1014
+Vector length:  512 bits
+PID:    1575
+Terminated by signal 15, no error, iterations=9448, signals=1028
+Vector length:  512 bits
+PID:    1577
+Terminated by signal 15, no error, iterations=9436, signals=1039
+Vector length:  512 bits
+PID:    1579
+Terminated by signal 15, no error, iterations=9421, signals=1039
+Vector length:  512 bits
+PID:    1581
+Terminated by signal 15, no error, iterations=9403, signals=1039
+Vector length:  512 bits
+PID:    1583
+Terminated by signal 15, no error, iterations=9385, signals=1036
+Vector length:  512 bits
+PID:    1585
+Terminated by signal 15, no error, iterations=9376, signals=1039
+Vector length:  512 bits
+PID:    1587
+Terminated by signal 15, no error, iterations=9361, signals=1039
+Vector length:  512 bits
+PID:    1589
+Terminated by signal 15, no error, iterations=9350, signals=1039
+
+
+If an error was detected, details of the mismatch will be printed
+instead of "no error".
+
+Ideally, the test should be allowed to run for many minutes or hours
+to maximise test coverage.
+
+
+KVM stress testing
+==================
+
+To try to reproduce the bugs that we have been observing, sve-stress
+should be run in parallel in two KVM guests, while simultaneously
+running on the host.
+
+1) Start 2 guests, using the following command for each:
+
+$ lkvm run --console=virtio -pconsole=hvc0 --sve Image
+
+(Depending on the hardware GIC implementation, you may also need
+--irqchip=gicv3.  New kvmtool defaults to that if appropriate, but I
+can't remember whether my branch is new enough for that.  Try without
+the option first.)
+
+Kvmtool occupies the terminal until you kill it (Ctrl+A x),
+or until the guest terminates.  It is therefore recommended to run
+each instance in separate terminal (use screen or ssh etc.)  This
+allows multiple guests to be run in parallel while running other
+commands on the host.
+
+Within the guest, the host filesystem is accessible, mounted on /host.
+
+2) Run the sve-stress on *each* guest with the Vector-Length set to 32:
+guest$ ./vlset --inherit 32 ./sve-stress
+
+3) Run the sve-stress on the host with the maximum Vector-Length:
+host$ ./vlset --inherit --max ./sve-stress
+
+
+Again, the test should be allowed to run for many minutes or hours to
+maximise test coverage.
+
+If no error is detected, you will see output from each sve-stress
+instance similar to that illustrated above; otherwise details of the
+observed mismatches will be printed.