diff mbox series

[v5,15/15] perf jevents: Run metric_test.py at compile-time

Message ID 20230126233645.200509-16-irogers@google.com (mailing list archive)
State New, archived
Headers show
Series jevents/pmu-events improvements | expand

Commit Message

Ian Rogers Jan. 26, 2023, 11:36 p.m. UTC 
Add a target that generates a log file for running metric_test.py and
make this a dependency on generating pmu-events.c. The log output is
displayed if the test fails like (the test was modified to make it
fail):

```
  TEST    /tmp/perf/pmu-events/metric_test.log
F......
======================================================================
FAIL: test_Brackets (__main__.TestMetricExpressions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tools/perf/pmu-events/metric_test.py", line 33, in test_Brackets
    self.assertEqual((a * b + c).ToPerfJson(), 'a * b + d')
AssertionError: 'a * b + c' != 'a * b + d'
- a * b + c
?         ^
+ a * b + d
?         ^

----------------------------------------------------------------------
Ran 7 tests in 0.004s

FAILED (failures=1)
make[3]: *** [pmu-events/Build:32: /tmp/perf/pmu-events/metric_test.log] Error 1
```

However, normal execution will just show the TEST line.

This is roughly modeled on fortify testing in the kernel lib directory.

Modify metric_test.py so that it is executable. This is necessary when
PYTHON isn't specified in the build, the normal case.

Use variables to make the paths to files clearer and more consistent.
---
 tools/perf/pmu-events/Build          | 13 +++++++++++--
 tools/perf/pmu-events/metric_test.py |  1 +
 2 files changed, 12 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 tools/perf/pmu-events/metric_test.py

Comments

Arnaldo Carvalho de Melo Feb. 3, 2023, 8:15 p.m. UTC | #1 
Em Thu, Jan 26, 2023 at 03:36:45PM -0800, Ian Rogers escreveu:
> Add a target that generates a log file for running metric_test.py and
> make this a dependency on generating pmu-events.c. The log output is
> displayed if the test fails like (the test was modified to make it
> fail):
> 
> ```
>   TEST    /tmp/perf/pmu-events/metric_test.log
> F......
> ======================================================================
> FAIL: test_Brackets (__main__.TestMetricExpressions)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "tools/perf/pmu-events/metric_test.py", line 33, in test_Brackets
>     self.assertEqual((a * b + c).ToPerfJson(), 'a * b + d')
> AssertionError: 'a * b + c' != 'a * b + d'
> - a * b + c
> ?         ^
> + a * b + d

Added this:

diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
index 05806ecfc33c12a1..f533e76fb48002b7 100644
--- a/tools/perf/.gitignore
+++ b/tools/perf/.gitignore
@@ -38,6 +38,7 @@ arch/*/include/generated/
 trace/beauty/generated/
 pmu-events/pmu-events.c
 pmu-events/jevents
+pmu-events/metric_test.log
 feature/
 libapi/
 libbpf/
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index b7d9c42062300d04..bac9272682b759e9 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -1103,6 +1103,7 @@ clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(
 		$(OUTPUT)util/intel-pt-decoder/inat-tables.c \
 		$(OUTPUT)tests/llvm-src-{base,kbuild,prologue,relocation}.c \
 		$(OUTPUT)pmu-events/pmu-events.c \
+		$(OUTPUT)pmu-events/metric_test.log \
 		$(OUTPUT)$(fadvise_advice_array) \
 		$(OUTPUT)$(fsconfig_arrays) \
 		$(OUTPUT)$(fsmount_arrays) \
Ian Rogers Feb. 4, 2023, 9:25 p.m. UTC | #2 
On Fri, Feb 3, 2023 at 12:15 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Thu, Jan 26, 2023 at 03:36:45PM -0800, Ian Rogers escreveu:
> > Add a target that generates a log file for running metric_test.py and
> > make this a dependency on generating pmu-events.c. The log output is
> > displayed if the test fails like (the test was modified to make it
> > fail):
> >
> > ```
> >   TEST    /tmp/perf/pmu-events/metric_test.log
> > F......
> > ======================================================================
> > FAIL: test_Brackets (__main__.TestMetricExpressions)
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> >   File "tools/perf/pmu-events/metric_test.py", line 33, in test_Brackets
> >     self.assertEqual((a * b + c).ToPerfJson(), 'a * b + d')
> > AssertionError: 'a * b + c' != 'a * b + d'
> > - a * b + c
> > ?         ^
> > + a * b + d
>
> Added this:
>
> diff --git a/tools/perf/.gitignore b/tools/perf/.gitignore
> index 05806ecfc33c12a1..f533e76fb48002b7 100644
> --- a/tools/perf/.gitignore
> +++ b/tools/perf/.gitignore
> @@ -38,6 +38,7 @@ arch/*/include/generated/
>  trace/beauty/generated/
>  pmu-events/pmu-events.c
>  pmu-events/jevents
> +pmu-events/metric_test.log
>  feature/
>  libapi/
>  libbpf/
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index b7d9c42062300d04..bac9272682b759e9 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -1103,6 +1103,7 @@ clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(
>                 $(OUTPUT)util/intel-pt-decoder/inat-tables.c \
>                 $(OUTPUT)tests/llvm-src-{base,kbuild,prologue,relocation}.c \
>                 $(OUTPUT)pmu-events/pmu-events.c \
> +               $(OUTPUT)pmu-events/metric_test.log \
>                 $(OUTPUT)$(fadvise_advice_array) \
>                 $(OUTPUT)$(fsconfig_arrays) \
>                 $(OUTPUT)$(fsmount_arrays) \

Acked, thanks!

Ian
diff mbox series

Patch

diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index a14de24ecb69..150765f2baee 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -6,6 +6,11 @@  JDIR_TEST	=  pmu-events/arch/test
 JSON_TEST	=  $(shell [ -d $(JDIR_TEST) ] &&			\
 			find $(JDIR_TEST) -name '*.json')
 JEVENTS_PY	=  pmu-events/jevents.py
+METRIC_PY	=  pmu-events/metric.py
+METRIC_TEST_PY	=  pmu-events/metric_test.py
+EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c
+PMU_EVENTS_C	=  $(OUTPUT)pmu-events/pmu-events.c
+METRIC_TEST_LOG	=  $(OUTPUT)pmu-events/metric_test.log
 
 ifeq ($(JEVENTS_ARCH),)
 JEVENTS_ARCH=$(SRCARCH)
@@ -18,11 +23,15 @@  JEVENTS_MODEL ?= all
 #
 
 ifeq ($(NO_JEVENTS),1)
-$(OUTPUT)pmu-events/pmu-events.c: pmu-events/empty-pmu-events.c
+$(PMU_EVENTS_C): $(EMPTY_PMU_EVENTS_C)
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,gen)cp $< $@
 else
-$(OUTPUT)pmu-events/pmu-events.c: $(JSON) $(JSON_TEST) $(JEVENTS_PY) pmu-events/metric.py
+$(METRIC_TEST_LOG): $(METRIC_TEST_PY) $(METRIC_PY)
+	$(call rule_mkdir)
+	$(Q)$(call echo-cmd,test)$(PYTHON) $< 2> $@ || (cat $@ && false)
+
+$(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG)
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) pmu-events/arch $@
 endif
diff --git a/tools/perf/pmu-events/metric_test.py b/tools/perf/pmu-events/metric_test.py
old mode 100644
new mode 100755
index e4c792428277..40a3c7d8b2bc
--- a/tools/perf/pmu-events/metric_test.py
+++ b/tools/perf/pmu-events/metric_test.py
@@ -1,3 +1,4 @@ 
+#!/usr/bin/env python3
 # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 import unittest
 from metric import Constant