diff mbox

[6/8] tools/kvm_stat: separate drilldown and fields filtering

Message ID 20180205130004.29691-7-raspl@linux.vnet.ibm.com (mailing list archive)
State New, archived
Headers show

Commit Message

Stefan Raspl Feb. 5, 2018, 1 p.m. UTC 
From: Stefan Raspl <stefan.raspl@de.ibm.com>

Drilldown (i.e. toggle display of child trace events) was implemented by
overriding the fields filter. This resulted in inconsistencies: E.g. when
drilldown was not active, adding a filter that also matches child trace
events would not only filter fields according to the filter, but also add
in the child trace events matching the filter. E.g. on x86, setting
'kvm_userspace_exit' as the fields filter after startup would result in
display of kvm_userspace_exit(DCR), although that wasn't previously
present - not exactly what one would expect from a filter.
This patch addresses the issue by keeping drilldown and fields filter
separate. While at it, we also fix a PEP8 issue by adding a blank line
at one place (since we're in the area...).
We implement this by adding a framework that also allows to define a
taxonomy among the debugfs events to identify child trace events. I.e.
drilldown using 'x' can now also work with debugfs. A respective parent-
child relationship is only known for S390 at the moment, but could be
added adjusting other platforms' ARCH.dbg_is_child() methods
accordingly.

Signed-off-by: Stefan Raspl <raspl@linux.vnet.ibm.com>
---
 tools/kvm/kvm_stat/kvm_stat | 132 +++++++++++++++++++++++++++++++-------------
 1 file changed, 93 insertions(+), 39 deletions(-)

Comments

Paolo Bonzini Feb. 13, 2018, 2:55 p.m. UTC | #1 
On 05/02/2018 14:00, Stefan Raspl wrote:
> From: Stefan Raspl <stefan.raspl@de.ibm.com>
> 
> Drilldown (i.e. toggle display of child trace events) was implemented by
> overriding the fields filter. This resulted in inconsistencies: E.g. when
> drilldown was not active, adding a filter that also matches child trace
> events would not only filter fields according to the filter, but also add
> in the child trace events matching the filter. E.g. on x86, setting
> 'kvm_userspace_exit' as the fields filter after startup would result in
> display of kvm_userspace_exit(DCR), although that wasn't previously
> present - not exactly what one would expect from a filter.
> This patch addresses the issue by keeping drilldown and fields filter
> separate. While at it, we also fix a PEP8 issue by adding a blank line
> at one place (since we're in the area...).
> We implement this by adding a framework that also allows to define a
> taxonomy among the debugfs events to identify child trace events. I.e.
> drilldown using 'x' can now also work with debugfs. A respective parent-
> child relationship is only known for S390 at the moment, but could be
> added adjusting other platforms' ARCH.dbg_is_child() methods
> accordingly.

Looks good, but please name the functions "tracepoint_is_child" and
"debugfs_is_child" rather than "trc_is_child".

Also, should "child_events" (and "pid" too) move from DebugfsProvider
and TracepointProvider up to "Provider"?

Thanks,

Paolo
Stefan Raspl Feb. 14, 2018, 9:31 p.m. UTC | #2 
On 13.02.2018 15:55, Paolo Bonzini wrote:
> On 05/02/2018 14:00, Stefan Raspl wrote:
>> From: Stefan Raspl <stefan.raspl@de.ibm.com>
>>
>> Drilldown (i.e. toggle display of child trace events) was implemented by
>> overriding the fields filter. This resulted in inconsistencies: E.g. when
>> drilldown was not active, adding a filter that also matches child trace
>> events would not only filter fields according to the filter, but also add
>> in the child trace events matching the filter. E.g. on x86, setting
>> 'kvm_userspace_exit' as the fields filter after startup would result in
>> display of kvm_userspace_exit(DCR), although that wasn't previously
>> present - not exactly what one would expect from a filter.
>> This patch addresses the issue by keeping drilldown and fields filter
>> separate. While at it, we also fix a PEP8 issue by adding a blank line
>> at one place (since we're in the area...).
>> We implement this by adding a framework that also allows to define a
>> taxonomy among the debugfs events to identify child trace events. I.e.
>> drilldown using 'x' can now also work with debugfs. A respective parent-
>> child relationship is only known for S390 at the moment, but could be
>> added adjusting other platforms' ARCH.dbg_is_child() methods
>> accordingly.
> 
> Looks good, but please name the functions "tracepoint_is_child" and
> "debugfs_is_child" rather than "trc_is_child".

Though I'd keep the names short - nevermind, will change.

> Also, should "child_events" (and "pid" too) move from DebugfsProvider
> and TracepointProvider up to "Provider"?

Good point, will do.

Ciao,
Stefan
diff mbox

Patch

diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat
index 8bdbd14c5bb6..c4e097bda083 100755
--- a/tools/kvm/kvm_stat/kvm_stat
+++ b/tools/kvm/kvm_stat/kvm_stat
@@ -228,6 +228,7 @@  IOCTL_NUMBERS = {
 }
 
 ENCODING = locale.getpreferredencoding(False)
+TRACE_FILTER = re.compile(r'^[^\(]*$')
 
 
 class Arch(object):
@@ -260,6 +261,11 @@  class Arch(object):
                     return ArchX86(SVM_EXIT_REASONS)
                 return
 
+    def trc_is_child(self, field):
+        if (TRACE_FILTER.match(field)):
+            return None
+        return field.split('(', 1)[0]
+
 
 class ArchX86(Arch):
     def __init__(self, exit_reasons):
@@ -267,6 +273,10 @@  class ArchX86(Arch):
         self.ioctl_numbers = IOCTL_NUMBERS
         self.exit_reasons = exit_reasons
 
+    def dbg_is_child(self, field):
+        """ Returns name of parent if 'field' is a child, None otherwise """
+        return None
+
 
 class ArchPPC(Arch):
     def __init__(self):
@@ -282,6 +292,10 @@  class ArchPPC(Arch):
         self.ioctl_numbers['SET_FILTER'] = 0x80002406 | char_ptr_size << 16
         self.exit_reasons = {}
 
+    def dbg_is_child(self, field):
+        """ Returns name of parent if 'field' is a child, None otherwise """
+        return None
+
 
 class ArchA64(Arch):
     def __init__(self):
@@ -289,6 +303,10 @@  class ArchA64(Arch):
         self.ioctl_numbers = IOCTL_NUMBERS
         self.exit_reasons = AARCH64_EXIT_REASONS
 
+    def dbg_is_child(self, field):
+        """ Returns name of parent if 'field' is a child, None otherwise """
+        return None
+
 
 class ArchS390(Arch):
     def __init__(self):
@@ -296,6 +314,12 @@  class ArchS390(Arch):
         self.ioctl_numbers = IOCTL_NUMBERS
         self.exit_reasons = None
 
+    def dbg_is_child(self, field):
+        """ Returns name of parent if 'field' is a child, None otherwise """
+        if field.startswith('instruction_'):
+            return 'exit_instruction'
+
+
 ARCH = Arch.get_arch()
 
 
@@ -503,6 +527,7 @@  class TracepointProvider(Provider):
         self.filters = self.__get_filters()
         self.update_fields(fields_filter)
         self.pid = pid
+        self.child_events = False
 
     @staticmethod
     def __get_filters():
@@ -522,7 +547,7 @@  class TracepointProvider(Provider):
         return filters
 
     def __get_available_fields(self):
-        """Returns a list of available event's of format 'event name(filter
+        """Returns a list of available events of format 'event name(filter
         name)'.
 
         All available events have directories under
@@ -550,7 +575,8 @@  class TracepointProvider(Provider):
     def update_fields(self, fields_filter):
         """Refresh fields, applying fields_filter"""
         self.fields = [field for field in self.__get_available_fields()
-                       if self.is_field_wanted(fields_filter, field)]
+                       if self.is_field_wanted(fields_filter, field) or
+                       ARCH.trc_is_child(field)]
 
     @staticmethod
     def __get_online_cpus():
@@ -671,8 +697,12 @@  class TracepointProvider(Provider):
         ret = defaultdict(int)
         for group in self.group_leaders:
             for name, val in group.read().items():
-                if name in self._fields:
-                    ret[name] += val
+                if name not in self._fields:
+                    continue
+                parent = ARCH.trc_is_child(name)
+                if parent:
+                    name += ' ' + parent
+                ret[name] += val
         return ret
 
     def reset(self):
@@ -691,6 +721,7 @@  class DebugfsProvider(Provider):
         self.do_read = True
         self.paths = []
         self.pid = pid
+        self.child_events = False
         if include_past:
             self.__restore()
 
@@ -705,7 +736,8 @@  class DebugfsProvider(Provider):
     def update_fields(self, fields_filter):
         """Refresh fields, applying fields_filter"""
         self._fields = [field for field in self.__get_available_fields()
-                        if self.is_field_wanted(fields_filter, field)]
+                        if self.is_field_wanted(fields_filter, field) or
+                        ARCH.dbg_is_child(field)]
 
     @property
     def fields(self):
@@ -766,14 +798,15 @@  class DebugfsProvider(Provider):
                     self._baseline[key] = 0
                 if self._baseline.get(key, -1) == -1:
                     self._baseline[key] = value
-                increment = (results.get(field, 0) + value -
-                             self._baseline.get(key, 0))
-                if by_guest:
-                    pid = key.split('-')[0]
-                    if pid in results:
-                        results[pid] += increment
-                    else:
-                        results[pid] = increment
+                parent = ARCH.dbg_is_child(field)
+                if parent:
+                    field = field + ' ' + parent
+                else:
+                    if by_guest:
+                        field = key.split('-')[0]    # set 'field' to 'pid'
+                increment = value - self._baseline.get(key, 0)
+                if field in results:
+                    results[field] += increment
                 else:
                     results[field] = increment
 
@@ -816,6 +849,7 @@  class Stats(object):
         self._fields_filter_comp = re.compile(options.fields)
         self.providers = self.__get_providers(options)
         self.values = {}
+        self._child_events = False
 
     def __get_providers(self, options):
         """Returns a list of data providers depending on the passed options."""
@@ -867,12 +901,29 @@  class Stats(object):
             for provider in self.providers:
                 provider.pid = self._pid_filter
 
+    @property
+    def child_events(self):
+        return self._child_events
+
+    @child_events.setter
+    def child_events(self, val):
+        self._child_events = val
+        for provider in self.providers:
+            provider.child_events = val
+
     def get(self, by_guest=0):
         """Returns a dict with field -> (value, delta to last value) of all
-        provider data."""
+        provider data.
+        Key formats:
+          * plain: 'key' is event name
+          * child-parent: 'key' is in format '<child> <parent>'
+          * pid: 'key' is the pid of the guest, and the record contains the
+               aggregated event data
+        These formats are generated by the providers, and handled in class TUI.
+        """
         for provider in self.providers:
             new = provider.read(by_guest=by_guest)
-            for key in new if by_guest else provider.fields:
+            for key in new:
                 oldval = self.values.get(key, EventStat(0, 0)).value
                 newval = new.get(key, 0)
                 newdelta = newval - oldval
@@ -905,10 +956,10 @@  class Stats(object):
         self.get(to_pid)
         return 0
 
+
 DELAY_DEFAULT = 3.0
 MAX_GUEST_NAME_LEN = 48
 MAX_REGEX_LEN = 44
-DEFAULT_REGEX = r'^[^\(]*$'
 SORT_DEFAULT = 0
 
 
@@ -1038,14 +1089,6 @@  class Tui(object):
 
         return name
 
-    def __update_drilldown(self):
-        """Sets or removes a filter that only allows fields without braces."""
-        if not self.stats.fields_filter:
-            self.stats.fields_filter = DEFAULT_REGEX
-
-        elif self.stats.fields_filter == DEFAULT_REGEX:
-            self.stats.fields_filter = ''
-
     def __update_pid(self, pid):
         """Propagates pid selection to stats object."""
         self.screen.addstr(4, 1, 'Updating pid filter...')
@@ -1067,8 +1110,7 @@  class Tui(object):
                                .format(pid, gname), curses.A_BOLD)
         else:
             self.screen.addstr(0, 0, 'kvm statistics - summary', curses.A_BOLD)
-        if self.stats.fields_filter and self.stats.fields_filter \
-           != DEFAULT_REGEX:
+        if self.stats.fields_filter:
             regex = self.stats.fields_filter
             if len(regex) > MAX_REGEX_LEN:
                 regex = regex[:MAX_REGEX_LEN] + '...'
@@ -1084,6 +1126,9 @@  class Tui(object):
         self.screen.refresh()
 
     def __refresh_body(self, sleeptime):
+        def is_child_field(field):
+            return field.find('(') != -1
+
         row = 3
         self.screen.move(row, 0)
         self.screen.clrtobot()
@@ -1091,7 +1136,11 @@  class Tui(object):
         total = 0.
         ctotal = 0.
         for key, values in stats.items():
-            if key.find('(') == -1:
+            if self._display_guests:
+                if self.get_gname_from_pid(key):
+                    total += values.value
+                continue
+            if not key.find(' ') != -1:
                 total += values.value
             else:
                 ctotal += values.value
@@ -1108,19 +1157,26 @@  class Tui(object):
                 # sort by overall value
                 return v.value
 
+        sorted_items = sorted(stats.items(), key=sortkey, reverse=True)
+
+        # print events
         tavg = 0
-        for key, values in sorted(stats.items(), key=sortkey, reverse=True):
+        for key, values in sorted_items:
             if row >= self.screen.getmaxyx()[0] - 1:
                 break
-            if not values.value and not values.delta:
-                break
+            if values == (0, 0):
+                continue
+            if not self.stats.child_events and key.find(' ') != -1:
+                continue
             if values.value is not None:
                 cur = int(round(values.delta / sleeptime)) if values.delta else ''
                 if self._display_guests:
                     key = self.get_gname_from_pid(key)
-                self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' %
-                                   (key, values.value, values.value * 100 / total,
-                                    cur))
+                    if not key:
+                        continue
+                self.screen.addstr(row, 1, '%-40s %10d%7.1f %8s' % (key
+                                   .split(' ')[0], values.value,
+                                   values.value * 100 / total, cur))
                 if cur != '' and key.find('(') == -1:
                     tavg += cur
             row += 1
@@ -1196,7 +1252,7 @@  class Tui(object):
             regex = self.screen.getstr().decode(ENCODING)
             curses.noecho()
             if len(regex) == 0:
-                self.stats.fields_filter = DEFAULT_REGEX
+                self.stats.fields_filter = ''
                 self.__refresh_header()
                 return
             try:
@@ -1314,7 +1370,7 @@  class Tui(object):
                         self._display_guests = not self._display_guests
                     self.__refresh_header()
                 if char == 'c':
-                    self.stats.fields_filter = DEFAULT_REGEX
+                    self.stats.fields_filter = ''
                     self.__refresh_header(0)
                     self.__update_pid(0)
                 if char == 'f':
@@ -1339,9 +1395,7 @@  class Tui(object):
                     curses.curs_set(0)
                     sleeptime = self._delay_initial
                 if char == 'x':
-                    self.__update_drilldown()
-                    # prevents display of current values on next refresh
-                    self.stats.get(self._display_guests)
+                    self.stats.child_events = not self.stats.child_events
             except KeyboardInterrupt:
                 break
             except curses.error:
@@ -1476,7 +1530,7 @@  Press any other key to refresh statistics immediately.
                          )
     optparser.add_option('-f', '--fields',
                          action='store',
-                         default=DEFAULT_REGEX,
+                         default='',
                          dest='fields',
                          help='''fields to display (regex)
                                  "-f help" for a list of available events''',