[v9,1/2] Documentation: fpga: dfl: add description for performance reporting support
diff mbox series

Message ID 1587102000-18262-2-git-send-email-hao.wu@intel.com
State New
Headers show
Series
  • add performance reporting support to FPGA DFL drivers
Related show

Commit Message

Wu, Hao April 17, 2020, 5:39 a.m. UTC
From: Xu Yilun <yilun.xu@intel.com>

This patch adds description for performance reporting support for
Device Feature List (DFL) based FPGA.

Signed-off-by: Xu Yilun <yilun.xu@intel.com>
Signed-off-by: Wu Hao <hao.wu@intel.com>
---
 Documentation/fpga/dfl.rst | 84 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)

Comments

Randy Dunlap April 18, 2020, 12:07 a.m. UTC | #1
Hi--

Here are a few comments for you:

On 4/16/20 10:39 PM, Wu Hao wrote:
> From: Xu Yilun <yilun.xu@intel.com>
> 
> This patch adds description for performance reporting support for
> Device Feature List (DFL) based FPGA.
> 
> Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> ---
>  Documentation/fpga/dfl.rst | 84 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 84 insertions(+)
> 
> diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst
> index 094fc8a..6ccaf77 100644
> --- a/Documentation/fpga/dfl.rst
> +++ b/Documentation/fpga/dfl.rst

> @@ -378,6 +383,85 @@ The device nodes used for ioctl() or mmap() can be referenced through::
>  	/sys/class/fpga_region/<regionX>/<dfl-port.n>/dev
>  
>  
> +Performance Counters
> +====================
> +Performance reporting is one private feature implemented in FME. It could
> +supports several independent, system-wide, device counter sets in hardware to
> +monitor and count for performance events, including "basic", "cache", "fabric",
> +"vtd" and "vtd_sip" counters. Users could use standard perf tool to monitor
> +FPGA cache hit/miss rate, transaction number, interface clock counter of AFU
> +and other FPGA performance events.
> +
> +Different FPGA devices may have different counter sets, it depends on hardware

                                                     sets, depending on hardware

> +implementation. e.g. some discrete FPGA cards don't have any cache. User could

                   E.g., some

> +use "perf list" to check which perf events are supported by target hardware.
> +
> +In order to allow user to use standard perf API to access these performance
> +counters, driver creates a perf PMU, and related sysfs interfaces in
> +/sys/bus/event_source/devices/dfl_fme* to describe available perf events and
> +configuration options.
> +
> +The "format" directory describes the format of the config field of struct
> +perf_event_attr. There are 3 bitfields for config, "evtype" defines which type

                                              config: "evtype"

> +the perf event belongs to. "event" is the identity of the event within its

                  belongs to; "event" is

> +category. "portid" is introduced to decide counters set to monitor on FPGA

   category; "portid" is

> +overall data or a specific port.
> +
> +The "events" directory describes the configuration templates for all available
> +events which can be used with perf tool directly. For example, fab_mmio_read
> +has the configuration "event=0x06,evtype=0x02,portid=0xff", which shows this
> +event belongs to fabric type (0x02), the local event id is 0x06 and it is for
> +overall monitoring (portid=0xff).
> +
> +Example usage of perf::
> +
> +  $# perf list |grep dfl_fme
> +
> +  dfl_fme0/fab_mmio_read/                              [Kernel PMU event]
> +  <...>
> +  dfl_fme0/fab_port_mmio_read,portid=?/                [Kernel PMU event]
> +  <...>
> +
> +  $# perf stat -a -e dfl_fme0/fab_mmio_read/ <command>
> +  or
> +  $# perf stat -a -e dfl_fme0/event=0x06,evtype=0x02,portid=0xff/ <command>
> +  or
> +  $# perf stat -a -e dfl_fme0/config=0xff2006/ <command>
> +
> +Another example, fab_port_mmio_read monitors mmio read of a specific port. So
> +its configuration template is "event=0x06,evtype=0x01,portid=?". The portid
> +should be explicitly set.
> +
> +Its usage of perf::
> +
> +  $# perf stat -a -e dfl_fme0/fab_port_mmio_read,portid=0x0/ <command>
> +  or
> +  $# perf stat -a -e dfl_fme0/event=0x06,evtype=0x02,portid=0x0/ <command>
> +  or
> +  $# perf stat -a -e dfl_fme0/config=0x2006/ <command>
> +
> +Please note for fabric counters, overall perf events (fab_*) and port perf
> +events (fab_port_*) actually share one set of counters in hardware, so it can't
> +monitor both at the same time. If this set of counters is configured to monitor
> +overall data, then per port perf data is not supported. See below example::
> +
> +  $# perf stat -e dfl_fme0/fab_mmio_read/,dfl_fme0/fab_port_mmio_write,\
> +                                                    portid=0/ sleep 1
> +
> +  Performance counter stats for 'system wide':
> +
> +                 3      dfl_fme0/fab_mmio_read/
> +   <not supported>      dfl_fme0/fab_port_mmio_write,portid=0x0/
> +
> +       1.001750904 seconds time elapsed
> +
> +The driver also provides a "cpumask" sysfs attribute, which contains only one
> +cpu id used to access these perf events. Counting on multiple CPUs is not
   CPU

> +allowed since they are system-wide counters on FPGA device.
> +
> +The current driver does not support sampling. So "perf record" is unsupported.
> +
> +
>  Add new FIUs support
>  ====================
>  It's possible that developers made some new function blocks (FIUs) under this
> 

thanks.
Wu, Hao April 20, 2020, 9:38 a.m. UTC | #2
> -----Original Message-----
> From: linux-fpga-owner@vger.kernel.org <linux-fpga-owner@vger.kernel.org>
> On Behalf Of Randy Dunlap
> Sent: Saturday, April 18, 2020 8:08 AM
> To: Wu, Hao <hao.wu@intel.com>; will@kernel.org; mdf@kernel.org;
> mark.rutland@arm.com; gregkh@linuxfoundation.org; linux-
> fpga@vger.kernel.org; linux-kernel@vger.kernel.org
> Cc: linux-api@vger.kernel.org; atull@kernel.org; Xu, Yilun
> <yilun.xu@intel.com>; trix@redhat.com; bhu@redhat.com
> Subject: Re: [PATCH v9 1/2] Documentation: fpga: dfl: add description for
> performance reporting support
> 
> Hi--
> 
> Here are a few comments for you:

Thanks a lot for the comments, will fix all of them in the next version. : )

Hao


> 
> On 4/16/20 10:39 PM, Wu Hao wrote:
> > From: Xu Yilun <yilun.xu@intel.com>
> >
> > This patch adds description for performance reporting support for
> > Device Feature List (DFL) based FPGA.
> >
> > Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > ---
> >  Documentation/fpga/dfl.rst | 84
> ++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 84 insertions(+)
> >
> > diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst
> > index 094fc8a..6ccaf77 100644
> > --- a/Documentation/fpga/dfl.rst
> > +++ b/Documentation/fpga/dfl.rst
> 
> > @@ -378,6 +383,85 @@ The device nodes used for ioctl() or mmap() can
> be referenced through::
> >  	/sys/class/fpga_region/<regionX>/<dfl-port.n>/dev
> >
> >
> > +Performance Counters
> > +====================
> > +Performance reporting is one private feature implemented in FME. It
> could
> > +supports several independent, system-wide, device counter sets in
> hardware to
> > +monitor and count for performance events, including "basic", "cache",
> "fabric",
> > +"vtd" and "vtd_sip" counters. Users could use standard perf tool to
> monitor
> > +FPGA cache hit/miss rate, transaction number, interface clock counter of
> AFU
> > +and other FPGA performance events.
> > +
> > +Different FPGA devices may have different counter sets, it depends on
> hardware
> 
>                                                      sets, depending on hardware
> 
> > +implementation. e.g. some discrete FPGA cards don't have any cache.
> User could
> 
>                    E.g., some
> 
> > +use "perf list" to check which perf events are supported by target
> hardware.
> > +
> > +In order to allow user to use standard perf API to access these
> performance
> > +counters, driver creates a perf PMU, and related sysfs interfaces in
> > +/sys/bus/event_source/devices/dfl_fme* to describe available perf events
> and
> > +configuration options.
> > +
> > +The "format" directory describes the format of the config field of struct
> > +perf_event_attr. There are 3 bitfields for config, "evtype" defines which
> type
> 
>                                               config: "evtype"
> 
> > +the perf event belongs to. "event" is the identity of the event within its
> 
>                   belongs to; "event" is
> 
> > +category. "portid" is introduced to decide counters set to monitor on FPGA
> 
>    category; "portid" is
> 
> > +overall data or a specific port.
> > +
> > +The "events" directory describes the configuration templates for all
> available
> > +events which can be used with perf tool directly. For example,
> fab_mmio_read
> > +has the configuration "event=0x06,evtype=0x02,portid=0xff", which
> shows this
> > +event belongs to fabric type (0x02), the local event id is 0x06 and it is for
> > +overall monitoring (portid=0xff).
> > +
> > +Example usage of perf::
> > +
> > +  $# perf list |grep dfl_fme
> > +
> > +  dfl_fme0/fab_mmio_read/                              [Kernel PMU event]
> > +  <...>
> > +  dfl_fme0/fab_port_mmio_read,portid=?/                [Kernel PMU event]
> > +  <...>
> > +
> > +  $# perf stat -a -e dfl_fme0/fab_mmio_read/ <command>
> > +  or
> > +  $# perf stat -a -e dfl_fme0/event=0x06,evtype=0x02,portid=0xff/
> <command>
> > +  or
> > +  $# perf stat -a -e dfl_fme0/config=0xff2006/ <command>
> > +
> > +Another example, fab_port_mmio_read monitors mmio read of a specific
> port. So
> > +its configuration template is "event=0x06,evtype=0x01,portid=?". The
> portid
> > +should be explicitly set.
> > +
> > +Its usage of perf::
> > +
> > +  $# perf stat -a -e dfl_fme0/fab_port_mmio_read,portid=0x0/ <command>
> > +  or
> > +  $# perf stat -a -e dfl_fme0/event=0x06,evtype=0x02,portid=0x0/
> <command>
> > +  or
> > +  $# perf stat -a -e dfl_fme0/config=0x2006/ <command>
> > +
> > +Please note for fabric counters, overall perf events (fab_*) and port perf
> > +events (fab_port_*) actually share one set of counters in hardware, so it
> can't
> > +monitor both at the same time. If this set of counters is configured to
> monitor
> > +overall data, then per port perf data is not supported. See below example::
> > +
> > +  $# perf stat -e
> dfl_fme0/fab_mmio_read/,dfl_fme0/fab_port_mmio_write,\
> > +                                                    portid=0/ sleep 1
> > +
> > +  Performance counter stats for 'system wide':
> > +
> > +                 3      dfl_fme0/fab_mmio_read/
> > +   <not supported>      dfl_fme0/fab_port_mmio_write,portid=0x0/
> > +
> > +       1.001750904 seconds time elapsed
> > +
> > +The driver also provides a "cpumask" sysfs attribute, which contains only
> one
> > +cpu id used to access these perf events. Counting on multiple CPUs is not
>    CPU
> 
> > +allowed since they are system-wide counters on FPGA device.
> > +
> > +The current driver does not support sampling. So "perf record" is
> unsupported.
> > +
> > +
> >  Add new FIUs support
> >  ====================
> >  It's possible that developers made some new function blocks (FIUs) under
> this
> >
> 
> thanks.
> --
> ~Randy

Patch
diff mbox series

diff --git a/Documentation/fpga/dfl.rst b/Documentation/fpga/dfl.rst
index 094fc8a..6ccaf77 100644
--- a/Documentation/fpga/dfl.rst
+++ b/Documentation/fpga/dfl.rst
@@ -118,6 +118,11 @@  More functions are exposed through sysfs
      management information (current temperature, thresholds, threshold status,
      etc.).
 
+ Performance reporting
+     performance counters are exposed through perf PMU APIs. Standard perf tool
+     can be used to monitor all available perf events. Please see performance
+     counter section below for more detailed information.
+
 
 FIU - PORT
 ==========
@@ -378,6 +383,85 @@  The device nodes used for ioctl() or mmap() can be referenced through::
 	/sys/class/fpga_region/<regionX>/<dfl-port.n>/dev
 
 
+Performance Counters
+====================
+Performance reporting is one private feature implemented in FME. It could
+supports several independent, system-wide, device counter sets in hardware to
+monitor and count for performance events, including "basic", "cache", "fabric",
+"vtd" and "vtd_sip" counters. Users could use standard perf tool to monitor
+FPGA cache hit/miss rate, transaction number, interface clock counter of AFU
+and other FPGA performance events.
+
+Different FPGA devices may have different counter sets, it depends on hardware
+implementation. e.g. some discrete FPGA cards don't have any cache. User could
+use "perf list" to check which perf events are supported by target hardware.
+
+In order to allow user to use standard perf API to access these performance
+counters, driver creates a perf PMU, and related sysfs interfaces in
+/sys/bus/event_source/devices/dfl_fme* to describe available perf events and
+configuration options.
+
+The "format" directory describes the format of the config field of struct
+perf_event_attr. There are 3 bitfields for config, "evtype" defines which type
+the perf event belongs to. "event" is the identity of the event within its
+category. "portid" is introduced to decide counters set to monitor on FPGA
+overall data or a specific port.
+
+The "events" directory describes the configuration templates for all available
+events which can be used with perf tool directly. For example, fab_mmio_read
+has the configuration "event=0x06,evtype=0x02,portid=0xff", which shows this
+event belongs to fabric type (0x02), the local event id is 0x06 and it is for
+overall monitoring (portid=0xff).
+
+Example usage of perf::
+
+  $# perf list |grep dfl_fme
+
+  dfl_fme0/fab_mmio_read/                              [Kernel PMU event]
+  <...>
+  dfl_fme0/fab_port_mmio_read,portid=?/                [Kernel PMU event]
+  <...>
+
+  $# perf stat -a -e dfl_fme0/fab_mmio_read/ <command>
+  or
+  $# perf stat -a -e dfl_fme0/event=0x06,evtype=0x02,portid=0xff/ <command>
+  or
+  $# perf stat -a -e dfl_fme0/config=0xff2006/ <command>
+
+Another example, fab_port_mmio_read monitors mmio read of a specific port. So
+its configuration template is "event=0x06,evtype=0x01,portid=?". The portid
+should be explicitly set.
+
+Its usage of perf::
+
+  $# perf stat -a -e dfl_fme0/fab_port_mmio_read,portid=0x0/ <command>
+  or
+  $# perf stat -a -e dfl_fme0/event=0x06,evtype=0x02,portid=0x0/ <command>
+  or
+  $# perf stat -a -e dfl_fme0/config=0x2006/ <command>
+
+Please note for fabric counters, overall perf events (fab_*) and port perf
+events (fab_port_*) actually share one set of counters in hardware, so it can't
+monitor both at the same time. If this set of counters is configured to monitor
+overall data, then per port perf data is not supported. See below example::
+
+  $# perf stat -e dfl_fme0/fab_mmio_read/,dfl_fme0/fab_port_mmio_write,\
+                                                    portid=0/ sleep 1
+
+  Performance counter stats for 'system wide':
+
+                 3      dfl_fme0/fab_mmio_read/
+   <not supported>      dfl_fme0/fab_port_mmio_write,portid=0x0/
+
+       1.001750904 seconds time elapsed
+
+The driver also provides a "cpumask" sysfs attribute, which contains only one
+cpu id used to access these perf events. Counting on multiple CPUs is not
+allowed since they are system-wide counters on FPGA device.
+
+The current driver does not support sampling. So "perf record" is unsupported.
+
+
 Add new FIUs support
 ====================
 It's possible that developers made some new function blocks (FIUs) under this