[PATCHSET,00/23] Remaining bits of data type profiling (v7)

Message

Namhyung Kim March 19, 2024, 5:50 a.m. UTC

Hello,

This is the last part of the data type profiling series.
So far we added the basic pointer variable support, and direct access to
global/local variables.  Now it's time to add instruction tracking. :)

For the history and background, you can refer to the previous version
[1] and the LWN article [2].

 * Changes from v6
  - add a new debug option for data type profiling
  - fix return value look up for function calls
  - handle general per-cpu access (by tracking ADD instructions)
  - add cache for global variables
  - fail insn tracking early when no further info is expected
  - and more small bug fixes

 * Changes from v5
  - rename register state from scratch to caller_saved
  - change comment in map__objdump_2rip()


Basically it needs to track variable (and its type) assignment to get
a type of memory access at the sampled instruction.  Compilers don't
generate DWARF information for every memory accesses so it cannot find
all the necessary information from DWARF.  Therefore, it follows the
path to the sample in the function, and update type information at
each location when the instruction moves it.

For the DWARF search, it has a list of scope entries (subroutines or
blocks) that covers the sample already.  So it can use the scopes to
find the shortest path to the sample instruction.

Let's say we have this.  It got 5 scopes but couldn't find a matching
variable for the sample.

      +----------------  scope[0] subprogram
      |
      | +--------------  scope[1] lexical_block
      | |
      | | +------------  scope[2] inlined_subroutine
      | | |
      | | | +----------  scope[3] inlined_subroutine
      | | | |
      | | | | +--------  scope[4] lexical_block
      | | | | |
      | | | | |     ***  target instruction
      ...

Then it starts with the closest scope (at index 4), and find the
shortest path from the start of the scope to the target instruction.
Along the way, it updates type information in the scope and see if the
location at the target instruction has the type.  If so, it can
return with the type.

Otherwise, it goes to the scope[3] and find the shortest path from the
start of scope[3] to the start of scope[4].  And then it can combine
the existing shortest path from the scope[4] to the target with the
new path.  Now it can start from the scope[3] with new variables and
types.  It can repeat this algorithm for the outer scopes.

I did it this way because mostly it was able to find a type in the
closest scope.  So it can avoid unnecessary work for outer scopes.

And it added a basic per-cpu variable support for this CPU on x86_64
which uses %gs segment register.  Also it can detect the stack-canary
pattern which is added by compiler to detect stack overflow.

The code is available at 'perf/data-profile-v7' branch in the tree
below.  I've dropped the debug patch at the end in this series but you
can find it in the git branch.

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung


Cc: Ben Woodard <[e-mail]>
Cc: Joe Mario <[e-mail]>
CC: Kees Cook <[e-mail]>
Cc: David Blaikie <[e-mail]>
Cc: Xu Liu <[e-mail]>
Cc: Kan Liang <[e-mail]>
Cc: Ravi Bangoria <[e-mail]>
Cc: Mark Wielaard <[e-mail]>
Cc: Jason Merrill <[e-mail]>
Cc: Jose E. Marchesi <[e-mail]>
Cc: William Huang <[e-mail]>


[1] https://lore.kernel.org/linux-perf-users/[e-mail]/
[2] https://lwn.net/Articles/955709/


Namhyung Kim (23):
  perf dwarf-aux: Remove unused pc argument
  perf dwarf-aux: Add die_collect_vars()
  perf dwarf-aux: Handle type transfer for memory access
  perf dwarf-aux: Add die_find_func_rettype()
  perf map: Add map__objdump_2rip()
  perf annotate-data: Introduce struct data_loc_info
  perf annotate: Add annotate_get_basic_blocks()
  perf annotate-data: Add debug messages
  perf annotate-data: Maintain variable type info
  perf annotate-data: Add update_insn_state()
  perf annotate-data: Add get_global_var_type()
  perf annotate-data: Handle global variable access
  perf annotate-data: Handle call instructions
  perf annotate-data: Implement instruction tracking
  perf annotate-data: Check register state for type
  perf annotate: Parse x86 segment register location
  perf annotate-data: Handle this-cpu variables in kernel
  perf annotate-data: Track instructions with a this-cpu variable
  perf annotate-data: Support general per-cpu access
  perf annotate-data: Handle ADD instructions
  perf annotate-data: Add stack canary type
  perf annotate-data: Add a cache for global variable types
  perf annotate-data: Do not retry for invalid types

 tools/perf/builtin-annotate.c   |    1 +
 tools/perf/util/annotate-data.c | 1320 +++++++++++++++++++++++++++++--
 tools/perf/util/annotate-data.h |   52 +-
 tools/perf/util/annotate.c      |  334 +++++++-
 tools/perf/util/annotate.h      |   31 +
 tools/perf/util/debug.c         |    3 +
 tools/perf/util/debug.h         |    1 +
 tools/perf/util/dso.c           |    2 +
 tools/perf/util/dso.h           |    6 +-
 tools/perf/util/dwarf-aux.c     |  282 ++++++-
 tools/perf/util/dwarf-aux.h     |   33 +-
 tools/perf/util/map.c           |   17 +
 tools/perf/util/map.h           |    3 +
 13 files changed, 1955 insertions(+), 130 deletions(-)


base-commit: 2ebf53c2894623d1ef3bcecf1464dbcc3fe1cb72

Comments

Arnaldo Carvalho de Melo March 19, 2024, 2:17 p.m. UTC | #1

On Mon, Mar 18, 2024 at 10:50:52PM -0700, Namhyung Kim wrote:
> 
> Hello,
> 
> This is the last part of the data type profiling series.
> So far we added the basic pointer variable support, and direct access to
> global/local variables.  Now it's time to add instruction tracking. :)
> 
> For the history and background, you can refer to the previous version
> [1] and the LWN article [2].

Most of my comments are minor and can be addressed later, I've applied
in my local repo, will go thru the usual build tests and then I'll do
further usage tests.

Thanks,

- Arnaldo