[v2,00/12] Build trace-cruncher as Python pakage

Message

Yordan Karadzhov Jan. 7, 2020, 5:03 p.m. UTC

This patch-set is an attempt to restructure the project and to make it
build as a native Python package. Although it looks like a complete
rewrite, this is essentially just a switching from using Cython to using
directly the C API of Python. Cython is still being used but only for
the implementation of the NumPy data wrapper. The new building procedure
is very simple and intuitive. The appropriate versions of trace-cmd and
KernelShark libraries are automatically download, patch and build. These
third-party libraries are installed as part of trace-cruncher itself and
do not interfere with any existing system-wide installations of trace-cmd
and KernelShark.

All patches are available at:
https://github.com/vmware/trace-cruncher/tree/refactoring_WIP (branch
"refactoring_WIP")

v2 changes:
  - Trying to address the building issues reported by Douglas Raillard in 
    particular the missing header file and the problem of linking with
    libkshark.so.X.Y.Z
  - Douglas suggested to use venv in order to avoid the patched versions
    of the third-party libraries to pollut the system-wide locations.
    Here I am suggesting a different solution that allows to make those
    third-party libraries part of the package itself. Not sure which of
    the two solution is the best. It will be great if we can have a
    discussion.
  - All obsoleted source files are removed and the README file is updated
    to describe the new building procedure.



Yordan Karadzhov (VMware) (12):
  trace-cruncher: Refactor the part of the interface that relies on
    libkshark
  trace-cruncher: Refactor the part of the interface that relies on
    libtraceevent
  trace-cruncher: Refactor NumPy based data wrapper
  trace-cruncher: Add "utils"
  trace-cruncher: Adapt sched_wakeup.py to use the new module
  trace-cruncher: Add Makefile
  trace-cruncher: Adapt gpareto_fit.py to use the new module
  trace-cruncher: Adapt page_faults.py to use the new module
  trace-cruncher: Automate the third-party build
  Update README.md
  trace-cruncher: Remove all leftover files.
  trace-cruncher: Improve Makefile Provide more robust and better
    looking build process.

 ...-_DEVEL-_LIBS-and-_RPATH_TO_ORIGIN-b.patch | 160 ++++++++
 0001-kernel-shark-Add-_DEVEL-build-flag.patch |  90 -----
 0002-kernel-shark-Add-reg_pid-plugin.patch    |   8 +-
 Makefile                                      |  36 ++
 README.md                                     |  50 +--
 clean.sh                                      |   6 -
 examples/gpareto_fit.py                       |  35 +-
 examples/ksharksetup.py                       |  24 --
 examples/page_faults.py                       |  64 ++--
 examples/sched_wakeup.py                      |  30 +-
 install_third_party.sh                        |  39 ++
 libkshark-py.c                                | 224 -----------
 libkshark_wrapper.pyx                         | 361 ------------------
 np_setup.py                                   |  90 -----
 setup.py                                      |  68 ++++
 src/common.h                                  |  20 +
 src/datawrapper.pyx                           | 201 ++++++++++
 src/ftracepy.c                                | 233 +++++++++++
 src/ksharkpy.c                                | 269 +++++++++++++
 src/trace2matrix.c                            |  29 ++
 tracecruncher/__init__.py                     |   4 +
 tracecruncher/utils.py                        |  54 +++
 22 files changed, 1188 insertions(+), 907 deletions(-)
 create mode 100644 0001-kernel-shark-Add-_DEVEL-_LIBS-and-_RPATH_TO_ORIGIN-b.patch
 delete mode 100644 0001-kernel-shark-Add-_DEVEL-build-flag.patch
 create mode 100644 Makefile
 delete mode 100755 clean.sh
 delete mode 100644 examples/ksharksetup.py
 create mode 100755 install_third_party.sh
 delete mode 100644 libkshark-py.c
 delete mode 100644 libkshark_wrapper.pyx
 delete mode 100755 np_setup.py
 create mode 100644 setup.py
 create mode 100644 src/common.h
 create mode 100644 src/datawrapper.pyx
 create mode 100644 src/ftracepy.c
 create mode 100644 src/ksharkpy.c
 create mode 100644 src/trace2matrix.c
 create mode 100644 tracecruncher/__init__.py
 create mode 100644 tracecruncher/utils.py