From patchwork Thu Dec 12 09:02:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yordan Karadzhov X-Patchwork-Id: 11287649 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D9CE109A for ; Thu, 12 Dec 2019 09:03:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3830D24654 for ; Thu, 12 Dec 2019 09:03:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Vu0rpbQN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728261AbfLLJDF (ORCPT ); Thu, 12 Dec 2019 04:03:05 -0500 Received: from mail-lj1-f196.google.com ([209.85.208.196]:42968 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728110AbfLLJDE (ORCPT ); Thu, 12 Dec 2019 04:03:04 -0500 Received: by mail-lj1-f196.google.com with SMTP id e28so1365060ljo.9 for ; Thu, 12 Dec 2019 01:03:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dYH87u3MAYmax+8WJxJzysRv03ukG556beLusQOoYWE=; b=Vu0rpbQNGwTxHYglI/U1Mmg7fozo7TF63qzCl7pgaEHo7T3Nox1mRzVjzucHkn1j2B jZpUDnv2G5yA9UYm7cbuTNpEGNnsWiC/4P7ToY4C4HubYaHFgrCZCXwUws6Qn166beFD 9nsLJiPOkDa1xVBEHA1aCLnCgdMph0JJrZQkrTz4JzE8upoECRhtlD4P8AdO7QRpizOb LtyMa9budO+rU0EHVEGtpj6ip9/w8Xgtewp5bMbGxQdGP/FyFMBUCHViXuDhea+OZPpl XC0cSQ8WvoalPAk7ylmA3OTCiRF5wtsJvtmh327WP0JBNNnMgLu57EuerKTCSqLbJhNs yaLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dYH87u3MAYmax+8WJxJzysRv03ukG556beLusQOoYWE=; b=rJimxXgRa+uZfmnhGFA8LpAKJjq2fID5HBeFTPrg98ITj7Bjuvgj3tg4pR0g3Hg5Q4 vZUZMHGzXH4JwO7tK45QQO4zgt5v46kn1lJl/A75myokTCsSBdFQZFDCV6mteJwzzCvO dFYunxEGXCzT/zyQV+hMGEWFL4Cjd/w2rKfeQNbcdAmssvjM8J5tHE3cdK6/d6zfSmga 5VudZ1tlQ1p6swdJcmgDqtrkKLE8rYQj6lnINgehA/Hfv6zanYLjmFzr+B6Cp3yVIgNq 2gRRvSRL+vidPTlubfNt3sKyBeqrw+q205woH6vMNxxOj7xoTJoubQMh+Jr73mH+Iuc1 Ovww== X-Gm-Message-State: APjAAAUln+pZh5H2RizqUIdnZR6Yd+0TsoG/mI8dnf10WVy8aOkrw6iW /oAmmr8keKA/pAOFiMHBF0QKXjhM X-Google-Smtp-Source: APXvYqx8z8YsnXMBChtl23AHOP3CxUck0m9lGrUSxuCVe1lb2mVXIrzZuf8Gy3MmzcIvZLo166MbjA== X-Received: by 2002:a2e:4704:: with SMTP id u4mr5196317lja.117.1576141379805; Thu, 12 Dec 2019 01:02:59 -0800 (PST) Received: from mamba.eng.vmware.com ([146.247.46.5]) by smtp.gmail.com with ESMTPSA id u16sm2849908lfi.36.2019.12.12.01.02.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2019 01:02:59 -0800 (PST) From: "Yordan Karadzhov (VMware)" To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, Valentin.Schneider@arm.com, douglas.raillard@arm.com, "Yordan Karadzhov (VMware)" Subject: [PATCH 1/5] Refactor the part of the interface that relies on libkshark Date: Thu, 12 Dec 2019 11:02:28 +0200 Message-Id: <20191212090232.24236-2-y.karadz@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191212090232.24236-1-y.karadz@gmail.com> References: <20191212090232.24236-1-y.karadz@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org This is the first patch from a patch-set that aims to refactor trace-cruncher completely. The goal it to be able to build the project as a native Python package, which contains several sub-packages implemented as C extensions via the Python's C API. In this patch the part of the interface that relies on libkshark gets re-implemented as an extension called "tracecruncher.ksharkpy". Note that this new extension has a stand-alone build that is completely decoupled from the existing build system used by trace-cruncher. Signed-off-by: Yordan Karadzhov (VMware) --- setup.py | 44 +++++++ src/common.h | 19 +++ src/ksharkpy.c | 268 ++++++++++++++++++++++++++++++++++++++ tracecruncher/__init__.py | 0 4 files changed, 331 insertions(+) create mode 100644 setup.py create mode 100644 src/common.h create mode 100644 src/ksharkpy.c create mode 100644 tracecruncher/__init__.py diff --git a/setup.py b/setup.py new file mode 100644 index 0000000..6a1d2e8 --- /dev/null +++ b/setup.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 + +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) +""" + + +from setuptools import setup, find_packages +from distutils.core import Extension +from Cython.Build import cythonize + +def main(): + kshark_path = '/usr/local/lib/kernelshark' + + module_ks = Extension('tracecruncher.ksharkpy', + sources=['src/ksharkpy.c'], + library_dirs=[kshark_path], + runtime_library_dirs=[kshark_path], + libraries=['kshark'], + define_macros=[ + ('LIB_KSHARK_PATH', '\"' + kshark_path + '/libkshark.so\"'), + ('KS_PLUGIN_DIR', '\"' + kshark_path + '/plugins\"') + ], + ) + + setup(name='tracecruncher', + version='0.1.0', + description='NumPy based interface for accessing tracing data in Python.', + author='Yordan Karadzhov (VMware)', + author_email='y.karadz@gmail.com', + url='https://github.com/vmware/trace-cruncher', + license='LGPL-2.1', + packages=find_packages(), + ext_modules=[module_ks], + classifiers=[ + 'Development Status :: 3 - Alpha', + 'Programming Language :: Python :: 3', + ] + ) + +if __name__ == '__main__': + main() diff --git a/src/common.h b/src/common.h new file mode 100644 index 0000000..d7d355a --- /dev/null +++ b/src/common.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: LGPL-2.1 */ + +/* + * Copyright (C) 2017 VMware Inc, Yordan Karadzhov + */ + +#ifndef _TC_COMMON_H +#define _TC_COMMON_H + +#define TRACECRUNCHER_ERROR tracecruncher_error +#define KSHARK_ERROR kshark_error + +#define KS_INIT_ERROR \ + PyErr_SetString(KSHARK_ERROR, "libshark failed to initialize"); + +#define KS_MEM_ERROR \ + PyErr_SetString(TRACECRUNCHER_ERROR, "failed to allocate memory"); + +#endif diff --git a/src/ksharkpy.c b/src/ksharkpy.c new file mode 100644 index 0000000..90a6c1f --- /dev/null +++ b/src/ksharkpy.c @@ -0,0 +1,268 @@ +// SPDX-License-Identifier: LGPL-2.1 + +/* + * Copyright (C) 2019 VMware Inc, Yordan Karadzhov (VMware) + */ + +/** Use GNU C Library. */ +#define _GNU_SOURCE 1 + +// C +#include +#include + +// Python +#include + +// KernelShark +#include "kernelshark/libkshark.h" +#include "kernelshark/libkshark-input.h" +#include "kernelshark/libkshark-plugin.h" +#include "kernelshark/libkshark-model.h" + +// trace-cruncher +#include "common.h" + +static PyObject *KSHARK_ERROR = NULL; +static PyObject *TRACECRUNCHER_ERROR = NULL; + +static PyObject *method_open(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + char *fname = NULL; + + static char *kwlist[] = {"fname", NULL}; + if(!PyArg_ParseTupleAndKeywords(args, + kwargs, + "s", + kwlist, + &fname)) { + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + if (!kshark_open(kshark_ctx, fname)) + return Py_False; + + return Py_True; +} + +static PyObject* method_close(PyObject* self, PyObject* noarg) +{ + struct kshark_context *kshark_ctx = NULL; + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + kshark_close(kshark_ctx); + + Py_RETURN_NONE; +} + +static int compare(const void *a, const void *b) +{ + int a_i, b_i; + + a_i = *(const int *) a; + b_i = *(const int *) b; + + if (a_i > b_i) + return +1; + + if (a_i < b_i) + return -1; + + return 0; +} + +static PyObject* method_get_tasks(PyObject* self, PyObject* noarg) +{ + struct kshark_context *kshark_ctx = NULL; + const char *comm; + int *pids; + ssize_t i, n; + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + n = kshark_get_task_pids(kshark_ctx, &pids); + if (n == 0) { + PyErr_SetString(KSHARK_ERROR, + "Failed to retrieve the PID-s of the tasks"); + return NULL; + } + + qsort(pids, n, sizeof(*pids), compare); + + PyObject *tasks, *pid_list, *pid_val; + + tasks = PyDict_New(); + for (i = 0; i < n; ++i) { + comm = tep_data_comm_from_pid(kshark_ctx->pevent, pids[i]); + pid_val = PyLong_FromLong(pids[i]); + pid_list = PyDict_GetItemString(tasks, comm); + if (!pid_list) { + pid_list = PyList_New(1); + PyList_SET_ITEM(pid_list, 0, pid_val); + PyDict_SetItemString(tasks, comm, pid_list); + } else { + PyList_Append(pid_list, pid_val); + } + } + + return tasks; +} + +static PyObject *method_register_plugin(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + char *plugin, *lib_file; + int ret; + + static char *kwlist[] = {"plugin", NULL}; + if (!PyArg_ParseTupleAndKeywords(args, + kwargs, + "s", + kwlist, + &plugin)) { + return NULL; + } + + if (asprintf(&lib_file, "%s/plugin-%s.so", KS_PLUGIN_DIR, plugin) < 0) { + KS_MEM_ERROR + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + ret = kshark_register_plugin(kshark_ctx, lib_file); + free(lib_file); + if (ret < 0) { + PyErr_Format(KSHARK_ERROR, + "libshark failed to load plugin '%s'", + plugin); + return NULL; + } + + if (kshark_handle_plugins(kshark_ctx, KSHARK_PLUGIN_INIT) < 0) { + PyErr_SetString(KSHARK_ERROR, + "libshark failed to handle its plugins"); + return NULL; + } + + Py_RETURN_NONE; +} + +static PyObject *method_new_session_file(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + struct kshark_config_doc *session; + struct kshark_config_doc *filters; + struct kshark_config_doc *markers; + struct kshark_config_doc *model; + struct kshark_config_doc *file; + struct kshark_trace_histo histo; + const char *session_file, *data_file; + + static char *kwlist[] = {"data_file", "session_file", NULL}; + if (!PyArg_ParseTupleAndKeywords(args, + kwargs, + "ss", + kwlist, + &data_file, + &session_file)) { + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + session = kshark_config_new("kshark.config.session", + KS_CONFIG_JSON); + + file = kshark_export_trace_file(data_file, KS_CONFIG_JSON); + kshark_config_doc_add(session, "Data", file); + + filters = kshark_export_all_filters(kshark_ctx, KS_CONFIG_JSON); + kshark_config_doc_add(session, "Filters", filters); + + ksmodel_init(&histo); + model = kshark_export_model(&histo, KS_CONFIG_JSON); + kshark_config_doc_add(session, "Model", model); + + markers = kshark_config_new("kshark.config.markers", KS_CONFIG_JSON); + kshark_config_doc_add(session, "Markers", markers); + + kshark_save_config_file(session_file, session); + kshark_free_config_doc(session); + + Py_RETURN_NONE; +} + +static PyMethodDef ksharkpy_methods[] = { + {"open", + (PyCFunction) method_open, + METH_VARARGS | METH_KEYWORDS, + "Open trace data file" + }, + {"close", + (PyCFunction) method_close, + METH_NOARGS, + "Close trace data file" + }, + {"get_tasks", + (PyCFunction) method_get_tasks, + METH_NOARGS, + "Get all tasks recorded in a trace file" + }, + {"register_plugin", + (PyCFunction) method_register_plugin, + METH_VARARGS | METH_KEYWORDS, + "Load a plugin" + }, + {"new_session_file", + (PyCFunction) method_new_session_file, + METH_VARARGS | METH_KEYWORDS, + "Create new session description file" + }, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef ksharkpy_module = { + PyModuleDef_HEAD_INIT, + "ksharkpy", + "", + -1, + ksharkpy_methods +}; + +PyMODINIT_FUNC PyInit_ksharkpy(void) +{ + PyObject *module = PyModule_Create(&ksharkpy_module); + + KSHARK_ERROR = PyErr_NewException("tracecruncher.ksharkpy.ks_error", + NULL, NULL); + PyModule_AddObject(module, "ks_error", KSHARK_ERROR); + + TRACECRUNCHER_ERROR = PyErr_NewException("tracecruncher.tc_error", + NULL, NULL); + PyModule_AddObject(module, "tc_error", TRACECRUNCHER_ERROR); + + return module; +} diff --git a/tracecruncher/__init__.py b/tracecruncher/__init__.py new file mode 100644 index 0000000..e69de29 From patchwork Thu Dec 12 09:02:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yordan Karadzhov X-Patchwork-Id: 11287651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 996326C1 for ; Thu, 12 Dec 2019 09:03:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 77D2C24658 for ; Thu, 12 Dec 2019 09:03:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VSPGvaFF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728110AbfLLJDF (ORCPT ); Thu, 12 Dec 2019 04:03:05 -0500 Received: from mail-lj1-f176.google.com ([209.85.208.176]:33006 "EHLO mail-lj1-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728218AbfLLJDE (ORCPT ); Thu, 12 Dec 2019 04:03:04 -0500 Received: by mail-lj1-f176.google.com with SMTP id 21so1404871ljr.0 for ; Thu, 12 Dec 2019 01:03:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qri2yookmjZui+Ee9/8tZUSYwPNQ8QXdi+ukzfXc778=; b=VSPGvaFFAjFmUlFjNT3a08SphsCCGeUw0TdwnER78vd3XejXfoWg7tNswwVoz8DkDz GGi+Fx4Hqjb88vF1qLujjSfdmobONHouxTxqt1NPgzzC2dhMKSBWNo1AcV4Fxa+YFMcD AgY1RiEPhTF+nciESKU+Hezpd3ogWXVwny5LlgGJGLEmNvTF3BC/RlpXeKEWqPRqblS1 i1hH9O/8n5LUWQb/AQhh4KfCjhVBBsJxfCDofH3+GUppULalj4rqUfJbmXQrnuXJfw3x kzqFm6SGP8oLvi32VPCNy1YJM4iD1pC09lOMMPEFKQ8ChM0A9GFL97FeTya7KUjdmLNf p1gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qri2yookmjZui+Ee9/8tZUSYwPNQ8QXdi+ukzfXc778=; b=HPH0ceynF19Cf4NxN4tGb8cGbzfOEgvoqEpAEVh0t5hL1f0nOj0Ycz6hws5gzWeziR OMiG8r0yuRUIK7aKCU+LEmJ6Rnw3BV7Av8lNPQop2GcVmdN+F9nNICLevPpsSME1GjGp AA2aYCz8xjSq+F+po/MTQMLZUJA6eZVir5Q6B9z19kmSHTqJnPykQiZWsvJ/04uPcbVD yVg6ycJO/CK/Zd5RHqnOqE6JjpqJbzTNK7U9WHaXsSJUovTzBd6ajaPYZT08JFWgZQ9o QQKTa5UcXzcnMRaQr5Cf5swQ6AFuV9qtEhFnOiMSLYYszkpw9xfOLgsqDsRaCE4bc6X2 ruJg== X-Gm-Message-State: APjAAAWwxHWXWEqe5wQHdIwL6jI7NQo4sk/dC9gIwvViRPKw4aqwNH7n lGFMa2mM27pSff8fqKtbkvZNzRoN X-Google-Smtp-Source: APXvYqyaLtmb8xc0Lmlv2gw8l0hXFSpo/lW+ZhVOBsnMPtdR5dxMZph+Op7Y4Fv0Sf+liG4rIKEtIg== X-Received: by 2002:a2e:9899:: with SMTP id b25mr4952634ljj.70.1576141381724; Thu, 12 Dec 2019 01:03:01 -0800 (PST) Received: from mamba.eng.vmware.com ([146.247.46.5]) by smtp.gmail.com with ESMTPSA id u16sm2849908lfi.36.2019.12.12.01.03.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2019 01:03:01 -0800 (PST) From: "Yordan Karadzhov (VMware)" To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, Valentin.Schneider@arm.com, douglas.raillard@arm.com, "Yordan Karadzhov (VMware)" Subject: [PATCH 2/5] Refactor the part of the interface that relies on libtraceevent Date: Thu, 12 Dec 2019 11:02:29 +0200 Message-Id: <20191212090232.24236-3-y.karadz@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191212090232.24236-1-y.karadz@gmail.com> References: <20191212090232.24236-1-y.karadz@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The part of the interface that relies on libtraceevent gets re-implemented as an extension called "tracecruncher.ftracepy". The new extension gets build together with the previously implemented "tracecruncher.ksharkpy" extension. Signed-off-by: Yordan Karadzhov (VMware) --- setup.py | 11 ++- src/common.h | 1 + src/ftracepy.c | 234 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 245 insertions(+), 1 deletion(-) create mode 100644 src/ftracepy.c diff --git a/setup.py b/setup.py index 6a1d2e8..dfe1d00 100644 --- a/setup.py +++ b/setup.py @@ -13,6 +13,8 @@ from Cython.Build import cythonize def main(): kshark_path = '/usr/local/lib/kernelshark' + traceevent_path = '/usr/local/lib/traceevent/' + tracecmd_path = '/usr/local/lib/trace-cmd/' module_ks = Extension('tracecruncher.ksharkpy', sources=['src/ksharkpy.c'], @@ -25,6 +27,13 @@ def main(): ], ) + module_ft = Extension('tracecruncher.ftracepy', + sources=['src/ftracepy.c'], + library_dirs=[kshark_path, traceevent_path, tracecmd_path], + runtime_library_dirs=[kshark_path, traceevent_path, tracecmd_path], + libraries=['kshark', 'traceevent', 'tracecmd'], + ) + setup(name='tracecruncher', version='0.1.0', description='NumPy based interface for accessing tracing data in Python.', @@ -33,7 +42,7 @@ def main(): url='https://github.com/vmware/trace-cruncher', license='LGPL-2.1', packages=find_packages(), - ext_modules=[module_ks], + ext_modules=[module_ks, module_ft], classifiers=[ 'Development Status :: 3 - Alpha', 'Programming Language :: Python :: 3', diff --git a/src/common.h b/src/common.h index d7d355a..632e17a 100644 --- a/src/common.h +++ b/src/common.h @@ -9,6 +9,7 @@ #define TRACECRUNCHER_ERROR tracecruncher_error #define KSHARK_ERROR kshark_error +#define FTRACE_ERROR ftrace_error #define KS_INIT_ERROR \ PyErr_SetString(KSHARK_ERROR, "libshark failed to initialize"); diff --git a/src/ftracepy.c b/src/ftracepy.c new file mode 100644 index 0000000..e23b875 --- /dev/null +++ b/src/ftracepy.c @@ -0,0 +1,234 @@ +// SPDX-License-Identifier: LGPL-2.1 + +/* + * Copyright (C) 2019 VMware Inc, Yordan Karadzhov (VMware) + */ + +// Python +#include + +// trace-cmd +#include "trace-cmd/trace-cmd.h" + +// KernelShark +#include "kernelshark/libkshark.h" +#include "kernelshark/libkshark-tepdata.h" + +// trace-cruncher +#include "common.h" + +static PyObject *KSHARK_ERROR = NULL; +static PyObject *FTRACE_ERROR = NULL; + +static PyObject *method_event_id(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + struct tep_event *event; + const char *system, *name; + + static char *kwlist[] = {"system", "event", NULL}; + if (!PyArg_ParseTupleAndKeywords(args, + kwargs, + "ss", + kwlist, + &system, + &name)) { + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + event = tep_find_event_by_name(kshark_ctx->pevent, system, name); + if (!event) { + PyErr_Format(FTRACE_ERROR, + "Failed to find event '%s/%s'", + system, name); + } + + return PyLong_FromLong(event->id); +} + +static PyObject *method_read_event_field(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + struct tep_format_field *evt_field; + struct tep_record *record; + struct tep_event *event; + unsigned long long val; + const char *field; + uint64_t offset; + int event_id, ret; + + static char *kwlist[] = {"offset", "event_id", "field", NULL}; + if(!PyArg_ParseTupleAndKeywords(args, + kwargs, + "Lis", + kwlist, + &offset, + &event_id, + &field)) { + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + event = tep_find_event(kshark_ctx->pevent, event_id); + if (!event) { + PyErr_Format(FTRACE_ERROR, + "Failed to find event '%i'", + event_id); + return NULL; + } + + evt_field = tep_find_any_field(event, field); + if (!evt_field) { + PyErr_Format(FTRACE_ERROR, + "Failed to find field '%s' of event '%i'", + field, event_id); + return NULL; + } + + record = tracecmd_read_at(kshark_ctx->handle, offset, NULL); + if (!record) { + PyErr_Format(FTRACE_ERROR, + "Failed to read record at offset '%i'", + offset); + return NULL; + } + + ret = tep_read_number_field(evt_field, record->data, &val); + free_record(record); + + if (ret != 0) { + PyErr_Format(FTRACE_ERROR, + "Failed to read field '%s' of event '%i'", + field, event_id); + return NULL; + } + + return PyLong_FromLong(val); +} + +static PyObject *method_get_function(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + unsigned long long address; + const char *func; + + static char *kwlist[] = {"address"}; + if (!PyArg_ParseTupleAndKeywords(args, + kwargs, + "i", + kwlist, + &address)) { + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + func = tep_find_function(kshark_ctx->pevent, address); + if (!func) + Py_RETURN_NONE; + + return PyUnicode_FromString(func); +} + +static PyObject *method_map_instruction_address(PyObject *self, PyObject *args, + PyObject *kwargs) +{ + struct kshark_context *kshark_ctx = NULL; + struct tracecmd_proc_addr_map *mem_map; + unsigned long long proc_addr, obj_addr; + int pid; + PyObject *ret; + + static char *kwlist[] = {"pid", "proc_addr"}; + if (!PyArg_ParseTupleAndKeywords(args, + kwargs, + "iL", + kwlist, + &pid, + &proc_addr)) { + return NULL; + } + + if (!kshark_instance(&kshark_ctx)) { + KS_INIT_ERROR + return NULL; + } + + mem_map = tracecmd_search_task_map(kshark_ctx->handle, + pid, proc_addr); + + if (!mem_map) + Py_RETURN_NONE; + + ret = PyDict_New(); + + PyDict_SetItemString(ret, "obj_file", + PyUnicode_FromString(mem_map->lib_name)); + + obj_addr = proc_addr - mem_map->start; + PyDict_SetItemString(ret, "address", PyLong_FromLong(obj_addr)); + + return ret; +} + +static PyMethodDef ftracepy_methods[] = { + {"event_id", + (PyCFunction) method_event_id, + METH_VARARGS | METH_KEYWORDS, + "Get the Id of the event from its name" + }, + {"read_event_field", + (PyCFunction) method_read_event_field, + METH_VARARGS | METH_KEYWORDS, + "Get the value of an event field having a given name" + }, + {"get_function", + (PyCFunction) method_get_function, + METH_VARARGS | METH_KEYWORDS, + "" + }, + {"map_instruction_address", + (PyCFunction) method_map_instruction_address, + METH_VARARGS | METH_KEYWORDS, + "" + }, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef ftracepy_module = { + PyModuleDef_HEAD_INIT, + "ftracepy", + "", + -1, + ftracepy_methods +}; + +PyMODINIT_FUNC PyInit_ftracepy(void) +{ + PyObject *module = PyModule_Create(&ftracepy_module); + + KSHARK_ERROR = PyErr_NewException("tracecruncher.ftracepy.ks_error", + NULL, NULL); + PyModule_AddObject(module, "ks_error", KSHARK_ERROR); + + FTRACE_ERROR = PyErr_NewException("tracecruncher.ftracepy.ft_error", + NULL, NULL); + PyModule_AddObject(module, "ft_error", FTRACE_ERROR); + + return module; +} From patchwork Thu Dec 12 09:02:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yordan Karadzhov X-Patchwork-Id: 11287653 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 578A66C1 for ; Thu, 12 Dec 2019 09:03:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2AC342464B for ; Thu, 12 Dec 2019 09:03:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hEgkwbLq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728274AbfLLJDG (ORCPT ); Thu, 12 Dec 2019 04:03:06 -0500 Received: from mail-lf1-f42.google.com ([209.85.167.42]:46174 "EHLO mail-lf1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728256AbfLLJDF (ORCPT ); Thu, 12 Dec 2019 04:03:05 -0500 Received: by mail-lf1-f42.google.com with SMTP id f15so1057148lfl.13 for ; Thu, 12 Dec 2019 01:03:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=a+BXVPMKBy+1VO/77mfq//mwpBhl95JVPpWHC+3ZHXw=; b=hEgkwbLqruqEqj+qK+BFcjQs1ABWDRFR06lA6WLsRrF01gWUDJt8c3WlIDOigtO5Pi 6Spb8KBNw57hZUOYfIJdYsM/9bRpxrPCrw8ZLJInFkotqh6nVms5L1iK0atKO8A4WDdN DaG0ktXyTLfl6LkjT8rY69iiATzxl0TATq21cv3puGVYmcTDPk2l/5JEpR5vjzRDoNnc vwkyRkbiVuL9JG/T5W/EAS6naBdd2iL1KawDMOXprQV0Nay2bfQfyTIfcdzSqwN1hA9W +BSyfPXnh8uKbR9Gfs7FjJhQ/TN2fCJGmraGNkyseQIwk1gF1ZlliSuIYeWyPohdjNVn d3Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=a+BXVPMKBy+1VO/77mfq//mwpBhl95JVPpWHC+3ZHXw=; b=QvoKvMCv3vxmEYooW7RNZ5LLLmbZP/Ki+3f0NnmmVEgIYt3govzqsssQ0/BzruL/jb VoDrc8ZIfMgmtKq0FoVMd0Z9/mkxApe6EgnT+jiF//akeOp0AJWEdQnztm5SrlUaGgUU ZBbUU+FlqNNwyVTdQCYmcO4dq1F2aD7qOipnt9y7+ehMmp/R68iCjvqD2v7vUwOxc68Y 4198/p2yEWweeDZF02xwniTuYq5T7I+8fBSRcgTFUIbx64aQJwivZm7wW5jMpC8+SX1d wb9Q9QPFXYCAJ+gy/tbjTemtFn6P4Xb/JSpD6YGNEtE0BmTJN21htKe2zw0NjD3ccOnJ Z3UQ== X-Gm-Message-State: APjAAAXkUlKxCuXOWEUTt6rM8h1quzRXVsyTZNN895ZY9VNjblYu4UGb 13lvrGgLrJaXZWS0ChT6xWqk5Tw9 X-Google-Smtp-Source: APXvYqw2/KGKHcqjq7gSukWNktx38T2brmarVgUtmlr4U4Mgv4cU0dqKMcMEHypyPMyU5NYOy36Sdw== X-Received: by 2002:a19:5212:: with SMTP id m18mr4905676lfb.7.1576141383171; Thu, 12 Dec 2019 01:03:03 -0800 (PST) Received: from mamba.eng.vmware.com ([146.247.46.5]) by smtp.gmail.com with ESMTPSA id u16sm2849908lfi.36.2019.12.12.01.03.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2019 01:03:02 -0800 (PST) From: "Yordan Karadzhov (VMware)" To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, Valentin.Schneider@arm.com, douglas.raillard@arm.com, "Yordan Karadzhov (VMware)" Subject: [PATCH 3/5] Refactor NumPy based data wrapper Date: Thu, 12 Dec 2019 11:02:30 +0200 Message-Id: <20191212090232.24236-4-y.karadz@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191212090232.24236-1-y.karadz@gmail.com> References: <20191212090232.24236-1-y.karadz@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The data wrapper is the only thing that remains being built with Cython. It is now a subpackage called "tracecruncher.datawrapper". Signed-off-by: Yordan Karadzhov (VMware) --- setup.py | 10 ++- src/datawrapper.pyx | 201 ++++++++++++++++++++++++++++++++++++++++++++ src/trace2matrix.c | 29 +++++++ 3 files changed, 239 insertions(+), 1 deletion(-) create mode 100644 src/datawrapper.pyx create mode 100644 src/trace2matrix.c diff --git a/setup.py b/setup.py index dfe1d00..5a07710 100644 --- a/setup.py +++ b/setup.py @@ -16,6 +16,14 @@ def main(): traceevent_path = '/usr/local/lib/traceevent/' tracecmd_path = '/usr/local/lib/trace-cmd/' + cythonize('src/datawrapper.pyx') + module_data = Extension('tracecruncher.datawrapper', + sources=['src/datawrapper.c'], + library_dirs=[kshark_path, traceevent_path, tracecmd_path], + runtime_library_dirs=[kshark_path, traceevent_path, tracecmd_path], + libraries=['kshark', 'traceevent', 'tracecmd'] + ) + module_ks = Extension('tracecruncher.ksharkpy', sources=['src/ksharkpy.c'], library_dirs=[kshark_path], @@ -42,7 +50,7 @@ def main(): url='https://github.com/vmware/trace-cruncher', license='LGPL-2.1', packages=find_packages(), - ext_modules=[module_ks, module_ft], + ext_modules=[module_data, module_ks, module_ft], classifiers=[ 'Development Status :: 3 - Alpha', 'Programming Language :: Python :: 3', diff --git a/src/datawrapper.pyx b/src/datawrapper.pyx new file mode 100644 index 0000000..070d4e4 --- /dev/null +++ b/src/datawrapper.pyx @@ -0,0 +1,201 @@ +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) +""" + +import ctypes + +# Import the Python-level symbols of numpy +import numpy as np +# Import the C-level symbols of numpy +cimport numpy as np + +import json + +from libcpp cimport bool + +from libc.stdlib cimport free + +from cpython cimport PyObject, Py_INCREF + +from libc cimport stdint +ctypedef stdint.int16_t int16_t +ctypedef stdint.uint16_t uint16_t +ctypedef stdint.int32_t int32_t +ctypedef stdint.uint32_t uint32_t +ctypedef stdint.int64_t int64_t +ctypedef stdint.uint64_t uint64_t + +cdef extern from 'numpy/ndarraytypes.h': + int NPY_ARRAY_CARRAY + +# Numpy must be initialized!!! +np.import_array() + +cdef extern from 'trace2matrix.c': + ssize_t trace2matrix(uint64_t **offset_array, + uint16_t **cpu_array, + uint64_t **ts_array, + uint16_t **pid_array, + int **event_array) + +data_column_types = { + 'cpu': np.NPY_UINT16, + 'pid': np.NPY_UINT16, + 'event': np.NPY_INT, + 'offset': np.NPY_UINT64, + 'time': np.NPY_UINT64 + } + +cdef class KsDataWrapper: + cdef int item_size + cdef int data_size + cdef int data_type + cdef void* data_ptr + + cdef init(self, int data_type, + int data_size, + int item_size, + void* data_ptr): + """ This initialization cannot be done in the constructor because + we use C-level arguments. + """ + self.item_size = item_size + self.data_size = data_size + self.data_type = data_type + self.data_ptr = data_ptr + + def __array__(self): + """ Here we use the __array__ method, that is called when numpy + tries to get an array from the object. + """ + cdef np.npy_intp shape[1] + shape[0] = self.data_size + + ndarray = np.PyArray_New(np.ndarray, + 1, shape, + self.data_type, + NULL, + self.data_ptr, + self.item_size, + NPY_ARRAY_CARRAY, + NULL) + + return ndarray + + def __dealloc__(self): + """ Free the data. This is called by Python when all the references to + the object are gone. + """ + free(self.data_ptr) + + +def load(ofst_data=True, cpu_data=True, ts_data=True, + pid_data=True, evt_data=True): + """ Python binding of the 'kshark_load_data_matrix' function that does not + copy the data. The input parameters can be used to avoid loading the + data from the unnecessary fields. + """ + cdef uint64_t *ofst_c + cdef uint16_t *cpu_c + cdef uint64_t *ts_c + cdef uint16_t *pid_c + cdef int *evt_c + + cdef np.ndarray ofst + cdef np.ndarray cpu + cdef np.ndarray ts + cdef np.ndarray pid + cdef np.ndarray evt + + if not ofst_data: + ofst_c = NULL + + if not cpu_data: + cpu_c = NULL + + if not ts_data: + ts_c = NULL + + if not pid_data: + pid_c = NULL + + if not evt_data: + evt_c = NULL + + data_dict = {} + + cdef ssize_t size + + size = trace2matrix(&ofst_c, &cpu_c, &ts_c, &pid_c, &evt_c) + if size <= 0: + raise Exception('No data has been loaded.') + + if cpu_data: + column = 'cpu' + array_wrapper_cpu = KsDataWrapper() + array_wrapper_cpu.init(data_type=data_column_types[column], + data_size=size, + item_size=0, + data_ptr= cpu_c) + + cpu = np.array(array_wrapper_cpu, copy=False) + cpu.base = array_wrapper_cpu + data_dict.update({column: cpu}) + Py_INCREF(array_wrapper_cpu) + + if pid_data: + column = 'pid' + array_wrapper_pid = KsDataWrapper() + array_wrapper_pid.init(data_type=data_column_types[column], + data_size=size, + item_size=0, + data_ptr=pid_c) + + pid = np.array(array_wrapper_pid, copy=False) + pid.base = array_wrapper_pid + data_dict.update({column: pid}) + Py_INCREF(array_wrapper_pid) + + if evt_data: + column = 'event' + array_wrapper_evt = KsDataWrapper() + array_wrapper_evt.init(data_type=data_column_types[column], + data_size=size, + item_size=0, + data_ptr=evt_c) + + evt = np.array(array_wrapper_evt, copy=False) + evt.base = array_wrapper_evt + data_dict.update({column: evt}) + Py_INCREF(array_wrapper_evt) + + if ofst_data: + column = 'offset' + array_wrapper_ofst = KsDataWrapper() + array_wrapper_ofst.init(data_type=data_column_types[column], + data_size=size, + item_size=0, + data_ptr= ofst_c) + + + ofst = np.array(array_wrapper_ofst, copy=False) + ofst.base = array_wrapper_ofst + data_dict.update({column: ofst}) + Py_INCREF(array_wrapper_ofst) + + if ts_data: + column = 'time' + array_wrapper_ts = KsDataWrapper() + array_wrapper_ts.init(data_type=data_column_types[column], + data_size=size, + item_size=0, + data_ptr= ts_c) + + ts = np.array(array_wrapper_ts, copy=False) + ts.base = array_wrapper_ts + data_dict.update({column: ts}) + Py_INCREF(array_wrapper_ts) + + return data_dict diff --git a/src/trace2matrix.c b/src/trace2matrix.c new file mode 100644 index 0000000..aaf8322 --- /dev/null +++ b/src/trace2matrix.c @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: LGPL-2.1 + +/* + * Copyright 2019 VMware Inc, Yordan Karadzhov + */ + +// KernelShark +#include "kernelshark/libkshark.h" + +ssize_t trace2matrix(uint64_t **offset_array, + uint16_t **cpu_array, + uint64_t **ts_array, + uint16_t **pid_array, + int **event_array) +{ + struct kshark_context *kshark_ctx = NULL; + ssize_t total = 0; + + if (!kshark_instance(&kshark_ctx)) + return -1; + + total = kshark_load_data_matrix(kshark_ctx, offset_array, + cpu_array, + ts_array, + pid_array, + event_array); + + return total; +} From patchwork Thu Dec 12 09:02:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yordan Karadzhov X-Patchwork-Id: 11287655 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A9D2D1593 for ; Thu, 12 Dec 2019 09:03:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8820024654 for ; Thu, 12 Dec 2019 09:03:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="s8zn7b6H" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728280AbfLLJDH (ORCPT ); Thu, 12 Dec 2019 04:03:07 -0500 Received: from mail-lf1-f50.google.com ([209.85.167.50]:46184 "EHLO mail-lf1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728218AbfLLJDH (ORCPT ); Thu, 12 Dec 2019 04:03:07 -0500 Received: by mail-lf1-f50.google.com with SMTP id f15so1057197lfl.13 for ; Thu, 12 Dec 2019 01:03:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qDwePEGln2rZJFTcfkO11fGvFsmIaTgTUWpXtppzZdw=; b=s8zn7b6H3ijz4U2YInQbfaPrJt8TxOfZzpRTLEPtFiyL7xf/7iFfMWsxtlFz1rX85B RlvTCrU+uLYxYyeF5ZKwHuV/GEnwLpu3MJr/J5sR5d/LxQnpAMkQXmlD1F3sDi4HVlHS 20Oj3TOE8o4xPS0ZHTHrXGxu8hnduyKzoaq0ANoGxP5UenWe3Sf2OB7BQ9zsnFge+UJE jnA64BTsP0k7nubLVVIqW8a1vGRNI9WBbRFlHRBIVKDOa5DBwqmrA8gEVE35dzhKc8Dv HD+L/O6fc5jrVzC5SYJTl9vboch7KsQS6I7kZyPLeELMGMruvAf3XE4d6sQdtF2eXUp+ 3lLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qDwePEGln2rZJFTcfkO11fGvFsmIaTgTUWpXtppzZdw=; b=rlswwIClsbI6u+VgMzdoaLI5bHMUSpeO+zuhm5U8XQ/bCJCSvHko7y0//FVuBjyCtU L+0zkSkGVt0VOrG0S2AFa+EeqykxhCAQ0t50iFP4FAYaPx6X6DuvOHCgpFnUHzHvqUUR BfK2kOJ1BV8kslhgVUM9NALJFZ8eyW5vpHh5d9QS4NgR+yDYCyHUA3MdvdEhVpUYnSwK zxUpwWytuhUEAX52enWSXedEGrOnihGb7dOKzJGzDs7ntYy+zl0+7rKKCDD0h+VsjMYD Cb3/1sPmewU85ixOfnP3hhNObgbUu7nW/V4OWLUU8O2PaijXc3qtXjVavvam/hC9IYk5 gMNw== X-Gm-Message-State: APjAAAWiaiCNNVNDsGFYTgJ8du3h174TqYCnCvLuT4knLHtIPzOI897J /YvO71VJrL7nE6/vT6q8eKsBvIlx X-Google-Smtp-Source: APXvYqwkOi8uEK18UNFkxtBQ2Un7ZABPVybLCYQaacVN6ugn6NJE7N1RKaLjwYmxK0QzjwVA45/08w== X-Received: by 2002:a19:6455:: with SMTP id b21mr4681359lfj.45.1576141384628; Thu, 12 Dec 2019 01:03:04 -0800 (PST) Received: from mamba.eng.vmware.com ([146.247.46.5]) by smtp.gmail.com with ESMTPSA id u16sm2849908lfi.36.2019.12.12.01.03.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2019 01:03:04 -0800 (PST) From: "Yordan Karadzhov (VMware)" To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, Valentin.Schneider@arm.com, douglas.raillard@arm.com, "Yordan Karadzhov (VMware)" Subject: [PATCH 4/5] Add "utils" Date: Thu, 12 Dec 2019 11:02:31 +0200 Message-Id: <20191212090232.24236-5-y.karadz@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191212090232.24236-1-y.karadz@gmail.com> References: <20191212090232.24236-1-y.karadz@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Place the rest of the code, that is pure Python in tracecrunche/utils.py Signed-off-by: Yordan Karadzhov (VMware) --- tracecruncher/utils.py | 54 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 tracecruncher/utils.py diff --git a/tracecruncher/utils.py b/tracecruncher/utils.py new file mode 100644 index 0000000..f785c01 --- /dev/null +++ b/tracecruncher/utils.py @@ -0,0 +1,54 @@ +""" +SPDX-License-Identifier: LGPL-2.1 + +Copyright 2019 VMware Inc, Yordan Karadzhov (VMware) +""" + +import json + +from . import datawrapper as dw +from . import ksharkpy as ks + +def size(data): + """ Get the number of trace records. + """ + for key in dw.data_column_types: + if data[key] is not None: + return data[key].size + + raise Exception('Data size is unknown.') + +def save_session(session, s): + """ Save a KernelShark session description of a JSON file. + """ + s.seek(0) + json.dump(session, s, indent=4) + s.truncate() + + +def new_gui_session(fname, sname): + """ Generate and save a default KernelShark session description + file (JSON). + """ + ks.new_session_file(fname, sname) + + with open(sname, 'r+') as s: + session = json.load(s) + + session['Filters']['filter mask'] = 7 + session['CPUPlots'] = [] + session['TaskPlots'] = [] + session['Splitter'] = [1, 1] + session['MainWindow'] = [1200, 800] + session['ViewTop'] = 0 + session['ColorScheme'] = 0.75 + session['Model']['bins'] = 1000 + + session['Markers']['markA'] = {} + session['Markers']['markA']['isSet'] = False + session['Markers']['markB'] = {} + session['Markers']['markB']['isSet'] = False + session['Markers']['Active'] = 'A' + + save_session(session, s) + From patchwork Thu Dec 12 09:02:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yordan Karadzhov X-Patchwork-Id: 11287657 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01E22109A for ; Thu, 12 Dec 2019 09:03:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D3A7724654 for ; Thu, 12 Dec 2019 09:03:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QcZMEG2b" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728218AbfLLJDI (ORCPT ); Thu, 12 Dec 2019 04:03:08 -0500 Received: from mail-lf1-f66.google.com ([209.85.167.66]:39807 "EHLO mail-lf1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728256AbfLLJDI (ORCPT ); Thu, 12 Dec 2019 04:03:08 -0500 Received: by mail-lf1-f66.google.com with SMTP id y1so1086607lfb.6 for ; Thu, 12 Dec 2019 01:03:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gC9y5Vi2lQqNZJf6Se2uz2rT5mhaFKhXP+phl5JdvNA=; b=QcZMEG2bnC/SUmm9tzYbMlM0DK9YcxJhXpfdRpgDLpGcwaGIJDJILDej8JQ5ub1b19 rIxVpcOmzomD7G0nywYsJPT66E7Cn8PnTBTXWWrFoQ/xh5VBo9j7KMJ4DOR1lu5y+D5z 6K0oXzoVc9P1PByrpnCBXQISOoiwibibw88DPWKDFdXebSkER+Pxn7Rnf2IMRymOGx+t fb+6NFRUQG2MZjE3CNlMIruPrvmj5ezOGNwlZh9QQwNHD9N8ogPa0mzAUazL+DDFrLzp FyzjOnFZ76CtooFOnDCkUZFFbAOvmxCdBvoBcohv6qrNc00Ggd5Q++TizZ3f1EN4YHHm UpYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gC9y5Vi2lQqNZJf6Se2uz2rT5mhaFKhXP+phl5JdvNA=; b=Se78ZqVR9zk//FNSMgqTN3rKU5rqPplx6i9b5kCt6AY7Q66APXGUpKSW5bYsgu6jGR i6LGQJbBWvb2QHQBsO1X/1FklOJUFaotnW9Q1MNcKcNe54zEQeEJSzFgZnh1xp7sxxaj oFmqbJPA4DeJeLYOq0eucB8hQ8+zaUy7NJMBBPYLnz7n4NlCSASwlGBjE5YPReJygObc tQbSqwJs9NiUd0ZtbQujIDddEDdQrT8cushEZg7QzKDKh7HU9o2/cC5glHgwQy2Ua8aQ DkZk7Fr4k9cLDHFR0TFiY78t9STSlDQ1+QVjLfvcH/7dRZW71jh2rY0UHFCeXQsHxAhm Whsg== X-Gm-Message-State: APjAAAW+ErQoEhO0PeFdkkR140T0m8HOa2/x/7k9aYcIYcogsaGOFNTE 1Ii6SLLAcA6lJyKaIoFREKzepR2T X-Google-Smtp-Source: APXvYqyolVADbiAYiwaeMlwsSQDhMPAajCcz0weC/TnR4nq/eZ1gQStfVBEK86JNWUo9VHJUWevjpA== X-Received: by 2002:ac2:5444:: with SMTP id d4mr5163131lfn.49.1576141386163; Thu, 12 Dec 2019 01:03:06 -0800 (PST) Received: from mamba.eng.vmware.com ([146.247.46.5]) by smtp.gmail.com with ESMTPSA id u16sm2849908lfi.36.2019.12.12.01.03.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2019 01:03:05 -0800 (PST) From: "Yordan Karadzhov (VMware)" To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, Valentin.Schneider@arm.com, douglas.raillard@arm.com, "Yordan Karadzhov (VMware)" Subject: [PATCH 5/5] Adapt the sched_wakeup.py example script to use the new tracecruncher module Date: Thu, 12 Dec 2019 11:02:32 +0200 Message-Id: <20191212090232.24236-6-y.karadz@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191212090232.24236-1-y.karadz@gmail.com> References: <20191212090232.24236-1-y.karadz@gmail.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Signed-off-by: Yordan Karadzhov (VMware) --- examples/sched_wakeup.py | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/examples/sched_wakeup.py b/examples/sched_wakeup.py index 52f2688..41e0c9c 100755 --- a/examples/sched_wakeup.py +++ b/examples/sched_wakeup.py @@ -15,18 +15,18 @@ import matplotlib.pyplot as plt import scipy.stats as st import numpy as np -from ksharksetup import setup -# Always call setup() before importing ksharkpy!!! -setup() +import tracecruncher.datawrapper as dw +import tracecruncher.ksharkpy as ks +import tracecruncher.ftracepy as ft +import tracecruncher.utils as tc -import ksharkpy as ks # Get the name of the user program. if len(sys.argv) >= 2: fname = str(sys.argv[1]) else: fname = input('choose a trace file: ') -status = ks.open_file(fname) +status = ks.open(fname) if not status: print ("Failed to open file ", fname) sys.exit() @@ -35,7 +35,7 @@ ks.register_plugin('reg_pid') # We do not need the Process Ids of the records. # Do not load the "pid" data. -data = ks.load_data(pid_data=False) +data = dw.load(pid_data=False) tasks = ks.get_tasks() # Get the name of the user program. @@ -48,8 +48,8 @@ else: task_pid = tasks[prog_name][0] # Get the Event Ids of the sched_switch and sched_waking events. -ss_eid = ks.event_id('sched', 'sched_switch') -w_eid = ks.event_id('sched', 'sched_waking') +ss_eid = ft.event_id('sched', 'sched_switch') +w_eid = ft.event_id('sched', 'sched_waking') # Gey the size of the data. i = data['offset'].size @@ -60,7 +60,7 @@ delta_max = i_ss_max = i_sw_max = 0 while i > 0: i = i - 1 if data['event'][i] == ss_eid: - next_pid = ks.read_event_field(offset=data['offset'][i], + next_pid = ft.read_event_field(offset=data['offset'][i], event_id=ss_eid, field='next_pid') @@ -77,9 +77,9 @@ while i > 0: break if data['event'][i] == ss_eid: - next_pid = ks.read_event_field(offset=data['offset'][i], - event_id=ss_eid, - field='next_pid') + next_pid = ft.read_event_field(offset=data['offset'][i], + event_id=ss_eid, + field='next_pid') if next_pid == task_pid: # Second sched_switch for the same task. ? time_ss = data['time'][i] @@ -89,7 +89,7 @@ while i > 0: continue if (data['event'][i] == w_eid): - waking_pid = ks.read_event_field(offset=data['offset'][i], + waking_pid = ft.read_event_field(offset=data['offset'][i], event_id=w_eid, field='pid') @@ -120,7 +120,7 @@ ax.hist(dt, bins=(100), histtype='step') plt.show() sname = 'sched.json' -ks.new_session(fname, sname) +tc.new_gui_session(fname, sname) with open(sname, 'r+') as s: session = json.load(s) @@ -143,6 +143,6 @@ with open(sname, 'r+') as s: session['ViewTop'] = int(i_sw_max) - 5 - ks.save_session(session, s) + tc.save_session(session, s) ks.close()