| Message ID | 20210422073902.484953-1-tz.stoyanov@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add trace file compression | expand |
On Thu, 22 Apr 2021 10:38:52 +0300 "Tzvetomir Stoyanov (VMware)" <tz.stoyanov@gmail.com> wrote: > A new PoC implementation for adding a compression to trace file version 7. > A basic infrastructure for compression support is added into the trace-cmd > library. The zlib is used for compression, but more libraries and algorithms > can be added. Only part of the trace file metadata is compressed: > - ftrace events format > - format of recorded events > - information about the mapping of function addresses to the function names > - trace_printk() format strings > - information of the mapping a PID to a process name > > Note: not all trace-cmd commands are tested with these changes. These are > verified to work with compressed trace files: > trace-cmd record > trace-cmd report > trace-cmd dump > > Todo list, in order to have complete trace file compression: > - Compress trace file options. > - Compress trace data. > - Add support for more compression libraries. > - Add new trace-cmd command to convert trace files from version 6 to > version 7 and vise versa. > - Add compression support to all trace-cmd commands. > - Update trace-cmd documentation. > > This patch set depends on "[PATCH 0/6] Bump trace file version" > and should be applied on top of it. > https://lore.kernel.org/linux-trace-devel/20210422071718.483383-1-tz.stoyanov@gmail.com/ > > Tzvetomir Stoyanov (VMware) (10): > trace-cmd: Add APIs for library initialization and free > trace-cmd library: Add support for compression algorithms > trace-cmd library: Compress part of the trace file > trace-cmd library: Read compressed trace file > trace-cmd library: Add new API to get compression of input handler > trace-cmd library: Inherit compression algorithm from input file > trace-cmd library: Extend the create file APIs to support different > compression > trace-cmd record: Add new parameter --compression > trace-cmd dump: Add support for trace files version 7 > trace-cmd library: Add support for zlib compression library I'm not able to apply these patches as I'm getting conflicts from your previous series that bumps the version number, as if what you sent to the mailing list was not what you have in your git repo. Can you please resend the version update patch series. -- Steve > > lib/trace-cmd/Makefile | 11 + > .../include/private/trace-cmd-private.h | 42 ++- > lib/trace-cmd/include/trace-cmd-local.h | 7 + > lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++ > lib/trace-cmd/trace-compress.c | 289 +++++++++++++++++ > lib/trace-cmd/trace-input.c | 304 +++++++++++++----- > lib/trace-cmd/trace-output.c | 268 ++++++++++++--- > lib/trace-cmd/trace-util.c | 12 + > tracecmd/trace-cmd.c | 11 +- > tracecmd/trace-dump.c | 164 ++++++++-- > tracecmd/trace-record.c | 18 +- > tracecmd/trace-restore.c | 4 +- > tracecmd/trace-usage.c | 1 + > 13 files changed, 1152 insertions(+), 151 deletions(-) > create mode 100644 lib/trace-cmd/trace-compress-zlib.c > create mode 100644 lib/trace-cmd/trace-compress.c >
On Thu, Apr 29, 2021 at 4:35 AM Steven Rostedt <rostedt@goodmis.org> wrote: > > On Thu, 22 Apr 2021 10:38:52 +0300 > "Tzvetomir Stoyanov (VMware)" <tz.stoyanov@gmail.com> wrote: > > > A new PoC implementation for adding a compression to trace file version 7. > > A basic infrastructure for compression support is added into the trace-cmd > > library. The zlib is used for compression, but more libraries and algorithms > > can be added. Only part of the trace file metadata is compressed: > > - ftrace events format > > - format of recorded events > > - information about the mapping of function addresses to the function names > > - trace_printk() format strings > > - information of the mapping a PID to a process name > > > > Note: not all trace-cmd commands are tested with these changes. These are > > verified to work with compressed trace files: > > trace-cmd record > > trace-cmd report > > trace-cmd dump > > > > Todo list, in order to have complete trace file compression: > > - Compress trace file options. > > - Compress trace data. > > - Add support for more compression libraries. > > - Add new trace-cmd command to convert trace files from version 6 to > > version 7 and vise versa. > > - Add compression support to all trace-cmd commands. > > - Update trace-cmd documentation. > > > > This patch set depends on "[PATCH 0/6] Bump trace file version" > > and should be applied on top of it. > > https://lore.kernel.org/linux-trace-devel/20210422071718.483383-1-tz.stoyanov@gmail.com/ > > > > Tzvetomir Stoyanov (VMware) (10): > > trace-cmd: Add APIs for library initialization and free > > trace-cmd library: Add support for compression algorithms > > trace-cmd library: Compress part of the trace file > > trace-cmd library: Read compressed trace file > > trace-cmd library: Add new API to get compression of input handler > > trace-cmd library: Inherit compression algorithm from input file > > trace-cmd library: Extend the create file APIs to support different > > compression > > trace-cmd record: Add new parameter --compression > > trace-cmd dump: Add support for trace files version 7 > > trace-cmd library: Add support for zlib compression library > > I'm not able to apply these patches as I'm getting conflicts from your > previous series that bumps the version number, as if what you sent to > the mailing list was not what you have in your git repo. > > Can you please resend the version update patch series. The order of applying the changes should be: 1. [PATCH v2] trace-cmd: Check if file version is supported https://lore.kernel.org/linux-trace-devel/20210419093156.200099-1-tz.stoyanov@gmail.com/ 2. [PATCH 0/6] Bump trace file version https://lore.kernel.org/linux-trace-devel/20210422071718.483383-1-tz.stoyanov@gmail.com/ 3. [POC PATCH 00/10] Add trace file compression https://lore.kernel.org/linux-trace-devel/20210422073902.484953-1-tz.stoyanov@gmail.com/ I wrote these dependencies in the cover-letters. BTW, I do not see (2) in patchwork, only in the mailing list. I'll send v2 of "Bump trace file version" including (1) in the patchset. > > -- Steve > > > > > > lib/trace-cmd/Makefile | 11 + > > .../include/private/trace-cmd-private.h | 42 ++- > > lib/trace-cmd/include/trace-cmd-local.h | 7 + > > lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++ > > lib/trace-cmd/trace-compress.c | 289 +++++++++++++++++ > > lib/trace-cmd/trace-input.c | 304 +++++++++++++----- > > lib/trace-cmd/trace-output.c | 268 ++++++++++++--- > > lib/trace-cmd/trace-util.c | 12 + > > tracecmd/trace-cmd.c | 11 +- > > tracecmd/trace-dump.c | 164 ++++++++-- > > tracecmd/trace-record.c | 18 +- > > tracecmd/trace-restore.c | 4 +- > > tracecmd/trace-usage.c | 1 + > > 13 files changed, 1152 insertions(+), 151 deletions(-) > > create mode 100644 lib/trace-cmd/trace-compress-zlib.c > > create mode 100644 lib/trace-cmd/trace-compress.c > > >
On Thu, 29 Apr 2021 06:51:59 +0300 Tzvetomir Stoyanov <tz.stoyanov@gmail.com> wrote: > The order of applying the changes should be: > 1. [PATCH v2] trace-cmd: Check if file version is supported > https://lore.kernel.org/linux-trace-devel/20210419093156.200099-1-tz.stoyanov@gmail.com/ > 2. [PATCH 0/6] Bump trace file version > https://lore.kernel.org/linux-trace-devel/20210422071718.483383-1-tz.stoyanov@gmail.com/ > 3. [POC PATCH 00/10] Add trace file compression > https://lore.kernel.org/linux-trace-devel/20210422073902.484953-1-tz.stoyanov@gmail.com/ > > I wrote these dependencies in the cover-letters. BTW, I do not see Yeah, I see that. But I was using patchwork to read the files and not the email. But there is a way (not totally obvious) to read the cover letter from patchwork. (just found it now!) > (2) in patchwork, only in the mailing list. > I'll send v2 of "Bump trace file version" including (1) in the patchset. As I stated in the other email. After I make comments, and expect a newer version, I'll set the older one to "changes needed", which makes it disappear from the default view. You can still see it if you deselect "Action Required". -- Steve
A new PoC implementation for adding a compression to trace file version 7. A basic infrastructure for compression support is added into the trace-cmd library. The zlib is used for compression, but more libraries and algorithms can be added. Only part of the trace file metadata is compressed: - ftrace events format - format of recorded events - information about the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Note: not all trace-cmd commands are tested with these changes. These are verified to work with compressed trace files: trace-cmd record trace-cmd report trace-cmd dump Todo list, in order to have complete trace file compression: - Compress trace file options. - Compress trace data. - Add support for more compression libraries. - Add new trace-cmd command to convert trace files from version 6 to version 7 and vise versa. - Add compression support to all trace-cmd commands. - Update trace-cmd documentation. This patch set depends on "[PATCH 0/6] Bump trace file version" and should be applied on top of it. https://lore.kernel.org/linux-trace-devel/20210422071718.483383-1-tz.stoyanov@gmail.com/ Tzvetomir Stoyanov (VMware) (10): trace-cmd: Add APIs for library initialization and free trace-cmd library: Add support for compression algorithms trace-cmd library: Compress part of the trace file trace-cmd library: Read compressed trace file trace-cmd library: Add new API to get compression of input handler trace-cmd library: Inherit compression algorithm from input file trace-cmd library: Extend the create file APIs to support different compression trace-cmd record: Add new parameter --compression trace-cmd dump: Add support for trace files version 7 trace-cmd library: Add support for zlib compression library lib/trace-cmd/Makefile | 11 + .../include/private/trace-cmd-private.h | 42 ++- lib/trace-cmd/include/trace-cmd-local.h | 7 + lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++ lib/trace-cmd/trace-compress.c | 289 +++++++++++++++++ lib/trace-cmd/trace-input.c | 304 +++++++++++++----- lib/trace-cmd/trace-output.c | 268 ++++++++++++--- lib/trace-cmd/trace-util.c | 12 + tracecmd/trace-cmd.c | 11 +- tracecmd/trace-dump.c | 164 ++++++++-- tracecmd/trace-record.c | 18 +- tracecmd/trace-restore.c | 4 +- tracecmd/trace-usage.c | 1 + 13 files changed, 1152 insertions(+), 151 deletions(-) create mode 100644 lib/trace-cmd/trace-compress-zlib.c create mode 100644 lib/trace-cmd/trace-compress.c