From patchwork Thu Nov 11 15:11:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA697C433F5 for ; Thu, 11 Nov 2021 15:11:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3E0661252 for ; Thu, 11 Nov 2021 15:11:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233970AbhKKPOs (ORCPT ); Thu, 11 Nov 2021 10:14:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233715AbhKKPOr (ORCPT ); Thu, 11 Nov 2021 10:14:47 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B96FC061766 for ; Thu, 11 Nov 2021 07:11:58 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id j21so25218962edt.11 for ; Thu, 11 Nov 2021 07:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=B82wbR+fQBe7n7Wo2UNgv0z81FwwEm6EZNFgYYFUGSs=; b=RCcZhvHM1gXmP3J0N9X1XUQhZV3a8cnkK3LbBrM9d8yxwBNPNGu8UXQRRG0tc/zrV/ RuH4aoqJibrsu/m4HOPMTQ+Xc/LBgUXEZWfTsJQcLO5pceNJbhd73g4wTT3n1t/wTH+3 p0Lesf3FMEduYIGtCYr2fU7rM8iGZGHIskTMKHE76yv6KC/ISTu6F8yRITnZkYwpl3xW HEJllN754gI+gP+/o95kxbAiD6VrJ9hQVAnf0+poTddWgez9MzhZgDoVoT/RKsqjw7KM 80B1UWRUrGHJcyKcEuUeiab2TLUIwzX4VvMI+30cMOHF9dZlnGt9JHa7d6TYIqfRXWHc uAEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B82wbR+fQBe7n7Wo2UNgv0z81FwwEm6EZNFgYYFUGSs=; b=U4QuFsi7le92sfTxCJG/Rjekpb3+XjB+5TDVaMBwfpSYoNUT5333tgpecIAFYLbJu3 VVOls7zXSEqc0ucriiIuIUzHXje/2q0NkWoRwwYK/zi5lc16OE4ZjHw4gsv8q/0bJG8e bAebty49Kd39BfMIQr+nfqvOe3inQ6TL6koc5tD67xOyEZH93p/6YbBN+a78wdFOJzYv 6ehwWJBgPOJKIFNEVe+7nZUTh48hF451g00I6Y2ERiJzV72fGLeKDhNV6SFR4G6S+g7t pgYoN7gV/4TwLbOxfE3ChAuB0UkrMnNdD5K/XRT6RovPAk25lusFF05Xv8TGMSS+WMju Qamw== X-Gm-Message-State: AOAM533/Gvzmne58FhTY9dWPCd9onXQIZEZGN7xeAIR49TPJl/EL+x52 FOc432F6R5cAL6Hhq5NmB5dGIXDpSZTIOg== X-Google-Smtp-Source: ABdhPJw+ZSxSPcy+IerVMusYce56InW1ZcY3SqTazPBP9SHD0iwuJuuYok45pdQdtoyJIqwbKg4zNA== X-Received: by 2002:a05:6402:1d50:: with SMTP id dz16mr10712899edb.191.1636643516411; Thu, 11 Nov 2021 07:11:56 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.11.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:11:55 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 01/20] trace-cmd library: Add support for compression algorithms Date: Thu, 11 Nov 2021 17:11:34 +0200 Message-Id: <20211111151153.86855-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added infrastructure to trace-cmd library for compression. Introduced various new APIs to work with this new functionality: struct tracecmd_compression tracecmd_compress_init() tracecmd_compress_free() tracecmd_compress_alloc() tracecmd_compress_destroy() tracecmd_compress_block() tracecmd_uncompress_block() tracecmd_compress_reset() tracecmd_compress_read() tracecmd_compress_pread() tracecmd_compress_write() tracecmd_compress_lseek() tracecmd_compress_proto_get_name() tracecmd_compress_is_supported() tracecmd_compress_protos_get() tracecmd_compress_proto_register() tracecmd_compress_copy_from() tracecmd_uncompress_copy_to() tracecmd_uncompress_chunk() tracecmd_load_chunks_info() The compression algorithms are not part of this patch. Added trace-cmd library constructor and destructor routines, used to initialize and free compression context. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 1 + .../include/private/trace-cmd-private.h | 41 + lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-compress.c | 907 ++++++++++++++++++ lib/trace-cmd/trace-util.c | 10 + 5 files changed, 962 insertions(+) create mode 100644 lib/trace-cmd/trace-compress.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index 17600318..bab4322d 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -25,6 +25,7 @@ ifeq ($(VSOCK_DEFINED), 1) OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif +OBJS += trace-compress.o # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 879640ac..0c7027dd 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -36,6 +36,7 @@ int *tracecmd_add_id(int *list, int id, int len); #define FILE_VERSION_MAX 7 #define FILE_VERSION_SECTIONS 7 +#define FILE_VERSION_COMPRESSION 7 enum { RINGBUF_TYPE_PADDING = 29, @@ -154,6 +155,7 @@ enum { TRACECMD_FL_IN_USECS = (1 << 2), TRACECMD_FL_RAW_TS = (1 << 3), TRACECMD_FL_SECTIONED = (1 << 4), + TRACECMD_FL_COMPRESSION = (1 << 5), }; struct tracecmd_ftrace { @@ -487,6 +489,45 @@ void tracecmd_tsync_free(struct tracecmd_time_sync *tsync); int tracecmd_write_guest_time_shift(struct tracecmd_output *handle, struct tracecmd_time_sync *tsync); +/* --- Compression --- */ +struct tracecmd_compress_chunk { + unsigned int size; + unsigned int zsize; + off64_t zoffset; + off64_t offset; +}; +struct tracecmd_compression; +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle); +void tracecmd_compress_destroy(struct tracecmd_compression *handle); +int tracecmd_compress_block(struct tracecmd_compression *handle); +int tracecmd_uncompress_block(struct tracecmd_compression *handle); +void tracecmd_compress_reset(struct tracecmd_compression *handle); +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len); +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset); +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size); +off64_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off64_t offset, int whence); +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version); +bool tracecmd_compress_is_supported(const char *name, const char *version); +int tracecmd_compress_protos_get(char ***names, char ***versions); +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)); +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_chunk(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk *chunk, char *data); +int tracecmd_load_chunks_info(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk **chunks_info); /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 4b26960f..fb008678 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -34,6 +34,9 @@ struct data_file_write { unsigned long long file_data_offset; }; +void tracecmd_compress_init(void); +void tracecmd_compress_free(void); + bool check_file_state(unsigned long file_version, int current_state, int new_state); bool check_out_state(struct tracecmd_output *handle, int new_state); diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c new file mode 100644 index 00000000..b198b8dc --- /dev/null +++ b/lib/trace-cmd/trace-compress.c @@ -0,0 +1,907 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include +#include + +#include "trace-cmd-private.h" +#include "trace-cmd-local.h" + +struct compress_proto { + struct compress_proto *next; + char *proto_name; + char *proto_version; + int weight; + + int (*compress_block)(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + int (*uncompress_block)(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + unsigned int (*compress_size)(unsigned int bytes); + bool (*is_supported)(const char *name, const char *version); +}; + +static struct compress_proto *proto_list; + +struct tracecmd_compression { + int fd; + unsigned int capacity; + unsigned long pointer; + char *buffer; + struct compress_proto *proto; + struct tep_handle *tep; + struct tracecmd_msg_handle *msg_handle; +}; + +static int read_fd(int fd, char *dst, int len) +{ + size_t size = 0; + int r; + + do { + r = read(fd, dst+size, len); + if (r > 0) { + size += r; + len -= r; + } else + break; + } while (r > 0); + + if (len) + return -1; + return size; +} + +static long long write_fd(int fd, const void *data, size_t size) +{ + long long tot = 0; + long long w; + + do { + w = write(fd, data + tot, size - tot); + tot += w; + + if (!w) + break; + if (w < 0) + return w; + } while (tot != size); + + return tot; +} + +static long long do_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + int ret; + + if (handle->msg_handle) { + ret = tracecmd_msg_data_send(handle->msg_handle, data, size); + if (ret) + return -1; + return size; + } + return write_fd(handle->fd, data, size); +} + +static inline int buffer_extend(struct tracecmd_compression *handle, unsigned int size) +{ + int extend; + char *buf; + + if (size <= handle->capacity) + return 0; + + extend = ((size - handle->capacity) / BUFSIZ + 1) * BUFSIZ; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return -1; + handle->buffer = buf; + handle->capacity += extend; + + return 0; +} + +/** + * tracecmd_compress_lseek - Move the read/write pointer into the compression buffer + * @handle: compression handler + * @offset: number of bytes to move the pointer, can be negative or positive + * @whence: the starting position of the pointer movement, + * + * Returns the new file pointer on success, or -1 in case of an error. + */ +off64_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off64_t offset, int whence) +{ + unsigned long p; + + if (!handle || !handle->buffer) + return (off64_t)-1; + + switch (whence) { + case SEEK_CUR: + p = handle->pointer + offset; + break; + case SEEK_END: + p = handle->capacity + offset; + break; + case SEEK_SET: + p = offset; + break; + default: + return (off64_t)-1; + } + + if (buffer_extend(handle, p)) + return (off64_t)-1; + + handle->pointer = p; + + return p; +} + +static int compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int s; + + if (handle->pointer + len > handle->capacity) + s = handle->capacity - handle->pointer; + else + s = len; + memcpy(dst, handle->buffer + handle->pointer, s); + + return s; +} + +/** + * tracecmd_compress_pread - pread() on compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * @offset: offset in the buffer of data to be read + * + * Read a @len of data from the compression buffer at given @offset, + * without updating the buffer pointer. + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset) +{ + int ret; + + if (!handle || !handle->buffer || offset > handle->capacity) + return -1; + + ret = tracecmd_compress_lseek(handle, offset, SEEK_SET); + if (ret < 0) + return ret; + return compress_read(handle, dst, len); +} + +/** + * tracecmd_compress_read - read() from compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * + * Read a @len of data from the compression buffer + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int ret; + + if (!handle || !handle->buffer) + return -1; + + ret = compress_read(handle, dst, len); + if (ret > 0) + handle->pointer += ret; + + return ret; +} + +/** + * tracecmd_compress_reset - Reset the compression buffer + * @handle: compression handler + * + * Reset the compression buffer, any data currently in the buffer will be destroyed. + * + */ +void tracecmd_compress_reset(struct tracecmd_compression *handle) +{ + if (!handle) + return; + + free(handle->buffer); + handle->buffer = NULL; + handle->pointer = 0; + handle->capacity = 0; +} + +/** + * tracecmd_uncompress_block - uncompress a memory block + * @handle: compression handler + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_uncompress_block(struct tracecmd_compression *handle) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + char *bytes = NULL; + char buf[4]; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + tracecmd_compress_reset(handle); + + if (read(handle->fd, buf, 4) != 4) + return -1; + s_compressed = tep_read_number(handle->tep, buf, 4); + if (read(handle->fd, buf, 4) != 4) + return -1; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + + handle->buffer = malloc(s_uncompressed); + if (!handle->buffer) + return -1; + bytes = malloc(s_compressed); + if (!bytes) + goto error; + + if (read_fd(handle->fd, bytes, s_compressed) < 0) + goto error; + ret = handle->proto->uncompress_block(bytes, s_compressed, + handle->buffer, &s_uncompressed); + if (ret) + goto error; + free(bytes); + handle->pointer = 0; + handle->capacity = s_uncompressed; + return 0; +error: + tracecmd_compress_reset(handle); + free(bytes); + return -1; +} + +/** + * tracecmd_compress_block - compress a memory block + * @handle: compression handler + * + * Compress the content of the internal memory buffer and write the compressed data in the file + * The tracecmd_compress_write() can be used to write data into the internal memory buffer, before + * calling this API. + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_compress_block(struct tracecmd_compression *handle) +{ + unsigned int size; + char *buf; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->compress_size || !handle->proto->compress_block) + return -1; + + size = handle->proto->compress_size(handle->pointer); + buf = malloc(size); + if (!buf) + return -1; + ret = handle->proto->compress_block(handle->buffer, handle->pointer, buf, &size); + if (ret < 0) + goto out; + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &handle->pointer, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write compressed data */ + ret = do_write(handle, buf, size); + ret = ((ret == size) ? 0 : -1); +out: + tracecmd_compress_reset(handle); + free(buf); + return ret; +} + +/** + * tracecmd_compress_write - write() to compression buffer + * @handle: compression handler + * @data: data to be written + * @size: size of @data + * + * Write @data of @size in the compression buffer + * + * Returns 0 on success, or -1 on failure. + */ +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + if (!handle) + return -1; + + if (buffer_extend(handle, (handle->pointer + size))) + return -1; + + memcpy(&handle->buffer[handle->pointer], data, size); + handle->pointer += size; + return 0; +} + +/** + * tracecmd_compress_init - initialize the library with available compression algorithms + */ +void tracecmd_compress_init(void) +{ + struct timeval time; + + gettimeofday(&time, NULL); + srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); + +} + +static struct compress_proto *compress_proto_select(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *selected = NULL; + + while (proto) { + if (!selected || selected->weight > proto->weight) + selected = proto; + proto = proto->next; + } + + return selected; +} + +/** + * tracecmd_compress_alloc - Allocate a new compression context + * @name: name of the compression algorithm, if NULL - auto select the best available algorithm + * @version: version of the compression algorithm, can be NULL + * @fd: file descriptor for reading / writing data + * @tep: tep handler, used to encode the data + * @msg_handle: message handler, use it for reading / writing data instead of @fd + * + * Returns NULL on failure or pointer to allocated compression context. + * The returned context must be freed by tracecmd_compress_destroy() + */ +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle) +{ + struct tracecmd_compression *new; + struct compress_proto *proto; + + if (name) { + proto = proto_list; + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + break; + proto = proto->next; + } + } else { + proto = compress_proto_select(); + } + if (!proto) + return NULL; + + new = calloc(1, sizeof(*new)); + if (!new) + return NULL; + new->fd = fd; + new->tep = tep; + new->msg_handle = msg_handle; + new->proto = proto; + return new; +} + +/** + * tracecmd_compress_destroy - Free a compression context + * @handle: handle to the compression context that will be freed + */ +void tracecmd_compress_destroy(struct tracecmd_compression *handle) +{ + tracecmd_compress_reset(handle); + free(handle); +} + +/** + * tracecmd_compress_is_supported - check if compression algorithm with given name and + * version is supported + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * + * Returns true if the algorithm with given name and version is supported or false if it is not. + */ +bool tracecmd_compress_is_supported(const char *name, const char *version) +{ + struct compress_proto *proto = proto_list; + + if (!name) + return NULL; + + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + return true; + proto = proto->next; + } + return false; +} + +/** + * tracecmd_compress_proto_get_name - get name and version of compression algorithm + * @compress: compression handler. + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version) +{ + if (!compress || !compress->proto) + return -1; + if (name) + *name = compress->proto->proto_name; + if (version) + *version = compress->proto->proto_version; + return 0; +} + +/** + * tracecmd_compress_proto_register - register a new compression algorithm + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * @weight: weight of the compression algorithm, lower is better. + * @compress: compression hook, called to compress a memory block. + * @uncompress: uncompression hook, called to uncompress a memory block. + * @compress_size: hook, called to get the required minimum size of the buffer for compression + * given number of bytes. + * @is_supported: check hook, called to check if compression with given name and version is + * supported by this plugin. + * + * Returns 0 on success, or -1 in case of an error. If algorithm with given name and version is + * already registered, -1 is returned. + */ +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*compress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)) +{ + struct compress_proto *new; + + if (!name || !compress || !uncompress) + return -1; + if (tracecmd_compress_is_supported(name, version)) + return -1; + + new = calloc(1, sizeof(*new)); + if (!new) + return -1; + + new->proto_name = strdup(name); + if (!new->proto_name) + goto error; + new->proto_version = strdup(version); + if (!new->proto_version) + goto error; + new->compress_block = compress; + new->uncompress_block = uncompress; + new->compress_size = compress_size; + new->is_supported = is_supported; + new->weight = weight; + new->next = proto_list; + proto_list = new; + return 0; + +error: + free(new->proto_name); + free(new->proto_version); + free(new); + return -1; +} + +/** + * tracecmd_compress_free - free the library resources, related to available compression algorithms + * + */ +void tracecmd_compress_free(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *del; + + while (proto) { + del = proto; + proto = proto->next; + free(del->proto_name); + free(del->proto_version); + free(del); + } + proto_list = NULL; +} + +/** + * tracecmd_compress_protos_get - get a list of all supported compression algorithms and versions + * @names: return, array with names of all supported compression algorithms + * @versions: return, array with versions of all supported compression algorithms + * + * On success, the size of @names and @versions arrays is returned. Those arrays are allocated by + * the API and must be freed with free() by the caller. Both arrays are with same size, each name + * from @names corresponds to a version from @versions. + * On error -1 is returned and @names and @versions arrays are not allocated. + */ +int tracecmd_compress_protos_get(char ***names, char ***versions) +{ + struct compress_proto *proto = proto_list; + char **n = NULL; + char **v = NULL; + int c, i; + + for (c = 0; proto; proto = proto->next) + c++; + + if (c < 1) + return c; + + n = calloc(c, sizeof(char *)); + if (!n) + goto error; + v = calloc(c, sizeof(char *)); + if (!v) + goto error; + + proto = proto_list; + for (i = 0; i < c && proto; i++) { + n[i] = proto->proto_name; + v[i] = proto->proto_version; + proto = proto->next; + } + + *names = n; + *versions = v; + return c; + +error: + free(n); + free(v); + return -1; +} + +/** + * tracecmd_compress_copy_from - Copy and compress data from a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data to copy from + * @chunk_size: size of one compression chunk + * @read_size: in - max bytes to read from @fd, 0 to read till the EOF + * out - size of the uncompressed data read from @fd + * @write_size: return, size of the compressed data written into @handle + * + * This function reads uncompressed data from given @fd, compresses the data using the @handle + * compression context and writes the compressed data into the fd associated with the @handle. + * The data is compressed on chunks with given @chunk_size size. + * The compressed data is written in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int rchunk = 0; + unsigned int chunks = 0; + unsigned int wsize = 0; + unsigned int rsize = 0; + unsigned int rmax = 0; + unsigned int csize; + unsigned int size; + unsigned int all; + unsigned int r; + off64_t offset; + char *buf_from; + char *buf_to; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->compress_block || !handle->proto->compress_size) + return 0; + if (read_size) + rmax = *read_size; + csize = handle->proto->compress_size(chunk_size); + buf_from = malloc(chunk_size); + if (!buf_from) + return -1; + buf_to = malloc(csize); + if (!buf_to) + return -1; + /* save the initial offset and write 0 chunks */ + offset = lseek64(handle->fd, 0, SEEK_CUR); + write_fd(handle->fd, &chunks, 4); + + do { + all = 0; + if (rmax > 0 && (rmax - rsize) < chunk_size) + rchunk = (rmax - rsize); + else + rchunk = chunk_size; + + do { + r = read(fd, buf_from + all, rchunk - all); + all += r; + + if (r <= 0) + break; + } while (all != rchunk); + + + if (r < 0 || (rmax > 0 && rsize >= rmax)) + break; + rsize += all; + size = csize; + if (all > 0) { + ret = handle->proto->compress_block(buf_from, all, buf_to, &size); + if (ret < 0) { + if (errno == EINTR) + continue; + break; + } + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &all, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write the compressed data */ + ret = write_fd(handle->fd, buf_to, size); + if (ret != size) + break; + /* data + compress header */ + wsize += (size + 8); + chunks++; + } + } while (all > 0); + free(buf_from); + free(buf_to); + if (all) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + endian4 = tep_read_number(handle->tep, &chunks, 4); + /* write chunks count*/ + write_fd(handle->fd, &chunks, 4); + lseek64(handle->fd, offset, SEEK_SET); + if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} + +/** + * tracecmd_load_chunks_info - Read compression chunks information from the file + * @handle: compression handler + * @chunks_info: return, array with compression chunks information + * + * This function reads information of all compression chunks in the current compression block from + * the file and fills that information in a newly allocated array @chunks_info which is returned. + * + * On success count of compression chunks is returned. Array of that count is allocated and + * returned in @chunks_info. Each entry describes one compression chunk. On error -1 is returned. + * In case of success, @chunks_info must be freed by free(). + */ +int tracecmd_load_chunks_info(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk **chunks_info) +{ + struct tracecmd_compress_chunk *chunks = NULL; + unsigned long long size = 0; + unsigned int count = 0; + off64_t offset; + int ret = -1; + char buf[4]; + int i; + + if (!handle) + return -1; + + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (offset == (off64_t)-1) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + count = tep_read_number(handle->tep, buf, 4); + if (!count) { + ret = 0; + goto out; + } + chunks = calloc(count, sizeof(struct tracecmd_compress_chunk)); + if (!chunks) + goto out; + for (i = 0; i < count; i++) { + chunks[i].zoffset = lseek64(handle->fd, 0, SEEK_CUR); + if (chunks[i].zoffset == (off_t)-1) + goto out; + if (read(handle->fd, buf, 4) != 4) + goto out; + chunks[i].zsize = tep_read_number(handle->tep, buf, 4); + chunks[i].offset = size; + if (read(handle->fd, buf, 4) != 4) + goto out; + chunks[i].size = tep_read_number(handle->tep, buf, 4); + size += chunks[i].size; + if (lseek64(handle->fd, chunks[i].zsize, SEEK_CUR) == (off64_t)-1) + goto out; + } + + ret = count; +out: + if (lseek64(handle->fd, offset, SEEK_SET) == (off64_t)-1) + ret = -1; + + if (ret > 0 && chunks_info) + *chunks_info = chunks; + else + free(chunks); + + return ret; +} + +/** + * tracecmd_uncompress_chunk - Uncompress given compression chunk. + * @handle: compression handler + * @chunk: chunk, that will be uncompressed in @data + * @data: Preallocated memory for uncompressed data. Must have enough space to hold + * the uncompressed data + * + * This function uncompresses the chunk described by @chunk and stores the uncompressed data in + * the preallocated memory @data. + * + * On success 0 is returned and the uncompressed data is stored in @data. On error -1 is returned. + */ +int tracecmd_uncompress_chunk(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk *chunk, char *data) +{ + char *bytes_in = NULL; + unsigned int size; + int ret = -1; + + if (!handle || !handle->proto || !handle->proto->uncompress_block || !chunk || !data) + return -1; + + if (lseek64(handle->fd, chunk->zoffset + 8, SEEK_SET) == (off_t)-1) + return -1; + bytes_in = malloc(chunk->zsize); + if (!bytes_in) + return -1; + if (read_fd(handle->fd, bytes_in, chunk->zsize) < 0) + goto out; + size = chunk->size; + if (handle->proto->uncompress_block(bytes_in, chunk->zsize, data, &size)) + goto out; + ret = 0; +out: + free(bytes_in); + return ret; +} + +/** + * tracecmd_uncompress_copy_to - Uncompress data and copy to a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data to copy into + * @read_size: return, size of the compressed data read from @handle + * @write_size: return, size of the uncompressed data written into @fd + * + * This function reads compressed data from the fd, associated with @handle, uncompresses it + * using the @handle compression context and writes the uncompressed data into the fd. + * The compressed data must be in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + unsigned int rsize = 0; + unsigned int wsize = 0; + char *bytes_out = NULL; + char *bytes_in = NULL; + int size_out = 0; + int size_in = 0; + int chunks; + char buf[4]; + char *tmp; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + chunks = tep_read_number(handle->tep, buf, 4); + rsize += 4; + while (chunks) { + if (read(handle->fd, buf, 4) != 4) + break; + s_compressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (read(handle->fd, buf, 4) != 4) + break; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (!bytes_in || size_in < s_compressed) { + tmp = realloc(bytes_in, s_compressed); + if (!tmp) + break; + bytes_in = tmp; + size_in = s_compressed; + } + + if (!bytes_out || size_out < s_uncompressed) { + tmp = realloc(bytes_out, s_uncompressed); + if (!tmp) + break; + bytes_out = tmp; + size_out = s_uncompressed; + } + + if (read_fd(handle->fd, bytes_in, s_compressed) < 0) + break; + rsize += s_compressed; + ret = handle->proto->uncompress_block(bytes_in, s_compressed, + bytes_out, &s_uncompressed); + if (ret) + break; + write_fd(fd, bytes_out, s_uncompressed); + wsize += s_uncompressed; + chunks--; + } + free(bytes_in); + free(bytes_out); + if (chunks) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 9c0f047b..dd6ce06e 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -625,6 +625,16 @@ bool tracecmd_is_version_supported(unsigned int version) return false; } +static void __attribute__ ((constructor)) tracecmd_lib_init(void) +{ + tracecmd_compress_init(); +} + +static void __attribute__((destructor)) tracecmd_lib_free(void) +{ + tracecmd_compress_free(); +} + __hidden bool check_file_state(unsigned long file_version, int current_state, int new_state) { switch (new_state) { From patchwork Thu Nov 11 15:11:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C59BC433EF for ; Thu, 11 Nov 2021 15:12:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 052CF61268 for ; Thu, 11 Nov 2021 15:12:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233715AbhKKPOs (ORCPT ); Thu, 11 Nov 2021 10:14:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233978AbhKKPOs (ORCPT ); Thu, 11 Nov 2021 10:14:48 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6A44C061767 for ; Thu, 11 Nov 2021 07:11:58 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id g14so25479325edz.2 for ; Thu, 11 Nov 2021 07:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wGtg1MjDRh2b0GvnBljxRlAXnNTb2QvLiEXxmBN//mU=; b=mrRHIxEvHwnEZodXYFF9449cpiR4iSSCGTnf4fWGWadqVXmpia/IKw2Kq4YNRrhoOr HxwEvGOxhXAuOr82G8VWlntIjnzzzWA76IQUJdlGWj3rD42Ktoj+xIyxlPy1Cfx3yl4O H22C87OtEj4JihxJvidubsGi8JBCfOb8fSbb9LFtKy4hmjk87+96mD864Lt15ZUsXLRn 4xJVOudFH0pGNI7+HDY3tp/udVy4iHnm/rR7Gf7JzazNBK2k7ZQafJMKwbjrVDXWBcX3 NWZUW5HBxq3h3b5rgeiAqYz1iJnoikTprhvenMR+2laq0JaD4EF58aQmkxMuF9hnrRfy OOoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wGtg1MjDRh2b0GvnBljxRlAXnNTb2QvLiEXxmBN//mU=; b=NaBGMOcxM5b6b2zgCuLPbHDoeU7cNHQplsbW1cA9b3DId0AQa0XIVrToLhK1cKaJMq EHlNoqtfnW5r2grmA1s8WycVDStlCiYulAulROrNZbe0a6xSGgS+WKVFGQz+xnm1j+D+ fGwxf+Hgn/lVtsoBZhkBM9/nEHF06mdXPgaKQL7r3vz0GOnr5GeaxQU4yYiNbXsuD+yN iHrI2oVq7A78+jgYHXyf+06neImexEYuXLGBNmhICZW/Ix4sD9AZfrwVCydElQGnktCK 1s//kmiU9K+2yq+Gm0+eeFzBkZFA6BPwbIcCkdk3kTxaBql/miZp07HpytFK1yO3A8fY uI4Q== X-Gm-Message-State: AOAM532tTV6UOu9rEiiYmIF6y8HT1Efly+gP4BiFAi/EIsPiVpYC58hQ KQZ5qNs1Ka/3ibHu6u9lKZSGPyol4MiDFw== X-Google-Smtp-Source: ABdhPJzUtdsfon4a/9v4/jnSZdwUGZeRLw4toyiXvTtHgm4KSObLEU3rIHLvtHNHBd+cSsfiZn4/9A== X-Received: by 2002:a17:906:3b84:: with SMTP id u4mr10186961ejf.310.1636643517521; Thu, 11 Nov 2021 07:11:57 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.11.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:11:56 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 02/20] trace-cmd library: Internal helpers for compressing data Date: Thu, 11 Nov 2021 17:11:35 +0200 Message-Id: <20211111151153.86855-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New library internal helper functions are introduced, to add compression functionality to the output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 7 ++++ lib/trace-cmd/trace-output.c | 56 +++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index fb008678..393d05e8 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -40,6 +40,13 @@ void tracecmd_compress_free(void); bool check_file_state(unsigned long file_version, int current_state, int new_state); bool check_out_state(struct tracecmd_output *handle, int new_state); +int out_uncompress_block(struct tracecmd_output *handle); +int out_compression_start(struct tracecmd_output *handle, bool compress); +int out_compression_end(struct tracecmd_output *handle, bool compress); +void out_compression_reset(struct tracecmd_output *handle, bool compress); +unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size); unsigned long long out_write_section_header(struct tracecmd_output *handle, unsigned short header_id, char *description, enum tracecmd_section_flags flags, bool option); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e06931ce..c9cb174a 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -64,6 +64,8 @@ struct tracecmd_output { unsigned long file_version; tsize_t options_start; bool big_endian; + bool do_compress; + struct tracecmd_compression *compress; struct list_head options; struct list_head buffers; @@ -88,18 +90,27 @@ struct list_event_system { static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { + if (handle->do_compress) + return tracecmd_compress_write(handle->compress, data, size); if (handle->msg_handle) return tracecmd_msg_data_send(handle->msg_handle, data, size); - return __do_write_check(handle->fd, data, size); } static inline off64_t do_lseek(struct tracecmd_output *handle, off_t offset, int whence) { + if (handle->do_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); if (handle->msg_handle) return msg_lseek(handle->msg_handle, offset, whence); - else - return lseek64(handle->fd, offset, whence); + return lseek64(handle->fd, offset, whence); +} + +static inline int do_preed(struct tracecmd_output *handle, void *dst, int len, off_t offset) +{ + if (handle->do_compress) + return tracecmd_compress_pread(handle->compress, dst, len, offset); + return pread(handle->fd, dst, len, offset); } static short convert_endian_2(struct tracecmd_output *handle, short val) @@ -127,6 +138,43 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +__hidden void out_compression_reset(struct tracecmd_output *handle, bool compress) +{ + if (!compress || !handle->compress) + return; + tracecmd_compress_reset(handle->compress); + handle->do_compress = false; +} + +__hidden int out_uncompress_block(struct tracecmd_output *handle) +{ + int ret = 0; + + if (!handle->compress) + return 0; + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->do_compress = true; + return ret; +} + +__hidden int out_compression_start(struct tracecmd_output *handle, bool compress) +{ + if (!compress || !handle->compress) + return 0; + tracecmd_compress_reset(handle->compress); + handle->do_compress = true; + return 0; +} + +__hidden int out_compression_end(struct tracecmd_output *handle, bool compress) +{ + if (!compress || !handle->compress) + return 0; + handle->do_compress = false; + return tracecmd_compress_block(handle->compress); +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen @@ -1544,7 +1592,7 @@ static int append_options_v6(struct tracecmd_output *handle) if (offset == (off_t)-1) return -1; - r = pread(handle->fd, &option, 2, offset); + r = do_preed(handle, &option, 2, offset); if (r != 2 || option != TRACECMD_OPTION_DONE) return -1; From patchwork Thu Nov 11 15:11:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1095C433EF for ; Thu, 11 Nov 2021 15:12:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9BB5D61390 for ; Thu, 11 Nov 2021 15:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233984AbhKKPOu (ORCPT ); Thu, 11 Nov 2021 10:14:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233982AbhKKPOt (ORCPT ); Thu, 11 Nov 2021 10:14:49 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 488AAC061766 for ; Thu, 11 Nov 2021 07:12:00 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id v11so25477412edc.9 for ; Thu, 11 Nov 2021 07:12:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d3t3kEYhcHJipDCE3CbYWFZXnPRzJHyBhw0h3SpbeO0=; b=qAhuXIYJfiwklHaH6shDXYrAWFowPAxv2uM9YXHoFL8YvPlGjoe4Czd7o2AlsEJ4R2 +cFBk031cidHf9ZZM8q0Rno3l0mJSWiQzhMPYvBvMKp5lMIl7vrCT2rCL4/NqAnii1AQ vXW9praqnRwz32dIMvkEW+e8qnwOLomkjIIZnW7HE9jGxyyi2XdzjWgU1HxdGXablP/0 4ISDFVU7e2vfG6xs5lGlTfCg1axrpgqzHpC0iuSRo0EDeAIPs4CNGyL1pah5ocPWdFRR pcsYwmwUAI7ZjZkgM1TN+JG1ipOfULXpIKufDZpiHGYuQUixU5+zyhYl1/gpVEySZ7tN W4ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d3t3kEYhcHJipDCE3CbYWFZXnPRzJHyBhw0h3SpbeO0=; b=RL7PBQBBPuxM89DTkXsCR/PbU4Z38MMHkbZX60pKUKrBandDiUSHZsPF1S0blkZnmP t4OPD6ItifGr7BqS2+wRHKJ0kpmYox8GXYac/UFJq/EZ3gUCi6CZhu3ccmp4Wzsvr2qp G1RSjOFr1qvGAnqH7oncYt6NCLV8Aa9fLGru0QeaSwFpP6q4icV2klp40iN6aj36hcrA IFrwwyvIWBuTnKeYruM7xkSmUKZqLkEJF1AuQOS4YbVWEDVimLvER+X0jkPuVUFSgzjf IH4YcFcNeahrTqz4tGisZm8o+RLD8huCXeRKx+aWRl1tYgFQange5j9iUDar6NA/zGTH Fhfw== X-Gm-Message-State: AOAM532FZaiNvSGmOfRvdGineZb8Tl2iE7a6mHcbOwyBCepxPypE2OW6 nqKGCZuyYcOPBgcoYYBCXKtWU6l/aDUKNw== X-Google-Smtp-Source: ABdhPJwsGLCNvX4wTIuivBsqoGAHd8Wwt2Tmk6Eo4TYDsPQUt6iZHDYeMePfSgYBftTmOhO13I05Pw== X-Received: by 2002:a17:907:774d:: with SMTP id kx13mr10415487ejc.239.1636643518884; Thu, 11 Nov 2021 07:11:58 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.11.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:11:58 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 03/20] trace-cmd library: Internal helpers for uncompressing data Date: Thu, 11 Nov 2021 17:11:36 +0200 Message-Id: <20211111151153.86855-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New library internal helper functions are introduced, to add compression functionality to the input trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 3 ++ lib/trace-cmd/trace-input.c | 49 ++++++++++++++++++++++--- 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 393d05e8..fd61150d 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -47,6 +47,9 @@ void out_compression_reset(struct tracecmd_output *handle, bool compress); unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, int fd, unsigned long long max, unsigned long long *write_size); +void in_uncompress_reset(struct tracecmd_input *handle); +int in_uncompress_block(struct tracecmd_input *handle); + unsigned long long out_write_section_header(struct tracecmd_output *handle, unsigned short header_id, char *description, enum tracecmd_section_flags flags, bool option); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f9978dde..f0b7b3eb 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -157,6 +157,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + bool read_compress; + struct tracecmd_compression *compress; + struct host_trace_info host; double ts2secs; char * cpustats; @@ -263,13 +266,13 @@ static const char *show_records(struct page **pages, int nr_pages) static int init_cpu(struct tracecmd_input *handle, int cpu); -static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) +static ssize_t do_read_fd(int fd, void *data, size_t size) { ssize_t tot = 0; ssize_t r; do { - r = read(handle->fd, data + tot, size - tot); + r = read(fd, data + tot, size - tot); tot += r; if (!r) @@ -281,6 +284,22 @@ static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) return tot; } +static inline int do_lseek(struct tracecmd_input *handle, int offset, int whence) +{ + if (handle->read_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek(handle->fd, offset, whence); +} + +static inline ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) +{ + if (handle->read_compress) + return tracecmd_compress_read(handle->compress, data, size); + else + return do_read_fd(handle->fd, data, size); +} + static ssize_t do_read_check(struct tracecmd_input *handle, void *data, size_t size) { @@ -305,9 +324,7 @@ static char *read_string(struct tracecmd_input *handle) for (;;) { r = do_read(handle, buf, BUFSIZ); - if (r < 0) - goto fail; - if (!r) + if (r <= 0) goto fail; for (i = 0; i < r; i++) { @@ -333,7 +350,7 @@ static char *read_string(struct tracecmd_input *handle) } /* move the file descriptor to the end of the string */ - r = lseek(handle->fd, -(r - (i+1)), SEEK_CUR); + r = do_lseek(handle, -(r - (i+1)), SEEK_CUR); if (r < 0) goto fail; @@ -397,6 +414,26 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +__hidden void in_uncompress_reset(struct tracecmd_input *handle) +{ + if (handle->compress) { + handle->read_compress = false; + tracecmd_compress_reset(handle->compress); + } +} + +__hidden int in_uncompress_block(struct tracecmd_input *handle) +{ + int ret = 0; + + if (handle->compress) { + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->read_compress = true; + } + return ret; +} + static struct file_section *section_get(struct tracecmd_input *handle, int id) { struct file_section *sec; From patchwork Thu Nov 11 15:11:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8273C433F5 for ; Thu, 11 Nov 2021 15:12:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91E82610D0 for ; Thu, 11 Nov 2021 15:12:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233982AbhKKPOv (ORCPT ); Thu, 11 Nov 2021 10:14:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233978AbhKKPOu (ORCPT ); Thu, 11 Nov 2021 10:14:50 -0500 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 720CAC061766 for ; Thu, 11 Nov 2021 07:12:01 -0800 (PST) Received: by mail-ed1-x531.google.com with SMTP id c8so25070993ede.13 for ; Thu, 11 Nov 2021 07:12:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xfcpFr0c8aAQ8Msg55A2Qs5AvZyFozphkKbqcj6Nqz4=; b=YdX9o+7ppJTONKs09DX9shEcxdfpNgCxmG0w9svNYKxxi55T4CUmUBP9J22mhio2l+ L0TtyBlvt/JAjycYVRsutC983K/bDxUpM51eeo/Bt6qdTH82fOj0MKqSaK7g54LNCQPc n/fhkR0RmS2UhPzcWaPJkKMW4EqlLId2kiXFodKGMOfEKDSbRYN2aJJpC6sc7xVtP3EF JdRwbuI8ahez5MhkD6LzdackzG1kyjGwOoPGds+3CWSsBI63vpC6KPoFPgc4VZSNV1Qk 7Ib3E77NoNKOc5YF9WIze7qQ5eGBSAE1wr9YBuHupuPFrpEfCTvjM2NU9G39Waqvf5A1 Z+/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xfcpFr0c8aAQ8Msg55A2Qs5AvZyFozphkKbqcj6Nqz4=; b=zHIXhb+KeDaYBBFlw347L+kJ9W6o2upm7LfoPWz2Hv6ly6rzE/aVw1VFrGYwTYJOav GYy4SbtDPRXn7umtYBeLb4vDjqC34KEKScXZGFKv1X01E+tBA8+puR25O3kl4mSGtShR WpSVAzhM22mt6YGvvbTRXwHKHNo1D1d/i1+70eCwQiZS68NHFCRqTz4LKezHEPpqO6NT 3nu6ClAabXEq9E31Aibfw+OJzXYCdXn7eQlzrQYdHn1Adyh3pUGqa5WsTHyocMu7awR8 R2pny+pFkfn/c7SfNK/nFgZYzIFWroe0S2xk0qE9m3Kd+/9ndgF6BxLUaHVe1/QSP4e3 IJDw== X-Gm-Message-State: AOAM532wZ4lHZoixnCDoo8J4M1X7cKeq1e5WdRTokS5cIq3ksM+7UT8l iwHZ+Oe7j2pFTlM43RE9TUD4DAGVgnJUrA== X-Google-Smtp-Source: ABdhPJzRYlKYQ0/Z5y3UDA430RRCQ2oFzBho0bh7naFTFoFhmLX+31Ehc6UzxRVe1oguYP7HEFGhEA== X-Received: by 2002:a17:906:fcd9:: with SMTP id qx25mr10547121ejb.326.1636643520034; Thu, 11 Nov 2021 07:12:00 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.11.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:11:59 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 04/20] trace-cmd library: Inherit compression algorithm from input file Date: Thu, 11 Nov 2021 17:11:37 +0200 Message-Id: <20211111151153.86855-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace file output handler is allocated, based on given trace file input handler - use the same compression algorithm. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 16 +++++++++++++++ lib/trace-cmd/trace-output.c | 20 +++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 0c7027dd..d5aea618 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -255,6 +255,8 @@ tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); size_t tracecmd_get_options_offset(struct tracecmd_input *handle); +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f0b7b3eb..a3cc600d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4593,6 +4593,22 @@ unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle) return handle->file_version; } +/** + * tracecmd_get_file_compress_proto - get name and version of compression algorithm, + * used to compress the trace file + * @handle: input handle for the trace.dat file + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version) +{ + return tracecmd_compress_proto_get_name(handle->compress, name, version); +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c9cb174a..8f822323 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1209,6 +1209,9 @@ int tracecmd_output_set_kallsyms(struct tracecmd_output *handler, const char *ka */ int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct tracecmd_input *ihandle) { + const char *cname = NULL; + const char *cver = NULL; + if (!handler || !ihandle || handler->file_state != TRACECMD_FILE_ALLOCATED) return -1; @@ -1220,6 +1223,15 @@ int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct trace handler->file_version = tracecmd_get_in_file_version(ihandle); handler->big_endian = tep_is_file_bigendian(handler->pevent); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handler->compress = tracecmd_compress_alloc(cname, cver, handler->fd, + handler->pevent, handler->msg_handle); + if (!handler->compress) + return -1; + if (handler->file_version < FILE_VERSION_COMPRESSION) + handler->file_version = FILE_VERSION_COMPRESSION; + } + return 0; } @@ -2170,6 +2182,8 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) { struct tracecmd_output *handle = NULL; struct tracecmd_input *ihandle; + const char *cname = NULL; + const char *cver = NULL; int fd2; /* Move the file descriptor to the beginning */ @@ -2209,6 +2223,12 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) list_head_init(&handle->options); list_head_init(&handle->buffers); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + goto out_free; + } tracecmd_close(ihandle); return handle; From patchwork Thu Nov 11 15:11:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0566C433EF for ; Thu, 11 Nov 2021 15:12:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD1E3610D0 for ; Thu, 11 Nov 2021 15:12:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233986AbhKKPOw (ORCPT ); Thu, 11 Nov 2021 10:14:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233978AbhKKPOv (ORCPT ); Thu, 11 Nov 2021 10:14:51 -0500 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A8C5C061766 for ; Thu, 11 Nov 2021 07:12:02 -0800 (PST) Received: by mail-ed1-x52c.google.com with SMTP id v11so25477786edc.9 for ; Thu, 11 Nov 2021 07:12:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hRYBW6PEQfSvGIKur4/uHch0dizs5En44cTOlYvvvzs=; b=LePIno0qxhvBsm4u96ahwxA+du95kFB0wbRv0c5Y/nTyPZErMQQCmMMuCfH+emj8hB nRL/k4I7dvQn0/gkCOgdLD9TS4MibmSnnv09onp9f6OLmPu4jXSsO2wjVH7navXRJFf8 HGOfoFgF0XMAIy7GQG9KaVrRkSCqcRuetgBV+fXXJePp8pGElWObQAXIsSnaOg7tp+7h bmOcK9e5QAzFyNtQhBT/MU3Fvm/whnzlEivAieY13nS7G7nRoERLlnv/BPKG2R9uf1Tr z4shiOYqn5MOSNGJUBPAhJxP1xnNejx1cpn3UsH0SuHJWcJCDlfiubCxkiJw5S6rfXBD FK5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hRYBW6PEQfSvGIKur4/uHch0dizs5En44cTOlYvvvzs=; b=brR1bB/eXZoewrt5T4QCDuK6xYUrsX3tyvilNCSr+J5kkv9cfHtq2QLeuw2lS+FluY YQ8VohxcmZOURx3MZ0911KA3FfZkOMH6e2Z375muBLUfXXKA3s5LnrywNZ1jE0KLiMMR +BIl1cN0VNdiTze2f0T9BkxgG7ljmIhrkBQfWU2AfB00cmV5Jbt44XGctHzcHIztuHlh X6AyJ/xYcMXOEHxrjRwJzvutCiSp2ZT3bfUhOLRdbQbpatLaHSoXoVzXVeukNodCYpTl VE9xByKYEML8swR5QUd57S2r+bzh4YfLiiRUgIuozzZCfRRB9i5s2WDTFvWz0ECG6cX5 bJqg== X-Gm-Message-State: AOAM5308VI62qMB/w7sIzBuxc+ax7Jwqzx5lQpFKirIZBDV+QT9u2TuJ R+TpgAMm4lK/oWFd0E0+KJejVMSfYiXXGQ== X-Google-Smtp-Source: ABdhPJzLJZbzeln+QtvNi7mCS4xRQ2/6quLFB5kzwR0ADdPT/WWOZzSTQeLswaMu2bbf6eh55ycQXg== X-Received: by 2002:a05:6402:10c5:: with SMTP id p5mr10767802edu.150.1636643521068; Thu, 11 Nov 2021 07:12:01 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:00 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 05/20] trace-cmd library: New API to configure compression on an output handler Date: Thu, 11 Nov 2021 17:11:38 +0200 Message-Id: <20211111151153.86855-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The new API can be used to configure compression algorithm on an output handler to a trace file. tracecmd_output_set_compression() The API for creation of latency trace file is extended with compression parameter. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 +- lib/trace-cmd/trace-output.c | 57 ++++++++++++++++++- tracecmd/trace-record.c | 2 +- 3 files changed, 59 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index d5aea618..b44a6e58 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -297,12 +297,13 @@ int tracecmd_output_set_trace_dir(struct tracecmd_output *handler, const char *t int tracecmd_output_set_kallsyms(struct tracecmd_output *handler, const char *kallsyms); int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct tracecmd_input *ihandle); int tracecmd_output_set_version(struct tracecmd_output *handler, int file_version); +int tracecmd_output_set_compression(struct tracecmd_output *handler, const char *compression); int tracecmd_output_write_init(struct tracecmd_output *handler); int tracecmd_output_write_headers(struct tracecmd_output *handler, struct tracecmd_event_list *list); struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, - int file_version); + int file_version, const char *compression); struct tracecmd_output *tracecmd_create_init_fd(int fd); struct tracecmd_output *tracecmd_create_init_file(const char *output_file); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8f822323..6119721c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -234,6 +234,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) } free(handle->trace_clock); + tracecmd_compress_destroy(handle->compress); free(handle); } @@ -1251,9 +1252,57 @@ int tracecmd_output_set_version(struct tracecmd_output *handler, int file_versio if (file_version < FILE_VERSION_MIN || file_version > FILE_VERSION_MAX) return -1; handler->file_version = file_version; + if (handler->file_version < FILE_VERSION_COMPRESSION) + handler->compress = NULL; return 0; } +/** + * tracecmd_output_set_compression - Set file compression algorithm of the output handler + * @handle: output handler to a trace file. + * @compression: name of the desired compression algorithm. Can be one of: + * - "none" - do not use compression + * - "all" - use the best available compression algorithm + * - or specific name of the desired compression algorithm + * + * This API must be called before tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 in case of an error: + * - the output file handler is not allocated or not in expected state. + * - the specified compression algorithm is not available + */ +int tracecmd_output_set_compression(struct tracecmd_output *handler, const char *compression) +{ + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; + + handler->compress = NULL; + if (compression && strcmp(compression, "none")) { + if (!strcmp(compression, "any")) { + handler->compress = tracecmd_compress_alloc(NULL, NULL, handler->fd, + handler->pevent, + handler->msg_handle); + if (!handler->compress) + tracecmd_warning("No compression algorithms are supported"); + } else { + handler->compress = tracecmd_compress_alloc(compression, NULL, handler->fd, + handler->pevent, + handler->msg_handle); + if (!handler->compress) { + tracecmd_warning("Compression algorithm %s is not supported", + compression); + return -1; + } + } + } + if (handler->compress && handler->file_version < FILE_VERSION_COMPRESSION) { + handler->file_version = FILE_VERSION_COMPRESSION; + if (handler->msg_handle) + tracecmd_msg_handle_cache(handler->msg_handle); + } + + return 0; +} /** * tracecmd_output_write_init - Write the initial magics in the trace file @@ -1839,7 +1888,7 @@ out_add_buffer_option_v7(struct tracecmd_output *handle, const char *name, } struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, - int file_version) + int file_version, const char *compression) { enum tracecmd_section_flags flags = 0; struct tracecmd_output *handle; @@ -1856,6 +1905,12 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in goto out_free; if (file_version && tracecmd_output_set_version(handle, file_version)) goto out_free; + if (compression) { + if (tracecmd_output_set_compression(handle, compression)) + goto out_free; + } else if (file_version >= FILE_VERSION_COMPRESSION) { + tracecmd_output_set_compression(handle, "any"); + } if (tracecmd_output_write_init(handle)) goto out_free; if (tracecmd_output_write_headers(handle, NULL)) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index fab34361..699fa511 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4516,7 +4516,7 @@ static void record_data(struct common_record_context *ctx) if (latency) { handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, - ctx->file_version); + ctx->file_version, NULL); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) From patchwork Thu Nov 11 15:11:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C96A0C433F5 for ; Thu, 11 Nov 2021 15:12:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE9D361108 for ; Thu, 11 Nov 2021 15:12:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233978AbhKKPOy (ORCPT ); Thu, 11 Nov 2021 10:14:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233959AbhKKPOx (ORCPT ); Thu, 11 Nov 2021 10:14:53 -0500 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7756DC061766 for ; Thu, 11 Nov 2021 07:12:04 -0800 (PST) Received: by mail-ed1-x533.google.com with SMTP id r12so25429578edt.6 for ; Thu, 11 Nov 2021 07:12:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fE4TGTHK0TVJo1GEztfPc8OHAPrM7NY2fZxIr74H3UA=; b=Drm210+sDD1RmwMQOGjoqiw2IAgab9XWwmej/OiE46tFeQQfQXyr96sl2eytpVmx/F s/3C4NEqkIWR+Q/RSOI8vCOPuo36jlLWuwknF5AwH3ggBEkoIT/kOnoxgE79XYQ+NFbh V+INj0tLY/eTZI2bJg9PhiXWcoqv3z/zC81M5GoA1eoqedHJI+XphBk9jAGNs0flrw5C J3YSwJuRAPIEl0VPew+9jssG7UGdd9hth3/3wtEvD/YYmIl1Wp88wW4n0DiUaAFR6KDO fAAVMK5u9hmJDuXe4zRvUCzC0g8RJ5eK2vw6NfeZ1XGcsG7pbB1kOiVsKu9FLeoxuyO5 V5gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fE4TGTHK0TVJo1GEztfPc8OHAPrM7NY2fZxIr74H3UA=; b=pmnHuS8eUuT/9FTNmStgpEbeka6QK53VBzeJr4oltlKTncr5c/L448R+lTaE8nQ3cq I6K3VNCPbi3V1Qr+PJWA2ySK1t5C3EtFgq2HBr2tEM+Z/gosx/AohQhuuVyDQUSPLMxo BNNqrUz7UbmfNiRnL8YRmNmxOsU6At83RFMGvxth9KvI/L5eS3vreAfVQo/Z2MGD2BFm l7DT7jyIii6JoWiAsh1Dnd2y0kFpboC0kgHoDjPnmjAQ0zNK8WM5g+DrdEz5JHPNPLen T+QLzYT/ayJyUJRDshigXfdvr4RnKFunCxDChYr3q+dTf3n7JcCnN1Euj2OAneHu72ME tomA== X-Gm-Message-State: AOAM533nMRhH0XnOXUt0ukmTDIA5fqy/lXFZv5pWPvEvoqCvny2uBR59 aty88A/CoymI3JMdatRcZgxiYunhUjz9Fg== X-Google-Smtp-Source: ABdhPJw5wKiRl35fTb6RhKpVNS1PoPmgUPC/GnzMC1hiJh7ynYiE2MUXsOOiAwsiVIYJ0PRHYKJhVQ== X-Received: by 2002:a05:6402:1a4f:: with SMTP id bf15mr1965160edb.260.1636643523061; Thu, 11 Nov 2021 07:12:03 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:02 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 06/20] trace-cmd library: Write compression header in the trace file Date: Thu, 11 Nov 2021 17:11:39 +0200 Message-Id: <20211111151153.86855-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If there is a compression configured on the output file handler and if the file version is at least 7, write compression header in the file. The compression header is two null terminated strings - name and version of the compression algorithm, used to compress some parts of the file. The header is located after the page size in the file. The new header is mandatory for trace files version 7. If no compression is used, the string "none" is saved as name of the compression algorithm and empty string as compression algorithm version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 6119721c..cbe53199 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1076,6 +1076,24 @@ out_free: return ret; } +static int write_compression_header(struct tracecmd_output *handle) +{ + const char *name = NULL; + const char *ver = NULL; + int ret; + + ret = tracecmd_compress_proto_get_name(handle->compress, &name, &ver); + if (ret < 0 || !name || !ver) { + name = "none"; + ver = ""; + } + if (do_write_check(handle, name, strlen(name) + 1)) + return -1; + if (do_write_check(handle, ver, strlen(ver) + 1)) + return -1; + return 0; +} + /** * tracecmd_output_allocate - allocate new output handler to a trace file * @handle: file descriptor to an empty file, it can be -1 if the handler @@ -1355,6 +1373,10 @@ int tracecmd_output_write_init(struct tracecmd_output *handler) endian4 = convert_endian_4(handler, handler->page_size); if (do_write_check(handler, &endian4, 4)) return -1; + if (handler->file_version >= FILE_VERSION_COMPRESSION) { + if (write_compression_header(handler)) + return -1; + } if (HAS_SECTIONS(handler)) { /* Write 0 as options offset and save its location */ offset = 0; From patchwork Thu Nov 11 15:11:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E382FC433EF for ; Thu, 11 Nov 2021 15:12:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CCFB661108 for ; Thu, 11 Nov 2021 15:12:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233987AbhKKPOz (ORCPT ); Thu, 11 Nov 2021 10:14:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233959AbhKKPOy (ORCPT ); Thu, 11 Nov 2021 10:14:54 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F2A9C061766 for ; Thu, 11 Nov 2021 07:12:05 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id j21so25220307edt.11 for ; Thu, 11 Nov 2021 07:12:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HNGUzsiWRk6b9TAhXtVjrJoc/Vivu+0BGwmGzGqVQJc=; b=GWEtYO9nf6itlmvV0Z10dO26NGBaaDS4krt31h1gUaE2mOxvpuNjPaol/r3hmjxm+0 +RgrLHjryYVofw6VV5VKmFhYCQJUzw/aPcFfWbMD0vtAKTius8tj3N4RZy1iX63Mxcju CsCojtlUDve6bX3xNfX8GA9wC3Rz5csPcLjNp8kcYDGi/UzpycE456ce0OmfQ3qTY8O1 IgyP1SHqw/ku8BeSbdwWxPHAEEtBkYJzG04YNCd6C0dus3322+VayDBkVLbF9AxRQzdS WU5i/Oiw76AAZrwIA60YynMOgPR7HUpPofn6ioGcdPvkV+pHeD+HeOGeqEDZMQQgi7Q5 9Ikw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HNGUzsiWRk6b9TAhXtVjrJoc/Vivu+0BGwmGzGqVQJc=; b=Eh2+EHlMrMYBxaht3QNQOKUT4HYBCRKTH/Qc+OOvwyVny1sBRLNKY5Z66r2KKhbpPs BTIa6TJCajNwZ3kx5QWvh/mHMO5I8hq4yI8xRvjBVmr7EYjKmiGx/4EVVkszL1umi6/T 6kYwmacqKByjaV/UISbz1yIlKqKIAfwYDwYciqUL8DiTfapDlyLrBKPjv2D7gZVjmtYf A84PZrzRLMsCk9qnwZYSt5IPl/7UBaDiTWrvF1cewOfQUzGz+7DU2bgoIM4T5fPfkTMR OiBqfVpYk8d4TwJFSdQ3dqCn7Lm2EAmVyr8Bv7DPS455NpO5DTiESQ/vPXnI+V6dwBIG C+XA== X-Gm-Message-State: AOAM532ywBuzTENnkBmsx2TrvXXLpCdqr/Fv20Ac7G4IrKw1B4M3bfW8 ZI/ihFzQj7fQjvnuoYTGuOXCqHOlSpfchQ== X-Google-Smtp-Source: ABdhPJxKm2Gc5tR9xDZ4s7meuwzGFwmfRphsNOKTrT+YYealza0gvPe8l4pJ3ERLuFPBETKA84YlYg== X-Received: by 2002:a05:6402:26ce:: with SMTP id x14mr10765468edd.326.1636643524180; Thu, 11 Nov 2021 07:12:04 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:03 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 07/20] trace-cmd library: Compress part of the trace file Date: Thu, 11 Nov 2021 17:11:40 +0200 Message-Id: <20211111151153.86855-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Compress part of the trace.dat file metadata. If there is compression support, compress these parts of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 81 +++++++++++++++++++++++++++++------- 1 file changed, 67 insertions(+), 14 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index cbe53199..8319d1de 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -394,6 +394,8 @@ out_write_section_header(struct tracecmd_output *handle, unsigned short header_i return -1; if (!HAS_SECTIONS(handle)) return 0; + if (!handle->compress) + flags &= ~TRACECMD_SEC_FL_COMPRESS; offset = do_lseek(handle, 0, SEEK_CUR); if (option) { endian8 = convert_endian_8(handle, offset); @@ -449,7 +451,7 @@ __hidden int out_update_section_header(struct tracecmd_output *handle, unsigned return 0; } -static int read_header_files(struct tracecmd_output *handle) +static int read_header_files(struct tracecmd_output *handle, bool compress) { enum tracecmd_section_flags flags = 0; tsize_t size, check_size, endian8; @@ -469,11 +471,14 @@ static int read_header_files(struct tracecmd_output *handle) if (!path) return -1; + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_HEADER_INFO, "headers", flags, true); if (offset == (off64_t)-1) return -1; + out_compression_start(handle, compress); ret = stat(path, &st); if (ret < 0) { /* old style did not show this info, just add zero */ @@ -487,6 +492,8 @@ static int read_header_files(struct tracecmd_output *handle) goto out_close; if (do_write_check(handle, &size, 8)) goto out_close; + if (out_compression_end(handle, compress)) + goto out_close; if (out_update_section_header(handle, offset)) goto out_close; return 0; @@ -539,6 +546,8 @@ static int read_header_files(struct tracecmd_output *handle) goto out_close; } put_tracing_file(path); + if (out_compression_end(handle, compress)) + goto out_close; if (out_update_section_header(handle, offset)) goto out_close; handle->file_state = TRACECMD_FILE_HEADERS; @@ -546,6 +555,7 @@ static int read_header_files(struct tracecmd_output *handle) return 0; out_close: + out_compression_reset(handle, compress); if (fd >= 0) close(fd); return -1; @@ -774,7 +784,7 @@ create_event_list_item(struct tracecmd_output *handle, tracecmd_warning("Insufficient memory"); } -static int read_ftrace_files(struct tracecmd_output *handle) +static int read_ftrace_files(struct tracecmd_output *handle, bool compress) { enum tracecmd_section_flags flags = 0; struct list_event_system *systems = NULL; @@ -788,15 +798,20 @@ static int read_ftrace_files(struct tracecmd_output *handle) return -1; } + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_FTRACE_EVENTS, "ftrace events", flags, true); if (offset == (off64_t)-1) return -1; create_event_list_item(handle, &systems, &list); - + out_compression_start(handle, compress); ret = copy_event_system(handle, systems); - + if (!ret) + ret = out_compression_end(handle, compress); + else + out_compression_reset(handle, compress); free_list_events(systems); if (ret) return ret; @@ -822,7 +837,7 @@ create_event_list(struct tracecmd_output *handle, } static int read_event_files(struct tracecmd_output *handle, - struct tracecmd_event_list *event_list) + struct tracecmd_event_list *event_list, bool compress) { enum tracecmd_section_flags flags = 0; struct list_event_system *systems; @@ -840,6 +855,8 @@ static int read_event_files(struct tracecmd_output *handle, return -1; } + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_EVENT_FORMATS, "events format", flags, true); if (offset == (off64_t)-1) @@ -860,7 +877,7 @@ static int read_event_files(struct tracecmd_output *handle, for (slist = systems; slist; slist = slist->next) count++; - + out_compression_start(handle, compress); ret = -1; endian4 = convert_endian_4(handle, count); if (do_write_check(handle, &endian4, 4)) @@ -875,6 +892,9 @@ static int read_event_files(struct tracecmd_output *handle, } ret = copy_event_system(handle, slist); } + if (ret) + goto out_free; + ret = out_compression_end(handle, compress); if (ret) goto out_free; ret = out_update_section_header(handle, offset); @@ -882,6 +902,8 @@ static int read_event_files(struct tracecmd_output *handle, out_free: if (!ret) handle->file_state = TRACECMD_FILE_ALL_EVENTS; + else + out_compression_reset(handle, compress); free_list_events(systems); @@ -928,7 +950,7 @@ err: tracecmd_warning("can't set kptr_restrict"); } -static int read_proc_kallsyms(struct tracecmd_output *handle) +static int read_proc_kallsyms(struct tracecmd_output *handle, bool compress) { enum tracecmd_section_flags flags = 0; unsigned int size, check_size, endian4; @@ -946,11 +968,14 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) if (handle->kallsyms) path = handle->kallsyms; + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_KALLSYMS, "kallsyms", flags, true); if (offset == (off64_t)-1) return -1; + out_compression_start(handle, compress); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -976,14 +1001,19 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) } set_proc_kptr_restrict(1); + ret = out_compression_end(handle, compress); + if (ret) + goto out; ret = out_update_section_header(handle, offset); out: if (!ret) handle->file_state = TRACECMD_FILE_KALLSYMS; + else + out_compression_reset(handle, compress); return ret; } -static int read_ftrace_printk(struct tracecmd_output *handle) +static int read_ftrace_printk(struct tracecmd_output *handle, bool compress) { enum tracecmd_section_flags flags = 0; unsigned int size, check_size, endian4; @@ -1002,10 +1032,13 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_PRINTK, "printk", flags, true); if (offset == (off64_t)-1) return -1; + out_compression_start(handle, compress); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -1028,12 +1061,15 @@ static int read_ftrace_printk(struct tracecmd_output *handle) out: put_tracing_file(path); + if (out_compression_end(handle, compress)) + return -1; if (out_update_section_header(handle, offset)) return -1; handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); + out_compression_reset(handle, compress); return -1; } @@ -1407,21 +1443,25 @@ int tracecmd_output_write_init(struct tracecmd_output *handler) int tracecmd_output_write_headers(struct tracecmd_output *handler, struct tracecmd_event_list *list) { + bool compress = false; + if (!handler || handler->file_state < TRACECMD_FILE_ALLOCATED) return -1; /* Write init data, if not written yet */ if (handler->file_state < TRACECMD_FILE_INIT && tracecmd_output_write_init(handler)) return -1; - if (read_header_files(handler)) + if (handler->compress) + compress = true; + if (read_header_files(handler, compress)) return -1; - if (read_ftrace_files(handler)) + if (read_ftrace_files(handler, compress)) return -1; - if (read_event_files(handler, list)) + if (read_event_files(handler, list, compress)) return -1; - if (read_proc_kallsyms(handler)) + if (read_proc_kallsyms(handler, compress)) return -1; - if (read_ftrace_printk(handler)) + if (read_ftrace_printk(handler, compress)) return -1; return 0; } @@ -1792,6 +1832,7 @@ static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char int tracecmd_write_cmdlines(struct tracecmd_output *handle) { enum tracecmd_section_flags flags = 0; + bool compress = false; tsize_t offset; int ret; @@ -1801,14 +1842,26 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return -1; } + if (handle->compress) + compress = true; + + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_CMDLINES, "command lines", flags, true); if (offset == (off64_t)-1) return -1; + out_compression_start(handle, compress); + ret = save_tracing_file_data(handle, "saved_cmdlines"); - if (ret < 0) + if (ret < 0) { + out_compression_reset(handle, compress); return ret; + } + + if (out_compression_end(handle, compress)) + return -1; if (out_update_section_header(handle, offset)) return -1; From patchwork Thu Nov 11 15:11:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF40BC433FE for ; Thu, 11 Nov 2021 15:12:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C538361284 for ; Thu, 11 Nov 2021 15:12:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233994AbhKKPO4 (ORCPT ); Thu, 11 Nov 2021 10:14:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233989AbhKKPOz (ORCPT ); Thu, 11 Nov 2021 10:14:55 -0500 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7299AC061766 for ; Thu, 11 Nov 2021 07:12:06 -0800 (PST) Received: by mail-ed1-x52d.google.com with SMTP id b15so25225707edd.7 for ; Thu, 11 Nov 2021 07:12:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0UHDeOsY2GloB7q9msVpOoxnu8GKfP9mRvkmC/6VgNc=; b=gu3wUWk8Vx28uJBZ6zsqGETFevggCXwz01YyCRxrJzqdiA03zcz/sNZbUe7PUWmMOf dI0JJLMIic/HP932kYU1NUn4OJKvPHKyxBtwC/i339P5qsPSrxHtHz3CEmNPNFmL7HzC PoXY1/BVarj1X4Y6Dgi90iLTXm8OAO/raRyFmoIBdLQsk5whuQeijWv7elgflxg3dotc kQtYc23cUKfWRkB3mGSqa9ZQ+kfErbmRzNB5nG4FqqP+qKNyXA78SLIqoqXpuDKVwfZo SyWpzkpLCZbCoQky0OpKVVQoGymjSyajUN0O5E/8iV5B+ZHnsBdMMoWQKWsJBJ6CycGV priA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0UHDeOsY2GloB7q9msVpOoxnu8GKfP9mRvkmC/6VgNc=; b=SxCQvN3TQrAFaicK8yfbey6h8GIe5hhFA2I/5IwVf47WrHgvJmQs4T0v77WjEJzCiL asn5uBMcDEnQFCE2zHhr0gf1qrnDLo7VfYJbFPGV1ShPtu98hfo7fVasO8mcov8+rdcb sR5Ty5I/sk3BxY0RpgwuLBXdkX20MsyTncmGW/pLlq9Pu19hrC1nmF4sNDKdLHM8YYGn Mjf3oCTdoSXrfHvSkakwC3zu/bE0bQIDmAoBerYpcBLwUUSlb7fsWTlVY+ZWeCu0N0EJ eHciPao8zCRF0U84xtQnEwcAsFjba9HUEoC13ReIipq7AVlmKrYW0atPo49XIPgEc9UY 0bSQ== X-Gm-Message-State: AOAM533oBUUtEGYU2RiXLkI5IbjN80JhzGDfVjBTtvlSzyWaAtizk3LP YfS1ts7aJVcI2vhrqEtGUWVE2wmJX1I1TQ== X-Google-Smtp-Source: ABdhPJxQ0vxMK8e0z3j5W3My9e6l56+P/kIneAgST3e3sZ5rTLw7QuG4+gXge+/YM3wTkSG4qeCEnw== X-Received: by 2002:a05:6402:2743:: with SMTP id z3mr10679718edd.167.1636643525004; Thu, 11 Nov 2021 07:12:05 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:04 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 08/20] trace-cmd library: Add local helper function for data compression Date: Thu, 11 Nov 2021 17:11:41 +0200 Message-Id: <20211111151153.86855-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The newly added helper functions read data from a file and compress it, before writing into the trace file. The trace data is compressed in chunks, which are page aligned. A new local define is introduced: PAGES_IN_CHUNK which can be used to tune how big a compression chunk is. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 71 +++++++++++++++++++++++++++++++++--- 1 file changed, 65 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8319d1de..627805a7 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -283,18 +283,27 @@ static unsigned long get_size(const char *file) return size; } -static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd) +static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd, unsigned long long max) { + tsize_t rsize = BUFSIZ; tsize_t size = 0; char buf[BUFSIZ]; stsize_t r; do { - r = read(fd, buf, BUFSIZ); + if (max && rsize > max) + rsize = max; + + r = read(fd, buf, rsize); if (r > 0) { size += r; if (do_write_check(handle, buf, r)) return 0; + if (max) { + max -= r; + if (!max) + break; + } } } while (r > 0); @@ -312,12 +321,62 @@ static tsize_t copy_file(struct tracecmd_output *handle, tracecmd_warning("Can't read '%s'", file); return 0; } - size = copy_file_fd(handle, fd); + size = copy_file_fd(handle, fd, 0); close(fd); return size; } +#define PAGES_IN_CHUNK 10 +__hidden unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size) +{ + unsigned long long rsize = 0; + unsigned long long wsize = 0; + unsigned long long size; + int ret; + + if (handle->compress) { + rsize = max; + ret = tracecmd_compress_copy_from(handle->compress, fd, + PAGES_IN_CHUNK * handle->page_size, + &rsize, &wsize); + if (ret < 0) + return 0; + + size = rsize; + if (write_size) + *write_size = wsize; + } else { + size = copy_file_fd(handle, fd, max); + if (write_size) + *write_size = size; + } + + return size; +} + +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + ret = out_copy_fd_compress(handle, fd, 0, write_size); + if (!ret) + tracecmd_warning("Can't compress '%s'", file); + + close(fd); + return ret; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -513,7 +572,7 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s' size=%lld read=%lld", path, size, check_size); @@ -539,7 +598,7 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s'", path); @@ -2181,7 +2240,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (data[i].size) { if (lseek64(data[i].fd, data[i].offset, SEEK_SET) == (off64_t)-1) goto out_free; - read_size = copy_file_fd(handle, data[i].fd); + read_size = copy_file_fd(handle, data[i].fd, data[i].size); if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", From patchwork Thu Nov 11 15:11:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD95CC433F5 for ; Thu, 11 Nov 2021 15:12:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9491961252 for ; Thu, 11 Nov 2021 15:12:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233989AbhKKPO5 (ORCPT ); Thu, 11 Nov 2021 10:14:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233959AbhKKPO4 (ORCPT ); Thu, 11 Nov 2021 10:14:56 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FE7CC061766 for ; Thu, 11 Nov 2021 07:12:07 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id f4so25348359edx.12 for ; Thu, 11 Nov 2021 07:12:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3wxDM7+gwTNRiXFLdyAKrqj34nBdSxDFv1ck0Ef2oFo=; b=EYkqJDNdAR5BmVfnmvIXj6ov1CcxcW6puTb/D0H+BdgHYUG4Mq9qpUPs72JPAxGZtx Dnyzopf5LNEl62D9OuihOZX63XK12Bw7sCOnnibw0JW9LYFF0ggYQ9533Q6aw4plVc4a 8azUK9N1VWpc3OkdVpcwYjMYe2ZLMxe0Uc/K0SxoVrMiH19uJfcWJYXA1F8GcKbmlko4 4pR8XWIgYxvwJdQAAXiUV75fkFUxGI0r42Yr2UmNdwSPm4TAsBB4p9rKTJsKbchb/MQj Sf/i77uthKk5D9vtKUOLslxEANPKI7s8vLJB8acbUKrDpMMhO92QrvJJ20sXZdjfPJF+ 29Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3wxDM7+gwTNRiXFLdyAKrqj34nBdSxDFv1ck0Ef2oFo=; b=MTyaMoPnfNr2FpTYCKq6owomGehof2YyU6hmEf2/hJLsEB2kNFpB2UtRm3RwSignK1 6QOnTzomNlkSdcUIywuD1fwMfNjQKXJx+NpP5fbit1gYvH8ask5GdufvboWTqZ+LLmJ9 TmubenpuMAusMsCgx0F4cE+PR8UKmqMH+/FKMQ1akX2vdk866Z1m2fUZ/6ZGrPm5v+w+ gpWsVKcP/1mwWs2NzA+08eerjAWDKOCEPTw8pjKD7aDkc9nwha+QE71t8euMT6wSlivY RREr+WAjZVk8hfwY2Y+kBLjIcq50FMtq/xZqmiGOBqIT5nuaDyCdZ6fNCA/3Qsl1Efmp Uq+A== X-Gm-Message-State: AOAM532LDCK8jEe/BxjiX0c5hGWXL1GeBnlew33nXzsWKRUvCS4PugnH Ba/wwD8QB7b/lCCmIkLiFVtJgjFuay8fmg== X-Google-Smtp-Source: ABdhPJycKUzfZo6z81Df6rU4KuCBC97AYlJ1TkqJPgzGBBwy7R6mXxyv4WkwSrlC10U9RA1dZz87kw== X-Received: by 2002:aa7:cac2:: with SMTP id l2mr10914770edt.168.1636643526162; Thu, 11 Nov 2021 07:12:06 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:05 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 09/20] trace-cmd library: Compress the trace data Date: Thu, 11 Nov 2021 17:11:42 +0200 Message-Id: <20211111151153.86855-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If the output file handler supports compression, use it to compress the flyrecord and latency trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 627805a7..c4a8058d 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2079,11 +2079,13 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (HAS_SECTIONS(handle) && !out_add_buffer_option_v7(handle, "", TRACECMD_OPTION_BUFFER_TEXT, offset, 0, NULL)) goto out_free; + if (handle->compress) + flags |= TRACECMD_SEC_FL_COMPRESS; offset = out_write_section_header(handle, TRACECMD_OPTION_BUFFER_TEXT, "buffer latency", flags, false); - copy_file(handle, path); + copy_file_compress(handle, path, NULL); if (out_update_section_header(handle, offset)) goto out_free; @@ -2188,6 +2190,8 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (!HAS_SECTIONS(handle) && do_write_check(handle, "flyrecord", 10)) goto out_free; + if (handle->compress) + flags |= TRACECMD_SEC_FL_COMPRESS; if (asprintf(&str, "buffer flyrecord %s", buff_name) < 1) goto out_free; offset = out_write_section_header(handle, TRACECMD_OPTION_BUFFER, str, flags, false); @@ -2240,14 +2244,15 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (data[i].size) { if (lseek64(data[i].fd, data[i].offset, SEEK_SET) == (off64_t)-1) goto out_free; - read_size = copy_file_fd(handle, data[i].fd, data[i].size); + read_size = out_copy_fd_compress(handle, data[i].fd, + data[i].size, &data_files[i].write_size); + if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", read_size, data_files[i].file_size); goto out_free; } - data_files[i].write_size = read_size; } else { data_files[i].write_size = 0; } From patchwork Thu Nov 11 15:11:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F38CC433F5 for ; Thu, 11 Nov 2021 15:12:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 49AE460296 for ; Thu, 11 Nov 2021 15:12:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233898AbhKKPO6 (ORCPT ); Thu, 11 Nov 2021 10:14:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPO5 (ORCPT ); Thu, 11 Nov 2021 10:14:57 -0500 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 729A5C061766 for ; Thu, 11 Nov 2021 07:12:08 -0800 (PST) Received: by mail-ed1-x52d.google.com with SMTP id c8so25072456ede.13 for ; Thu, 11 Nov 2021 07:12:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H1BTFzuf5o5+0+Pd0gbo1N6C81aahbk2Yrr+z91o030=; b=bhMPPhYeaG9xkDaN+JI4FD3+yF9EscpPRwMK1s43avqL/8cmOzY5GjzXhNEO2KHvEV iYPMeK/JAxhgPN6K0bDxjRZMCfFnVildmrX1mPSkgrqJeBHdU1G7nV37QIuuS5ErJpdz mSlJT7YIzAbBXh+/vrOmP1eYjeqfCQsZRPvySh5Ks4e12EFOQBa6sNv84cYhYxCeRV5J C+8G06xFJGVJcQLQoKk64TmZ6ulnEmurX4KDGAEShuxu8K5J8fpRC9vDdhdy5U5xEz/M Y5Ym1RPNDsqaqiGDgBdNYBvKCbDMT1ktBTLQDIGYFP+SPhe5h87vLAA8b5VFEoLaU2zI 1ubA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H1BTFzuf5o5+0+Pd0gbo1N6C81aahbk2Yrr+z91o030=; b=5sNxA/uwJ/0LtYRbNJY4/rgT7FIkCXXWvJCaubEKrFFnQ7ISsdZ9WSe9QBt3UOfeuE bBZ5Gsxt2XWq5/ENjCyTGicGLWzHCVjb65t7Hb5xu/Rv3nJdeQxhecFtfk0gsOi5T7CX cbdWmI1ktjq/VnUxQMwfgM/Ma04dEW3zu14+ODioTOeOXaOHVy/I68qmVo0wLgmsgRUf 0SNMFZ24IFQRTPKnHmLMkalzjPHN24eGFEborOjOEd/58uBTE0GytMh0lg42CIVBt7ua s/0ZR2sTiL1GH7Y3hozMICLfghjq4CTxhaCBLQUJOGFKsvhghe3NeuBxyax6OFBQYZ5N ucbQ== X-Gm-Message-State: AOAM530ZZxebequ6O0VmBlZuY0dDnlMV7BVwuEfUIo7AHoOa6UQi67uZ +T2YWKse3WmM6MzG7leRMiAwyhW7XVGyXw== X-Google-Smtp-Source: ABdhPJxPu1miA7Ya3yDl1S/6Oe3/D62hghgW0DuW8+9JvWOSGdHCVDLXXjEUVgyIpmG1atbbXO/QxQ== X-Received: by 2002:a05:6402:14c3:: with SMTP id f3mr10580008edx.67.1636643527069; Thu, 11 Nov 2021 07:12:07 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:06 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 10/20] trace-cmd library: Decompress the options section, if it is compressed Date: Thu, 11 Nov 2021 17:11:43 +0200 Message-Id: <20211111151153.86855-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, options section can be compressed. Extended the options handling decompression if needed . Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 45 ++++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 13 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index a3cc600d..b7ad946c 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3025,6 +3025,7 @@ static int handle_options(struct tracecmd_input *handle) unsigned short id, flags; char *cpustats = NULL; struct hook_list *hook; + bool compress = false; char *buf; int cpus; int ret; @@ -3036,23 +3037,32 @@ static int handle_options(struct tracecmd_input *handle) return -1; if (id != TRACECMD_OPTION_DONE) return -1; + if (flags & TRACECMD_SEC_FL_COMPRESS) + compress = true; } + if (compress && in_uncompress_block(handle)) + return -1; for (;;) { - if (read2(handle, &option)) - return -1; + ret = read2(handle, &option); + if (ret) + goto out; if (!HAS_SECTIONS(handle) && option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (read4(handle, &size)) - return -1; + ret = read4(handle, &size); + if (ret) + goto out; buf = malloc(size); - if (!buf) - return -ENOMEM; - if (do_read_check(handle, buf, size)) - return -1; + if (!buf) { + ret = -ENOMEM; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret) + goto out; switch (option) { case TRACECMD_OPTION_DATE: @@ -3106,15 +3116,17 @@ static int handle_options(struct tracecmd_input *handle) buf + 8, 4); ret = tsync_cpu_offsets_load(handle, buf + 12, size - 12); if (ret < 0) - return ret; + goto out; tracecmd_enable_tsync(handle, true); break; case TRACECMD_OPTION_CPUSTAT: buf[size-1] = '\n'; cpustats = realloc(handle->cpustats, handle->cpustats_size + size + 1); - if (!cpustats) - return -ENOMEM; + if (!cpustats) { + ret = -ENOMEM; + goto out; + } memcpy(cpustats + handle->cpustats_size, buf, size); handle->cpustats_size += size; cpustats[handle->cpustats_size] = 0; @@ -3124,7 +3136,7 @@ static int handle_options(struct tracecmd_input *handle) case TRACECMD_OPTION_BUFFER_TEXT: ret = handle_buffer_option(handle, option, buf, size); if (ret < 0) - return ret; + goto out; break; case TRACECMD_OPTION_TRACECLOCK: if (!handle->ts2secs) @@ -3183,6 +3195,8 @@ static int handle_options(struct tracecmd_input *handle) tep_read_number(handle->pevent, buf, 8), 0); break; case TRACECMD_OPTION_DONE: + if (compress) + in_uncompress_reset(handle); ret = handle_option_done(handle, buf, size); free(buf); return ret; @@ -3195,7 +3209,12 @@ static int handle_options(struct tracecmd_input *handle) } - return 0; + ret = 0; + +out: + if (compress) + in_uncompress_reset(handle); + return ret; } static int read_options_type(struct tracecmd_input *handle) From patchwork Thu Nov 11 15:11:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5014BC433EF for ; Thu, 11 Nov 2021 15:12:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3AC8360F58 for ; Thu, 11 Nov 2021 15:12:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233995AbhKKPO7 (ORCPT ); Thu, 11 Nov 2021 10:14:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPO6 (ORCPT ); Thu, 11 Nov 2021 10:14:58 -0500 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BF5BC061766 for ; Thu, 11 Nov 2021 07:12:09 -0800 (PST) Received: by mail-ed1-x530.google.com with SMTP id v11so25479210edc.9 for ; Thu, 11 Nov 2021 07:12:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ysWlhe9zjnYL/T8prK0xXq8+XvsRI88igIHmwsQhv9M=; b=Sj0LecgrU4eGL4h+57FzIJfhXU0l4IfIs6UGss/0dCU/1eG/19c4kQCN1ezcBfnpXy aNDHm48iYs37SzO6Dp6cEXPd8knpg66Mo5HIOHiv4Lqb+Pb6mEEwzwmdGVhP3Q1MHi2e 23nANApGm49VTt7DWKNLBDKiSqG+SB9OfIkNXmdBoU+/OdNdC4uAp62sn+pc89fouLKN vLYl8b6Bb17USp6UNFKSa9Q951uxd3W/CeMDh1+YtxvmEtDlEWle2J0eRuyelalavqJw OBQzsRwDZAUwoGRQnbqBrPvQMSWjTiRo4qSWWiU3G7uiM5wck5w9fyhJ0x4J1h1cs2m1 2+iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ysWlhe9zjnYL/T8prK0xXq8+XvsRI88igIHmwsQhv9M=; b=y0Q4/nGZx6BDguueSGxHaG+N7VADTjTQsqL5FuQx+9ZVd/AzdLsB9LYcNKAvV3s8eS CVYqTDPEpLADHOaiNJGT7tUuAg3vuT2xXVqGj0L1kq5nP2cOpJZYObIDjdEUlGowMIEz FbI4BwQU8K93KfXZ0EduXVRn6ifT9CW5JhlpqK4m/F/7hPvtg2XoW45XL3FVy0BkwVeb bCKENtSVDXsu5EjQc+b/BUZNcZJCOcubRLDmqIa8tpaFqTsOou3H4XCss5cbJWk2m7Rh YWuYiu1e5z2FNp/gVPNCCs9M0tNqvlA3wM62BDxKcZ9VrGoba+A8MnDHnSfNeDvN5Jdi up4g== X-Gm-Message-State: AOAM532c2Jm+Gmgmd0ADUPUSRZNp4N5g0kp3uxVMuvc3aQRdl99pFnj4 DDAuZK5cS/CT1iSuRZ/Y7lBHPpHbDo4Npg== X-Google-Smtp-Source: ABdhPJzfxHi9ex3Zjh8iBcOoiNfF2OUfwE9rh9a2RCZ65C4/t24xsGu7G6ruuwgJxPjPABM8al521A== X-Received: by 2002:a17:906:6c83:: with SMTP id s3mr10527597ejr.13.1636643528047; Thu, 11 Nov 2021 07:12:08 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:07 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 11/20] trace-cmd library: Read compression header Date: Thu, 11 Nov 2021 17:11:44 +0200 Message-Id: <20211111151153.86855-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 introduced new mandatory compression header, storing information about the compression algorithm used to compress the trace file. Added code to read that header and to initialize compression context according to it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b7ad946c..2badc341 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -190,6 +190,7 @@ __thread struct tracecmd_input *tracecmd_curr_thread_handle; #define CHECK_READ_STATE(H, S) ((H)->file_version < FILE_VERSION_SECTIONS && (H)->file_state >= (S)) #define HAS_SECTIONS(H) ((H)->flags & TRACECMD_FL_SECTIONED) +#define HAS_COMPRESSION(H) ((H)->flags & TRACECMD_FL_COMPRESSION) static int read_options_type(struct tracecmd_input *handle); @@ -3784,7 +3785,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) char test[] = TRACECMD_MAGIC; unsigned int page_size; size_t offset; - char *version; + char *version = NULL; + char *zver = NULL; + char *zname = NULL; char buf[BUFSIZ]; unsigned long ver; @@ -3822,9 +3825,12 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) } handle->file_version = ver; free(version); + version = NULL; if (handle->file_version >= FILE_VERSION_SECTIONS) handle->flags |= TRACECMD_FL_SECTIONED; + if (handle->file_version >= FILE_VERSION_COMPRESSION) + handle->flags |= TRACECMD_FL_COMPRESSION; if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3854,6 +3860,26 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); lseek64(handle->fd, offset, SEEK_SET); + if (HAS_COMPRESSION(handle)) { + zname = read_string(handle); + if (!zname) + goto failed_read; + zver = read_string(handle); + if (!zver) + goto failed_read; + if (strcmp(zname, "none")) { + handle->compress = tracecmd_compress_alloc(zname, zver, + handle->fd, + handle->pevent, NULL); + if (!handle->compress) { + tracecmd_warning("Unsupported file compression %s %s", zname, zver); + goto failed_read; + } + } + free(zname); + free(zver); + } + if (HAS_SECTIONS(handle)) { if (read8(handle, &(handle->options_start))) { tracecmd_warning("Filed to read the offset of the first option section"); @@ -3866,6 +3892,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) return handle; failed_read: + free(version); + free(zname); + free(zver); free(handle); return NULL; @@ -4064,7 +4093,8 @@ void tracecmd_close(struct tracecmd_input *handle) if (handle->flags & TRACECMD_FL_BUFFER_INSTANCE) tracecmd_close(handle->parent); else { - /* Only main handle frees plugins and pevent */ + /* Only main handle frees plugins, pevent and compression context */ + tracecmd_compress_destroy(handle->compress); tep_unload_plugins(handle->plugin_list, handle->pevent); tep_free(handle->pevent); } From patchwork Thu Nov 11 15:11:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE3BEC433FE for ; Thu, 11 Nov 2021 15:12:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C60ED60E75 for ; Thu, 11 Nov 2021 15:12:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233959AbhKKPPA (ORCPT ); Thu, 11 Nov 2021 10:15:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234001AbhKKPO7 (ORCPT ); Thu, 11 Nov 2021 10:14:59 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7977AC061766 for ; Thu, 11 Nov 2021 07:12:10 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id j21so25221237edt.11 for ; Thu, 11 Nov 2021 07:12:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=68lYZsA3ue8BHX/KQfu7Y5XJCSNyGtkBGxThRpuUnB8=; b=FeXZuBxpvpfsEne0aw72KJ+lCm4j+RBk7lWkVd82z32sFSSjTkYmLLeoeN6qAQI0q/ EvuUupFBBQUUK6UQdiebKvBeuNJO1eBtPQt7dAJI9WVu/LHnuXqlhAbFS2E02+T2j9wA UlwjZ/Ph7rXxbhHOAbJN40CnynqYbULWycgYkEiSJ4i4Nw0ty/9sDu1K9UUIaZTSqsPb MXwGqfnQWRS/8l34aznFX4gwbQMi804G1rcY4BuPS82M5O0V8D2Jtpfp5sAXdz7M/ffb 3apc1ERtBplikc5qaSIOhBcBx1TpCiUv6GUzMh3hULI01ceq0MvhN3Hgd5JVKvsJLLsU hnFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=68lYZsA3ue8BHX/KQfu7Y5XJCSNyGtkBGxThRpuUnB8=; b=OZ/VzCq0aWQmG4Z/yAErUXvDZ52mg5PgOb0rkL4+q7rexKe5EApC6+EnmU0bRYXAS7 j+5V/hc9dt7jLg4oRMO5qGnNUrsZdIKY4vtrW0/cCX13XTBmCtEfpJSig1W4xGP5t+c6 v+SFnjHTviSPjbIZOPzU7K1MmPtgFejTL3i0ruQE2xIQAbvCTrmRgbRyvfIfznj8VOAb dsLCn6tsdLMddPwfM17sPZkiwQOiNm6unXV0tVhYlTEwL8xdoBKnKHv7xRvpdRYYgVSi gf40mcc4bzyQqKK6nnh5Im5LzlQDuI4TwlelKMvkNvanH+fy8I6CREz8knV0yps5iLJI i/Cg== X-Gm-Message-State: AOAM5322R0GymeIKPJDFrxwm1oyVupBeXVrgcVry96QGciRdHje05QxO wW8IBK4YraIS97UL6/mik7SqUGxg/mWftg== X-Google-Smtp-Source: ABdhPJww32hUvxjIS3CVQxwEsN51lBab8Hm9YnFb+C9c9UXgA5aYWRCd/H3EalqNIWmvztBVWvrZFg== X-Received: by 2002:a17:907:980e:: with SMTP id ji14mr10070688ejc.346.1636643529037; Thu, 11 Nov 2021 07:12:09 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:08 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 12/20] trace-cmd library: Extend the input handler with trace data decompression context Date: Thu, 11 Nov 2021 17:11:45 +0200 Message-Id: <20211111151153.86855-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The CPU trace data is compressed in chunks, as chunk's size is multiple trace pages. The input handler is extended with the necessary structures, to control the data decompression. There are two approaches for data decompression, both are supported and can be used in different use cases: - in-memory decompression, page by page. - using a temporary file Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 59 ++++++++++++++++++++++++++++++------- 1 file changed, 48 insertions(+), 11 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 2badc341..f46feac5 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,24 @@ struct page { #endif }; +struct zchunk_cache { + struct list_head list; + struct tracecmd_compress_chunk *chunk; + void *map; + int ref; +}; + +struct cpu_zdata { + /* uncompressed cpu data */ + int fd; + char file[26]; /* strlen(COMPR_TEMP_FILE) */ + unsigned int count; + unsigned int last_chunk; + struct list_head cache; + struct tracecmd_compress_chunk *chunks; +}; + +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +90,7 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + struct cpu_zdata compress; }; struct cpu_file_data { @@ -151,6 +170,8 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + bool read_zpage; /* uncompress pages in memory, do not use tmp files */ + bool cpu_compressed; int file_version; unsigned int cpustats_size; struct cpu_data *cpu_data; @@ -3294,6 +3315,7 @@ static int init_cpu_data(struct tracecmd_input *handle) endian = KBUFFER_ENDIAN_LITTLE; for (cpu = 0; cpu < handle->cpus; cpu++) { + handle->cpu_data[cpu].compress.fd = -1; handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; @@ -4030,6 +4052,7 @@ static inline void free_buffer(struct input_buffer_instance *buf) */ void tracecmd_close(struct tracecmd_input *handle) { + struct zchunk_cache *cache; struct file_section *del_sec; int cpu; int i; @@ -4049,17 +4072,31 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].compress.fd >= 0) { + close(handle->cpu_data[cpu].compress.fd); + unlink(handle->cpu_data[cpu].compress.file); + } + while (!list_empty(&handle->cpu_data[cpu].compress.cache)) { + cache = container_of(handle->cpu_data[cpu].compress.cache.next, + struct zchunk_cache, list); + list_del(&cache->list); + free(cache->map); + free(cache); + } + free(handle->cpu_data[cpu].compress.chunks); } } From patchwork Thu Nov 11 15:11:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 035C0C433F5 for ; Thu, 11 Nov 2021 15:12:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E1DA56044F for ; Thu, 11 Nov 2021 15:12:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234001AbhKKPPB (ORCPT ); Thu, 11 Nov 2021 10:15:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPA (ORCPT ); Thu, 11 Nov 2021 10:15:00 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AC58C061766 for ; Thu, 11 Nov 2021 07:12:11 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id v11so25479627edc.9 for ; Thu, 11 Nov 2021 07:12:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y1gQJSSWIHQEZDOOQU4vx9BZINk/a3FDLCxMqa9nJ88=; b=cVAEMf1JFezohykmZ2jNrklBuGm4faU0mh6/EGj/8UfSLiwCPQNhAtUzZh2aCeHac5 C1V4GEQ2l1ZAzI0kawGmTeDNzpdMkTnKME/hpUf9RMPAQLAmY3EB0aYA7/NefmnOQQya Sma66kvWCrPVFLLFmKMuei6G50F3gc1j6p3YDcsCatGyBCVOPjXomTAUf+RxG/cYD8hF t9vktqpxRhkjJhRy+MkZzYvAi9htrCaQs3HSfr2dq7iJLCtW2+kLdFKlmTy9ZgEhrOOI DeUv5TmVE4DsuYCDGbJqyrOAHrNKBMDLj07tCik93m+fwgvTC+OiwttYaoSojyk+4Vg9 OKuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y1gQJSSWIHQEZDOOQU4vx9BZINk/a3FDLCxMqa9nJ88=; b=mTlniie7oVvu58N8n9T503kiSos0FS3LFMS0xzkjPuzw9/Sj7E4LJceGAq+X3bAJA0 nc1erjBN89jAK7ifIi3ph67+abNbbmPEShDm4SsL1iyYOaw8Bqg4P9SqmYuwRFa8J32s 3x2KfIUUSZbLHFnW6sGVBZzHlXJpOuvEdPDkfz0AomoPBX2cGvm67dPs45GmDSGGk9OE Pw0VQcmTmNyHDYbuk071TrxJW9EjKBRLz3Oi5COI5VbCR/VhrQcrx2wEG78zLqIvTeQ/ xzasP35QF41K0q6FNIX3EkIH0/Z6Q+cchoYwBJyiWiDHvzZ4MePwvnrMxiKZfMaXRFBs u5VQ== X-Gm-Message-State: AOAM533ms0y0g94khhpiTO8AR2KEGKEyIANt7mMCDTkWr6NuEkMcM6rw G8PlKn0qJNx5pxDiaEtsUzaP6bw5BtVj4w== X-Google-Smtp-Source: ABdhPJw7NBP7RCZij5w1A9J52C90rpxC1tPcEaFe2FjwVB4z28MVEciluS1LYjrkw7Khh+Ls/ab/ow== X-Received: by 2002:a17:907:6e1c:: with SMTP id sd28mr10452437ejc.28.1636643530025; Thu, 11 Nov 2021 07:12:10 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:09 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 13/20] trace-cmd library: Initialize CPU data decompression logic Date: Thu, 11 Nov 2021 17:11:46 +0200 Message-Id: <20211111151153.86855-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org On CPU data initialization stage, initialize decompression context for both in-memory and temporary file decompression logics. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 72 +++++++++++++++++++++++++++++++++++-- 1 file changed, 69 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f46feac5..84895051 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -1259,6 +1259,7 @@ static void *allocate_page_map(struct tracecmd_input *handle, off64_t map_offset; void *map; int ret; + int fd; if (handle->read_page) { map = malloc(handle->page_size); @@ -1298,12 +1299,15 @@ static void *allocate_page_map(struct tracecmd_input *handle, map_size -= map_offset + map_size - (cpu_data->file_offset + cpu_data->file_size); + if (cpu_data->compress.fd >= 0) + fd = cpu_data->compress.fd; + else + fd = handle->fd; again: page_map->size = map_size; page_map->offset = map_offset; - page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, - handle->fd, map_offset); + page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, fd, map_offset); if (page_map->map == MAP_FAILED) { /* Try a smaller map */ @@ -2491,16 +2495,76 @@ tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record) /* Not reached */ } -static int init_cpu(struct tracecmd_input *handle, int cpu) +static int init_cpu_zfile(struct tracecmd_input *handle, int cpu) +{ + struct cpu_data *cpu_data; + unsigned long long size; + off64_t offset; + + cpu_data = &handle->cpu_data[cpu]; + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + strcpy(cpu_data->compress.file, COMPR_TEMP_FILE); + cpu_data->compress.fd = mkstemp(cpu_data->compress.file); + if (cpu_data->compress.fd < 0) + return -1; + if (tracecmd_uncompress_copy_to(handle->compress, cpu_data->compress.fd, NULL, &size)) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + cpu_data->offset = 0; + cpu_data->file_offset = 0; + cpu_data->file_size = size; + cpu_data->size = size; + return 0; +} + +static int init_cpu_zpage(struct tracecmd_input *handle, int cpu) { struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + int count; int i; + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + + count = tracecmd_load_chunks_info(handle->compress, &cpu_data->compress.chunks); + if (count < 0) + return -1; + cpu_data->compress.count = count; + cpu_data->compress.last_chunk = 0; + + cpu_data->file_offset = 0; + cpu_data->file_size = 0; + for (i = 0; i < count; i++) + cpu_data->file_size += cpu_data->compress.chunks[i].size; cpu_data->offset = cpu_data->file_offset; cpu_data->size = cpu_data->file_size; + return 0; +} + +static int init_cpu(struct tracecmd_input *handle, int cpu) +{ + struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + int ret; + int i; + + if (handle->cpu_compressed && cpu_data->file_size > 0) { + if (handle->read_zpage) + ret = init_cpu_zpage(handle, cpu); + else + ret = init_cpu_zfile(handle, cpu); + if (ret) + return ret; + } else { + cpu_data->offset = cpu_data->file_offset; + cpu_data->size = cpu_data->file_size; + } cpu_data->timestamp = 0; list_head_init(&cpu_data->page_maps); + list_head_init(&cpu_data->compress.cache); if (!cpu_data->size) { printf("CPU %d is empty\n", cpu); @@ -3372,6 +3436,8 @@ static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buff return -1; if (read_section_header(handle, &id, &flags, NULL, NULL)) return -1; + if (flags & TRACECMD_SEC_FL_COMPRESS) + handle->cpu_compressed = true; if (buffer->latency) { handle->file_state = TRACECMD_FILE_CPU_LATENCY; return init_latency_data(handle) == 0 ? 1 : -1; From patchwork Thu Nov 11 15:11:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC0F0C433FE for ; Thu, 11 Nov 2021 15:12:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B442660E75 for ; Thu, 11 Nov 2021 15:12:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234004AbhKKPPC (ORCPT ); Thu, 11 Nov 2021 10:15:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPB (ORCPT ); Thu, 11 Nov 2021 10:15:01 -0500 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AF12C061766 for ; Thu, 11 Nov 2021 07:12:12 -0800 (PST) Received: by mail-ed1-x531.google.com with SMTP id f4so25349244edx.12 for ; Thu, 11 Nov 2021 07:12:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SCMPfk3ygWNU2j/fHCDKLlaqbzZUU2EQUTUGmHpYEJs=; b=dL0zVi2ULCoIA79XgE3TV4imgUwjs0MFxC4zgICv1oSV1tM0gbJkXou8dZvWAcUkaj JwTbqQ5JweW91cHlOCkJmNkaD/RjA8W1QM0dWoeODXfdxU6g4HsOdHbnSN3Eit8T7Ft/ KhipRdO/RbwKn7xdWIR9ltGEiEOuVUuXfxCpsGYvPGxCgCb0RU/zs9FNYiDYqD3SxD68 D/8j4S/azPftCAnS/68FTrWtdqTMZF5tUsgIqVjLX7GqIJKu4c8aUUY4q5EKk3oRBGy1 tcRtlVDVbitZNPX7mDGTuOl2zKgyrb99SLj8XO5VfuuDT0X/35ZqNVjk3h3mwTeIWrwL x0wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SCMPfk3ygWNU2j/fHCDKLlaqbzZUU2EQUTUGmHpYEJs=; b=pOtKOcvE3fgr+VwBfCKHaUEaqvKq7511x+/bPDFBDzbrcVowZu9RtQyn5ZND6UO29z E1yEo3JRoLA8WQxr4/3/JWU5APTuyO07UHxAw3ZzpuN4HohTNuVdtIqi0422Y/C0dRju HQLmP7B7jfQEWU0kgz8bGHWjKbqyI3tNovfpYf7x2zwtwpbBnW4gAli0bgtGhonFdnxv 7B/kbVtPfJukAB7aiMUuct0AJBAPuu4VJ+rZujQtCsO/P/mwpWv7HwXf4zXMTOnjIodr 9FMgt1LCYMMsb9PPXxbn5tdYRrwYDMYgzFKB/OZ8WJp2TIGVlWEd69w9AmojuNBcY9xo /RxQ== X-Gm-Message-State: AOAM530od9AIU/L14RdHSgqFNhoZXyUjJCb8/w5CxGVQxYXyJxgdOABc /eG6WtW+7YIfoUFKS/kaFSfhNyvPu/pKbQ== X-Google-Smtp-Source: ABdhPJzYAzo1Gg3CM8GpQaspsrYn/3N48oVefZoQbLzhRAVty5DL197kUkZ2gl68+Bm2wcm5BBz+4g== X-Received: by 2002:a17:906:1b1b:: with SMTP id o27mr10377285ejg.279.1636643530984; Thu, 11 Nov 2021 07:12:10 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:10 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 14/20] trace-cmd library: Add logic for in-memory decompression Date: Thu, 11 Nov 2021 17:11:47 +0200 Message-Id: <20211111151153.86855-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org There are two approaches to read compressed trace data: - use a temporary file to decompress entire trace data before reading - use in-memory decompression of requested trace data chunk only In-memory decompression seems to be more efficient, but selecting which approach to use depends in the use case. A compression chunk consists of multiple trace pages, that's why a small cache with uncompressed chunks is implemented. The chunk stays in the cache until there are pages which have reference to it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 110 ++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 84895051..9e27337c 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -29,6 +29,9 @@ #define COMMIT_MASK ((1 << 27) - 1) +/* force uncompressing in memory */ +#define INMEMORY_DECOMPRESS + /* for debugging read instead of mmap */ static int force_read = 0; @@ -1250,6 +1253,105 @@ static void free_page_map(struct page_map *page_map) free(page_map); } +#define CHUNK_CHECK_OFFSET(C, O) ((O) >= (C)->offset && (O) < ((C)->offset + (C)->size)) +static struct tracecmd_compress_chunk *get_zchunk(struct cpu_data *cpu, off64_t offset) +{ + struct cpu_zdata *cpuz = &cpu->compress; + int min, mid, max; + + if (!cpuz->chunks) + return NULL; + if (offset > (cpuz->chunks[cpuz->count - 1].offset + cpuz->chunks[cpuz->count - 1].size)) + return NULL; + + /* check if the requested offset is in the last requested chunk or in the next chunk */ + if (CHUNK_CHECK_OFFSET(cpuz->chunks + cpuz->last_chunk, offset)) + return cpuz->chunks + cpuz->last_chunk; + cpuz->last_chunk++; + if (cpuz->last_chunk < cpuz->count && + CHUNK_CHECK_OFFSET(cpuz->chunks + cpuz->last_chunk, offset)) + return cpuz->chunks + cpuz->last_chunk; + + /* do a binary search to find the chunk holding the given offset */ + min = 0; + max = cpuz->count - 1; + mid = (min + max)/2; + while (min <= max) { + if (offset < cpuz->chunks[mid].offset) + max = mid - 1; + else if (offset > (cpuz->chunks[mid].offset + cpuz->chunks[mid].size)) + min = mid + 1; + else + break; + mid = (min + max)/2; + } + cpuz->last_chunk = mid; + return cpuz->chunks + mid; +} + +static void free_zpage(struct cpu_data *cpu_data, void *map) +{ + struct zchunk_cache *cache; + + list_for_each_entry(cache, &cpu_data->compress.cache, list) { + if (map <= cache->map && map > (cache->map + cache->chunk->size)) + goto found; + } + return; + +found: + cache->ref--; + if (cache->ref) + return; + list_del(&cache->list); + free(cache->map); + free(cache); +} + +static void *read_zpage(struct tracecmd_input *handle, int cpu, off64_t offset) +{ + struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + struct tracecmd_compress_chunk *chunk; + struct zchunk_cache *cache; + void *map = NULL; + int pindex; + int size; + + /* Look in the cache of already loaded chunks */ + list_for_each_entry(cache, &cpu_data->compress.cache, list) { + if (CHUNK_CHECK_OFFSET(cache->chunk, offset)) { + cache->ref++; + goto out; + } + } + + chunk = get_zchunk(cpu_data, offset); + if (!chunk) + return NULL; + size = handle->page_size > chunk->size ? handle->page_size : chunk->size; + map = malloc(size); + if (!map) + return NULL; + if (tracecmd_uncompress_chunk(handle->compress, chunk, map) < 0) + goto error; + + cache = calloc(1, sizeof(struct zchunk_cache)); + if (!cache) + goto error; + cache->ref = 1; + cache->chunk = chunk; + cache->map = map; + list_add(&cache->list, &cpu_data->compress.cache); + + /* a chunk can hold multiple pages, get the requested one */ +out: + pindex = (offset - cache->chunk->offset) / handle->page_size; + return cache->map + (pindex * handle->page_size); +error: + free(map); + return NULL; +} + static void *allocate_page_map(struct tracecmd_input *handle, struct page *page, int cpu, off64_t offset) { @@ -1261,6 +1363,9 @@ static void *allocate_page_map(struct tracecmd_input *handle, int ret; int fd; + if (handle->cpu_compressed && handle->read_zpage) + return read_zpage(handle, cpu, offset); + if (handle->read_page) { map = malloc(handle->page_size); if (!map) @@ -1403,6 +1508,8 @@ static void __free_page(struct tracecmd_input *handle, struct page *page) if (handle->read_page) free(page->map); + else if (handle->read_zpage) + free_zpage(cpu_data, page->map); else free_page_map(page->page_map); @@ -3889,6 +3996,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; +#ifdef INMEMORY_DECOMPRESS + handle->read_zpage = 1; +#endif if (do_read_check(handle, buf, 3)) goto failed_read; From patchwork Thu Nov 11 15:11:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE0E7C433EF for ; Thu, 11 Nov 2021 15:12:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9D769603E9 for ; Thu, 11 Nov 2021 15:12:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234005AbhKKPPD (ORCPT ); Thu, 11 Nov 2021 10:15:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPC (ORCPT ); Thu, 11 Nov 2021 10:15:02 -0500 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B5D5C061766 for ; Thu, 11 Nov 2021 07:12:13 -0800 (PST) Received: by mail-ed1-x52c.google.com with SMTP id f4so25349385edx.12 for ; Thu, 11 Nov 2021 07:12:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WPuH75SD2kG5GcNF4ZEfZX2r12d6dnM33DDW8nEDB2E=; b=WW5q3jBaY8iz+pnMN4KI/xW1ieMBKKjKnVswajRr5fT02Dfi73nytdc4H0QnvyYmFw PEQG6Dj+2uHelhgTNIEQEkLNO/8+2WaDsmunE/HAhLF1U326PN3VxdrfKsPh30f0AkAb e0nd9xD6Muu9TqOv1j/xfMrCNLU3W97oCzDFWBbH+R/JIFzFc9vfRcmjKgUje52Sluom mFi9vBlL4+JD8FvPgYdnSr1jf6u9rNZJCXCQnvCkVV+qTOk1hUmdpKgBDYbRo4cYCX5A uF2su23eCxuH2kc+zXphYmkAdGxWxYJTzIP8il21coSB4OalIsU1b/Y4WeijKJcf5/bf cifA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WPuH75SD2kG5GcNF4ZEfZX2r12d6dnM33DDW8nEDB2E=; b=62L2JaWxTpA60CCJHcIWYiRCzvMaQx8o4tk+HcGRF1+XPWmVBWlgiEEdWyd/IiY2R5 80QlWQiHVGK7hA6UUoI16RkiTKv695HRutV8Q42b8FZigeSdRnr6uIirkTt3LrTL+mQZ RBHA20a58pnOHDFimPNopktikyIsgxLxtcxLs70v8L4oiFRh/x3AZnI7EqMwws+dDsQj Bibzzm15pjls9FcCpPcYWUbN1RjUpYZ9CKezkwTiROgUQiW/5pCGZ4iko7PGxq1NoKFN scjiSaapnXZiUxKTlXiomA/VJnOvgyRCZnsf/WPZxsJwVRBW/MkgeKAVRiq+ikYbjRhy BhxQ== X-Gm-Message-State: AOAM532u4DdBTRbABClWVMVqvT1Owntl9ZOuWslKjMv3VgA/L/YEzba1 IPT330QwGnsN0E40t6ts34DROBf3Fznp/A== X-Google-Smtp-Source: ABdhPJwLotPr1NotaTPUC/i3fz4jJllxb8ww6Diy5DThlTkW0GLAXqvKXA6qkFu65V30gyYQ1+X9JA== X-Received: by 2002:a17:907:d22:: with SMTP id gn34mr10045539ejc.195.1636643531881; Thu, 11 Nov 2021 07:12:11 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:11 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 15/20] trace-cmd library: Read compressed latency data Date: Thu, 11 Nov 2021 17:11:48 +0200 Message-Id: <20211111151153.86855-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extended the latency read logic for reading comperssed latency trace data. Both decompressing approaches are supported: in-memory and using temporary file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 74 +++++++++++++++++++++++++++++++++---- 1 file changed, 66 insertions(+), 8 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 9e27337c..43094b11 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -177,6 +177,7 @@ struct tracecmd_input { bool cpu_compressed; int file_version; unsigned int cpustats_size; + struct cpu_zdata latz; struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -3444,20 +3445,52 @@ static int read_options_type(struct tracecmd_input *handle) int tracecmd_latency_data_read(struct tracecmd_input *handle, char **buf, size_t *size) { + struct cpu_zdata *zdata = &handle->latz; + void *data; + int rsize; + int fd = -1; + int id; + if (!handle || !buf || !size) return -1; if (handle->file_state != TRACECMD_FILE_CPU_LATENCY) return -1; - /* Read data from a file */ - if (!(*buf)) { - *size = BUFSIZ; - *buf = malloc(*size); - if (!(*buf)) + if (!handle->cpu_compressed) { + fd = handle->fd; + } else if (!handle->read_zpage) { + if (zdata->fd < 0) return -1; + fd = zdata->fd; } - return do_read(handle, *buf, *size); + /* Read data from a file */ + if (fd >= 0) { + if (!(*buf)) { + *size = BUFSIZ; + *buf = malloc(*size); + if (!(*buf)) + return -1; + } + return do_read_fd(fd, *buf, *size); + } + + /* Uncompress data in memory */ + if (zdata->last_chunk >= zdata->count) + return 0; + id = zdata->last_chunk; + if (!*buf || *size < zdata->chunks[id].size) { + data = realloc(*buf, zdata->chunks[id].size); + if (!data) + return -1; + *buf = data; + *size = zdata->chunks[id].size; + } + if (tracecmd_uncompress_chunk(handle->compress, &zdata->chunks[id], *buf)) + return -1; + rsize = zdata->chunks[id].size; + zdata->last_chunk++; + return rsize; } static int init_cpu_data(struct tracecmd_input *handle) @@ -3525,7 +3558,27 @@ static int init_cpu_data(struct tracecmd_input *handle) int init_latency_data(struct tracecmd_input *handle) { - /* To do */ + unsigned long long wsize; + int ret; + + if (!handle->cpu_compressed) + return 0; + + if (handle->read_zpage) { + handle->latz.count = tracecmd_load_chunks_info(handle->compress, &handle->latz.chunks); + if (handle->latz.count < 0) + return -1; + } else { + strcpy(handle->latz.file, COMPR_TEMP_FILE); + handle->latz.fd = mkstemp(handle->latz.file); + if (handle->latz.fd < 0) + return -1; + ret = tracecmd_uncompress_copy_to(handle->compress, handle->latz.fd, NULL, &wsize); + if (ret) + return -1; + lseek64(handle->latz.fd, 0, SEEK_SET); + } + return 0; } @@ -3993,6 +4046,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->fd = fd; handle->ref = 1; + handle->latz.fd = -1; /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; @@ -4282,7 +4336,11 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->trace_clock); free(handle->version); close(handle->fd); - + free(handle->latz.chunks); + if (handle->latz.fd >= 0) { + close(handle->latz.fd); + unlink(handle->latz.file); + } while (handle->sections) { del_sec = handle->sections; handle->sections = handle->sections->next; From patchwork Thu Nov 11 15:11:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81AA9C433F5 for ; Thu, 11 Nov 2021 15:12:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6C45E60F58 for ; Thu, 11 Nov 2021 15:12:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234006AbhKKPPD (ORCPT ); Thu, 11 Nov 2021 10:15:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPD (ORCPT ); Thu, 11 Nov 2021 10:15:03 -0500 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 300DFC061766 for ; Thu, 11 Nov 2021 07:12:14 -0800 (PST) Received: by mail-ed1-x529.google.com with SMTP id m14so25744044edd.0 for ; Thu, 11 Nov 2021 07:12:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5ZdgtACZNoxhLbpI1BAp/Q/VnJlL8lle6m0luqtHneY=; b=FeNb5RMisx8ZBM0iv3+4aEmSip/gjPLcPUboMr6YgxLTh1d9F+3sRz1CgHmN0iry3Z ZZDbkYQrboX55f6IPFU0iOQkDv7Z/4cdiTU9MP2wCTmpAIaZyAhyGbAQuxUZYUA5Xbyw B5ARWp3oEFECAhNChWqsUoh9m5KHnv+gGt60JVo1+ZUqq4tFCdZzXiC/IRg8kNk/x8eI MygyRCYDq7i2Zx/v/YnfZOYItkILMamONXx7u3EOZMbKliB9/Uc2N/qh8aITUv9uDNhG oGCYkqDGqTcvQ0blyVhU2+2aCdFwxdbJB6tWYwO2NKU62IunGxGNGPCoeEB4omRCPavU QT9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5ZdgtACZNoxhLbpI1BAp/Q/VnJlL8lle6m0luqtHneY=; b=kuowAMNzHMEwqVJvVHSPZZETsCkysBcnavQ5C7/6gqK+uFhOZf5QtGM4NMvr4PsTe7 glwKVKHZvw2IBSxP1udrjmKCYR6i+Q5xz3LlZJW/9CDgHtLjAt0rmlaO0tdok+8Ac1gp uSrMbKbSmIjZrAs86Sa/LVXC/K/FsOw/EyG2emyDgTveZ9KUFXn4t4eCWyp9BL+jHXc4 lkAFrOLLhaIDGQ5o3BkKiAKUgiYYME/6zDVdkowz/knkDs19+Z7Zla9DBY5RqWrAxSAF 4Yo0sr5jKsu8qiKAlHuE8uJxnQ9Bz4w/vX5MtECJ8zv57IUka4r2wigg5dB1SKeXbpOc Hguw== X-Gm-Message-State: AOAM532QGlezBaU9sO1hq0HVXcl+dW6DLdb4Brn4QaHkFzX2WdW/HAgY LYzaeFQ68HptY7c1FxpdBeGprcrb3M9B0Q== X-Google-Smtp-Source: ABdhPJyKrQCjDXHpuauBJEmBQTX/w2jAiph3gTTsS3Ymr+h2HIER1kE8UbdyjlyZypOGA7/md0wUOQ== X-Received: by 2002:a05:6402:5216:: with SMTP id s22mr10708918edd.291.1636643532813; Thu, 11 Nov 2021 07:12:12 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:12 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 16/20] trace-cmd library: Decompress file sections on reading Date: Thu, 11 Nov 2021 17:11:49 +0200 Message-Id: <20211111151153.86855-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading sections from trace file v7 - check section flags to find out if the section is compressed and decompress it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 43094b11..7bcf4cf4 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -481,12 +481,15 @@ static struct file_section *section_open(struct tracecmd_input *handle, int id) if (lseek64(handle->fd, sec->data_offset, SEEK_SET) == (off64_t)-1) return NULL; + if ((sec->flags & TRACECMD_SEC_FL_COMPRESS) && in_uncompress_block(handle)) + return NULL; return sec; } static void section_close(struct tracecmd_input *handle, struct file_section *sec) { - /* To Do */ + if (sec->flags & TRACECMD_SEC_FL_COMPRESS) + in_uncompress_reset(handle); } static int section_add_or_update(struct tracecmd_input *handle, int id, int flags, @@ -1109,6 +1112,8 @@ static int handle_section(struct tracecmd_input *handle, struct file_section *se return -1; section->data_offset = lseek64(handle->fd, 0, SEEK_CUR); + if ((section->flags & TRACECMD_SEC_FL_COMPRESS) && in_uncompress_block(handle)) + return -1; switch (section->id) { case TRACECMD_OPTION_HEADER_INFO: @@ -1134,6 +1139,9 @@ static int handle_section(struct tracecmd_input *handle, struct file_section *se break; } + if (section->flags & TRACECMD_SEC_FL_COMPRESS) + in_uncompress_reset(handle); + return ret; } From patchwork Thu Nov 11 15:11:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3F72C433EF for ; Thu, 11 Nov 2021 15:12:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B65160E75 for ; Thu, 11 Nov 2021 15:12:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234007AbhKKPPF (ORCPT ); Thu, 11 Nov 2021 10:15:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPE (ORCPT ); Thu, 11 Nov 2021 10:15:04 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 459EAC061766 for ; Thu, 11 Nov 2021 07:12:15 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id w1so25473663edd.10 for ; Thu, 11 Nov 2021 07:12:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2FdTYThrsW3IOIH66mpQnV9KJ7jDkulNvcqYodkm6VM=; b=dI0z9ZNSdpcYrw+hQ0jHbiqr3gJ/1aCk5FeYrq+sXFMu6X/2lGnA78m779I/XzvExE ASSCEGW/nFr6Qvp6JHuNMrfB3XUj7wY6LBJzsSe+EnWhyb2u8x27PYw4PkOwEHi5FiBM RHSbG2j/HyXIpDio3SRCZCsFF/v5D0V9rNStrEr0q6L7C8nmID9CUfh2YWvG3v/OQIxz qQiRIwkWM9NZwtmvfxmGOoJrxpy9zQzbAmzMOanCzgZ3EcKohLnb4Q035ogi7sROVKrE aTXSwiH/cfjhvce1hvTEhPXU8fUgfsbE7blkrQUFlmtecdYw4lXqZ8NTcpTkUdeL1Gcb L4WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2FdTYThrsW3IOIH66mpQnV9KJ7jDkulNvcqYodkm6VM=; b=SpescENPaLg4ofsDaQPAG+fnYV99Vh9E7MmULdfGTDBIQ0kUsS0WWE+htva+4eDSOb mPm3DZi/TGF/8WYFlxUHRS5RT5021XZ8zPj59v3aMZLAlqhsiv8n/zE9Q00ai8xDLpTv YybJVQdxQF3lbfDHQvUFgzbmpoLBfmbhJLAGjDIWhOBpewZenJ7ybLOkp0uGqMyxf1cn HOckwspK52/YtsK1Yeu9Fb4hbHEDsCux7k1hxKdcXIUCXqMbIQtLH5oYF4BZA3c1MA1B 9LRA9Et7cSZ6Cx2KEgDgTQ7w9v7nl71/tpABNFCKCMlqh38APGZAtQ+/eF/kzrmVgbNg 5QdQ== X-Gm-Message-State: AOAM531ZXIb+jmtdt0/urFL4cBPjbSuV+BwfNEwsjQL2RiT8twRh4GQk Yiu2O98D0o5jn+Lg2E02NB1PvWMCkOfJog== X-Google-Smtp-Source: ABdhPJxC+tc9//kQglJNzsmy/UUWFZ7bt1tdxDAOSOEqVMavjJ+qurAwBgOYs8GWHeojWGF8Sx5jPQ== X-Received: by 2002:aa7:c0cc:: with SMTP id j12mr2997876edp.124.1636643533869; Thu, 11 Nov 2021 07:12:13 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:13 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 17/20] trace-cmd library: Add zlib compression algorithm Date: Thu, 11 Nov 2021 17:11:50 +0200 Message-Id: <20211111151153.86855-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added implementation for zlib compression algorithm, using libz library, if available. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Makefile | 7 ++ lib/trace-cmd/Makefile | 7 ++ lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-compress-zlib.c | 109 ++++++++++++++++++++++++ lib/trace-cmd/trace-compress.c | 3 + tracecmd/Makefile | 4 + 6 files changed, 134 insertions(+) create mode 100644 lib/trace-cmd/trace-compress-zlib.c diff --git a/Makefile b/Makefile index 91f62859..c9faf8db 100644 --- a/Makefile +++ b/Makefile @@ -300,6 +300,13 @@ ifeq ($(PERF_DEFINED), 1) CFLAGS += -DPERF endif +ZLIB_INSTALLED := $(shell if (printf "$(pound)include \n void main(){deflateInit(NULL, Z_BEST_COMPRESSION);}" | $(CC) -o /dev/null -x c - -lz >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) +ifeq ($(ZLIB_INSTALLED), 1) +export ZLIB_INSTALLED +CFLAGS += -DHAVE_ZLIB +$(info Have zlib compression support) +endif + CUNIT_INSTALLED := $(shell if (printf "$(pound)include \n void main(){CU_initialize_registry();}" | $(CC) -o /dev/null -x c - -lcunit >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) export CUNIT_INSTALLED diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index bab4322d..1ee6ac4c 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -26,6 +26,9 @@ OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif OBJS += trace-compress.o +ifeq ($(ZLIB_INSTALLED), 1) +OBJS += trace-compress-zlib.o +endif # Additional util objects OBJS += trace-blk-hack.o @@ -47,6 +50,10 @@ $(LIBTRACECMD_STATIC): $(OBJS) LIBS = $(LIBTRACEEVENT_LDLAGS) $(LIBTRACEFS_LDLAGS) -lpthread +ifeq ($(ZLIB_INSTALLED), 1) +LIBS += -lz +endif + $(LIBTRACECMD_SHARED_VERSION): $(LIBTRACECMD_SHARED) @ln -sf $( + * + */ +#include +#include +#include +#include + +#include "trace-cmd-private.h" + +#define __ZLIB_NAME "zlib" +#define __ZLIB_WEIGTH 10 + +static int zlib_compress(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + ret = compress2((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes, Z_BEST_COMPRESSION); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_STREAM_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static int zlib_decompress(const char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + ret = uncompress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_DATA_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static unsigned int zlib_compress_bound(unsigned int in_bytes) +{ + return compressBound(in_bytes); +} + +static bool zlib_is_supported(const char *name, const char *version) +{ + const char *zver; + + if (!name) + return false; + if (strlen(name) != strlen(__ZLIB_NAME) || strcmp(name, __ZLIB_NAME)) + return false; + + if (!version) + return true; + + zver = zlibVersion(); + if (!zver) + return false; + + /* Compare the major version number */ + if (atoi(version) <= atoi(zver)) + return true; + + return false; +} + +int tracecmd_zlib_init(void) +{ + return tracecmd_compress_proto_register(__ZLIB_NAME, zlibVersion(), __ZLIB_WEIGTH, + zlib_compress, zlib_decompress, + zlib_compress_bound, zlib_is_supported); +} diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index b198b8dc..c1f37cdf 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -356,6 +356,9 @@ void tracecmd_compress_init(void) gettimeofday(&time, NULL); srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); +#ifdef HAVE_ZLIB + tracecmd_zlib_init(); +#endif } static struct compress_proto *compress_proto_select(void) diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 80c69bbb..b7a23dc4 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -51,6 +51,10 @@ CONFIG_INCLUDES = CONFIG_LIBS = -lrt -lpthread $(TRACE_LIBS) CONFIG_FLAGS = +ifeq ($(ZLIB_INSTALLED), 1) +CONFIG_LIBS += -lz +endif + all: $(TARGETS) $(bdir): From patchwork Thu Nov 11 15:11:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9982BC433F5 for ; Thu, 11 Nov 2021 15:12:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 843E0603E9 for ; Thu, 11 Nov 2021 15:12:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234010AbhKKPPG (ORCPT ); Thu, 11 Nov 2021 10:15:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPF (ORCPT ); Thu, 11 Nov 2021 10:15:05 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C212C061766 for ; Thu, 11 Nov 2021 07:12:16 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id m14so25744355edd.0 for ; Thu, 11 Nov 2021 07:12:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0vF4/kyI9N2FIR8o+/109ZpzkBALfqAJzj8PQ0/ofkw=; b=fQuMNzVI4YLcCCb83XJB9spVHaJh7M68i3qCR+OU8w03n71RhFuvOucbnB6pqoopdk TGP0JfqASPnVkmLnYOhqur/7KTDIu2zr5uaYt2dd9A9GyRrzaSQf5JWS8sDqtbYHcg9g HKOta12P1tRbnAOAL2yTx82qIWshXXYaZ0ftu24xx6ultZNDlhn1KJovEdurZUciGcAJ oEBCW2wQA2c4caqWpOzH1Nbqu4t/CSpr25ZG+MswZJKuJgDdms01HHVe2FXaI5CSVDIy Ps1yEleLXbnerZMVPzxfkpuzjgrQOHt5XvYd0dn9Yr1VkpSz/p0OWQPRkdAiNSf7eHBr dCQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0vF4/kyI9N2FIR8o+/109ZpzkBALfqAJzj8PQ0/ofkw=; b=RpgXsMgpa17k53jd/upBsnq+hgThSez0O8f5b4v+javAK+Oke9d9PWvYvqdQQMWETP 0iu5kr/kDz+jLt+I00Kb6mhliIBNo43xnQuEovhq2rkDe+K27tglHOpUvZ08Oax0Gkp4 2MZ6SkQOwXwpLlxholA8H14v69UGPZWshXa7ws2cIfkWyZ6N9QwrrT3UlbOrDrnzNy5W tsEEcIQxYMe1nwSzyfmX9zz/ATTBjkMShSbRhP9UlCSu40eg4f9nJ6FbCyRCeYyW/CbY V3FDg/YTX4kveQ92kZiJeZtC4OycccykQjYhtVe79Cj6taCHS/SDRLrP22dluiX3upDM Vymw== X-Gm-Message-State: AOAM533sG7YCCkILsqfO7q0qpFLIKvjjR9O3pkhqDazmehkNQ643wtVv SgMEy2ygiFa8BdUG/sEZ7dh4/WvKRyADDQ== X-Google-Smtp-Source: ABdhPJyn4nSyRnpfE27IYIChKwAoXqKSV9cPtmpE9I75M7QzgnUS4COobqrz9/xKJRg4S0Ipaj0Vkw== X-Received: by 2002:aa7:cb92:: with SMTP id r18mr10880541edt.282.1636643534866; Thu, 11 Nov 2021 07:12:14 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:14 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 18/20] trace-cmd list: Show supported compression algorithms Date: Thu, 11 Nov 2021 17:11:51 +0200 Message-Id: <20211111151153.86855-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add new parameter "trace-cmd list -c" to show supported compression algorithms. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd/trace-cmd-list.1.txt | 3 +++ tracecmd/trace-list.c | 26 ++++++++++++++++++++ tracecmd/trace-usage.c | 1 + 3 files changed, 30 insertions(+) diff --git a/Documentation/trace-cmd/trace-cmd-list.1.txt b/Documentation/trace-cmd/trace-cmd-list.1.txt index a5c6b16c..b77e3460 100644 --- a/Documentation/trace-cmd/trace-cmd-list.1.txt +++ b/Documentation/trace-cmd/trace-cmd-list.1.txt @@ -71,6 +71,9 @@ OPTIONS List defined clocks that can be used with trace-cmd record -C. The one in brackets ([]) is the active clock. +*-c*:: + List the available trace file compression algorithms. + SEE ALSO -------- trace-cmd(1), trace-cmd-record(1), trace-cmd-report(1), trace-cmd-start(1), diff --git a/tracecmd/trace-list.c b/tracecmd/trace-list.c index d060c810..900da73b 100644 --- a/tracecmd/trace-list.c +++ b/tracecmd/trace-list.c @@ -549,6 +549,24 @@ static void show_plugins(void) tep_free(pevent); } +static void show_compression(void) +{ + char **versions, **names; + int c, i; + + c = tracecmd_compress_protos_get(&names, &versions); + if (c <= 0) { + printf("No compression algorithms are supported\n"); + return; + } + printf("Supported compression algorithms:\n"); + for (i = 0; i < c; i++) + printf("\t%s, %s\n", names[i], versions[i]); + + free(names); + free(versions); +} + void trace_list(int argc, char **argv) { int events = 0; @@ -562,6 +580,7 @@ void trace_list(int argc, char **argv) int flags = 0; int systems = 0; int show_all = 1; + int compression = 0; int i; const char *arg; const char *funcre = NULL; @@ -626,6 +645,10 @@ void trace_list(int argc, char **argv) systems = 1; show_all = 0; break; + case 'c': + compression = 1; + show_all = 0; + break; case '-': if (strcmp(argv[i], "--debug") == 0) { tracecmd_set_debug(true); @@ -670,6 +693,8 @@ void trace_list(int argc, char **argv) show_clocks(); if (systems) show_systems(); + if (compression) + show_compression(); if (show_all) { printf("event systems:\n"); show_systems(); @@ -679,6 +704,7 @@ void trace_list(int argc, char **argv) show_tracers(); printf("\noptions:\n"); show_options(); + show_compression(); } return; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index ac12b066..34c6cc35 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -348,6 +348,7 @@ static struct usage_help usage_help[] = { " -O list plugin options\n" " -B list defined buffer instances\n" " -C list the defined clocks (and active one)\n" + " -c list the supported trace file compression algorithms\n" }, { "restore", From patchwork Thu Nov 11 15:11:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDB38C433FE for ; Thu, 11 Nov 2021 15:12:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B69F460F58 for ; Thu, 11 Nov 2021 15:12:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234009AbhKKPPH (ORCPT ); Thu, 11 Nov 2021 10:15:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233991AbhKKPPG (ORCPT ); Thu, 11 Nov 2021 10:15:06 -0500 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C7C9C061766 for ; Thu, 11 Nov 2021 07:12:17 -0800 (PST) Received: by mail-ed1-x52b.google.com with SMTP id o8so25333999edc.3 for ; Thu, 11 Nov 2021 07:12:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ArPF4W8+YICbDVWNiHuRa29uK5W+Pn0glkD+rw5Vb30=; b=X/tDEsZy0BCVoroFX2fIrLpgaDmJ/Top//+yhhvEww+7CPToXm14NMV5BkNEAVoAsU A2ikhAHYffr/uxbogCmD85xtZt9GwSdqtqsErIQqHkzYZ6YkXVZBLfUOz5JyUlAtg9Kv qE5rDZgOgi8HnLQEWiUjPvVVzVLcovaYEy5pBIb73NGdS/qa+M+gnqZ89moNZV8j6N9X oriLFNE2spnSC2xFKvQFsJI0l/LWwRDZR69r4ur1FKxxLHIsgj8AUbdmS11k97gmBFPs TCvMxEEhI8Tk8/ueg+xLCc7nLxgK8k5ypx4GcN8tfgvRHs43Dpve9aqxsy6VR5ZLkRmj wLPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ArPF4W8+YICbDVWNiHuRa29uK5W+Pn0glkD+rw5Vb30=; b=wcLOhetQP5muwrnbet5YtylEbC+VD3IHMEwAIb/CsC6Rf/PQwPCr5orjpmGP+f3EFr PcSpSpCf3KS9J9KZMIHuP5cTcewGrWKFS0rPmrGXyieIhJyGE2NmJAXG051tCy31WqdK n5sNHXC82rJyZkqu3yN9te7qcAjoB1IBvf8jW8CWjBv/wUS3BN4GddnOSFiO2crQzMfB y9BO3FPee/SOirlV4FY7H8EsabxSniDszQEyhb/2tmacS+FBsniHVP8VIUxiOtnYiESL BAOU9GyTuWS8bf2Zm21EI4xb3ZvNr7o5XsF1h2oZImQHJICeJQ+17Uxk/8+gTeKVA192 pQqw== X-Gm-Message-State: AOAM531sFXncdAR0KxFsqXsRAEeCejjpOL+53P+p5crtQuO1t+/v35tH a4UtHFERRXY1xKCoQ48GMqE+pt4x92ua+g== X-Google-Smtp-Source: ABdhPJwqrgm/idjLVLUx71ZRMKec5bem0reeV/0BFmSLc5kidAZvO8OfQ4SYY8eXZ463O9YajLL+KQ== X-Received: by 2002:aa7:c34c:: with SMTP id j12mr10921755edr.31.1636643535915; Thu, 11 Nov 2021 07:12:15 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:15 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 19/20] trace-cmd record: Add compression to the trace context Date: Thu, 11 Nov 2021 17:11:52 +0200 Message-Id: <20211111151153.86855-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org As the trace-cmd library supports trace file compression, trace-cmd record command should have a way to configure this functionality. Trace context is extended to hold the compression algorithm, used to compress the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 699fa511..98d663f1 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -199,6 +199,7 @@ struct common_record_context { char *date2ts; char *user; const char *clock; + const char *compression; struct tsc_nsec tsc2nsec; int data_flags; int tsync_loop_interval; @@ -3702,6 +3703,12 @@ static struct tracecmd_output *create_net_output(struct common_record_context *c goto error; if (tracecmd_output_set_msg(out, msg_handle)) goto error; + if (ctx->compression) { + if (tracecmd_output_set_compression(out, ctx->compression)) + goto error; + } else if (ctx->file_version >= FILE_VERSION_COMPRESSION) { + tracecmd_output_set_compression(out, "any"); + } if (tracecmd_output_write_headers(out, listed_events)) goto error; @@ -3748,6 +3755,12 @@ setup_connection(struct buffer_instance *instance, struct common_record_context goto error; if (tracecmd_output_set_version(network_handle, ctx->file_version)) goto error; + if (ctx->compression) { + if (tracecmd_output_set_compression(network_handle, ctx->compression)) + goto error; + } else if (ctx->file_version >= FILE_VERSION_COMPRESSION) { + tracecmd_output_set_compression(network_handle, "any"); + } if (tracecmd_output_write_headers(network_handle, listed_events)) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4481,6 +4494,12 @@ static struct tracecmd_output *create_output(struct common_record_context *ctx) goto error; if (ctx->file_version && tracecmd_output_set_version(out, ctx->file_version)) goto error; + if (ctx->compression) { + if (tracecmd_output_set_compression(out, ctx->compression)) + goto error; + } else if (ctx->file_version >= FILE_VERSION_COMPRESSION) { + tracecmd_output_set_compression(out, "any"); + } if (tracecmd_output_write_headers(out, listed_events)) goto error; return out; @@ -4516,7 +4535,7 @@ static void record_data(struct common_record_context *ctx) if (latency) { handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, - ctx->file_version, NULL); + ctx->file_version, ctx->compression); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) From patchwork Thu Nov 11 15:11:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31360C433F5 for ; Thu, 11 Nov 2021 15:12:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B18A60F94 for ; Thu, 11 Nov 2021 15:12:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234015AbhKKPPI (ORCPT ); Thu, 11 Nov 2021 10:15:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234012AbhKKPPH (ORCPT ); Thu, 11 Nov 2021 10:15:07 -0500 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40012C061766 for ; Thu, 11 Nov 2021 07:12:18 -0800 (PST) Received: by mail-ed1-x533.google.com with SMTP id r12so25431942edt.6 for ; Thu, 11 Nov 2021 07:12:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=A0QbvQM6j2lXvTsVng1o2qq/PqDIx5UMKtM885ZAIJk=; b=EsVUA/LBZ4//3TJWWeUDmHsCxICRKgj1prdqFUKRH6TnpeQjc1XcCSecYNh/Mj4hI1 Hm7x99aP5mzSbKMadV2WkcCmlGJdLl6ZoIoK4T4HjhKoioTZcHsTfhFo04m0K0YSTkKf RPt5gnJksEtybakSyzynrWh2SblRvzyVgPX1SYPCVoiTA6aoqUzauBJIQD2r9BBeKCAL UUANjZ+F1OG3D+p5RmeNGn/wpGv9zA9FYtS6Ae12UkCuJcah9K2fI4WODYByxRlZYM4h OAhmups1FXAGXVa5A3jrO4Ovv/VWAZHm9JIWKYVcZ/6whWa/50zKCpXI0WqaRfhsoyn2 AtmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A0QbvQM6j2lXvTsVng1o2qq/PqDIx5UMKtM885ZAIJk=; b=J+elFJ/p6RzcYNFnjduLhsdLmN8IsXkNzFc2VP3iFGuEZpmKcpGKNvr4R8YHZnp8+k 6WVDAcEFWeJTHopYtqf/dGZl9MPMhmTzr4aesQ16Vzx9bq8WMvBB4tJwvtlDBBoLZA/0 Ttcs54IEDpeKsa7XpigigZGn3sb0Ih9egrNW8SCVmMCRnsNFddPJKKTbaiR/Tt7p5og+ LneDZQClJPHls62/AWz6cUtoBA2xk6TVbiHl2NHEAxLLtuT5RL0+PSlJ/NX/GvuqQ+Fb L92jKrpXGu4FrKyrON0Jd8lsW5YuEbSJKqYquVME8I+bfrcFHaCRFKlVIccK6sJdUyXs hI/Q== X-Gm-Message-State: AOAM533xc/H+d5bcINA87VCBm5zdvIafp0W4NPvDOo5CX+le7+LkUML/ ig/oDNijrIlIATqDDzard0m2Uz323Du5GA== X-Google-Smtp-Source: ABdhPJxZaxBiH5gh8SkFcIYU21aWRRNEGUdLVxDceF8kCPRgmGBvA8XZVqEisIQQ0Hdf0HGY2LKWIg== X-Received: by 2002:a05:6402:50d0:: with SMTP id h16mr10678054edb.70.1636643536893; Thu, 11 Nov 2021 07:12:16 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.12.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:12:16 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 20/20] trace-cmd report: Add new parameter for trace file compression Date: Thu, 11 Nov 2021 17:11:53 +0200 Message-Id: <20211111151153.86855-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211111151153.86855-1-tz.stoyanov@gmail.com> References: <20211111151153.86855-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A new parameter is added, which can be used to set desired compression the output trace file. "trace-cmd report --compression " Where the string can be compression algorithm name, "any" for the best available algorithm or "none" for no compression. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 13 +++++++++++++ tracecmd/trace-usage.c | 5 +++++ 2 files changed, 18 insertions(+) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 98d663f1..cbf51418 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -5810,6 +5810,7 @@ void init_top_instance(void) } enum { + OPT_compression = 237, OPT_file_ver = 238, OPT_verbose = 239, OPT_tsc2nsec = 240, @@ -6250,6 +6251,7 @@ static void parse_record_options(int argc, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, {"poll", no_argument, NULL, OPT_poll}, {"verbose", optional_argument, NULL, OPT_verbose}, + {"compression", required_argument, NULL, OPT_compression}, {"file-version", required_argument, NULL, OPT_file_ver}, {NULL, 0, NULL, 0} }; @@ -6676,6 +6678,17 @@ static void parse_record_options(int argc, cmd_check_die(ctx, CMD_set, *(argv+1), "--poll"); recorder_flags |= TRACECMD_RECORD_POLL; break; + case OPT_compression: + cmd_check_die(ctx, CMD_start, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_set, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_extract, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_stream, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_profile, *(argv+1), "--compression"); + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + ctx->compression = strdup(optarg); + break; case OPT_file_ver: cmd_check_die(ctx, CMD_start, *(argv+1), "--file_version"); cmd_check_die(ctx, CMD_set, *(argv+1), "--file_version"); diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 34c6cc35..77898c1c 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -70,6 +70,11 @@ static struct usage_help usage_help[] = { " at the beginnig and at the end of the trace\n" " --poll don't block while reading from the trace buffer\n" " --file-version set the desired trace file version\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { "set",