From patchwork Wed Jan 19 08:24:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AD2AC433FE for ; Wed, 19 Jan 2022 08:25:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352346AbiASIZM (ORCPT ); Wed, 19 Jan 2022 03:25:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352368AbiASIZM (ORCPT ); Wed, 19 Jan 2022 03:25:12 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C93D5C06161C for ; Wed, 19 Jan 2022 00:25:11 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id c24so5170471edy.4 for ; Wed, 19 Jan 2022 00:25:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iFXgQn14zZ1pBYi8HBoCz7AWkIK64BoR5fXq1Tc9x90=; b=OhAY0FhdJ71QxOIf05ke4MR5M8dYfSbSWHXNuqlD2Xgt5ohWkk3JTzlHOPBIdkZ2Eh kHeI76YYaf9FUFBo7XzYlE/Ex7heKztnQrASJ+ALlAqa0dfPCSHSzjEYYLqWYsPoDFLB umZbHmunXcWofen11kGtFkQ9OKQcXNy65p1fk0c1+n5hq5fGWMLoOVASwwphj7nvTeOx byyQeLEbHb+/L7G2VWZpGratKYvkPbIIpt2iJxZkIoUgBtLAL/kYCdcbNCojztERg/EC KGAllgKdAYySUy2u4MOviOETiDuPwctIfrmrNsNsfpF8mqqIBPxMYjDxL4mjukiw/8lr VnpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iFXgQn14zZ1pBYi8HBoCz7AWkIK64BoR5fXq1Tc9x90=; b=FhQzHIWTiSYdXDJPkSgqI+7kgQF6EbiLefFeLxCnjDYL9EKMDa87NpwC6R+R2c1SP5 OpH8GtPUbeeNX20D01H7x+9vl5CEPECVPpEuuk43SJalk4LpEitN0yLl/g5xlb3ejEvk zP7ggDagSPLFgz8nfTnmfCPTVoZuSRb+pKpaI90P12WgkkelF3oZssfE+MudZ+ZyGklR L1kfvBxA+ewMpBVvRgTxPX+DLu5gqVxa60G0kphSJIkYsqvpYTD9pe6Z7i6Ghcf1IsO+ ODAsMy1hqHeGDRNmW8rUCEWoDGcDDjXUwFFGxualkyZPRuZj2NXU+Td/tGAFRqcfe2U9 uuSw== X-Gm-Message-State: AOAM532sYyMxED/6/n5Q9zlKDAVsuQJd3DbT+IjuQ3CrLZ0Gojute1g7 eWM0Mpu4yebQuh0fFtwULzKWoqro9Pg= X-Google-Smtp-Source: ABdhPJw306PoWKp1Pzqi/Wj9GimMSDuDss6TciQlQJO14vlb+NHUlRsrvDry4Q4cNUUSEt1I7tW6iQ== X-Received: by 2002:a05:6402:f06:: with SMTP id i6mr28835115eda.268.1642580710435; Wed, 19 Jan 2022 00:25:10 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:09 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 01/25] trace-cmd library: Define trace file version 7 Date: Wed, 19 Jan 2022 10:24:43 +0200 Message-Id: <20220119082507.245600-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a define for file version 7, but keep the default file version to 6. Defined the new file version as the first version that supports trace file sections. A new trace-cmd library API is introduced, to get the default trace file version if the library: tracecmd_default_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 6 ++++++ lib/trace-cmd/include/trace-cmd-local.h | 9 ++------- lib/trace-cmd/trace-output.c | 4 +++- lib/trace-cmd/trace-util.c | 12 +++++++++++- tracecmd/trace-record.c | 2 ++ 5 files changed, 24 insertions(+), 9 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 1a7d8691..2c9d1452 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -31,6 +31,11 @@ struct tep_plugin_list *trace_load_plugins(struct tep_handle *tep, int flags); int *tracecmd_add_id(int *list, int id, int len); +#define FILE_VERSION_MIN 6 +#define FILE_VERSION_MAX 7 + +#define FILE_VERSION_SECTIONS 7 + enum { RINGBUF_TYPE_PADDING = 29, RINGBUF_TYPE_TIME_EXTEND = 30, @@ -43,6 +48,7 @@ void tracecmd_set_debug(bool set_debug); bool tracecmd_get_debug(void); bool tracecmd_is_version_supported(unsigned int version); +int tracecmd_default_file_version(void); struct tracecmd_output; struct tracecmd_recorder; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index a0d6f0a6..4f8f1d76 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -9,18 +9,13 @@ #include #include "trace-cmd-private.h" +#define FILE_VERSION_DEFAULT 6 + /* Can be overridden */ void tracecmd_warning(const char *fmt, ...); void tracecmd_critical(const char *fmt, ...); void tracecmd_info(const char *fmt, ...); -/* trace.dat file format version */ -#define FILE_VERSION 6 - -#define _STR(x) #x -#define STR(x) _STR(x) -#define FILE_VERSION_STRING STR(FILE_VERSION) - #ifndef htonll # if __BYTE_ORDER == __LITTLE_ENDIAN #define htonll(x) __bswap_64(x) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index f5e2574c..a524415c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -83,6 +83,8 @@ struct list_event_system { char *name; }; +#define HAS_SECTIONS(H) ((H)->file_version >= FILE_VERSION_SECTIONS) + static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { @@ -919,7 +921,7 @@ struct tracecmd_output *tracecmd_output_create_fd(int fd) handle->fd = fd; - handle->file_version = FILE_VERSION; + handle->file_version = FILE_VERSION_DEFAULT; handle->page_size = getpagesize(); handle->big_endian = tracecmd_host_bigendian(); diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 3b2d40ee..21f1b065 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -618,9 +618,19 @@ unsigned long long tracecmd_generate_traceid(void) return hash; } +/* + * tracecmd_default_file_version - Get default trace file version of the library + * + * Returns the default trace file version + */ +int tracecmd_default_file_version(void) +{ + return FILE_VERSION_DEFAULT; +} + bool tracecmd_is_version_supported(unsigned int version) { - if (version <= FILE_VERSION) + if (version >= FILE_VERSION_MIN && version <= FILE_VERSION_MAX) return true; return false; } diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index b4200db1..7b2b59bb 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -214,6 +214,7 @@ struct common_record_context { int topt; int run_command; int saved_cmdlines_size; + int file_version; }; static void add_reset_file(const char *file, const char *val, int prio) @@ -5972,6 +5973,7 @@ static void init_common_record_context(struct common_record_context *ctx, ctx->instance = &top_instance; ctx->curr_cmd = curr_cmd; local_cpu_count = tracecmd_count_cpus(); + ctx->file_version = tracecmd_default_file_version(); init_top_instance(); } From patchwork Wed Jan 19 08:24:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D8DBC43217 for ; Wed, 19 Jan 2022 08:25:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352358AbiASIZQ (ORCPT ); Wed, 19 Jan 2022 03:25:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352368AbiASIZO (ORCPT ); Wed, 19 Jan 2022 03:25:14 -0500 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5B51C061574 for ; Wed, 19 Jan 2022 00:25:13 -0800 (PST) Received: by mail-ed1-x52d.google.com with SMTP id cx27so7677667edb.1 for ; Wed, 19 Jan 2022 00:25:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iAfupZ0NC9/moJ+kh+FwhIx5DA4MFkvqu1YS/2vOrIA=; b=E6XrL74IozhqouZ09PV2Q7r5gX3lZIyd85wbj0EASuVz40h0BrsipKxSfwGpRw7aaN 7DTz8TcBw+JJ8t2192R0p0pHIBy2FhtFZWX+CzUA6ql8+aYiaS1iZccKHClC6XW6UZOC coLEuW5b3+M4lYwpDweLXfSs3JgiyYOVi6fZDrLL8ILkZScnu3R9fXD6Z8gdTlLuyFME 4kCfVyJ6trWlvj2yCyF3PkwjhpM9F0gcH8+HiYqUEcjKKPycFMd+yL9tAWC2JKXosmFD SoefbJO/E2YlkiTBU1Ea79/yR7nWmZZiVc4YxgQ0iOXZ8udj1O8Aajkgsxndt7/1SqBK Mv2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iAfupZ0NC9/moJ+kh+FwhIx5DA4MFkvqu1YS/2vOrIA=; b=6b69gSrM4jrcTtSw6CarpW9z1we/BFkGv7yKc1AVjrcr0yCzZT7otD0QHYRX/DYPiE MsycktwUbzV7AFt9KRc81Zs3FGH/r46fNx/A5yy6dBS3ljcCu7boZ44f6mruvqPNs25C WFsq+9OcmRid1JnFjosHY33Zqa1ZlP6rlRnXN7VnF1XJ9JTbErha9kumvbZdI3BUTa0V /DGSpOQeOnGLGSmnIgbLg73y7lCxR2mEEo4UiKXpgPb6t0ch6tDHm1QFKmcL+jOf0EtV SdaSsux8JehAF10A6WoFNrdtC5MUK9jcvo+YXXUfas1qjI5uDoHH4pnIzzj9Kw2oeund MN9w== X-Gm-Message-State: AOAM531iP+XjeM5IxPac2iLtIQ9ajitpGfRBpp5MVxwH/IwdIAnn1J23 f20F6Ryt4VuU/4xFjfk+1guW5PKF7iI= X-Google-Smtp-Source: ABdhPJwjQdsuyV9ehhvT5FEpjCE67E7lAY4ag/TE7LwzcKYg/N6kwO59+rhum3m1xGBH6uhi7f3pzw== X-Received: by 2002:a17:907:d93:: with SMTP id go19mr5845328ejc.430.1642580711694; Wed, 19 Jan 2022 00:25:11 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:11 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 02/25] trace-cmd library: Add cache functionality to network message handler Date: Wed, 19 Jan 2022 10:24:44 +0200 Message-Id: <20220119082507.245600-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Network message handler is used to send trace metadata through a network socket, instead writing it into a trace file. There are use cases, that could require to do a lseek() on the metadata. It is hard to implement lseek on a network socket, that's why for such use cases a cache to a local file is introduced. Once the metadata is constructed, the local cache is send to the socket. A new library API is used to enable the local cache: tracecmd_msg_handle_cache() The local cache is flushed on the socket when the tracecmd_msg_finish_sending_data() is called. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 + lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-msg.c | 127 +++++++++++++----- lib/trace-cmd/trace-output.c | 45 ++++--- 4 files changed, 127 insertions(+), 51 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 2c9d1452..3e366fe0 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -347,12 +347,16 @@ enum tracecmd_msg_flags { }; /* for both client and server */ +#define MSG_CACHE_FILE "/tmp/trace_msg_cacheXXXXXX" struct tracecmd_msg_handle { int fd; short cpu_count; short version; /* Current protocol version */ unsigned long flags; bool done; + bool cache; + int cfd; + char cfile[sizeof(MSG_CACHE_FILE)]; }; struct tracecmd_tsync_protos { @@ -361,6 +365,7 @@ struct tracecmd_tsync_protos { struct tracecmd_msg_handle * tracecmd_msg_handle_alloc(int fd, unsigned long flags); +int tracecmd_msg_handle_cache(struct tracecmd_msg_handle *msg_handle); /* Closes the socket and frees the handle */ void tracecmd_msg_handle_close(struct tracecmd_msg_handle *msg_handle); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 4f8f1d76..7f280533 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -47,5 +47,6 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name); +off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-msg.c b/lib/trace-cmd/trace-msg.c index 6667028e..4910ed5c 100644 --- a/lib/trace-cmd/trace-msg.c +++ b/lib/trace-cmd/trace-msg.c @@ -154,33 +154,55 @@ static inline int msg_buf_len(struct tracecmd_msg *msg) return ntohl(msg->hdr.size) - MSG_HDR_LEN - ntohl(msg->hdr.cmd_size); } -static int msg_write(int fd, struct tracecmd_msg *msg) +static int __msg_write(int fd, struct tracecmd_msg *msg, bool network) { - int cmd = ntohl(msg->hdr.cmd); int msg_size, data_size; int ret; - - if (cmd < 0 || cmd >= MSG_NR_COMMANDS) - return -EINVAL; - - dprint("msg send: %d (%s) [%d]\n", - cmd, cmd_to_name(cmd), ntohl(msg->hdr.size)); - + int cmd; + + if (network) { + cmd = ntohl(msg->hdr.cmd); + if (cmd < 0 || cmd >= MSG_NR_COMMANDS) + return -EINVAL; + dprint("msg send: %d (%s) [%d]\n", + cmd, cmd_to_name(cmd), ntohl(msg->hdr.size)); + } msg_size = MSG_HDR_LEN + ntohl(msg->hdr.cmd_size); data_size = ntohl(msg->hdr.size) - msg_size; if (data_size < 0) return -EINVAL; - ret = __do_write_check(fd, msg, msg_size); - if (ret < 0) - return ret; - + if (network) { + ret = __do_write_check(fd, msg, msg_size); + if (ret < 0) + return ret; + } if (!data_size) return 0; return __do_write_check(fd, msg->buf, data_size); } +__hidden off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off64_t offset, int whence) +{ + /* + * lseek works only if the handle is in cache mode, + * cannot seek on a network socket + */ + if (!msg_handle->cache || msg_handle->cfd < 0) + return (off64_t)-1; + return lseek64(msg_handle->cfd, offset, whence); +} + +static int msg_write(struct tracecmd_msg_handle *msg_handle, struct tracecmd_msg *msg) +{ + if (msg_handle->cache && msg_handle->cfd >= 0) + return __msg_write(msg_handle->cfd, msg, false); + + + return __msg_write(msg_handle->fd, msg, true); +} + enum msg_trace_flags { MSG_TRACE_USE_FIFOS = 1 << 0, }; @@ -274,11 +296,11 @@ static void msg_free(struct tracecmd_msg *msg) memset(msg, 0, sizeof(*msg)); } -static int tracecmd_msg_send(int fd, struct tracecmd_msg *msg) +static int tracecmd_msg_send(struct tracecmd_msg_handle *msg_handle, struct tracecmd_msg *msg) { int ret = 0; - ret = msg_write(fd, msg); + ret = msg_write(msg_handle, msg); if (ret < 0) ret = -ECOMM; @@ -287,11 +309,11 @@ static int tracecmd_msg_send(int fd, struct tracecmd_msg *msg) return ret; } -static int msg_send_nofree(int fd, struct tracecmd_msg *msg) +static int msg_send_nofree(struct tracecmd_msg_handle *msg_handle, struct tracecmd_msg *msg) { int ret = 0; - ret = msg_write(fd, msg); + ret = msg_write(msg_handle, msg); if (ret < 0) ret = -ECOMM; @@ -454,7 +476,7 @@ static int tracecmd_msg_send_notsupp(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; tracecmd_msg_init(MSG_NOT_SUPP, &msg); - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } static int handle_unexpected_msg(struct tracecmd_msg_handle *msg_handle, @@ -472,7 +494,6 @@ int tracecmd_msg_send_init_data(struct tracecmd_msg_handle *msg_handle, unsigned int **client_ports) { struct tracecmd_msg msg; - int fd = msg_handle->fd; unsigned int *ports; int i, cpus, ret; char *p, *buf_end; @@ -485,13 +506,13 @@ int tracecmd_msg_send_init_data(struct tracecmd_msg_handle *msg_handle, if (ret < 0) goto out; - ret = tracecmd_msg_send(fd, &msg); + ret = tracecmd_msg_send(msg_handle, &msg); if (ret < 0) goto out; msg_free(&msg); - ret = tracecmd_msg_wait_for_msg(fd, &msg); + ret = tracecmd_msg_wait_for_msg(msg_handle->fd, &msg); if (ret < 0) goto out; @@ -564,12 +585,53 @@ tracecmd_msg_handle_alloc(int fd, unsigned long flags) handle->fd = fd; handle->flags = flags; + handle->cfd = -1; + handle->cache = false; return handle; } +int tracecmd_msg_handle_cache(struct tracecmd_msg_handle *msg_handle) +{ + if (msg_handle->cfd < 0) { + strcpy(msg_handle->cfile, MSG_CACHE_FILE); + msg_handle->cfd = mkstemp(msg_handle->cfile); + if (msg_handle->cfd < 0) + return -1; + unlink(msg_handle->cfile); + } + msg_handle->cache = true; + return 0; +} + +static int flush_cache(struct tracecmd_msg_handle *msg_handle) +{ + char buf[MSG_MAX_DATA_LEN]; + int ret; + + if (!msg_handle->cache || msg_handle->cfd < 0) + return 0; + msg_handle->cache = false; + if (lseek64(msg_handle->cfd, 0, SEEK_SET) == (off64_t)-1) + return -1; + do { + ret = read(msg_handle->cfd, buf, MSG_MAX_DATA_LEN); + if (ret <= 0) + break; + ret = tracecmd_msg_data_send(msg_handle, buf, ret); + if (ret < 0) + break; + } while (ret >= 0); + + close(msg_handle->cfd); + msg_handle->cfd = -1; + return ret; +} + void tracecmd_msg_handle_close(struct tracecmd_msg_handle *msg_handle) { close(msg_handle->fd); + if (msg_handle->cfd >= 0) + close(msg_handle->cfd); free(msg_handle); } @@ -666,7 +728,7 @@ int tracecmd_msg_send_port_array(struct tracecmd_msg_handle *msg_handle, if (ret < 0) return ret; - ret = tracecmd_msg_send(msg_handle->fd, &msg); + ret = tracecmd_msg_send(msg_handle, &msg); if (ret < 0) return ret; @@ -678,7 +740,7 @@ int tracecmd_msg_send_close_msg(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; tracecmd_msg_init(MSG_CLOSE, &msg); - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } int tracecmd_msg_send_close_resp_msg(struct tracecmd_msg_handle *msg_handle) @@ -686,14 +748,13 @@ int tracecmd_msg_send_close_resp_msg(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; tracecmd_msg_init(MSG_CLOSE_RESP, &msg); - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } int tracecmd_msg_data_send(struct tracecmd_msg_handle *msg_handle, const char *buf, int size) { struct tracecmd_msg msg; - int fd = msg_handle->fd; int n; int ret; int count = 0; @@ -721,7 +782,7 @@ int tracecmd_msg_data_send(struct tracecmd_msg_handle *msg_handle, memcpy(msg.buf, buf + count, n); n = 0; } - ret = msg_write(fd, &msg); + ret = msg_write(msg_handle, &msg); if (ret < 0) break; } @@ -735,8 +796,9 @@ int tracecmd_msg_finish_sending_data(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; int ret; + flush_cache(msg_handle); tracecmd_msg_init(MSG_FIN_DATA, &msg); - ret = tracecmd_msg_send(msg_handle->fd, &msg); + ret = tracecmd_msg_send(msg_handle, &msg); if (ret < 0) return ret; return 0; @@ -752,10 +814,7 @@ int tracecmd_msg_read_data(struct tracecmd_msg_handle *msg_handle, int ofd) while (!tracecmd_msg_done(msg_handle)) { ret = tracecmd_msg_recv_wait(msg_handle->fd, &msg); if (ret < 0) { - if (ret == -ETIMEDOUT) - tracecmd_warning("Connection timed out\n"); - else - tracecmd_warning("reading client"); + tracecmd_warning("reading client %d (%s)", ret, strerror(ret)); return ret; } @@ -959,7 +1018,7 @@ int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, if (ret < 0) return ret; - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } static int get_trace_req_protos(char *buf, int length, @@ -1151,7 +1210,7 @@ int tracecmd_msg_send_time_sync(struct tracecmd_msg_handle *msg_handle, msg.hdr.size = htonl(ntohl(msg.hdr.size) + payload_size); msg.buf = payload; - return msg_send_nofree(msg_handle->fd, &msg); + return msg_send_nofree(msg_handle, &msg); } /** @@ -1283,7 +1342,7 @@ int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, if (ret < 0) return ret; - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a524415c..6ed4bbf0 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -94,6 +94,14 @@ do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) return __do_write_check(handle->fd, data, size); } +static inline off64_t do_lseek(struct tracecmd_output *handle, off_t offset, int whence) +{ + if (handle->msg_handle) + return msg_lseek(handle->msg_handle, offset, whence); + else + return lseek64(handle->fd, offset, whence); +} + static short convert_endian_2(struct tracecmd_output *handle, short val) { if (!handle->pevent) @@ -955,6 +963,9 @@ int tracecmd_output_set_msg(struct tracecmd_output *handle, struct tracecmd_msg_ return -1; handle->msg_handle = msg_handle; + /* Force messages to be cached in a temp file before sending through the socket */ + if (handle->msg_handle && HAS_SECTIONS(handle)) + tracecmd_msg_handle_cache(handle->msg_handle); return 0; } @@ -1281,7 +1292,7 @@ int tracecmd_write_options(struct tracecmd_output *handle) return -1; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) @@ -1314,9 +1325,9 @@ int tracecmd_append_options(struct tracecmd_output *handle) if (handle->file_state != TRACECMD_FILE_OPTIONS) return -1; - if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + if (do_lseek(handle, 0, SEEK_END) == (off_t)-1) return -1; - offset = lseek64(handle->fd, -2, SEEK_CUR); + offset = do_lseek(handle, -2, SEEK_CUR); if (offset == (off_t)-1) return -1; @@ -1334,7 +1345,7 @@ int tracecmd_append_options(struct tracecmd_output *handle) return -1; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) @@ -1525,10 +1536,10 @@ static int update_buffer_cpu_offset(struct tracecmd_output *handle, return -1; } - current = lseek64(handle->fd, 0, SEEK_CUR); + current = do_lseek(handle, 0, SEEK_CUR); /* Go to the option data, where will write the offest */ - if (lseek64(handle->fd, b_offset, SEEK_SET) == (off64_t)-1) { + if (do_lseek(handle, b_offset, SEEK_SET) == (off64_t)-1) { tracecmd_warning("could not seek to %lld\n", b_offset); return -1; } @@ -1537,7 +1548,7 @@ static int update_buffer_cpu_offset(struct tracecmd_output *handle, return -1; /* Go back to end of file */ - if (lseek64(handle->fd, current, SEEK_SET) == (off64_t)-1) { + if (do_lseek(handle, current, SEEK_SET) == (off64_t)-1) { tracecmd_warning("could not seek to %lld\n", offset); return -1; } @@ -1586,7 +1597,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, goto out_free; } - data_offs = lseek64(handle->fd, 0, SEEK_CUR); + data_offs = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, "flyrecord", 10)) goto out_free; @@ -1601,10 +1612,10 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, * updated them with the correct data later. */ endian8 = 0; - data_files[i].file_data_offset = lseek64(handle->fd, 0, SEEK_CUR); + data_files[i].file_data_offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; - data_files[i].file_write_size = lseek64(handle->fd, 0, SEEK_CUR); + data_files[i].file_write_size = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; } @@ -1615,12 +1626,12 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, goto out_free; for (i = 0; i < cpus; i++) { - data_files[i].data_offset = lseek64(handle->fd, 0, SEEK_CUR); + data_files[i].data_offset = do_lseek(handle, 0, SEEK_CUR); /* Page align offset */ data_files[i].data_offset += handle->page_size - 1; data_files[i].data_offset &= ~(handle->page_size - 1); - ret = lseek64(handle->fd, data_files[i].data_offset, SEEK_SET); + ret = do_lseek(handle, data_files[i].data_offset, SEEK_SET); if (ret == (off64_t)-1) goto out_free; @@ -1644,21 +1655,21 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, } /* Write the real CPU data offset in the file */ - if (lseek64(handle->fd, data_files[i].file_data_offset, SEEK_SET) == (off64_t)-1) + if (do_lseek(handle, data_files[i].file_data_offset, SEEK_SET) == (off64_t)-1) goto out_free; endian8 = convert_endian_8(handle, data_files[i].data_offset); if (do_write_check(handle, &endian8, 8)) goto out_free; /* Write the real CPU data size in the file */ - if (lseek64(handle->fd, data_files[i].file_write_size, SEEK_SET) == (off64_t)-1) + if (do_lseek(handle, data_files[i].file_write_size, SEEK_SET) == (off64_t)-1) goto out_free; endian8 = convert_endian_8(handle, data_files[i].write_size); if (do_write_check(handle, &endian8, 8)) goto out_free; offset = data_files[i].data_offset + data_files[i].write_size; - if (lseek64(handle->fd, offset, SEEK_SET) == (off64_t)-1) + if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) goto out_free; if (!tracecmd_get_quiet(handle)) @@ -1667,7 +1678,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, } free(data_files); - if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) + if (do_lseek(handle, 0, SEEK_END) == (off64_t)-1) return -1; handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; @@ -1675,7 +1686,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, return 0; out_free: - lseek64(handle->fd, 0, SEEK_END); + do_lseek(handle, 0, SEEK_END); free(data_files); return -1; } From patchwork Wed Jan 19 08:24:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25DD0C433EF for ; Wed, 19 Jan 2022 08:25:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346595AbiASIZP (ORCPT ); Wed, 19 Jan 2022 03:25:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352358AbiASIZO (ORCPT ); Wed, 19 Jan 2022 03:25:14 -0500 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5141CC06161C for ; Wed, 19 Jan 2022 00:25:14 -0800 (PST) Received: by mail-ed1-x535.google.com with SMTP id q25so7643986edb.2 for ; Wed, 19 Jan 2022 00:25:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Zi7+vslc/ZiY52+jOy+DyEXY9P5mYpQ8HCQAQVaqY/4=; b=BRFH9ZEOL7dkDWP/wMBu+uBCfNWEVBMNS/mgWTHjYFtuUdiJeD4SIx9iZRDnszv4tu M5Ja/9Vinkh9Bs5dDqKYvXyHVJcgg+OaG+xAxL8TOTy07mLCYC/ozjfm6H99rye/Bpef COEAE111HtAmeSyeqNVBxaPj9t6UVkcL+vxPMN34PaZiKzmla/365VS4nPbZuMNG3iae 2grNsJ6dCSScrAXBim7LEsAyQyYU/0n8KQ8k2z5BWo6lvY7Hfk1+1o6q/5mDlGyis7Kp q4rZImOezYVnNUkrNt7IyN/hMsimcEE9+uSKOlWuuwl8pBr5rBfvOBmkqhCDYIR1j05l Bw0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zi7+vslc/ZiY52+jOy+DyEXY9P5mYpQ8HCQAQVaqY/4=; b=OpteHxz6keeQEBSb5Zo4lwMzmjhYWRxVr1u/betX9EGP91gR6qx3ZF83xJqjzJcrYH BryebYSdBDH9qkMxpn1+3hD5mO6Pja0FCEYDOPRHrl0/qOdE8Jy3AzO/id3bUYlpxrTk 3S+JzRCFlC0m2h2JYAYiKXfRqbkXNz9QDOIC4Ij277XLbDFzNfiTzdR331AibzibCk1u yvsrgUAj0JFJZutkpvCkUskl88rCDo1DCpRByCFP+vmYbK9aLEml1YJT44XxgwiP5NJK 8OHkl6//bZGmPof5b+06FN5xquNExp2JO/5V9GjZSohGyOBfT1aU+UCGFLwLgOnW3GnM YCow== X-Gm-Message-State: AOAM530sL24yw3BUVEmC/nIk8COFSJMdDMOEO9MaSIzngzPFMjGyK+zN gfvvMfZ6n9Hz4Hiq9WuyK6c= X-Google-Smtp-Source: ABdhPJyKkjJg/OnnzruScYrn2jEmbbMcSgNQU/kwMjwMcTdI5frv7CKXzqTr4UD8PSME2NPLWukoHQ== X-Received: by 2002:aa7:c448:: with SMTP id n8mr27312781edr.378.1642580712886; Wed, 19 Jan 2022 00:25:12 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:12 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 03/25] trace-cmd library: New APIs to get and set version of output handler Date: Wed, 19 Jan 2022 10:24:45 +0200 Message-Id: <20220119082507.245600-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org There are cases where the version of the file, associated with given output handle must be get or set. Added new APIs for such use cases: tracecmd_get_out_file_version() tracecmd_output_set_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-output.c | 28 +++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 3e366fe0..77374000 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -285,6 +285,7 @@ int tracecmd_output_set_msg(struct tracecmd_output *handle, int tracecmd_output_set_trace_dir(struct tracecmd_output *handle, const char *tracing_dir); int tracecmd_output_set_kallsyms(struct tracecmd_output *handle, const char *kallsyms); int tracecmd_output_set_from_input(struct tracecmd_output *handle, struct tracecmd_input *ihandle); +int tracecmd_output_set_version(struct tracecmd_output *handle, int file_version); int tracecmd_output_write_headers(struct tracecmd_output *handle, struct tracecmd_event_list *list); @@ -318,6 +319,7 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 6ed4bbf0..4d165ac2 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1065,6 +1065,25 @@ int tracecmd_output_set_from_input(struct tracecmd_output *handle, struct tracec return 0; } +/** + * tracecmd_output_set_version - Set file version of the output handle + * @handle: output handle to a trace file. + * @file_version: desired file version + * + * This API must be called before tracecmd_output_write_headers(). + * + * Returns 0 on success, or -1 if the output file handle is not allocated or not in expected state. + */ +int tracecmd_output_set_version(struct tracecmd_output *handle, int file_version) +{ + if (!handle || handle->file_state != TRACECMD_FILE_ALLOCATED) + return -1; + if (file_version < FILE_VERSION_MIN || file_version > FILE_VERSION_MAX) + return -1; + handle->file_version = file_version; + return 0; +} + /** * output_write_init - Write the initial data into the trace file * @handle: output handle to a trace file. @@ -1880,3 +1899,12 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); } + +/** + * tracecmd_get_out_file_version - return the trace.dat file version + * @handle: output handle for the trace.dat file + */ +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle) +{ + return handle->file_version; +} From patchwork Wed Jan 19 08:24:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BF12C433FE for ; Wed, 19 Jan 2022 08:25:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352368AbiASIZQ (ORCPT ); Wed, 19 Jan 2022 03:25:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352348AbiASIZP (ORCPT ); Wed, 19 Jan 2022 03:25:15 -0500 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82DD2C061574 for ; Wed, 19 Jan 2022 00:25:15 -0800 (PST) Received: by mail-ed1-x531.google.com with SMTP id b13so7807900edn.0 for ; Wed, 19 Jan 2022 00:25:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N7QBpH5k2pjW5kIzyo+rJLcLvrdcWSp1lEuVsY13NwY=; b=QlHCwcka7zwYJZX//2Ksf53rI9eTC4oMGTPHs6PYh6xH94PVhRB/eUKRGiTn2bSd0h nNsbYDle2p7JHJXmhQrX7Aw/RqgKTdvc5NYAkYdJe4uxQmaeNILz12LDUAM+G0RuiZ12 UFVNvfCEPS7bbM3cHSrh9n102A0096Fnp9JSSXUh1qizyvaBBnQdjGnq8d4kx0R+iUqv bXhGxB1sLdd8IjTk0vKJOVw9VMKAYCADAt6dCUfLnndaE9Ebz+BL9w5Pgapqfsguun8I 05QM7LbroujOMS6BKc/tU/Zyx+1yi//ceR0D+CNFzidWzvIW9461PPSwVHves8WR2dKM 9lUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N7QBpH5k2pjW5kIzyo+rJLcLvrdcWSp1lEuVsY13NwY=; b=4WHZp+LswEmLRMpeISK41o7WrwxeDBAPgAxRNSVROHK4p6TV+ZQ/cWtvrGaDrhlzT7 mdybqo8EebBbFjx127Yy5Ax4hp6ELhhFGT2foVhRs257OetlE0kPR8dfQ4mdxSz4d4h2 rQSXc3Sv2v8DmmxDPMdmowLH/Ov+VjntjuOPTXqFh0TzLriZXZf3rOA2IbFu/FMmbg36 m66iPbeXxHdDO3y8mcqVJFqRqgtYqBiUMCJ40vWzlIA5MjoktbfepbNjT3ZUyI21YvuZ /tIVlIiThAqMtfoqn82vIn9ocRgD8GoQQ83SHk6bxoM0LmidAR4dmzkVkvgYSDgpmnaI /Prw== X-Gm-Message-State: AOAM5325+c1h4mWc4vAkd25QnL/rfNUlItLAWWPdZDUEIQ2OQ3h9TZGM yT1UkZsCuJp8F4kESOvNzhqZj/Y7lGk= X-Google-Smtp-Source: ABdhPJzyE0sRsNoz6Fpcty7yRYNOF7qqDmFUXf7c5WmIpPHwLnRs+i4xvEvwHHeo/DOIasNH1jy+/A== X-Received: by 2002:a17:907:72c4:: with SMTP id du4mr1027506ejc.564.1642580714104; Wed, 19 Jan 2022 00:25:14 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:13 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 04/25] trace-cmd library: Add strings section in trace file version 7 Date: Wed, 19 Jan 2022 10:24:46 +0200 Message-Id: <20220119082507.245600-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In the trace file metadata there are various dynamic strings. Collecting all these strings in a dedicated section in the file simplifies parsing of the metadata. The string section is added in trace files version 7, at the end of the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-output.c | 68 +++++++++++++++++++ tracecmd/trace-record.c | 1 + 3 files changed, 71 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 77374000..e1467971 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -143,6 +143,7 @@ enum { TRACECMD_OPTION_TIME_SHIFT, TRACECMD_OPTION_GUEST, TRACECMD_OPTION_TSC2NSEC, + TRACECMD_OPTION_STRINGS, }; enum { @@ -306,6 +307,7 @@ int tracecmd_write_buffer_info(struct tracecmd_output *handle); int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); int tracecmd_write_options(struct tracecmd_output *handle); +int tracecmd_write_meta_strings(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 4d165ac2..d3c73ceb 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -62,6 +62,12 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + + /* size of meta-data strings, not yet stored in the file */ + unsigned long strings_p; + /* current virtual offset of meta-data string */ + unsigned long strings_offs; + size_t options_start; bool big_endian; @@ -69,6 +75,9 @@ struct tracecmd_output { struct list_head buffers; struct tracecmd_msg_handle *msg_handle; char *trace_clock; + + /* meta-data strings, not yet stored in the file */ + char *strings; }; struct list_event { @@ -85,6 +94,8 @@ struct list_event_system { #define HAS_SECTIONS(H) ((H)->file_version >= FILE_VERSION_SECTIONS) +static int save_string_section(struct tracecmd_output *handle); + static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { @@ -127,6 +138,22 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +static long add_string(struct tracecmd_output *handle, const char *string) +{ + int size = strlen(string) + 1; + int pos = handle->strings_p; + char *strings; + + strings = realloc(handle->strings, pos + size); + if (!strings) + return -1; + handle->strings = strings; + memcpy(handle->strings + pos, string, size); + handle->strings_p += size; + + return handle->strings_offs + pos; +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen @@ -185,6 +212,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) free(option); } + free(handle->strings); free(handle->trace_clock); free(handle); } @@ -194,6 +222,11 @@ void tracecmd_output_close(struct tracecmd_output *handle) if (!handle) return; + if (HAS_SECTIONS(handle)) { + /* write strings section */ + save_string_section(handle); + } + if (handle->fd >= 0) { close(handle->fd); handle->fd = -1; @@ -332,6 +365,32 @@ int tracecmd_ftrace_enable(int set) return ret; } +static int save_string_section(struct tracecmd_output *handle) +{ + if (!handle->strings || !handle->strings_p) + return 0; + + if (!check_out_state(handle, TRACECMD_OPTION_STRINGS)) { + tracecmd_warning("Cannot write strings, unexpected state 0x%X", + handle->file_state); + return -1; + } + + if (do_write_check(handle, handle->strings, handle->strings_p)) + goto error; + + handle->strings_offs += handle->strings_p; + free(handle->strings); + handle->strings = NULL; + handle->strings_p = 0; + handle->file_state = TRACECMD_OPTION_STRINGS; + return 0; + +error: + return -1; +} + + static int read_header_files(struct tracecmd_output *handle) { tsize_t size, check_size, endian8; @@ -1328,6 +1387,15 @@ int tracecmd_write_options(struct tracecmd_output *handle) return 0; } +int tracecmd_write_meta_strings(struct tracecmd_output *handle) +{ + if (!HAS_SECTIONS(handle)) + return 0; + + return save_string_section(handle); +} + + int tracecmd_append_options(struct tracecmd_output *handle) { struct tracecmd_option *options; diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 7b2b59bb..f599610e 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4093,6 +4093,7 @@ static void setup_agent(struct buffer_instance *instance, tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); tracecmd_write_options(network_handle); + tracecmd_write_meta_strings(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); instance->network_handle = network_handle; } From patchwork Wed Jan 19 08:24:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DCD3C433F5 for ; Wed, 19 Jan 2022 08:25:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352348AbiASIZR (ORCPT ); Wed, 19 Jan 2022 03:25:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZR (ORCPT ); Wed, 19 Jan 2022 03:25:17 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B7C9C061574 for ; Wed, 19 Jan 2022 00:25:16 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id c71so7587694edf.6 for ; Wed, 19 Jan 2022 00:25:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d6fNTUIWdW75sP4ix2ktwJ53a4hbge6GLMCdTHFHxVQ=; b=h7xFh9/eYcDj1sf9hQThEpDg9fEd4Z5PIAH04ONzks34en9+7Ql4ntMiTXhb5sEaV3 1n3KDIC0oHh98R7tyAZsMgp8Nxo9vov8XJmkob/VplpkuMA2gZkVbJ+9UVT3oxREt3Wd rrAtg+lzVAeZ7TKerZB7KjzhASWW3XBpmmPxhz7FEXFNZ8E9mNInO023X5fp4iURHZM+ cTbgfs8ElC5P0xPkWqe7DaJicfe0p96O8h0dB+qEEm7hJ53cO6BR9qPUulK2r+HKR5XB zYyi39mqmsXllAfdOIa1yk9o0geg9Jw5QRhfYNotHeiu8VixrTjzeW7sXdvgFji2dIKM UU3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d6fNTUIWdW75sP4ix2ktwJ53a4hbge6GLMCdTHFHxVQ=; b=hcNF9fyPik1eFbo6iassHOeJEVktO24v/ql9eO/1tmaZI6nNru78WYOu76AJRLlxt+ gy1jatBUA1+um0YTCzP5oCuqTxDfnDZ+1AJpCsSFI0abWIzdzEpCMlg2QkxTVGlxZIbF 28dYJD6Vglq+9CpjXcOfs5rHlqC+WoLyLkBav/Jvyf5CNqax1bOkb7GB1jnqSvXQ8iMR 4nae7qU7/Z5sGCgmKaMQMWXksFbi7ppldzm4Tc9vjEcD2txpqz22h/2JZuUZ71zkHuaW P8z9owAJZMVQgzUq2VM+YGkOsfZ13E9vFGiKzjLdjKWIUHcJusNbe154Lozh0RfVWt50 KqMA== X-Gm-Message-State: AOAM533DQWOfkJGLwHp07fi8e8PkKkU5DN4efI72O7opDWxTIZ7gzybp 89iNLUF6rFZjylTtgE0ofz1lrfBM58E= X-Google-Smtp-Source: ABdhPJzKJ0XZLbkWhZ2Jne6pJEFmCYYw3FSMwzl0RvGjNPltms3XZGp44U/FH6f6n/IIuY3PPf8fDQ== X-Received: by 2002:a17:906:3103:: with SMTP id 3mr21270767ejx.289.1642580715148; Wed, 19 Jan 2022 00:25:15 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:14 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 05/25] trace-cmd library: Add internal helper function for writing headers before file sections Date: Wed, 19 Jan 2022 10:24:47 +0200 Message-Id: <20220119082507.245600-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Introduce headers before each file section, in trace file version 7. The section header has the following format: <2 bytes>, header ID <2 bytes>, section flags: 1: the section is compressed <4 bytes>, offset within the strings section, where the section description string is located. <8 bytes>, size of the section Signed-off-by: Tzvetomir Stoyanov (VMware) --- include/trace-cmd/trace-cmd.h | 5 ++ .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/include/trace-cmd-local.h | 5 ++ lib/trace-cmd/trace-output.c | 77 ++++++++++++++++++- 4 files changed, 87 insertions(+), 1 deletion(-) diff --git a/include/trace-cmd/trace-cmd.h b/include/trace-cmd/trace-cmd.h index 7fea4e01..5d71e8ba 100644 --- a/include/trace-cmd/trace-cmd.h +++ b/include/trace-cmd/trace-cmd.h @@ -15,6 +15,11 @@ enum tracecmd_open_flags { TRACECMD_FL_LOAD_NO_PLUGINS = 1 << 0, /* Do not load plugins */ TRACECMD_FL_LOAD_NO_SYSTEM_PLUGINS = 1 << 1, /* Do not load system plugins */ }; + +enum tracecmd_section_flags { + TRACECMD_SEC_FL_COMPRESS = 1 << 0, /* the section is compressed */ +}; + struct tracecmd_input *tracecmd_open_head(const char *file, int flags); struct tracecmd_input *tracecmd_open(const char *file, int flags); struct tracecmd_input *tracecmd_open_fd(int fd, int flags); diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index e1467971..2eb077c8 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -144,6 +144,7 @@ enum { TRACECMD_OPTION_GUEST, TRACECMD_OPTION_TSC2NSEC, TRACECMD_OPTION_STRINGS, + TRACECMD_OPTION_MAX, }; enum { diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 7f280533..4a0a691c 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,11 @@ struct data_file_write { bool check_file_state(unsigned long file_version, int current_state, int new_state); bool check_out_state(struct tracecmd_output *handle, int new_state); +unsigned long long +out_write_section_header(struct tracecmd_output *handle, unsigned short header_id, + char *description, int flags, bool option); +int out_update_section_header(struct tracecmd_output *handle, unsigned long long offset); + struct cpu_data_source { int fd; int size; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index d3c73ceb..e1803dbd 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -365,6 +365,82 @@ int tracecmd_ftrace_enable(int set) return ret; } +__hidden unsigned long long +out_write_section_header(struct tracecmd_output *handle, unsigned short header_id, + char *description, int flags, bool option) +{ + tsize_t endian8; + tsize_t offset; + long long size; + short endian2; + int endian4; + int desc; + + if (header_id >= TRACECMD_OPTION_MAX) + return -1; + if (!HAS_SECTIONS(handle)) + return 0; + offset = do_lseek(handle, 0, SEEK_CUR); + if (option) { + endian8 = convert_endian_8(handle, offset); + if (!tracecmd_add_option(handle, header_id, 8, &endian8)) + return -1; + } + /* Section ID */ + endian2 = convert_endian_2(handle, header_id); + if (do_write_check(handle, &endian2, 2)) + return (off64_t)-1; + + /* Section flags */ + endian2 = convert_endian_2(handle, flags); + if (do_write_check(handle, &endian2, 2)) + return (off64_t)-1; + + /* Section description */ + if (description) + desc = add_string(handle, description); + else + desc = -1; + endian4 = convert_endian_4(handle, desc); + if (do_write_check(handle, &endian4, 4)) + return (off64_t)-1; + + offset = do_lseek(handle, 0, SEEK_CUR); + size = 0; + /* Reserve for section size */ + if (do_write_check(handle, &size, 8)) + return (off64_t)-1; + return offset; +} + +__hidden int out_update_section_header(struct tracecmd_output *handle, tsize_t offset) +{ + tsize_t current; + tsize_t endian8; + tsize_t size; + + if (!HAS_SECTIONS(handle) || offset == 0) + return 0; + + current = do_lseek(handle, 0, SEEK_CUR); + /* The real size is the difference between the saved offset and + * the current offset - 8 bytes, the reserved space for the section size. + */ + size = current - offset; + if (size < 8) + return -1; + size -= 8; + if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) + return -1; + + endian8 = convert_endian_8(handle, size); + if (do_write_check(handle, &endian8, 8)) + return -1; + if (do_lseek(handle, current, SEEK_SET) == (off64_t)-1) + return -1; + return 0; +} + static int save_string_section(struct tracecmd_output *handle) { if (!handle->strings || !handle->strings_p) @@ -390,7 +466,6 @@ error: return -1; } - static int read_header_files(struct tracecmd_output *handle) { tsize_t size, check_size, endian8; From patchwork Wed Jan 19 08:24:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0342CC433FE for ; Wed, 19 Jan 2022 08:25:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352386AbiASIZS (ORCPT ); Wed, 19 Jan 2022 03:25:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZS (ORCPT ); Wed, 19 Jan 2022 03:25:18 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C52FEC061574 for ; Wed, 19 Jan 2022 00:25:17 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id c71so7587955edf.6 for ; Wed, 19 Jan 2022 00:25:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XuWycg7VJ8blRzdxJxOEjzVQefSc9ceTfyQHR7U9emg=; b=CmY/0ScUGzuQaxMqjFRv2mIQkorrb78f0j2OKWE9C5rB0LDdD0fOG95FQSJvSluIot zFeXV0AQxD310S2BWXCHycZoT0woaCt9fbFmFHesa01ikQca7TAJTgH+iR2oYBaoFTeB dKwX6xpeNDI1lhGFpFzDU+lo2jegD1hxaXjd10kjAAhBXZzVrh5hSDOgW+u/4ICw8FlU P5gomdAbTmNqX1Iq3MfV7Cxfajyjq3m8Qon9puYQbOscNqUUu7ClJWXX8XN1XjSMtPsx VhU5DeUbKiCNq/LAdDts9jcOhU/eKtkEvaLZJn0/kkaOHlFeL7V+sbXBBDHHIBfy8Pk/ DxUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XuWycg7VJ8blRzdxJxOEjzVQefSc9ceTfyQHR7U9emg=; b=ygt3944SeUTM0ADw4l9xMAKGqe7WLPpCrzA1TXdHCjv7tit+nhQ0cgJq4+MZTAzPLL Xxh0XXCP71+4+AvSnCYLtxxCeK8rrVoF7cOmSKvzgrKVipT8zWsiH5RSn+o33SLuCeTB jCuFxFUEvwaEKXAQhb0wwL0dHKmOOjIBxRnLj+uKi6M6DRTi7jCRcJBW+ErDGA+7Rb70 pjSrKQSJywuIMjqTrXlbq+KMrZ1AAqPfZpVDL5ULXT+bvoi8O33kOk7cQ6BdSKEI+i8W 1iwhKuHYnsaRmpVAcJOKWuikYFePvY7dS+DXVC5Y1B1Qfi1YVqww9DL4LW1Cq5yJnk1M /a/g== X-Gm-Message-State: AOAM530h+3/ASlIE4NVNlR2Mqf684G/fKPf0kiGX8HGGXeSdQ9g1Ar+P GzCBO87EE2mYz0xQ7gufPCWR9E+Ih+w= X-Google-Smtp-Source: ABdhPJwWcCTNPHXOSckckTfS/kJA7jBk1hrt4+0locLDfKgFHfzJxclPhAmG8Xe3VgMXBlf+u4b02Q== X-Received: by 2002:a17:906:14d8:: with SMTP id y24mr3805101ejc.62.1642580716321; Wed, 19 Jan 2022 00:25:16 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:15 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 06/25] trace-cmd library: Write header before file sections Date: Wed, 19 Jan 2022 10:24:48 +0200 Message-Id: <20220119082507.245600-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Headers are added before these file sections, in trace file version 7: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name - strings New options are defined for each of these sections, holding the section's offset into the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 6 + lib/trace-cmd/trace-output.c | 120 ++++++++++++++---- 2 files changed, 103 insertions(+), 23 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 2eb077c8..eaf51f5f 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -144,6 +144,12 @@ enum { TRACECMD_OPTION_GUEST, TRACECMD_OPTION_TSC2NSEC, TRACECMD_OPTION_STRINGS, + TRACECMD_OPTION_HEADER_INFO, + TRACECMD_OPTION_FTRACE_EVENTS, + TRACECMD_OPTION_EVENT_FORMATS, + TRACECMD_OPTION_KALLSYMS, + TRACECMD_OPTION_PRINTK, + TRACECMD_OPTION_CMDLINES, TRACECMD_OPTION_MAX, }; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e1803dbd..d2f90f29 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -443,6 +443,9 @@ __hidden int out_update_section_header(struct tracecmd_output *handle, tsize_t o static int save_string_section(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; + tsize_t offset; + if (!handle->strings || !handle->strings_p) return 0; @@ -452,9 +455,16 @@ static int save_string_section(struct tracecmd_output *handle) return -1; } + offset = out_write_section_header(handle, TRACECMD_OPTION_STRINGS, "strings", flags, false); + if (offset == (off64_t)-1) + return -1; + if (do_write_check(handle, handle->strings, handle->strings_p)) goto error; + if (out_update_section_header(handle, offset)) + return -1; + handle->strings_offs += handle->strings_p; free(handle->strings); handle->strings = NULL; @@ -468,10 +478,12 @@ error: static int read_header_files(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; tsize_t size, check_size, endian8; struct stat st; + tsize_t offset; char *path; - int fd; + int fd = -1; int ret; if (!check_out_state(handle, TRACECMD_FILE_HEADERS)) { @@ -484,26 +496,33 @@ static int read_header_files(struct tracecmd_output *handle) if (!path) return -1; + offset = out_write_section_header(handle, TRACECMD_OPTION_HEADER_INFO, + "headers", flags, true); + if (offset == (off64_t)-1) + return -1; + ret = stat(path, &st); if (ret < 0) { /* old style did not show this info, just add zero */ put_tracing_file(path); if (do_write_check(handle, "header_page", 12)) - return -1; + goto out_close; size = 0; if (do_write_check(handle, &size, 8)) - return -1; + goto out_close; if (do_write_check(handle, "header_event", 13)) - return -1; + goto out_close; if (do_write_check(handle, &size, 8)) - return -1; + goto out_close; + if (out_update_section_header(handle, offset)) + goto out_close; return 0; } fd = open(path, O_RDONLY); if (fd < 0) { tracecmd_warning("can't read '%s'", path); - return -1; + goto out_close; } /* unfortunately, you can not stat debugfs files for size */ @@ -519,18 +538,18 @@ static int read_header_files(struct tracecmd_output *handle) if (size != check_size) { tracecmd_warning("wrong size for '%s' size=%lld read=%lld", path, size, check_size); errno = EINVAL; - return -1; + goto out_close; } put_tracing_file(path); path = get_tracing_file(handle, "events/header_event"); if (!path) - return -1; + goto out_close; fd = open(path, O_RDONLY); if (fd < 0) { tracecmd_warning("can't read '%s'", path); - return -1; + goto out_close; } size = get_size_fd(fd); @@ -544,16 +563,18 @@ static int read_header_files(struct tracecmd_output *handle) close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s'", path); - return -1; + goto out_close; } put_tracing_file(path); - + if (out_update_section_header(handle, offset)) + goto out_close; handle->file_state = TRACECMD_FILE_HEADERS; return 0; out_close: - close(fd); + if (fd >= 0) + close(fd); return -1; } @@ -782,8 +803,10 @@ create_event_list_item(struct tracecmd_output *handle, static int read_ftrace_files(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; struct list_event_system *systems = NULL; struct tracecmd_event_list list = { .glob = "ftrace/*" }; + tsize_t offset; int ret; if (!check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS)) { @@ -792,11 +815,20 @@ static int read_ftrace_files(struct tracecmd_output *handle) return -1; } + offset = out_write_section_header(handle, TRACECMD_OPTION_FTRACE_EVENTS, + "ftrace events", flags, true); + if (offset == (off64_t)-1) + return -1; + create_event_list_item(handle, &systems, &list); ret = copy_event_system(handle, systems); free_list_events(systems); + if (ret) + return ret; + if (out_update_section_header(handle, offset)) + return -1; handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; @@ -819,11 +851,13 @@ create_event_list(struct tracecmd_output *handle, static int read_event_files(struct tracecmd_output *handle, struct tracecmd_event_list *event_list) { + enum tracecmd_section_flags flags = 0; struct list_event_system *systems; struct list_event_system *slist; struct tracecmd_event_list *list; struct tracecmd_event_list all_events = { .glob = "*/*" }; int count = 0; + tsize_t offset; int endian4; int ret; @@ -832,6 +866,11 @@ static int read_event_files(struct tracecmd_output *handle, handle->file_state); return -1; } + + offset = out_write_section_header(handle, TRACECMD_OPTION_EVENT_FORMATS, + "events format", flags, true); + if (offset == (off64_t)-1) + return -1; /* * If any of the list is the special keyword "all" then * just do all files. @@ -863,9 +902,14 @@ static int read_event_files(struct tracecmd_output *handle, } ret = copy_event_system(handle, slist); } + if (ret) + goto out_free; + ret = out_update_section_header(handle, offset); - handle->file_state = TRACECMD_FILE_ALL_EVENTS; out_free: + if (!ret) + handle->file_state = TRACECMD_FILE_ALL_EVENTS; + free_list_events(systems); return ret; @@ -913,8 +957,10 @@ err: static int read_proc_kallsyms(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; unsigned int size, check_size, endian4; const char *path = "/proc/kallsyms"; + tsize_t offset; struct stat st; int ret; @@ -927,19 +973,24 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) if (handle->kallsyms) path = handle->kallsyms; + offset = out_write_section_header(handle, TRACECMD_OPTION_KALLSYMS, + "kallsyms", flags, true); + if (offset == (off64_t)-1) + return -1; + ret = stat(path, &st); if (ret < 0) { /* not found */ size = 0; endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; - return 0; + ret = do_write_check(handle, &endian4, 4); + goto out; } size = get_size(path); endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; + ret = do_write_check(handle, &endian4, 4); + if (ret) + goto out; set_proc_kptr_restrict(0); check_size = copy_file(handle, path); @@ -947,18 +998,23 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) errno = EINVAL; tracecmd_warning("error in size of file '%s'", path); set_proc_kptr_restrict(1); - return -1; + ret = -1; + goto out; } set_proc_kptr_restrict(1); - handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = out_update_section_header(handle, offset); +out: + if (!ret) + handle->file_state = TRACECMD_FILE_KALLSYMS; + return ret; } static int read_ftrace_printk(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; unsigned int size, check_size, endian4; + tsize_t offset; struct stat st; char *path; int ret; @@ -973,6 +1029,10 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + offset = out_write_section_header(handle, TRACECMD_OPTION_PRINTK, "printk", flags, true); + if (offset == (off64_t)-1) + return -1; + ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -994,8 +1054,10 @@ static int read_ftrace_printk(struct tracecmd_output *handle) } out: - handle->file_state = TRACECMD_FILE_PRINTK; put_tracing_file(path); + if (out_update_section_header(handle, offset)) + return -1; + handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); @@ -1599,6 +1661,8 @@ static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char int tracecmd_write_cmdlines(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; + tsize_t offset; int ret; if (!check_out_state(handle, TRACECMD_FILE_CMD_LINES)) { @@ -1606,9 +1670,19 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) handle->file_state); return -1; } + + offset = out_write_section_header(handle, TRACECMD_OPTION_CMDLINES, + "command lines", flags, true); + if (offset == (off64_t)-1) + return -1; + ret = save_tracing_file_data(handle, "saved_cmdlines"); if (ret < 0) return ret; + + if (out_update_section_header(handle, offset)) + return -1; + handle->file_state = TRACECMD_FILE_CMD_LINES; return 0; } From patchwork Wed Jan 19 08:24:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BB14C433EF for ; Wed, 19 Jan 2022 08:25:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352380AbiASIZT (ORCPT ); Wed, 19 Jan 2022 03:25:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZT (ORCPT ); Wed, 19 Jan 2022 03:25:19 -0500 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8050C061574 for ; Wed, 19 Jan 2022 00:25:18 -0800 (PST) Received: by mail-ed1-x52d.google.com with SMTP id j23so2428123edp.5 for ; Wed, 19 Jan 2022 00:25:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lFnxoFsz5kg5KsMzJxsxj/4uxkULtcneOSCJSmq/cKw=; b=LGJT1lw5BsXcKrW85XRRbSm/LAfl43LZEhrGXyvhYoe5fRC7Brkmo1NEV3IPQSU8Tc JHpfbmOQPb2qiaxo8vXyt+Jq6eB6XF9xYX7iaDWSapdvCiDljAcmimF6sBJs0NNZsX9N 8dP0BkdCPOqgeYcBkrNyaL0iCaijpbVobuWBnA+nHzlEDJX34OeeOQgyZusLh6zgynvL 7LnGqGMppK92kXCHqtRvAV+TkaFy2WCahjMH9iuOXyy4VsWlTgDnv/K1tAu0O2FyVIuf s7m7qlJ7wEGzhX+ZqzaM+iArBlRYjkhABw1HPX/jGR0Ykqfgu5NQ+Mqjzhnfzvg6iYEH xu3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lFnxoFsz5kg5KsMzJxsxj/4uxkULtcneOSCJSmq/cKw=; b=Kp0cwTmVIR4i/HHwFXDrhi2lIGypenBFMO5TLDTDXNYuRIptOyd11UVLW0lW0x9RlH ej6cb/3LfuVfsCiV01E84aihWx3rpz1vlCZ3ubf5hEd61WITek2D62RHGC3L/sh9SePj cwugn2uGDeuAiIyBfAN5ncfIGrsfVlqsbpapnkU8DxOYvc1bB0abuUjARSdVnDq5+5Lf pe2F2mD4SCEMI3Sq6SaBUjHB0l602cf70EMWudnjzAADvxkoG4cMmsKLHPM/IfHgPcnL OxxRGhAWoIWnxg66csViwIEFTIZ3Rc7Rqm5UwofT/BOGVtY2AkM0j2O3Xov/LUu8O/Gt bHSw== X-Gm-Message-State: AOAM53393VN2/GDVHzJSsUTmAzqErlsPUZCl3y6RZS4aXXwNCSso2uBm HXFUXIXqqksjbPHz40v2dnKOhBwj4Vg= X-Google-Smtp-Source: ABdhPJyTM8yUFxSv9Py6QDY7LYnPP6euoHueeDDIhM9p0K6j/z7MrzogMUEMQUrXWs+gqjwInbZoPg== X-Received: by 2002:a17:906:49c4:: with SMTP id w4mr23158142ejv.745.1642580717271; Wed, 19 Jan 2022 00:25:17 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:16 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 07/25] trace-cmd library: Add multiple options sections in trace file version 7 Date: Wed, 19 Jan 2022 10:24:49 +0200 Message-Id: <20220119082507.245600-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Introduced chain of options sections in trace file version 7. Extended the "DONE" option to hold the offset into the file to the next options section. Format of trace file version 7 is extended with a new mandatory field after the compression algorithm header: <8 bytes>, unsigned long long integer - offset into the trace file where the first options section is located. This allows to place this section anywhere in the file. As all other sections have corresponding options, describing their offsets into the trace file, this change makes the structure of trace file version 7 flexible. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 105 ++++++++++++++++++++++++++++++++--- 1 file changed, 98 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index d2f90f29..8a8d4e55 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -68,7 +68,7 @@ struct tracecmd_output { /* current virtual offset of meta-data string */ unsigned long strings_offs; - size_t options_start; + unsigned long long options_start; bool big_endian; struct list_head options; @@ -94,6 +94,7 @@ struct list_event_system { #define HAS_SECTIONS(H) ((H)->file_version >= FILE_VERSION_SECTIONS) +static int write_options(struct tracecmd_output *handle); static int save_string_section(struct tracecmd_output *handle); static stsize_t @@ -223,6 +224,9 @@ void tracecmd_output_close(struct tracecmd_output *handle) return; if (HAS_SECTIONS(handle)) { + /* write any unsaved options at the end of trace files with sections */ + write_options(handle); + /* write strings section */ save_string_section(handle); } @@ -1300,6 +1304,7 @@ int tracecmd_output_set_version(struct tracecmd_output *handle, int file_version */ static int output_write_init(struct tracecmd_output *handle) { + unsigned long long offset; char buf[BUFSIZ]; int endian4; @@ -1333,6 +1338,14 @@ static int output_write_init(struct tracecmd_output *handle) endian4 = convert_endian_4(handle, handle->page_size); if (do_write_check(handle, &endian4, 4)) return -1; + if (HAS_SECTIONS(handle)) { + /* Write 0 as options offset and save its location */ + offset = 0; + handle->options_start = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &offset, 8)) + return -1; + } + handle->file_state = TRACECMD_FILE_INIT; return 0; } @@ -1401,7 +1414,7 @@ tracecmd_add_option_v(struct tracecmd_output *handle, * We can only add options before tracing data were written. * This may change in the future. */ - if (handle->file_state > TRACECMD_FILE_OPTIONS) + if (!HAS_SECTIONS(handle) && handle->file_state > TRACECMD_FILE_OPTIONS) return NULL; for (i = 0; i < count; i++) @@ -1414,8 +1427,7 @@ tracecmd_add_option_v(struct tracecmd_output *handle, return NULL; } } - - option = malloc(sizeof(*option)); + option = calloc(1, sizeof(*option)); if (!option) { tracecmd_warning("Could not allocate space for option"); free(data); @@ -1478,7 +1490,7 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) return 0; } -int tracecmd_write_options(struct tracecmd_output *handle) +static int write_options_v6(struct tracecmd_output *handle) { struct tracecmd_option *options; unsigned short option; @@ -1496,7 +1508,7 @@ int tracecmd_write_options(struct tracecmd_output *handle) if (do_write_check(handle, "options ", 10)) return -1; - + handle->options_start = do_lseek(handle, 0, SEEK_CUR); list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) @@ -1520,6 +1532,72 @@ int tracecmd_write_options(struct tracecmd_output *handle) return -1; handle->file_state = TRACECMD_FILE_OPTIONS; + return 0; +} + +static int write_options(struct tracecmd_output *handle) +{ + struct tracecmd_option *options; + unsigned long long endian8; + unsigned short endian2; + unsigned int endian4; + bool new = false; + tsize_t offset; + + /* Check if there are unsaved options */ + list_for_each_entry(options, &handle->options, list) { + if (!options->offset) { + new = true; + break; + } + } + if (!new) + return 0; + offset = do_lseek(handle, 0, SEEK_CUR); + + /* Append to the previous options section, if any */ + if (handle->options_start) { + if (do_lseek(handle, handle->options_start, SEEK_SET) == (off64_t)-1) + return -1; + endian8 = convert_endian_8(handle, offset); + if (do_write_check(handle, &endian8, 8)) + return -1; + if (do_lseek(handle, offset, SEEK_SET) == (off_t)-1) + return -1; + } + + offset = out_write_section_header(handle, TRACECMD_OPTION_DONE, "options", 0, false); + if (offset == (off_t)-1) + return -1; + + list_for_each_entry(options, &handle->options, list) { + /* Option is already saved, skip it */ + if (options->offset) + continue; + endian2 = convert_endian_2(handle, options->id); + if (do_write_check(handle, &endian2, 2)) + return -1; + endian4 = convert_endian_4(handle, options->size); + if (do_write_check(handle, &endian4, 4)) + return -1; + /* Save the data location */ + options->offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, options->data, options->size)) + return -1; + } + + endian2 = convert_endian_2(handle, TRACECMD_OPTION_DONE); + if (do_write_check(handle, &endian2, 2)) + return -1; + endian4 = convert_endian_4(handle, 8); + if (do_write_check(handle, &endian4, 4)) + return -1; + endian8 = 0; + handle->options_start = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &endian8, 8)) + return -1; + if (out_update_section_header(handle, offset)) + return -1; return 0; } @@ -1532,8 +1610,14 @@ int tracecmd_write_meta_strings(struct tracecmd_output *handle) return save_string_section(handle); } +int tracecmd_write_options(struct tracecmd_output *handle) +{ + if (!HAS_SECTIONS(handle)) + return write_options_v6(handle); + return write_options(handle); +} -int tracecmd_append_options(struct tracecmd_output *handle) +static int append_options_v6(struct tracecmd_output *handle) { struct tracecmd_option *options; unsigned short option; @@ -1584,6 +1668,13 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } +int tracecmd_append_options(struct tracecmd_output *handle) +{ + if (!HAS_SECTIONS(handle)) + return append_options_v6(handle); + return write_options(handle); +} + static struct tracecmd_option * add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { From patchwork Wed Jan 19 08:24:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B33C2C433F5 for ; Wed, 19 Jan 2022 08:25:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352387AbiASIZU (ORCPT ); Wed, 19 Jan 2022 03:25:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZU (ORCPT ); Wed, 19 Jan 2022 03:25:20 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95585C06161C for ; Wed, 19 Jan 2022 00:25:19 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id cx27so7678916edb.1 for ; Wed, 19 Jan 2022 00:25:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZPvlC3t3hhq/8hKs8oI0cCeyGegZXfQvrYNXq/lPZkg=; b=pzS690Ch2BhXbsOYegDuJpPr+uUQz21YQf7ibKtBI2++OCo5hlg3Lf6zc/m5CPgARW odV76TAtn3zLAdcWDiC1Is4ZFv6d1qCcx0beL8wZit0xEi/GVo+DRYppcqg+EGhlQDcJ cIjaSTmPj3AuGu4sQaGFg/i3lgG4QExPz7UkHAb9Z2pp17J59QB2tLFX64dcafpErUbd ahaE81YRA4KGvocECVpPqmlJQvG8yHMAJT9e6YIr2DJpQTu1jc8GZe4cSZ3AHjkRTvZH zA+DjcB3Ko/qB7RB4uPIdovv7jW8BSdPVwVQbB7RBkjMnGD2i/HLLrvm7fJTCPuTHOMq KAYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZPvlC3t3hhq/8hKs8oI0cCeyGegZXfQvrYNXq/lPZkg=; b=G3VqOoQg3XDVvPPHVVvDS8jkr7bIayWlXpDMdmbHOM12nIn8LUtsxzSDjVkxcUOwb6 88IPQx56H1T/NbtOLMpBrRy3xu7DsBKBIhhLNBZcPOcWDlbDcfqCMFECX70dSg+pmfaL X+X0QPCAFZXeKzdoVt9ZTzXhCwPj7i+s8eDatLw3ZyMI1NFXYBrv772KONgRQPBs7qOR duWo3FQm55aMHsdsUaBEVBdViRFikngR/FQyHLPuLcFG48/NhqhT7OPwhXGwS4FqPOQT ZMAFgWNt3DqjznPFuYWnOSTbTt/JswbxZfUHcxdXW/NLoxcfdsbipTa47kLlwHLkIGMC Cnfg== X-Gm-Message-State: AOAM5336ebNFx/PmxgyxBs6C60PFMh65vcPF+y7++kjLy1Lt4Sq4Zxrf sO7XZ67Yo6A/b48PisP3Nm4pyPuyK8M= X-Google-Smtp-Source: ABdhPJxgPNf1vD7L1LXZoEe8+JYztNTjY09rQYU/yQDdBMcAsi/py7JXf7h+F47gJ4uNwSZgdgKoAg== X-Received: by 2002:a17:907:2d26:: with SMTP id gs38mr24160590ejc.280.1642580718224; Wed, 19 Jan 2022 00:25:18 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:17 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 08/25] trace-cmd library: Do not write CPU count section in trace files version 7 Date: Wed, 19 Jan 2022 10:24:50 +0200 Message-Id: <20220119082507.245600-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Writing CPU count as a separate section in the trace file is redundant, as there is already an option for that. Use that option in trace files version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8a8d4e55..c075ab6d 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1482,10 +1482,16 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) handle->file_state); return -1; } - cpus = convert_endian_4(handle, cpus); - ret = do_write_check(handle, &cpus, 4); - if (ret < 0) - return ret; + + if (!HAS_SECTIONS(handle)) { + cpus = convert_endian_4(handle, cpus); + ret = do_write_check(handle, &cpus, 4); + if (ret < 0) + return ret; + } else { + tracecmd_add_option(handle, TRACECMD_OPTION_CPUCOUNT, sizeof(int), &cpus); + } + handle->file_state = TRACECMD_FILE_CPU_COUNT; return 0; } From patchwork Wed Jan 19 08:24:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBD40C433EF for ; Wed, 19 Jan 2022 08:25:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352390AbiASIZV (ORCPT ); Wed, 19 Jan 2022 03:25:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZV (ORCPT ); Wed, 19 Jan 2022 03:25:21 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A823BC061574 for ; Wed, 19 Jan 2022 00:25:20 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id j23so2428598edp.5 for ; Wed, 19 Jan 2022 00:25:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TS4JGZ/yPon2sVLY8ZFd6tAPvQZwFyppW805WNo7k0o=; b=aMZXBaFTeOio+8GDMeETX0fyGToP/NncFErMlLt63cWC/NsAaI8tUlC6rpSrlfZ451 ByH60M7DDp1AK+dchso9gUKZ0j54S9E/30pmzSMTU/OA9JcU+76xX/Mt59IhcL45aE8x 2txnHtJU0FkznsITYbjC8nqD49r9+Gaahuqr3NfEkJe3KqSrQVad819z0SnKfqfDn3w7 indqynmXAj48sv6L35E8XWhJvyU/e5nA/Txxvfymnf30nOx+vLPm0NSJZZq3BT3BbZ+u 09Psg7JPsYumgdGwSadVxMBRKZCx4QYwBEXUV6nfV2K1LgbFDzo3eN5rjniNSemYY2l+ lvSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TS4JGZ/yPon2sVLY8ZFd6tAPvQZwFyppW805WNo7k0o=; b=VXKeNpC0uqUivpNE8C0xN7k77K676eA42n3nwkIIkvS+L8TP61R1E5ZZwvwS7BA8hi FhNSyRVv4BjsU5nS5Dzb8i6YFBzS8QmqBMURV970OCJEitRUOlExqFRj3swuiD/E/Apb kKp+vUHVfYG4dVuheJTlWvX3sN9Z/Sg41sdEgYrBoRm9YmkLJaJKB/RhTypMOGVWFIQ8 S7hvSK4Mfb2TWS+nPofybEk+T2xhFe0QWeg/xQGjzVJM9yBivxv33oPWyHbLsCTW3HZ4 CBtlRg6XdN0OGWVSM2Ja4cHZajIvLgtFWCCiV+i7u/jPOlMHOdfaOuOZHa0MLzjoW3Fn ldtw== X-Gm-Message-State: AOAM531lRhgm1hjTyrI4fcAvIzj2pscuGI9tetO8fL1KwomUJg7RiqoW cFopGB3t19cQ7qm0ivF3Nwjgzg0SLSk= X-Google-Smtp-Source: ABdhPJxazv1ef+SnrM/jmJcVuCUT4YYrfjhGBVuA527a+EKEq65nq6L0RmsK5pZsdj+diK+bdHyfew== X-Received: by 2002:a17:907:2da6:: with SMTP id gt38mr22541028ejc.601.1642580719241; Wed, 19 Jan 2022 00:25:19 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:18 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 09/25] trace-cmd library: Move CPU flyrecord trace metadata into the buffer option, for trace file version 7 Date: Wed, 19 Jan 2022 10:24:51 +0200 Message-Id: <20220119082507.245600-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extended the BUFFER trace option in trace file version 7 with CPU flyrecord trace metadata. In the version 6 of the trace file, this metadata is located at several places in the file. Part of the metadata is only for the top trace instance, thus limiting per-instance configuration. Moving all CPU trace related metadata in the BUFFER option simplifies the parsing and makes per-instance configuration more flexible. In the new file structure, the top instance is treated as any other instances. The format of the extended BUFFER option is: - offset of the buffer in the trace file - name of the buffer - trace clock, used in this buffer for events timestamps - count of CPUs with trace data - array, describing each CPU with trace data: - CPU id - offset of CPU trace data in the trace file - size of the recorded CPU trace data Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 202 +++++++++++++++++++++++++---------- 1 file changed, 148 insertions(+), 54 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c075ab6d..a7329d84 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1682,7 +1682,7 @@ int tracecmd_append_options(struct tracecmd_output *handle) } static struct tracecmd_option * -add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) +add_buffer_option_v6(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; @@ -1732,8 +1732,11 @@ int tracecmd_write_buffer_info(struct tracecmd_output *handle) struct tracecmd_option *option; struct tracecmd_buffer *buf; + if (HAS_SECTIONS(handle)) + return 0; + list_for_each_entry(buf, &handle->buffers, list) { - option = add_buffer_option(handle, buf->name, buf->cpus); + option = add_buffer_option_v6(handle, buf->name, buf->cpus); if (!option) return -1; buf->option = option; @@ -1784,6 +1787,109 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return 0; } +static char *get_clock(struct tracecmd_output *handle) +{ + struct tracefs_instance *inst; + + if (handle->trace_clock) + return handle->trace_clock; + + /* + * If no clock is set on this handle, get the trace clock of + * the top instance in the handle's tracing dir + */ + if (!handle->tracing_dir) { + handle->trace_clock = tracefs_get_clock(NULL); + return handle->trace_clock; + } + + inst = tracefs_instance_alloc(handle->tracing_dir, NULL); + if (!inst) + return NULL; + handle->trace_clock = tracefs_get_clock(inst); + tracefs_instance_free(inst); + return handle->trace_clock; +} + +__hidden struct tracecmd_option * +out_add_buffer_option(struct tracecmd_output *handle, const char *name, + unsigned short id, unsigned long long data_offset, + int cpus, struct data_file_write *cpu_data) +{ + struct tracecmd_option *option; + int i, j = 0, k = 0; + int *cpu_ids = NULL; + struct iovec *vect; + char *clock; + + if (!HAS_SECTIONS(handle)) + return NULL; + + clock = get_clock(handle); + + /* + * Buffer flyrecord option: + * - trace data offset in the file + * - buffer name + * - buffer clock + * - CPU count + * - for each CPU: + * - CPU id + * - CPU trace data offset in the file + * - CPU trace data size + */ + + /* + * Buffer latency option: + * - trace data offset in the file + * - buffer name + * - buffer clock + */ + + /* + * 4 : offset, name, clock, count + * 3 : cpu offset, name, clock + */ + vect = calloc(4 + (cpus * 3), sizeof(struct iovec)); + if (!vect) + return NULL; + if (cpus) { + cpu_ids = calloc(cpus, sizeof(int)); + if (!cpu_ids) { + free(vect); + return NULL; + } + } + vect[j].iov_base = (void *) &data_offset; + vect[j++].iov_len = 8; + vect[j].iov_base = (void *) name; + vect[j++].iov_len = strlen(name) + 1; + vect[j].iov_base = (void *) clock; + vect[j++].iov_len = strlen(clock) + 1; + if (id == TRACECMD_OPTION_BUFFER) { + vect[j].iov_base = (void *) &k; + vect[j++].iov_len = 4; + for (i = 0; i < cpus; i++) { + if (!cpu_data[i].file_size) + continue; + cpu_ids[i] = i; + vect[j].iov_base = &cpu_ids[i]; + vect[j++].iov_len = 4; + vect[j].iov_base = &cpu_data[i].data_offset; + vect[j++].iov_len = 8; + vect[j].iov_base = &cpu_data[i].write_size; + vect[j++].iov_len = 8; + k++; + } + } + + option = tracecmd_add_option_v(handle, id, vect, j); + free(vect); + free(cpu_ids); + + return option; +} + struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) { struct tracecmd_output *handle; @@ -1854,8 +1960,8 @@ out: return ret; } -static int update_buffer_cpu_offset(struct tracecmd_output *handle, - const char *name, tsize_t offset) +static int update_buffer_cpu_offset_v6(struct tracecmd_output *handle, + const char *name, tsize_t offset) { tsize_t b_offset; tsize_t current; @@ -1888,26 +1994,6 @@ static int update_buffer_cpu_offset(struct tracecmd_output *handle, return 0; } - -static char *get_clock(struct tracecmd_output *handle) -{ - struct tracefs_instance *inst; - - if (handle->trace_clock) - return handle->trace_clock; - - /* - * If no clock is set on this handle, get the trace clock of - * the top instance in the handle's tracing dir - */ - inst = tracefs_instance_alloc(handle->tracing_dir, NULL); - if (!inst) - return NULL; - handle->trace_clock = tracefs_get_clock(inst); - tracefs_instance_free(inst); - return handle->trace_clock; -} - __hidden int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name) { @@ -1931,7 +2017,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, } data_offs = do_lseek(handle, 0, SEEK_CUR); - if (do_write_check(handle, "flyrecord", 10)) + if (!HAS_SECTIONS(handle) && do_write_check(handle, "flyrecord", 10)) goto out_free; data_files = calloc(cpus, sizeof(*data_files)); @@ -1944,19 +2030,23 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, * Place 0 for the data offset and size, and save the offsets to * updated them with the correct data later. */ - endian8 = 0; - data_files[i].file_data_offset = do_lseek(handle, 0, SEEK_CUR); - if (do_write_check(handle, &endian8, 8)) - goto out_free; - data_files[i].file_write_size = do_lseek(handle, 0, SEEK_CUR); - if (do_write_check(handle, &endian8, 8)) - goto out_free; + if (!HAS_SECTIONS(handle)) { + endian8 = 0; + data_files[i].file_data_offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + data_files[i].file_write_size = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + } } - update_buffer_cpu_offset(handle, buff_name, data_offs); - clock = get_clock(handle); - if (clock && save_clock(handle, clock)) - goto out_free; + if (!HAS_SECTIONS(handle)) { + update_buffer_cpu_offset_v6(handle, buff_name, data_offs); + clock = get_clock(handle); + if (clock && save_clock(handle, clock)) + goto out_free; + } for (i = 0; i < cpus; i++) { data_files[i].data_offset = do_lseek(handle, 0, SEEK_CUR); @@ -1987,29 +2077,33 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, data_files[i].write_size = 0; } - /* Write the real CPU data offset in the file */ - if (do_lseek(handle, data_files[i].file_data_offset, SEEK_SET) == (off64_t)-1) - goto out_free; - endian8 = convert_endian_8(handle, data_files[i].data_offset); - if (do_write_check(handle, &endian8, 8)) - goto out_free; - - /* Write the real CPU data size in the file */ - if (do_lseek(handle, data_files[i].file_write_size, SEEK_SET) == (off64_t)-1) - goto out_free; - endian8 = convert_endian_8(handle, data_files[i].write_size); - if (do_write_check(handle, &endian8, 8)) - goto out_free; - - offset = data_files[i].data_offset + data_files[i].write_size; - if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) - goto out_free; - + if (!HAS_SECTIONS(handle)) { + /* Write the real CPU data offset in the file */ + if (do_lseek(handle, data_files[i].file_data_offset, SEEK_SET) == (off64_t)-1) + goto out_free; + endian8 = convert_endian_8(handle, data_files[i].data_offset); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + /* Write the real CPU data size in the file */ + if (do_lseek(handle, data_files[i].file_write_size, SEEK_SET) == (off64_t)-1) + goto out_free; + endian8 = convert_endian_8(handle, data_files[i].write_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + data_files[i].write_size; + if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) + goto out_free; + } if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", (unsigned long long)data_files[i].write_size); } + if (HAS_SECTIONS(handle) && + !out_add_buffer_option(handle, buff_name, + TRACECMD_OPTION_BUFFER, data_offs, cpus, data_files)) + goto out_free; + free(data_files); if (do_lseek(handle, 0, SEEK_END) == (off64_t)-1) return -1; From patchwork Wed Jan 19 08:24:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06FACC433F5 for ; Wed, 19 Jan 2022 08:25:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352396AbiASIZW (ORCPT ); Wed, 19 Jan 2022 03:25:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZW (ORCPT ); Wed, 19 Jan 2022 03:25:22 -0500 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A8C8C061574 for ; Wed, 19 Jan 2022 00:25:21 -0800 (PST) Received: by mail-ed1-x52e.google.com with SMTP id b13so7809129edn.0 for ; Wed, 19 Jan 2022 00:25:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IXkaw51kn7nUyPsJb6+9iK15PVRNW362NBd7OxujqKM=; b=bOE4it0IV37OaZTQbwRAOpGEOJis4mIOiN5q5RBqFF9Cb1TUWJcLKeH8SHxdQGxeAW bVoSSjWcJyo0IS8qAi1zXiDjmxr0y4j3N9rj95o42aoc5R0z3woFPc1D24kIACbYyCg+ BvifqYHQPZ/4JsG8JPtLu9BcgAOa+P8ixv2TyxEcChBQx1cL9sBBEXK3JUvcHj4o9oxw jDKh4pF411jtnCgmpQozKB4lUCZ8zIsRPHo5yOK8oOtOdUEG0csmWVig6Q/5DbnBzATE C6d1F08dGJkBT9FK8/Prgb+NXrH1qBe/hwnbaPrCisZS49p+V64FCdQ9VYqkSTWPlMEj xYvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IXkaw51kn7nUyPsJb6+9iK15PVRNW362NBd7OxujqKM=; b=gvxzDh4clvRWvQKpxbPHchJHG8rYfzpvO49URRecSrcllunm+7cns3jabOKC9ZhPFf jwsWTnFGptmoZJ4YQ1g63KpnwTz+dTOajboPUDpKw/WqWif0ch/Zk9mfRfATU2dN7LhR sB8lvjBfEJVeaz0NRe/Dt63G72nJwENAh6pHD3zci7U9InfLgpX8z6wgKZxVU547ircf dBOYmVkt2Jl+DfYDSG6haYlYu5m90dPrxNr5vYAlvbPUTqVh4PJES0O6y4CVkxOKzUGG I/229ULhgx/ulGrBbFw9qsemLxd42Vojg1/7iT/Sy9C63cuf+2wKfjmlUkUdba9P6sl7 Se/g== X-Gm-Message-State: AOAM5329rLMFo+RB/KujNrpdcMOSRTzrYPmyQQwfJUOjY2BUSCU8S68K M4lL0CHw3YqYjc9U7ZTRXmLXCWfCFGY= X-Google-Smtp-Source: ABdhPJznAzIxwbqTwQxS1Ouyg4k/CWR+jeAVVLGldn6O2xhbza+oQ29R3U9g+s90j2qnWoCLWanCnw== X-Received: by 2002:a17:907:7ba2:: with SMTP id ne34mr12505789ejc.440.1642580720177; Wed, 19 Jan 2022 00:25:20 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:19 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 10/25] trace-cmd library: Add section header before flyrecord trace data Date: Wed, 19 Jan 2022 10:24:52 +0200 Message-Id: <20220119082507.245600-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, write a section header at the beginning of each section with flyrecord trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a7329d84..30bf0dd1 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1998,10 +1998,12 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name) { struct data_file_write *data_files = NULL; + enum tracecmd_section_flags flags = 0; tsize_t data_offs, offset; unsigned long long endian8; unsigned long long read_size; char *clock; + char *str; int ret; int i; @@ -2020,6 +2022,13 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (!HAS_SECTIONS(handle) && do_write_check(handle, "flyrecord", 10)) goto out_free; + if (asprintf(&str, "buffer flyrecord %s", buff_name) < 1) + goto out_free; + offset = out_write_section_header(handle, TRACECMD_OPTION_BUFFER, str, flags, false); + free(str); + if (offset == (off_t)-1) + goto out_free; + data_files = calloc(cpus, sizeof(*data_files)); if (!data_files) goto out_free; @@ -2108,6 +2117,9 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (do_lseek(handle, 0, SEEK_END) == (off64_t)-1) return -1; + if (out_update_section_header(handle, offset)) + goto out_free; + handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; return 0; From patchwork Wed Jan 19 08:24:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BAA6C433EF for ; Wed, 19 Jan 2022 08:25:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352399AbiASIZ0 (ORCPT ); Wed, 19 Jan 2022 03:25:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352397AbiASIZX (ORCPT ); Wed, 19 Jan 2022 03:25:23 -0500 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7276C061574 for ; Wed, 19 Jan 2022 00:25:22 -0800 (PST) Received: by mail-ed1-x529.google.com with SMTP id m11so7402639edi.13 for ; Wed, 19 Jan 2022 00:25:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0pLou1Be3byddCLV3Mi5US8QWZoFw9sYxhCJMlrcNzE=; b=aaju71/qqc1BPa7QmWMawS3MUwRNmc4rlt6NtNsmLuT4imrUOzPtwIyTKd8MNgpeNz HaXgV+beHoXNMWLc275/C8HhwxuhA0rPaIj+53KKptyq8kVF5K5GxpplpItLB1vUSOJO +OsXk6n1S25+qjQSQDNUcXCtujzO3r+nFRIDqOBryQG0XYQ79yZkSMEeH1wAks0JfF3J jqBJ/D8TYrCFYQccQgMz3hqmqvilQKAzhKEOKOFMXd53M+I6IvD6LyMYbgacylGO9jGM JAiXvBtVGpgV8dFmrLrMXgTBcOnkSnd3QT0p8dLhEwWcjXjp+QKCE2/zExZTZMKnKFKW J2fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0pLou1Be3byddCLV3Mi5US8QWZoFw9sYxhCJMlrcNzE=; b=b8oNWewfWXQQpfiT90wgQ91zZA0rLlRB3AprE19Y6ae8ZHaiZ4/RZC63aD3h98ySNV kTcQwSV3mrBoc5d+QZAnOnJfovND7IoY32q3w1+JT354EVDj7yxRcn8rriHT39ruhbRt biNGYQtk8Ls9UPaznsvTT+2uoqDTMMGN44RIqUG5dJ7SUrRTk0cWDdzafBAyKQaUUD7w EFCC62gFGcGW9vREE0oCQGHDAOlPYO1Ma873KNXk2PCgubyfdO59z1ZBYOpdf8eHSDr5 +nOE6T7F5AP6BwbMNlraCO6NyJCH89oTpLkwkL9hGLmNFPAT41ra57jjIXKeW15yA9F0 l3VQ== X-Gm-Message-State: AOAM533ogYwonWb9CQSzXAmPS9A+K3frZM/o6PrW6mN1wIP0BhMmUV/H SxujV3v/WYFUf1k66YG5uFiApL/yUso= X-Google-Smtp-Source: ABdhPJzxV1cgF5xKdwRJm41ZEgI03eepi351hX4UmBp5f6ELdDG+1sseTCeZofDs9u/gZ1hDFAxjBw== X-Received: by 2002:a17:907:1c8a:: with SMTP id nb10mr23781839ejc.5.1642580721250; Wed, 19 Jan 2022 00:25:21 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:20 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 11/25] trace-cmd library: Fit CPU latency trace data in the new trace file version 7 format Date: Wed, 19 Jan 2022 10:24:53 +0200 Message-Id: <20220119082507.245600-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 format is based on sections. To fit the latency trace data in this structure, a new section and option for it is defined: BUFFER_TEXT It is similar to the BUFFER section which holds the flyrecord binary data, but has a latency specific design for text data. The BUFFER_TEXT section has: - section header, as all other sections - compression of the trace data, optional - corresponding trace option, pointing to the section Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-output.c | 20 +++++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index eaf51f5f..953297f3 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -150,6 +150,7 @@ enum { TRACECMD_OPTION_KALLSYMS, TRACECMD_OPTION_PRINTK, TRACECMD_OPTION_CMDLINES, + TRACECMD_OPTION_BUFFER_TEXT, TRACECMD_OPTION_MAX, }; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 30bf0dd1..7e1849c8 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1892,7 +1892,9 @@ out_add_buffer_option(struct tracecmd_output *handle, const char *name, struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) { + enum tracecmd_section_flags flags = 0; struct tracecmd_output *handle; + tsize_t offset; char *path; handle = tracecmd_output_create(output_file); @@ -1909,7 +1911,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_cpus(handle, cpus) < 0) goto out_free; - + if (tracecmd_write_buffer_info(handle) < 0) + goto out_free; if (tracecmd_write_options(handle) < 0) goto out_free; @@ -1919,19 +1922,32 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in goto out_free; } - if (do_write_check(handle, "latency ", 10)) + if (!HAS_SECTIONS(handle) && do_write_check(handle, "latency ", 10)) goto out_free; path = get_tracing_file(handle, "trace"); if (!path) goto out_free; + offset = do_lseek(handle, 0, SEEK_CUR); + if (HAS_SECTIONS(handle) && + !out_add_buffer_option(handle, "", TRACECMD_OPTION_BUFFER_TEXT, offset, 0, NULL)) + goto out_free; + + offset = out_write_section_header(handle, TRACECMD_OPTION_BUFFER_TEXT, + "buffer latency", flags, false); + copy_file(handle, path); + if (out_update_section_header(handle, offset)) + goto out_free; put_tracing_file(path); handle->file_state = TRACECMD_FILE_CPU_LATENCY; + if (HAS_SECTIONS(handle)) + tracecmd_write_options(handle); + return handle; out_free: From patchwork Wed Jan 19 08:24:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88301C433FE for ; Wed, 19 Jan 2022 08:25:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352393AbiASIZa (ORCPT ); Wed, 19 Jan 2022 03:25:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZY (ORCPT ); Wed, 19 Jan 2022 03:25:24 -0500 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9ABFEC06161C for ; Wed, 19 Jan 2022 00:25:23 -0800 (PST) Received: by mail-ed1-x533.google.com with SMTP id c71so7589121edf.6 for ; Wed, 19 Jan 2022 00:25:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ydecgpCjx+5Q/YsLWcwO/BA1/5xVLF2taCoipK09S+U=; b=IULEt4+nk9SWk7BfVdv4e8jOBbmvBtUZD2hnRA6JpgqAXSGCrobM5WA2yUNIFN8DeR RMvu4g67CfP0IDP4ljhgN8tH9cratT7CibUBXkqY0obqfhoa6KNHML9jmSwU7B2IBLQU zbZx6g3AGvCz/H3M5xPS4pQBObGSGLFVx2vFqouE/VOOFgKWDw/LJj0qpUKl3/gijRq4 Om2/8GejZUHCHgW6ynpTi9HiFvH4/nv/TvgqhuqJyasy0T+l6OEX9vc6+0m0ST82KX6H taN0QHHqVVIRuCpz7SCcwwg32hCsO1Z1G2Xq6n34/fTn8v2y+0KCYVj8avye2tAjQIes 9lTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ydecgpCjx+5Q/YsLWcwO/BA1/5xVLF2taCoipK09S+U=; b=miPnZW5zFHZhNXKmuUZNuHtsAanV0NiQZXyGVFtS58v3hIKcni0n9QRxpyikVMm1H+ nTQNbNoKRGsEEixhZH3ToDth0fjVLpL1cGWQispfbtic0l+B0ieW8GKxBBVNQ0uVEmZW XiUBHtvtbe3afN9KrfSZ7CebCnc3eRBu/S8sRPz3Ol3GlZL20mvRlMaA/KBov+gc/3/N 2Ri1SglenDR4cLNnKI6lxLDJt9uqLqQk/vyNEYZctTzVNnueculZy5ik4A0HBCUyWFEL BMjhw3hxJCJBvqrgSGSrkcQGfHZvXg8+XfWU0sOe2lGuTG1blfQMayCPOOaPhAIaOFx3 XA9Q== X-Gm-Message-State: AOAM532+Px/pprh6kHkcSXMQ9sS6ant6HyOqBzyYMGHtsiay58m1z9qy B8TqKIv3SvHt/G/QCfV4d7yrofhma0M= X-Google-Smtp-Source: ABdhPJyav0MEo24LX8IGdLNuSbWw2S+Hmx+xRK1bHvxSDbq+lsG2fF5620jJ/iBICjZal5SWkKtMZA== X-Received: by 2002:a17:907:972a:: with SMTP id jg42mr24619692ejc.757.1642580722265; Wed, 19 Jan 2022 00:25:22 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:21 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 12/25] trace-cmd library: Add macro to check file state on reading Date: Wed, 19 Jan 2022 10:24:54 +0200 Message-Id: <20220119082507.245600-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 has flexible structure. It allows reading almost any part of the file at any time, unlike the version 6 file when reading state must be validated at each step. Added a macro to handle these checks. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index c7998301..50eaa5c0 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -166,6 +166,8 @@ struct tracecmd_input { __thread struct tracecmd_input *tracecmd_curr_thread_handle; +#define CHECK_READ_STATE(H, S) ((H)->file_version < FILE_VERSION_SECTIONS && (H)->file_state >= (S)) + static int read_options_type(struct tracecmd_input *handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag) @@ -382,7 +384,7 @@ static int read_header_files(struct tracecmd_input *handle) char *header; char buf[BUFSIZ]; - if (handle->file_state >= TRACECMD_FILE_HEADERS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_HEADERS)) return 0; if (do_read_check(handle, buf, 12)) @@ -588,7 +590,7 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) int unique; int ret; - if (handle->file_state >= TRACECMD_FILE_FTRACE_EVENTS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_FTRACE_EVENTS)) return 0; if (regex) { @@ -661,7 +663,7 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) int unique; int ret; - if (handle->file_state >= TRACECMD_FILE_ALL_EVENTS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_ALL_EVENTS)) return 0; if (regex) { @@ -745,7 +747,7 @@ static int read_proc_kallsyms(struct tracecmd_input *handle) unsigned int size; char *buf; - if (handle->file_state >= TRACECMD_FILE_KALLSYMS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_KALLSYMS)) return 0; if (read4(handle, &size) < 0) @@ -778,7 +780,7 @@ static int read_ftrace_printk(struct tracecmd_input *handle) unsigned int size; char *buf; - if (handle->file_state >= TRACECMD_FILE_PRINTK) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_PRINTK)) return 0; if (read4(handle, &size) < 0) @@ -826,7 +828,7 @@ static int read_cpus(struct tracecmd_input *handle) { unsigned int cpus; - if (handle->file_state >= TRACECMD_FILE_CPU_COUNT) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_CPU_COUNT)) return 0; if (read4(handle, &cpus) < 0) @@ -2829,7 +2831,7 @@ static int read_options_type(struct tracecmd_input *handle) { char buf[10]; - if (handle->file_state >= TRACECMD_FILE_CPU_LATENCY) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_CPU_LATENCY)) return 0; if (do_read_check(handle, buf, 10)) @@ -2994,7 +2996,7 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) unsigned long long size; char *cmdlines; - if (handle->file_state >= TRACECMD_FILE_CMD_LINES) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_CMD_LINES)) return 0; if (read_data_and_size(handle, &cmdlines, &size) < 0) From patchwork Wed Jan 19 08:24:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 488DBC4332F for ; Wed, 19 Jan 2022 08:25:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352398AbiASIZ3 (ORCPT ); Wed, 19 Jan 2022 03:25:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352393AbiASIZZ (ORCPT ); Wed, 19 Jan 2022 03:25:25 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACBF1C06173E for ; Wed, 19 Jan 2022 00:25:24 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id c71so7589266edf.6 for ; Wed, 19 Jan 2022 00:25:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1qLdqkJrMHt2l1qzDHy4fC1S3/yiXLV0StkQXfhhwrA=; b=ZKukvd+q6+3VzCJk61ckb/x4wuU0lbiLFlOHS9ItppikO1s0tAZoaEshmWuV6s7GVk EGYIrjKC+zzBhL0eJztYsFGJFPV//rx0m4ymBGTOsatsMTz7kzBXm+NdhLkA6awunwyZ v93Z6oxtraI4uySLleuiH9HEtepQ6kSsA35oZgJhqjkRuD6xT1psP4tQ3pC9q7YDu/4a 5oG62mEELKR65z6J98sn5GOvx2f2q3fG5CHlE8U7jRD8A5umdZWc+jN0k+VvL8MrIDm7 /NWM6sEg+PdS5Ktf4jt5Giu8Nvp0birWBHNGfSvzqwAsecJ+LX7LeSSZvLqV2T85sbA1 RyEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1qLdqkJrMHt2l1qzDHy4fC1S3/yiXLV0StkQXfhhwrA=; b=EEYm/+GGzPOHGpvoSIe8yw2MoV7SaljyUCgf3+0g5LzQORGttRCOPRn561BNjlKPwt ZsJZy7ZUF+kXSQW9ogA6Wro/po8bDJWJM4MqKKJZN8lOzHVGQztWJCQK6y3i88QbfHnQ fn7FlVH3zlhnwsnNAOSe/xp0a7gG72oVDtvuMfAWexNXc9ObFtcrfz2v3hFsbKv5wkHn eYfr39rfRbUtSGl/dcHqwXidazt4P3eODbTFVelUJSTcF8FFOtw4MvHsHNJqinLAWHko H7SP/LbPwX39trjKzgcw3UJ+ygzuHBZNLBTCNlP1lGWQfVvXAwHA9Mp9RlbiXh+290It jFwg== X-Gm-Message-State: AOAM530+lzNqm/9QnEL3jV/YMUI/oVrI53e3k4OnVpRFawvSJma+ns0H M9n3kwapJ9a0RTmdSg1wPbSdh4uQ7pE= X-Google-Smtp-Source: ABdhPJxvcww5vO34ozVlcX03vTwMlRqabEBYjMtVRLsfDPJkP59fARnSIJ3pJ7pZlp9ApicvUeLBwA== X-Received: by 2002:a17:907:728e:: with SMTP id dt14mr18780105ejc.723.1642580723330; Wed, 19 Jan 2022 00:25:23 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:22 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 13/25] trace-cmd library: Introduce sections in trace file reading logic Date: Wed, 19 Jan 2022 10:24:55 +0200 Message-Id: <20220119082507.245600-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 is based on sections. Added an internal sections database and new helper functions to add, read, open and close file sections. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 71 +++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 50eaa5c0..22071caf 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -115,6 +115,14 @@ struct tsc2nsec { unsigned long long offset; }; +struct file_section { + unsigned long long section_offset; + unsigned long long data_offset; + int id; + int flags; + struct file_section *next; +}; + struct tracecmd_input { struct tep_handle *pevent; unsigned long file_state; @@ -154,6 +162,7 @@ struct tracecmd_input { struct hook_list *hooks; struct pid_addr_maps *pid_maps; /* file information */ + struct file_section *sections; size_t header_files_start; size_t ftrace_files_start; size_t event_files_start; @@ -377,6 +386,60 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +static struct file_section *section_get(struct tracecmd_input *handle, int id) +{ + struct file_section *sec; + + for (sec = handle->sections; sec; sec = sec->next) { + if (sec->id == id) + return sec; + } + + return NULL; +} + +static struct file_section *section_open(struct tracecmd_input *handle, int id) +{ + struct file_section *sec = section_get(handle, id); + + if (!sec) + return NULL; + + if (lseek64(handle->fd, sec->data_offset, SEEK_SET) == (off64_t)-1) + return NULL; + return sec; +} + +static void section_close(struct tracecmd_input *handle, struct file_section *sec) +{ + /* To Do */ +} + +static int section_add_or_update(struct tracecmd_input *handle, int id, int flags, + unsigned long long section_offset, + unsigned long long data_offset) +{ + struct file_section *sec = section_get(handle, id); + + if (!sec) { + sec = calloc(1, sizeof(struct file_section)); + if (!sec) + return -1; + sec->next = handle->sections; + handle->sections = sec; + sec->id = id; + } + + if (section_offset) + sec->section_offset = section_offset; + if (data_offset) + sec->data_offset = data_offset; + if (flags >= 0) + sec->flags = flags; + + return 0; +} + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -3493,6 +3556,7 @@ void tracecmd_ref(struct tracecmd_input *handle) */ void tracecmd_close(struct tracecmd_input *handle) { + struct file_section *del_sec; int cpu; int i; @@ -3532,6 +3596,12 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->version); close(handle->fd); + while (handle->sections) { + del_sec = handle->sections; + handle->sections = handle->sections->next; + free(del_sec); + } + for (i = 0; i < handle->nr_buffers; i++) free(handle->buffers[i].name); free(handle->buffers); @@ -3976,6 +4046,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->nr_buffers = 0; new_handle->buffers = NULL; new_handle->version = NULL; + new_handle->sections = NULL; new_handle->guest = NULL; new_handle->ref = 1; if (handle->trace_clock) { From patchwork Wed Jan 19 08:24:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D21C4C43219 for ; Wed, 19 Jan 2022 08:25:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352397AbiASIZa (ORCPT ); Wed, 19 Jan 2022 03:25:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352395AbiASIZ0 (ORCPT ); Wed, 19 Jan 2022 03:25:26 -0500 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5932C061574 for ; Wed, 19 Jan 2022 00:25:25 -0800 (PST) Received: by mail-ed1-x534.google.com with SMTP id a18so7548072edj.7 for ; Wed, 19 Jan 2022 00:25:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oUzMNjjDE3+FGvq6RtcbtYF1ZP00ibo0U6bMcHqrQWk=; b=Fohszgo8QQrHtqjy2C6bbTecVIvm20pMpshbfmhSuCtLp5/EBshRSIctM0kYKNj+yI 9HltLpV7Yeae2sHEKZVNH1ydQ7Y86yrxoJNH/Fl8X3eQmSdRInK++D9oUS90biXo5Gst hylZjk9GnnyJ7bHxukBoahVCYWWwjt1ldjwa7Mzggv77Sv2ADhXTTpB17hKIvUesERvs Y03CdKdYFeaPc4zRcmvYLhz+zEWizWuYT1qUR9G8Ov4hhtJyxOHe96ldv3o/RSHDRiRa EkH7DR04lji0t/kPCqj18Vc/r6kWgtmWRvW6NNNy4CWfRqwv7I8wfN6t92Ae2DU2Zibu I3YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oUzMNjjDE3+FGvq6RtcbtYF1ZP00ibo0U6bMcHqrQWk=; b=37zUXc4Bks381x8ny7aQI9pWy1oLW1WUJ4yGmJq4juPGru/Czv705arKF8bcj82ZCT s9XRZhTEVPZfJeGgAR+DtvY34DOtqlUqwI8WcM0yKRGGtS/GCA3wOHR+jyuMG1awZ5Ex GkPO4daBhHDRP/sAtPwzWBLzFkhAg/Nb/pHY63ekVR5/+tVXNq6LID+GC1drXxqdX15r e2A4NI1Zefp4YBZhSlJPq441p+s+KMSas+0crKs5O14r+a3iTItr2o20oWn3ztKK7KVf TdOaZ7asD+gMomfnrdA1ES8zQI2L0toiA/36FQYfFMnGUhenbt2LZ8cnV8noXgZ1wkuC Drhw== X-Gm-Message-State: AOAM530H78cYgA30NPMyvDu4gjaNAL/LnC308K7saLSux0BgCaLS6IkI 473dNeHKVbFZcQkdLYs1ahh7E7ZhX+Q= X-Google-Smtp-Source: ABdhPJx5ciiyoDt46oBL7GsaF/Io4wsTv/e34LFh1C8/FqGSDGFJKOYbhlbnNr/bXFNMIx90bYFcdg== X-Received: by 2002:a17:906:175a:: with SMTP id d26mr16560459eje.611.1642580724311; Wed, 19 Jan 2022 00:25:24 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:23 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 14/25] trace-cmd library: Initialize internal sections database on file read Date: Wed, 19 Jan 2022 10:24:56 +0200 Message-Id: <20220119082507.245600-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add sections in internal database on file reading and parsing. In trace file version 7, sections are initialized when parsing corresponding trace options. In version 6 files, sections are retrieved on file reading, as there they are in a fixed position in the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 65 +++++++++++++++---- 2 files changed, 52 insertions(+), 14 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 953297f3..94b33949 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -159,6 +159,7 @@ enum { TRACECMD_FL_BUFFER_INSTANCE = (1 << 1), TRACECMD_FL_IN_USECS = (1 << 2), TRACECMD_FL_RAW_TS = (1 << 3), + TRACECMD_FL_SECTIONED = (1 << 4), }; struct tracecmd_ftrace { diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 22071caf..e3ed2bbb 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -165,8 +165,7 @@ struct tracecmd_input { struct file_section *sections; size_t header_files_start; size_t ftrace_files_start; - size_t event_files_start; - size_t options_start; + unsigned long long options_start; size_t total_file_size; /* For custom profilers. */ @@ -176,6 +175,7 @@ struct tracecmd_input { __thread struct tracecmd_input *tracecmd_curr_thread_handle; #define CHECK_READ_STATE(H, S) ((H)->file_version < FILE_VERSION_SECTIONS && (H)->file_state >= (S)) +#define HAS_SECTIONS(H) ((H)->flags & TRACECMD_FL_SECTIONED) static int read_options_type(struct tracecmd_input *handle); @@ -450,6 +450,10 @@ static int read_header_files(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_HEADERS)) return 0; + if (!HAS_SECTIONS(handle)) + section_add_or_update(handle, TRACECMD_OPTION_HEADER_INFO, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (do_read_check(handle, buf, 12)) return -1; @@ -493,9 +497,6 @@ static int read_header_files(struct tracecmd_input *handle) free(header); - handle->ftrace_files_start = - lseek64(handle->fd, 0, SEEK_CUR); - handle->file_state = TRACECMD_FILE_HEADERS; return 0; @@ -656,6 +657,10 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) if (CHECK_READ_STATE(handle, TRACECMD_FILE_FTRACE_EVENTS)) return 0; + if (!HAS_SECTIONS(handle)) + section_add_or_update(handle, TRACECMD_OPTION_FTRACE_EVENTS, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (regex) { sreg = &spreg; ereg = &epreg; @@ -696,8 +701,6 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) goto out; } - handle->event_files_start = - lseek64(handle->fd, 0, SEEK_CUR); handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; ret = 0; out: @@ -729,6 +732,10 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) if (CHECK_READ_STATE(handle, TRACECMD_FILE_ALL_EVENTS)) return 0; + if (!HAS_SECTIONS(handle)) + section_add_or_update(handle, TRACECMD_OPTION_EVENT_FORMATS, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (regex) { sreg = &spreg; ereg = &epreg; @@ -812,6 +819,9 @@ static int read_proc_kallsyms(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_KALLSYMS)) return 0; + if (!HAS_SECTIONS(handle)) + section_add_or_update(handle, TRACECMD_OPTION_KALLSYMS, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); if (read4(handle, &size) < 0) return -1; @@ -846,6 +856,10 @@ static int read_ftrace_printk(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_PRINTK)) return 0; + if (!HAS_SECTIONS(handle)) + section_add_or_update(handle, TRACECMD_OPTION_PRINTK, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (read4(handle, &size) < 0) return -1; if (!size) { @@ -2878,6 +2892,18 @@ static int handle_options(struct tracecmd_input *handle) handle->tsc_calc.offset = tep_read_number(handle->pevent, buf + 8, 8); break; + case TRACECMD_OPTION_HEADER_INFO: + case TRACECMD_OPTION_FTRACE_EVENTS: + case TRACECMD_OPTION_EVENT_FORMATS: + case TRACECMD_OPTION_KALLSYMS: + case TRACECMD_OPTION_PRINTK: + case TRACECMD_OPTION_CMDLINES: + if (size < 8) + break; + section_add_or_update(handle, option, -1, + tep_read_number(handle->pevent, buf, 8), 0); + break; + default: tracecmd_warning("unknown option %d", option); break; @@ -3062,6 +3088,11 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_CMD_LINES)) return 0; + if (!HAS_SECTIONS(handle)) + section_add_or_update(handle, TRACECMD_OPTION_CMDLINES, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + + if (read_data_and_size(handle, &cmdlines, &size) < 0) return -1; cmdlines[size] = 0; @@ -3351,6 +3382,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) struct tracecmd_input *handle; char test[] = TRACECMD_MAGIC; unsigned int page_size; + size_t offset; char *version; char buf[BUFSIZ]; unsigned long ver; @@ -3390,6 +3422,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->file_version = ver; free(version); + if (handle->file_version >= FILE_VERSION_SECTIONS) + handle->flags |= TRACECMD_FL_SECTIONED; + if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3414,14 +3449,16 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) read4(handle, &page_size); handle->page_size = page_size; - handle->header_files_start = - lseek64(handle->fd, 0, SEEK_CUR); - - handle->total_file_size = - lseek64(handle->fd, 0, SEEK_END); + offset = lseek64(handle->fd, 0, SEEK_CUR); + handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); + lseek64(handle->fd, offset, SEEK_SET); - handle->header_files_start = - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + if (HAS_SECTIONS(handle)) { + if (read8(handle, &(handle->options_start))) { + tracecmd_warning("Filed to read the offset of the first option section"); + goto failed_read; + } + } handle->file_state = TRACECMD_FILE_INIT; From patchwork Wed Jan 19 08:24:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25DCCC43217 for ; Wed, 19 Jan 2022 08:25:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352395AbiASIZa (ORCPT ); Wed, 19 Jan 2022 03:25:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352400AbiASIZ1 (ORCPT ); Wed, 19 Jan 2022 03:25:27 -0500 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2BFDC06173F for ; Wed, 19 Jan 2022 00:25:26 -0800 (PST) Received: by mail-ed1-x52f.google.com with SMTP id m11so7403551edi.13 for ; Wed, 19 Jan 2022 00:25:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NX3ri/ZFqmLhlylTQASE48AM7fbXmIafPIVc+13EIEI=; b=KnW2S1UGhK3QS3zovY2zB0s1Wd9bI1eFTs04oMG03nky9yVZEk+PUGBx4d+zKwlCdS 8fTP6AdqNKNAolDAB4o5JuMgaSA23nHKhEHQOwqNPwzcAzEEtGhKhIJDGCG7gC7rP4me pKIR1GofP2HB5xjNEc1SlPg2Gbajd2eaNvfPBxGNunnloyfK6qX6m5lIrk4I+6JLWWcb D/BNXiiefX5uM2nbqOmHX1KjVjj+hDYRMhIKGPupmTapj3CSAyrNZuSKoHLDwsx4vXZg 9HXCx/rbehATUaxVQ5B1nhvQ4PCf8FMqwGqCAaeyC3l3kyy5LSqFNzIwdfaJl4wtuivL 9gFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NX3ri/ZFqmLhlylTQASE48AM7fbXmIafPIVc+13EIEI=; b=KUNqAfStcfZm5Kh20z3Kf5QWfv9sjmj8zDGJYNyRrsRfGGi+9l6mqFG4qA0ROvEWCT odInf599wPXWmNnno5KvxpnTuARBMxGDKgnhTh3nXhjlufat3n4LvRWX/5jbxzV3hQcU le2AZ/9p3969qtBQjEXNQPqApPZD3O4idBFclRWGIlU0QuOGxvpK8UaAs35wm/Ori+93 KgM5EdW2UA+1qVTHwhmuB432u3tv3uE7/d2j7k0MfIGkFzhbqnmQmBG/ORn0ZT5ibCyT TLAR6gBQQHpPo0attizHAZub4XZmC6z96zWCCGnYokZlSUVyTbtrAGnv4dCSLrnd/f9M p1wQ== X-Gm-Message-State: AOAM5301Ci3tt6rnlcl4BIxG+Aq8wUHbwsIPqQSKbuNy4PbtJxSD3D21 4nzap0s1iC1xuDHpKFpA1ay0+TxQu6g= X-Google-Smtp-Source: ABdhPJyvfQhiqvtA6BeCP/P8hLcRoNtGbyYoiAj8KabsgljcgasOZLQQbyDr6NMbyj4qhuugLkQqkQ== X-Received: by 2002:a17:906:3052:: with SMTP id d18mr23622327ejd.675.1642580725513; Wed, 19 Jan 2022 00:25:25 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:25 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 15/25] trace-cmd library: Use sections database when reading parts of the trace file Date: Wed, 19 Jan 2022 10:24:57 +0200 Message-Id: <20220119082507.245600-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Utilize the internal database with file sections, when reading parts of a trace file. This logic unifies the way trace file version 6 and 7 are processed. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e3ed2bbb..e8ece688 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3270,21 +3270,26 @@ int tracecmd_make_pipe(struct tracecmd_input *handle, int cpu, int fd, int cpus) */ void tracecmd_print_events(struct tracecmd_input *handle, const char *regex) { - int ret; + struct file_section *sec; if (!regex) regex = ".*"; - if (!handle->ftrace_files_start) { - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + sec = section_open(handle, TRACECMD_OPTION_HEADER_INFO); + if (sec) { read_header_files(handle); + section_close(handle, sec); + } + sec = section_open(handle, TRACECMD_OPTION_FTRACE_EVENTS); + if (sec) { + read_ftrace_files(handle, regex); + section_close(handle, sec); + } + sec = section_open(handle, TRACECMD_OPTION_EVENT_FORMATS); + if (sec) { + read_event_files(handle, regex); + section_close(handle, sec); } - ret = read_ftrace_files(handle, regex); - if (ret < 0) - return; - - read_event_files(handle, regex); - return; } /* Show the cpu data stats */ @@ -3888,6 +3893,7 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state) { + struct file_section *sec; int ret; if (!start_state) @@ -3903,13 +3909,17 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, if (handle->file_state >= start_state) { /* Set the handle to just before the start state */ - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + sec = section_open(handle, TRACECMD_OPTION_HEADER_INFO); + if (!sec) + return -1; /* Now that the file handle has moved, change its state */ handle->file_state = TRACECMD_FILE_INIT; } /* Try to bring the input up to the start state - 1 */ ret = tracecmd_read_headers(handle, start_state - 1); + if (sec) + section_close(handle, sec); if (ret < 0) goto out; From patchwork Wed Jan 19 08:24:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAB53C4321E for ; Wed, 19 Jan 2022 08:25:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352402AbiASIZb (ORCPT ); Wed, 19 Jan 2022 03:25:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352401AbiASIZ2 (ORCPT ); Wed, 19 Jan 2022 03:25:28 -0500 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2729C06161C for ; Wed, 19 Jan 2022 00:25:27 -0800 (PST) Received: by mail-ed1-x52c.google.com with SMTP id b13so7810333edn.0 for ; Wed, 19 Jan 2022 00:25:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=A3838AQ6PwuFvs4A3Z75lD6iChvJ325ckhNRXS1psH8=; b=Hk3/ZibB5vjT9SNkKK/xVVLau1NH9bjLGtX4e2OHU6QeUO3SGmAqozB94PdO3x7hS1 KCoYnq0u30wWlX31vmJbwkIhpgGwTnFUEINqku9KMItxduTkAD46cJffXDSFxuQ8XLpU usANUDHhvUTUHNr00jIhH1zyffYbB/Ols4VAiWzr3E5WRnGlEICSm99Pm1xluDAR2QxC wSAB6s2wWMJBXT/+qemcRxdcDb8FLzIQezMWFlSPwNz2Z6Qx/AJF6yFQjUK4S4fBzGss e3bkTGgyswE5u5CCe5T5mmnNw5liWA4QLF0C1Ed7laWPa8Hh03kWtGPY+VP0hulJFJXA 8gMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=A3838AQ6PwuFvs4A3Z75lD6iChvJ325ckhNRXS1psH8=; b=4uWlhcUqw4Wsm5jaisz3Cb+3oX9ukeU3mKZE30qr1zlBGMbDM0lVL/HaYcDpCi0gGU Qtnobg9pXFwg5YrhDWCWEG6DvEOUb3+ycBlfOTj/5nN7UIQA4Kj0qSEsxSYjVsQLRkjl PzUmpKr1xeX8ipyGZk5JogZviq0xanJxX3X0Pd5w05++7ywW+DRJvOmB8Cr4DPMPP7vF YKaV5ywzzuOx4Y4Yhcr6uac7wUPJib+4vrRC3b1YKlf4LyoVVz9EkLqxTf4vmbbG8IgX HUZaAfuf2l8BOLU3poP9PN9POkLHygli1a0yz5CzZpmh1UuYbSU+zCq0bKucHI9F0Blu Uo+w== X-Gm-Message-State: AOAM531QYXaJCn2C9RcFTrq6pH4d+BlCT7il3504xaJ1tPDHInYGyu2M Tx0Z8bUMMABeNFPz3sGJMHqrfZiJ2uU= X-Google-Smtp-Source: ABdhPJyK9Eqki68cmgIdbvnnq0xcaq0ZZVqmVea1b6wR1X3GRZGKhGPQEkM67B5DqVJ2/NKBYy8rrg== X-Received: by 2002:a17:906:e282:: with SMTP id gg2mr24465476ejb.607.1642580726579; Wed, 19 Jan 2022 00:25:26 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:26 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 16/25] trace-cmd library: Read headers from trace file version 7 Date: Wed, 19 Jan 2022 10:24:58 +0200 Message-Id: <20220119082507.245600-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 has more flexible structure than version 6, headers are not located at fixed position in the file. A new logic is implemented to read and parse the headers from this new format. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 137 ++++++++++++++++++++++++++++++++---- 1 file changed, 122 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e8ece688..b7785700 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -163,8 +163,7 @@ struct tracecmd_input { struct pid_addr_maps *pid_maps; /* file information */ struct file_section *sections; - size_t header_files_start; - size_t ftrace_files_start; + bool options_init; unsigned long long options_start; size_t total_file_size; @@ -919,19 +918,7 @@ static int read_cpus(struct tracecmd_input *handle) return 0; } -/** - * tracecmd_read_headers - read the header information from trace.dat - * @handle: input handle for the trace.dat file - * @state: The state to read up to or zero to read up to options. - * - * This reads the trace.dat file for various information. Like the - * format of the ring buffer, event formats, ftrace formats, kallsyms - * and printk. This may be called multiple times with different @state - * values, to read partial data at a time. It will always continue - * where it left off. - */ -int tracecmd_read_headers(struct tracecmd_input *handle, - enum tracecmd_file_states state) +static int read_headers_v6(struct tracecmd_input *handle, enum tracecmd_file_states state) { int ret; @@ -997,6 +984,126 @@ int tracecmd_read_headers(struct tracecmd_input *handle, return 0; } +static int handle_options(struct tracecmd_input *handle); + +static int read_section_header(struct tracecmd_input *handle, unsigned short *id, + unsigned short *flags, unsigned long long *size, const char **description) +{ + unsigned short fl; + unsigned short sec_id; + unsigned long long sz; + int desc; + + if (read2(handle, &sec_id)) + return -1; + if (read2(handle, &fl)) + return -1; + if (read4(handle, (unsigned int *)&desc)) + return -1; + if (read8(handle, &sz)) + return -1; + + if (id) + *id = sec_id; + if (flags) + *flags = fl; + if (size) + *size = sz; + + return 0; +} + +static int handle_section(struct tracecmd_input *handle, struct file_section *section) +{ + unsigned short id, flags; + unsigned long long size; + int ret; + + if (lseek64(handle->fd, section->section_offset, SEEK_SET) == (off_t)-1) + return -1; + if (read_section_header(handle, &id, &flags, &size, NULL)) + return -1; + section->flags = flags; + if (id != section->id) + return -1; + + section->data_offset = lseek64(handle->fd, 0, SEEK_CUR); + + switch (section->id) { + case TRACECMD_OPTION_HEADER_INFO: + ret = read_header_files(handle); + break; + case TRACECMD_OPTION_FTRACE_EVENTS: + ret = read_ftrace_files(handle, NULL); + break; + case TRACECMD_OPTION_EVENT_FORMATS: + ret = read_event_files(handle, NULL); + break; + case TRACECMD_OPTION_KALLSYMS: + ret = read_proc_kallsyms(handle); + break; + case TRACECMD_OPTION_PRINTK: + ret = read_ftrace_printk(handle); + break; + case TRACECMD_OPTION_CMDLINES: + ret = read_and_parse_cmdlines(handle); + break; + default: + ret = 0; + break; + } + + return ret; +} + +static int read_headers(struct tracecmd_input *handle) +{ + struct file_section *section; + + if (handle->options_init) + return 0; + + if (!handle->options_start) + return -1; + + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off64_t)-1) { + tracecmd_warning("Filed to goto options offset %lld", handle->options_start); + return -1; + } + + if (handle_options(handle)) + return -1; + + section = handle->sections; + while (section) { + if (handle_section(handle, section)) + return -1; + section = section->next; + } + + handle->options_init = true; + return 0; +} + +/** + * tracecmd_read_headers - read the header information from trace.dat + * @handle: input handle for the trace.dat file + * @state: The state to read up to or zero to read up to options. + * + * This reads the trace.dat file for various information. Like the + * format of the ring buffer, event formats, ftrace formats, kallsyms + * and printk. This may be called multiple times with different @state + * values, to read partial data at a time. It will always continue + * where it left off. + */ +int tracecmd_read_headers(struct tracecmd_input *handle, + enum tracecmd_file_states state) +{ + if (!HAS_SECTIONS(handle)) + return read_headers_v6(handle, state); + return read_headers(handle); +} + static unsigned long long calc_page_offset(struct tracecmd_input *handle, unsigned long long offset) { From patchwork Wed Jan 19 08:24:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C5AEC433EF for ; Wed, 19 Jan 2022 08:25:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352400AbiASIZb (ORCPT ); Wed, 19 Jan 2022 03:25:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352402AbiASIZ3 (ORCPT ); Wed, 19 Jan 2022 03:25:29 -0500 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2E8DC06173E for ; Wed, 19 Jan 2022 00:25:28 -0800 (PST) Received: by mail-ed1-x52f.google.com with SMTP id q25so7646953edb.2 for ; Wed, 19 Jan 2022 00:25:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QS6PJhSLqh0bsb/HXlHjf6qtLioExB0jDJ2s/M3szr4=; b=YU0YnIIaqyX+A7ZBfkUpTf9GEDtNkhjko4U/sV+S0SQhnSjMDEcnJLFtp0Gm1tFkp/ nvnI0B6OvH2NSYC1t0bR/cHux91Ae/qnMwItCwPbtDcmo35UYD/WNZmf7UH6+aIZkDPm a93Yn9BmZAvZO9q2UMgq51iclwa2PMtY82s5d1P2uE41QJsgrVbxYjrYnpgK63qHeaBR lSCIXiGecjVg8wbODwcfYdjLZZP9QhcD//RMhVis3GwJgGVyjC7IWakyQHxxGhcFk4w/ aYoX5T/lQt89z6NRTyLSbvk78VAJKkqphxZ5h0NqawHk5lpUHpcaObh7+8l8iW5eqjg0 BGZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QS6PJhSLqh0bsb/HXlHjf6qtLioExB0jDJ2s/M3szr4=; b=RpShCWGIFJ4B6z6TgTCUZeIdRbiZftcfptiYTGQoSf5PM5VsbrmUbSnjpcWKuRCQzd Ka7aj0JD8Cbl0foLOyT3rWepyVSD7iOcI6G8oeD8clMppmOLX4v+Qgtbcea0II2+kvNa f84LsXnTHpv27AKyt8hHD8tZMGBTrlPaNM4DZqwm7xZZU63YFPaK8kDfyQ47gJ38uNEZ QWKEoFSp41LAsTjLNI0a16LkIh4/zUhxe/Gsjy7xYGYSmo2B1/u+5zcQB/3h7Uvm17Lk vZc8yCqfUb156xed0SpeQVBQNHu9WVNbOpKIhB+BtKjsjI1lqGyadw+eHOhXTSD1mDVl hPcA== X-Gm-Message-State: AOAM532M/4tGr06dbDFxOzoD0mulMze1a4XfG7fvVEA7MAZF1X8cA92I D7rpduvfIFD5wpwHFITTKHYSC3tk+jA= X-Google-Smtp-Source: ABdhPJzsh52yW28V9CNTfubJNQrv2UkON00ZpCpoOEFJVi0JUG7T9RyF8jjJCkyfHa4grjr+WRG2+Q== X-Received: by 2002:a17:906:4945:: with SMTP id f5mr2507128ejt.429.1642580727467; Wed, 19 Jan 2022 00:25:27 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:27 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 17/25] trace-cmd library: Read strings sections on file load Date: Wed, 19 Jan 2022 10:24:59 +0200 Message-Id: <20220119082507.245600-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Internal strings database is added to trace input handle, containing all metadata strings from trace file. When a trace file is opened, all strings sections are located and the internal strings database is initialised. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 65 +++++++++++++++++++++++++ lib/trace-cmd/trace-output.c | 1 + 3 files changed, 67 insertions(+) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 4a0a691c..ac7e7f17 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -53,5 +53,6 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name); off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); +unsigned int get_meta_strings_size(struct tracecmd_input *handle); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b7785700..2e10ebad 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -147,6 +147,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + unsigned int strings_size; /* size of the metadata strings */ + char *strings; /* metadata strings */ + struct host_trace_info host; double ts2secs; char * cpustats; @@ -986,6 +989,14 @@ static int read_headers_v6(struct tracecmd_input *handle, enum tracecmd_file_sta static int handle_options(struct tracecmd_input *handle); +static const char *get_metadata_string(struct tracecmd_input *handle, int offset) +{ + if (!handle || !handle->strings || offset < 0 || handle->strings_size >= offset) + return NULL; + + return handle->strings + offset; +} + static int read_section_header(struct tracecmd_input *handle, unsigned short *id, unsigned short *flags, unsigned long long *size, const char **description) { @@ -1009,6 +1020,8 @@ static int read_section_header(struct tracecmd_input *handle, unsigned short *id *flags = fl; if (size) *size = sz; + if (description) + *description = get_metadata_string(handle, desc); return 0; } @@ -2841,6 +2854,11 @@ tracecmd_search_task_map(struct tracecmd_input *handle, return lib; } +__hidden unsigned int get_meta_strings_size(struct tracecmd_input *handle) +{ + return handle->strings_size; +} + static int handle_options(struct tracecmd_input *handle) { long long offset; @@ -3474,6 +3492,50 @@ struct hook_list *tracecmd_hooks(struct tracecmd_input *handle) return handle->hooks; } +static int init_metadata_strings(struct tracecmd_input *handle, int size) +{ + char *tmp; + + tmp = realloc(handle->strings, handle->strings_size + size); + if (!tmp) + return -1; + + handle->strings = tmp; + if (do_read_check(handle, handle->strings + handle->strings_size, size)) + return -1; + + handle->strings_size += size; + + return 0; +} + +static int read_metadata_strings(struct tracecmd_input *handle) +{ + unsigned short flags; + int found = 0; + unsigned short id; + unsigned long long size; + off64_t offset; + + offset = lseek64(handle->fd, 0, SEEK_CUR); + do { + if (read_section_header(handle, &id, &flags, &size, NULL)) + break; + if (id == TRACECMD_OPTION_STRINGS) { + found++; + init_metadata_strings(handle, size); + } else { + if (lseek64(handle->fd, size, SEEK_CUR) == (off_t)-1) + break; + } + } while (1); + + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + + return found ? 0 : -1; +} + /** * tracecmd_alloc_fd - create a tracecmd_input handle from a file descriptor * @fd: the file descriptor for the trace.dat file @@ -3570,6 +3632,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) tracecmd_warning("Filed to read the offset of the first option section"); goto failed_read; } + read_metadata_strings(handle); } handle->file_state = TRACECMD_FILE_INIT; @@ -3742,6 +3805,7 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->cpu_data); free(handle->uname); free(handle->trace_clock); + free(handle->strings); free(handle->version); close(handle->fd); @@ -4201,6 +4265,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->buffers = NULL; new_handle->version = NULL; new_handle->sections = NULL; + new_handle->strings = NULL; new_handle->guest = NULL; new_handle->ref = 1; if (handle->trace_clock) { diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 7e1849c8..2b5671ae 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2255,6 +2255,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); handle->options_start = tracecmd_get_options_offset(ihandle); + handle->strings_offs = get_meta_strings_size(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Wed Jan 19 08:25:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA63C4167B for ; Wed, 19 Jan 2022 08:25:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352401AbiASIZb (ORCPT ); Wed, 19 Jan 2022 03:25:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352370AbiASIZa (ORCPT ); Wed, 19 Jan 2022 03:25:30 -0500 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7D40C061574 for ; Wed, 19 Jan 2022 00:25:29 -0800 (PST) Received: by mail-ed1-x535.google.com with SMTP id 30so7614393edv.3 for ; Wed, 19 Jan 2022 00:25:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BTn2Y51qmp2VvereioebFaCTG95UE8sdJogNHdDX/O4=; b=lpPg9au8Xpq+BHHaQnOdoLhm4Mmjnq1nKZmRDYb9Gx/u0lYLWSqthOIXMFKWtuq02L s2GZ9HA8uurGciUuRMgCY9PTmSbXNOVY5TSP1COD3+3KkUVHE4aEOG25EKp3CHwDjbJz XjfZK6MA25RWhidtfL3Dq3Y9K4dRxM1vYu2Apk2BOcYXhCXFjx5CSw1A1l7t0TGejOKb HFjL9btJC7B/e8Sh420uF22aC0pBITBXn2hBF2BxczUf9eepCXEmil0M6OgWBE03CojV mU/8YRY6yMsH0rFQ1RhJHVPy+bzgiv3V1A5do9eOxh4IdskMSBy8/2q3ufGBsYnJy3F9 csLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BTn2Y51qmp2VvereioebFaCTG95UE8sdJogNHdDX/O4=; b=SKpKmjrziEkQUUSxAKxiV3RF4bgEHC7ZdJ+QbOPpSWYiPpe1bF85Ec9LBMoXF79lBA is8VGjikpweJAApMBzLDNO5JX252BzOk8VZ1NFmgBgFmJTJin720oT8sGrOTJ2cKUMeh Vmy90F39iTIu6rRcfzg67WQxyXdBzrb/gk2t9HoYtkNRIAiWbmCVveNXhwPRsQKnWgNz LJ5tNdFqvSXMSwGFGD2pso2rFV76CEjrWJ8LHbf0LMCY+eOX01qhzxXWJcUF54HzJNeX bC3136G0iRE67NSU8IKkv/hp0lk1UdPTrcsx+gRHawWP97Kr/Si6Qyrfm5IFHRwtqclg nE6Q== X-Gm-Message-State: AOAM531/z9PM6bFxXaNPt1sAAKYCMMhFzmmSLHLP36LvCMm3UloCTdGS JCugsiVvBlt11OCxJhz+6lkdKbfpuA8= X-Google-Smtp-Source: ABdhPJxcbWylgxli69Jc9lw4tURQ9dl+S5CMWjqwMJ4pjd6YPGqkCbW6Z23IM0nd2P5PCIXZZiCtew== X-Received: by 2002:a17:907:d93:: with SMTP id go19mr5846000ejc.430.1642580728443; Wed, 19 Jan 2022 00:25:28 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:28 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 18/25] trace-cmd library: Read extended BUFFER option Date: Wed, 19 Jan 2022 10:25:00 +0200 Message-Id: <20220119082507.245600-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The BUFFER option is extended in trace file version 7. It holds CPU metadata related to the recorded CPU traces. Also, there is a new BUFFER_TEXT option for describing the latency trace data. A new logic is implemented for these new options. In trace file version 7, the top buffer is saved as other buffers in the file, no special treatment. But saving the top buffer in the list of buffers in the input handler causes problems. It breaks the legacy logic of trace-cmd library users, which have special logic for trace buffers processing. That's why "top_buffer" member is added in the input handler structure, to hold the top buffer. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 145 +++++++++++++++++++++++++++++++----- 1 file changed, 128 insertions(+), 17 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 2e10ebad..1eb92f2a 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -74,9 +74,19 @@ struct cpu_data { int pipe_fd; }; +struct cpu_file_data { + int cpu; + unsigned long long offset; + unsigned long long size; +}; + struct input_buffer_instance { char *name; size_t offset; + char *clock; + bool latency; + int cpus; + struct cpu_file_data *cpu_data; }; struct ts_offset_sample { @@ -156,6 +166,7 @@ struct tracecmd_input { char * uname; char * version; char * trace_clock; + struct input_buffer_instance top_buffer; struct input_buffer_instance *buffers; int parsing_failures; struct guest_trace_info *guest; @@ -2859,13 +2870,115 @@ __hidden unsigned int get_meta_strings_size(struct tracecmd_input *handle) return handle->strings_size; } +static inline int save_read_number(struct tep_handle *tep, char *data, int *data_size, + int *read_pos, int bytes, unsigned long long *num) +{ + if (bytes > *data_size) + return -1; + + *num = tep_read_number(tep, (data + *read_pos), bytes); + *read_pos += bytes; + *data_size -= bytes; + return 0; +} + +static inline char *save_read_string(char *data, int *data_size, int *read_pos) +{ + char *str; + + if (*data_size < 1) + return NULL; + + str = strdup(data + *read_pos); + if (!str) + return NULL; + *data_size -= (strlen(str) + 1); + if (*data_size < 0) { + free(str); + return NULL; + } + *read_pos += (strlen(str) + 1); + + return str; +} + +static int handle_buffer_option(struct tracecmd_input *handle, + unsigned short id, char *data, int size) +{ + struct input_buffer_instance *buff; + unsigned long long tmp; + int rsize = 0; + char *name; + int i; + + if (save_read_number(handle->pevent, data, &size, &rsize, 8, &tmp)) + return -1; + + name = save_read_string(data, &size, &rsize); + if (!name) + return -1; + + if (*name == '\0') { + /* top buffer */ + buff = &handle->top_buffer; + } else { + buff = realloc(handle->buffers, sizeof(*handle->buffers) * (handle->nr_buffers + 1)); + if (!buff) { + free(name); + return -1; + } + handle->buffers = buff; + handle->nr_buffers++; + + buff = &handle->buffers[handle->nr_buffers - 1]; + } + memset(buff, 0, sizeof(struct input_buffer_instance)); + buff->name = name; + buff->offset = tmp; + + if (!HAS_SECTIONS(handle)) + return 0; + + /* file sections specific data */ + buff->clock = save_read_string(data, &size, &rsize); + if (!buff->clock) + return -1; + + if (*name == '\0' && !handle->trace_clock) + handle->trace_clock = strdup(buff->clock); + + if (id == TRACECMD_OPTION_BUFFER) { + if (save_read_number(handle->pevent, data, &size, &rsize, 4, &tmp)) + return -1; + buff->cpus = tmp; + if (!buff->cpus) + return 0; + buff->cpu_data = calloc(buff->cpus, sizeof(struct cpu_file_data)); + if (!buff->cpu_data) + return -1; + for (i = 0; i < buff->cpus; i++) { + if (save_read_number(handle->pevent, data, &size, &rsize, 4, &tmp)) + return -1; + buff->cpu_data[i].cpu = tmp; + if (save_read_number(handle->pevent, data, + &size, &rsize, 8, &buff->cpu_data[i].offset)) + return -1; + if (save_read_number(handle->pevent, data, + &size, &rsize, 8, &buff->cpu_data[i].size)) + return -1; + } + } else { + buff->latency = true; + } + return 0; +} + static int handle_options(struct tracecmd_input *handle) { long long offset; unsigned short option; unsigned int size; char *cpustats = NULL; - struct input_buffer_instance *buffer; struct hook_list *hook; char *buf; int cpus; @@ -2956,21 +3069,10 @@ static int handle_options(struct tracecmd_input *handle) handle->cpustats = cpustats; break; case TRACECMD_OPTION_BUFFER: - /* A buffer instance is saved at the end of the file */ - handle->nr_buffers++; - handle->buffers = realloc(handle->buffers, - sizeof(*handle->buffers) * handle->nr_buffers); - if (!handle->buffers) - return -ENOMEM; - buffer = &handle->buffers[handle->nr_buffers - 1]; - buffer->name = strdup(buf + 8); - if (!buffer->name) { - free(handle->buffers); - handle->buffers = NULL; - return -ENOMEM; - } - offset = *(unsigned long long *)buf; - buffer->offset = tep_read_number(handle->pevent, &offset, 8); + case TRACECMD_OPTION_BUFFER_TEXT: + ret = handle_buffer_option(handle, option, buf, size); + if (ret < 0) + return ret; break; case TRACECMD_OPTION_TRACECLOCK: if (!handle->ts2secs) @@ -3759,6 +3861,13 @@ void tracecmd_ref(struct tracecmd_input *handle) handle->ref++; } +static inline void free_buffer(struct input_buffer_instance *buf) +{ + free(buf->name); + free(buf->clock); + free(buf->cpu_data); +} + /** * tracecmd_close - close and free the trace.dat handle * @handle: input handle for the trace.dat file @@ -3815,8 +3924,9 @@ void tracecmd_close(struct tracecmd_input *handle) free(del_sec); } + free_buffer(&handle->top_buffer); for (i = 0; i < handle->nr_buffers; i++) - free(handle->buffers[i].name); + free_buffer(&handle->buffers[i]); free(handle->buffers); tracecmd_free_hooks(handle->hooks); @@ -4260,6 +4370,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) return NULL; *new_handle = *handle; + memset(&new_handle->top_buffer, 0, sizeof(new_handle->top_buffer)); new_handle->cpu_data = NULL; new_handle->nr_buffers = 0; new_handle->buffers = NULL; From patchwork Wed Jan 19 08:25:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E02FC4167D for ; Wed, 19 Jan 2022 08:25:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352370AbiASIZc (ORCPT ); Wed, 19 Jan 2022 03:25:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352404AbiASIZb (ORCPT ); Wed, 19 Jan 2022 03:25:31 -0500 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5ECAC061574 for ; Wed, 19 Jan 2022 00:25:30 -0800 (PST) Received: by mail-ed1-x534.google.com with SMTP id 30so7614570edv.3 for ; Wed, 19 Jan 2022 00:25:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7PpO5wes0CmxlKpspqIMBO3n609WxZs0QKDjgxIZHIQ=; b=TwDoH0JGp2jTcL/Cb3ykrz0BjuUPPOSNp0dWQRIY5s8RgCsRIGFdtMp3nbzfu2QIUL XsqKiSSNmLLLn5hvNdg5Lr7F/sWbuDqt8dsnItV7EI0b3YfKCrSxU/7Fl/DgJ/GrnnZI 7XJu729PMrIG0CEBT7jmAs7lZ7pFjJscDDSivLYfg4V/R4y6eKQvhhn7eU+KdoDvwwKw lWYf6OWXPiRlZgUC/leacMfpnelZi6AZyFMPEiLFOPlLFN7ADaNQpt5G58YFb49vBuO7 y2AAo2NDY8JNnS/xjtozNNLmnjF1UCTsSoMdDW5CXoTVDWjPhXa+Xal8Hhh+hqGS7JyO IIOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7PpO5wes0CmxlKpspqIMBO3n609WxZs0QKDjgxIZHIQ=; b=SqoWWBTnuX0x3rip+zWyPZkLaTkCDUXWNtaZ+UbhZlJU/OsbcQXNj4Tv4hCFIN+PLN c4P1Q2MGbO74HUlaiBrW0P9J/5FLAgd3d10mHzt0WTCUqofL14PBuvTpKY9Un9fkoGL0 aUvmbeYTEXuk8PCO5NoFDCV5eTp9TDPVq/SUEflUyI2YxE0HemBnPr8iOFrM7MCp0YtJ 8gKKm7bug7azPCNBJ5eF1bc13i10IVSqfeeNSa5M6r+2hT9o6iNsusTDNcgz9wFa2Df1 ir+H44sMPNsp/Br1OiOcvKJPtjKB26Cj3aJGELlZ7UtqnVMCxDJzt2CSyLD9k6bkAJuz 7+sA== X-Gm-Message-State: AOAM530pz0ZU7NsGyW2yPpA39amaJjynQu3mHzhEXVPJe6qOCHn9qNri jI6Iad+8G2rWQ2GhLHD0eDHG+4pHSo0= X-Google-Smtp-Source: ABdhPJx9n9u6sttvEPGEOT4z7LnCXAHllltIN0JaLeL5URyi7cHSFoMyTi7hiyw/nUrDfhk2UMukMA== X-Received: by 2002:a05:6402:31e9:: with SMTP id dy9mr28977294edb.65.1642580729320; Wed, 19 Jan 2022 00:25:29 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:28 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 19/25] trace-cmd library: Handle the extended DONE option Date: Wed, 19 Jan 2022 10:25:01 +0200 Message-Id: <20220119082507.245600-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7 the DONE option is extended to store the offset in the file to the next options section. This way a list of options sections can be stored in the file. Added logic to recursively read all option sections from the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 43 +++++++++++++++++++++++-- lib/trace-cmd/trace-output.c | 2 +- 3 files changed, 43 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index ac7e7f17..b4f3d8c8 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -53,6 +53,7 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name); off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); +unsigned long long get_last_option_offset(struct tracecmd_input *handle); unsigned int get_meta_strings_size(struct tracecmd_input *handle); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 1eb92f2a..7f418a44 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -179,6 +179,7 @@ struct tracecmd_input { struct file_section *sections; bool options_init; unsigned long long options_start; + unsigned long long options_last_offset; size_t total_file_size; /* For custom profilers. */ @@ -2870,6 +2871,32 @@ __hidden unsigned int get_meta_strings_size(struct tracecmd_input *handle) return handle->strings_size; } +__hidden unsigned long long get_last_option_offset(struct tracecmd_input *handle) +{ + return handle->options_last_offset; +} + +static int handle_option_done(struct tracecmd_input *handle, char *buf, int size) +{ + unsigned long long offset; + + if (size < 8) + return -1; + + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (offset >= size) + handle->options_last_offset = offset - size; + + offset = tep_read_number(handle->pevent, buf, 8); + if (!offset) + return 0; + + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + + return handle_options(handle); +} + static inline int save_read_number(struct tep_handle *tep, char *data, int *data_size, int *read_pos, int bytes, unsigned long long *num) { @@ -2978,19 +3005,27 @@ static int handle_options(struct tracecmd_input *handle) long long offset; unsigned short option; unsigned int size; + unsigned short id, flags; char *cpustats = NULL; struct hook_list *hook; char *buf; int cpus; int ret; - handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + if (!HAS_SECTIONS(handle)) { + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + } else { + if (read_section_header(handle, &id, &flags, NULL, NULL)) + return -1; + if (id != TRACECMD_OPTION_DONE) + return -1; + } for (;;) { if (read2(handle, &option)) return -1; - if (option == TRACECMD_OPTION_DONE) + if (!HAS_SECTIONS(handle) && option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ @@ -3130,6 +3165,10 @@ static int handle_options(struct tracecmd_input *handle) section_add_or_update(handle, option, -1, tep_read_number(handle->pevent, buf, 8), 0); break; + case TRACECMD_OPTION_DONE: + ret = handle_option_done(handle, buf, size); + free(buf); + return ret; default: tracecmd_warning("unknown option %d", option); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 2b5671ae..fe4fcf64 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2254,7 +2254,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); - handle->options_start = tracecmd_get_options_offset(ihandle); + handle->options_start = get_last_option_offset(ihandle); handle->strings_offs = get_meta_strings_size(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Wed Jan 19 08:25:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9550C4332F for ; Wed, 19 Jan 2022 08:25:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352404AbiASIZc (ORCPT ); Wed, 19 Jan 2022 03:25:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352406AbiASIZc (ORCPT ); Wed, 19 Jan 2022 03:25:32 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF5B4C061574 for ; Wed, 19 Jan 2022 00:25:31 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id a18so7549405edj.7 for ; Wed, 19 Jan 2022 00:25:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gcD35ez3yGA4E7930iSZGDFqcnANq8aEACWjru4ELRk=; b=mPSuRCV2tBmJ/FAqiw22DLVnGwN0DNjFIZMyOC56ni9BeW4A2Vg8XJBo72jhExQat3 K1E2pMVAjsAlndGmoPnpIrGEGsUiHAD69YRT2pyMOvgruj9cjjmvTjZa1cFMiSp8v8R8 8Aa/5edlYzeZ68Oq0blLUqANl1kkGkJbcUJgdjWR5dwF7mB4Aplx0X7FPH0JQe5Io73E ZEWb2edVWjbf1J7v9hC0nFTjKMOzB1x+PwxVcEL370rZTGCfrZFgdPOA2Czb3jjpVmZ2 2T+R0ZKopIvMgQKrnGKh/MGBjm1s6yO8e35Bz8h9pCOGdC95wxWVGXxVOlBsHF/9LIPh iDXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gcD35ez3yGA4E7930iSZGDFqcnANq8aEACWjru4ELRk=; b=MA0oM+q43/oiKzIh4pT+9FkpIKLCEqs1GNIox516TM8tqlZ3ucoD10laWyPZUCj75D /bQUZvQCBx110ygmf89oab1qw7KEwcvcs5rI5Md6AB8cqVpY7R6XIecFPZ8DwuuQUfON 7g+Sj+AY3NFepr9vmol/PwW3ILpgMoVmMfw/3eJNDxx9MVUKeVc5yTSPD3kVpjRlsEL1 lOwMI6Tc41CfTIYSMyTkyfdf3f3s1jMfnXxvvufDb8isD9aNMCBGO5HK/00j8qEm1gUZ XLn8YQiSfWyJC7+fPk3wjTFLQ3yeSFdUUX7FhHMIVh5q151hJYeEdqKppEtrA5833kmu Yxkg== X-Gm-Message-State: AOAM532uGNDlYVKjErD8g1Vv940TXdQh8A79elter3CnbX2qyYShHOTB A7xvOJR3sWpW2orcnPW33Gc= X-Google-Smtp-Source: ABdhPJyb/xprRDHysCbU37JR7nYRwXqDoASuLmGahSRhj5+kbbOH3OiDD5GE8nYe6+6+Zkl0ejJXNA== X-Received: by 2002:a17:907:1b1f:: with SMTP id mp31mr22400700ejc.247.1642580730215; Wed, 19 Jan 2022 00:25:30 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:29 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 20/25] trace-cmd library: Initialize CPU data for reading from version 7 trace files Date: Wed, 19 Jan 2022 10:25:02 +0200 Message-Id: <20220119082507.245600-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In version 7 trace files, CPU trace data is written in slightly different way than in version 6 files. Added new CPU data initialization flow, to handle version 7 files: - the top trace instance is saved in the same way as the other trace instances. - per CPU trace metadata is stored in the buffer option. - trace data section has section header. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 178 +++++++++++++++++++++++++----------- 1 file changed, 123 insertions(+), 55 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 7f418a44..103ff7bf 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3214,34 +3214,18 @@ static int read_options_type(struct tracecmd_input *handle) return 0; } -static int read_cpu_data(struct tracecmd_input *handle) +static int init_cpu_data(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; enum kbuffer_long_size long_size; enum kbuffer_endian endian; - unsigned long long size; unsigned long long max_size = 0; unsigned long long pages; - int cpus; int cpu; - /* - * Check if this is a latency report or not. - */ - if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) - return 1; - /* We expect this to be flyrecord */ if (handle->file_state != TRACECMD_FILE_CPU_FLYRECORD) return -1; - cpus = handle->cpus; - - handle->cpu_data = malloc(sizeof(*handle->cpu_data) * handle->cpus); - if (!handle->cpu_data) - return -1; - memset(handle->cpu_data, 0, sizeof(*handle->cpu_data) * handle->cpus); - if (force_read) handle->read_page = true; @@ -3256,32 +3240,14 @@ static int read_cpu_data(struct tracecmd_input *handle) endian = KBUFFER_ENDIAN_LITTLE; for (cpu = 0; cpu < handle->cpus; cpu++) { - unsigned long long offset; - - handle->cpu_data[cpu].cpu = cpu; - handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; - if (tep_is_old_format(pevent)) + if (tep_is_old_format(handle->pevent)) kbuffer_set_old_format(handle->cpu_data[cpu].kbuf); - read8(handle, &offset); - read8(handle, &size); - - handle->cpu_data[cpu].file_offset = offset; - handle->cpu_data[cpu].file_size = size; - if (size > max_size) - max_size = size; - - if (size && (offset + size > handle->total_file_size)) { - /* this happens if the file got truncated */ - printf("File possibly truncated. " - "Need at least %llu, but file size is %zu.\n", - offset + size, handle->total_file_size); - errno = EINVAL; - goto out_free; - } + if (handle->cpu_data[cpu].file_size > max_size) + max_size = handle->cpu_data[cpu].file_size; } /* Calculate about a meg of pages for buffering */ @@ -3299,6 +3265,101 @@ static int read_cpu_data(struct tracecmd_input *handle) goto out_free; } + return 0; + + out_free: + for ( ; cpu >= 0; cpu--) { + free_page(handle, cpu); + kbuffer_free(handle->cpu_data[cpu].kbuf); + handle->cpu_data[cpu].kbuf = NULL; + } + return -1; +} + +static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buffer_instance *buffer) +{ + unsigned long long offset; + unsigned long long size; + unsigned short id, flags; + int cpu; + + if (handle->cpu_data) + return -1; + + if (lseek64(handle->fd, buffer->offset, SEEK_SET) == (off_t)-1) + return -1; + if (read_section_header(handle, &id, &flags, NULL, NULL)) + return -1; + + handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; + handle->cpus = buffer->cpus; + if (handle->max_cpu < handle->cpus) + handle->max_cpu = handle->cpus; + + handle->cpu_data = calloc(handle->cpus, sizeof(*handle->cpu_data)); + if (!handle->cpu_data) + return -1; + + for (cpu = 0; cpu < handle->cpus; cpu++) { + handle->cpu_data[cpu].cpu = buffer->cpu_data[cpu].cpu; + offset = buffer->cpu_data[cpu].offset; + size = buffer->cpu_data[cpu].size; + handle->cpu_data[cpu].file_offset = offset; + handle->cpu_data[cpu].file_size = size; + if (size && (offset + size > handle->total_file_size)) { + /* this happens if the file got truncated */ + printf("File possibly truncated. " + "Need at least %llu, but file size is %zu.\n", + offset + size, handle->total_file_size); + errno = EINVAL; + return -1; + } + } + + return init_cpu_data(handle); +} + +static int read_cpu_data(struct tracecmd_input *handle) +{ + unsigned long long size; + int cpus; + int cpu; + + /* + * Check if this is a latency report or not. + */ + if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) + return 1; + + /* We expect this to be flyrecord */ + if (handle->file_state != TRACECMD_FILE_CPU_FLYRECORD) + return -1; + + cpus = handle->cpus; + + handle->cpu_data = malloc(sizeof(*handle->cpu_data) * handle->cpus); + if (!handle->cpu_data) + return -1; + memset(handle->cpu_data, 0, sizeof(*handle->cpu_data) * handle->cpus); + + for (cpu = 0; cpu < handle->cpus; cpu++) { + unsigned long long offset; + + handle->cpu_data[cpu].cpu = cpu; + read8(handle, &offset); + read8(handle, &size); + handle->cpu_data[cpu].file_offset = offset; + handle->cpu_data[cpu].file_size = size; + if (size && (offset + size > handle->total_file_size)) { + /* this happens if the file got truncated */ + printf("File possibly truncated. " + "Need at least %llu, but file size is %zu.\n", + offset + size, handle->total_file_size); + errno = EINVAL; + return -1; + } + } + /* * It is possible that an option changed the number of CPUs. * If that happened, then there's "empty" cpu data saved for @@ -3318,15 +3379,7 @@ static int read_cpu_data(struct tracecmd_input *handle) } } - return 0; - - out_free: - for ( ; cpu >= 0; cpu--) { - free_page(handle, cpu); - kbuffer_free(handle->cpu_data[cpu].kbuf); - handle->cpu_data[cpu].kbuf = NULL; - } - return -1; + return init_cpu_data(handle); } static int read_data_and_size(struct tracecmd_input *handle, @@ -3424,14 +3477,7 @@ static int read_and_parse_trace_clock(struct tracecmd_input *handle, return 0; } -/** - * tracecmd_init_data - prepare reading the data from trace.dat - * @handle: input handle for the trace.dat file - * - * This prepares reading the data from trace.dat. This is called - * after tracecmd_read_headers() and before tracecmd_read_data(). - */ -int tracecmd_init_data(struct tracecmd_input *handle) +static int init_data_v6(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; int ret; @@ -3453,7 +3499,29 @@ int tracecmd_init_data(struct tracecmd_input *handle) tracecmd_parse_trace_clock(handle, clock, 8); } } + return ret; +} + +static int init_data(struct tracecmd_input *handle) +{ + return init_buffer_cpu_data(handle, &handle->top_buffer); +} +/** + * tracecmd_init_data - prepare reading the data from trace.dat + * @handle: input handle for the trace.dat file + * + * This prepares reading the data from trace.dat. This is called + * after tracecmd_read_headers() and before tracecmd_read_data(). + */ +int tracecmd_init_data(struct tracecmd_input *handle) +{ + int ret; + + if (!HAS_SECTIONS(handle)) + ret = init_data_v6(handle); + else + ret = init_data(handle); tracecmd_blk_hack(handle); return ret; From patchwork Wed Jan 19 08:25:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D569CC43217 for ; Wed, 19 Jan 2022 08:25:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352407AbiASIZe (ORCPT ); Wed, 19 Jan 2022 03:25:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352405AbiASIZd (ORCPT ); Wed, 19 Jan 2022 03:25:33 -0500 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7F06C061574 for ; Wed, 19 Jan 2022 00:25:32 -0800 (PST) Received: by mail-ed1-x52f.google.com with SMTP id a18so7549583edj.7 for ; Wed, 19 Jan 2022 00:25:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8XtAVvd5pbChMCkMkMESo1+yjK+cNkhfagDnRZi2opM=; b=a/OvHkAdUUmz/hNOIDayRPStJR69eTVxN0tVFRExg/VOD340/lXlPsyWc2DJyzaRcG AloVaHkzS47w2YUI2gxMtlkc6+eDbD1puJfN8BnFGxbgKwujO95UYg5hHMe01JL3nkEs cb7LzUYtarvE7k9neASL1htv6csvPMbFcrP92k5h6qH+G7dd93rxzmXOdViZU7wiwyuF PRp9KTxjlKrn6tQv/px+OWavS/JUr0+Chtz7AqSKEkPo0YOtcRgdaJO6WA9vX+CwTFcx JkiVL4X/YnHPvXC1RVgLIPyl9UVTNFj82zpiv9//1gUrTc0mVaNU1eEV6jJSRN9TYMV7 /eRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8XtAVvd5pbChMCkMkMESo1+yjK+cNkhfagDnRZi2opM=; b=MqiIO4r27jPXQumsrfnVBrOvZHPac8fnpqo6wNHTBkurEuuV/bTgRBbYqGR4et+0cR 0I7btLnT+4yIU5of7LqEi9npcqOVRSK4HN00GCyrY49BX2nfwX4dHD40/JitQs6+4klZ cIv0t7jKTF+gdW11A21KA3He9nAMx6IMkgl6S9lB/gCP8acFcCeDqu+MQWuy2kJGswBa XD0qf05Ys0fEHQ9opR3q6DeetgWOCLJ9gIoCXTxvzBCyu0Z1/SfGVZeZ8rdrUtF6uT0T Dvck50tZlFa+iSExcJSBD1qvHwIc2Xkwe9PTYvbBpKE5lgPHQQDYchA9dUp52hJ1Fslm O7pg== X-Gm-Message-State: AOAM5324DPOo8Y2L1S9V9MGnIBk8YbMtVf6PLJij692zsFCArqZb6ZWc YLUQ+pffcgPVuhnvXl3FVwhfvOWBpN4= X-Google-Smtp-Source: ABdhPJxYzcoC/6igFrJh6x44ND1eG9w9n72bIjTddetaFIiiWOoWcbwyEdDYoCOHdgpDUNKfQO+Tnw== X-Received: by 2002:a50:ce4a:: with SMTP id k10mr29586648edj.31.1642580731440; Wed, 19 Jan 2022 00:25:31 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:30 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 21/25] trace-cmd library: Handle latency trace in version 7 files Date: Wed, 19 Jan 2022 10:25:03 +0200 Message-Id: <20220119082507.245600-22-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Latency trace data is saved the same was as flyrecord buffer data in trace files version 7. There is a BUFFER_TEXT option which holds the latency specific trace metadata and points to the section in the file with the trace data. A new API is added to read latency data: tracecmd_latency_data_read() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 +++- lib/trace-cmd/trace-input.c | 29 ++++++++++++++++++- lib/trace-cmd/trace-output.c | 5 +++- tracecmd/trace-record.c | 3 +- 4 files changed, 38 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 94b33949..1efafba1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -227,6 +227,8 @@ tracecmd_peek_data_ref(struct tracecmd_input *handle, int cpu) return rec; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, char **buf, size_t *size); + struct tep_record * tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record); @@ -301,7 +303,8 @@ int tracecmd_output_write_headers(struct tracecmd_output *handle, struct tracecmd_output *tracecmd_output_create(const char *output_file); struct tracecmd_output *tracecmd_output_create_fd(int fd); -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + int file_version); struct tracecmd_option *tracecmd_add_option(struct tracecmd_output *handle, unsigned short id, int size, diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 103ff7bf..6f540c90 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3214,6 +3214,24 @@ static int read_options_type(struct tracecmd_input *handle) return 0; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, char **buf, size_t *size) +{ + if (!handle || !buf || !size) + return -1; + if (handle->file_state != TRACECMD_FILE_CPU_LATENCY) + return -1; + + /* Read data from a file */ + if (!(*buf)) { + *size = BUFSIZ; + *buf = malloc(*size); + if (!(*buf)) + return -1; + } + + return do_read(handle, *buf, *size); +} + static int init_cpu_data(struct tracecmd_input *handle) { enum kbuffer_long_size long_size; @@ -3276,6 +3294,12 @@ static int init_cpu_data(struct tracecmd_input *handle) return -1; } +int init_latency_data(struct tracecmd_input *handle) +{ + /* To do */ + return 0; +} + static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buffer_instance *buffer) { unsigned long long offset; @@ -3290,7 +3314,10 @@ static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buff return -1; if (read_section_header(handle, &id, &flags, NULL, NULL)) return -1; - + if (buffer->latency) { + handle->file_state = TRACECMD_FILE_CPU_LATENCY; + return init_latency_data(handle) == 0 ? 1 : -1; + } handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; handle->cpus = buffer->cpus; if (handle->max_cpu < handle->cpus) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index fe4fcf64..15baef8f 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1890,7 +1890,8 @@ out_add_buffer_option(struct tracecmd_output *handle, const char *name, return option; } -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + int file_version) { enum tracecmd_section_flags flags = 0; struct tracecmd_output *handle; @@ -1901,6 +1902,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (!handle) return NULL; + if (file_version && tracecmd_output_set_version(handle, file_version)) + goto out_free; if (tracecmd_output_write_headers(handle, NULL)) goto out_free; /* diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index f599610e..2dea79dc 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4501,7 +4501,8 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count); + handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, + ctx->file_version); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) From patchwork Wed Jan 19 08:25:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BEA5C433FE for ; Wed, 19 Jan 2022 08:25:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352405AbiASIZe (ORCPT ); Wed, 19 Jan 2022 03:25:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352406AbiASIZe (ORCPT ); Wed, 19 Jan 2022 03:25:34 -0500 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE1A9C06161C for ; Wed, 19 Jan 2022 00:25:33 -0800 (PST) Received: by mail-ed1-x532.google.com with SMTP id m11so7404855edi.13 for ; Wed, 19 Jan 2022 00:25:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cKB2R3K8A3E5rbF1P/bWmGpEwbkcmZTpoDw4W+286aQ=; b=aTlr6NOWQlZ3tNBBtgy5H4ya6Sk+OaFU7zIVeW/839vqmgzChH4fUiC9BBkmW88Djm Y59rItqvX+uz4kW65IwEvPYzGqZp0K4RHiEXQJDBUCXJfiHw7SDJwzFOULdc3YkPaWwe lpnApLTtrkFocOIiQ5Rd94Z3YCmB175L1THxXtayv2gJI0ztKa7N+v+IChZUGg2NO1rV K34hyKxk49zzTtkYxacIDrdH0qRouMhhzK0r5xiWCx/AH6LMHgBoToBsSTr3yanRsnAK jL7yv07k841+FMkM5cTpihRoWqrOi4CqlXfyJEPKtFwah9RYMlBwDvS1Wa01yX9vNakR TOtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cKB2R3K8A3E5rbF1P/bWmGpEwbkcmZTpoDw4W+286aQ=; b=7eLAQMNFtA93Py+RtujBhtb2t98S2dA5dTXJWGqH6AYNYMqBeXRH4a6UGR4KyfPpry DJSeVzyZDFaxG6xwt38VQ05Uftwqi3MRY7IfAlMLM+3uA2nslt6xeiG8yOint+X8uAcO ZKGRSTN/3sFcAfPhLxnmhxgp6vsQZ49Rw5iQsNjjNWmMgvrvtzUaE1OfWHWc6n/rDokX 6LAOjy3jatsJF03tCFXHYZ2L6Bk87vE3mPbMG7IXnINDnJsi5CRds5ZM7HiH55z9IoJj u4g7AoptMsdgw+gCLg42uPlgsNg2h4JIDsu5g0i5qd/xfAbyAETMokl1eWE3JCWXESZW 6J5Q== X-Gm-Message-State: AOAM530SKqkJE3QT+iTwDEJyOMouF93M6ob+XAd3n3TjXsserz4TxSJ7 RvaQP8RoJqRomw/bn14BMb6HSu2Lv3k= X-Google-Smtp-Source: ABdhPJwWdLLIxKV8eJm3qbKN/QthSStLzDhemK//aP0i18pYLpGBbyS4/zTo5Ot3rjK4ghBAs8FX/A== X-Received: by 2002:a17:907:9687:: with SMTP id hd7mr23283173ejc.12.1642580732428; Wed, 19 Jan 2022 00:25:32 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:31 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 22/25] trace-cmd library: Handle buffer trace data init for version 7 files Date: Wed, 19 Jan 2022 10:25:04 +0200 Message-Id: <20220119082507.245600-23-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org CPU data initialization is different for trace files version 6 and 7. When a new input handler to trace buffer is created, initialize the CPU data according to the file version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 55 +++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 26 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 6f540c90..ecb56826 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4534,34 +4534,37 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->flags |= TRACECMD_FL_BUFFER_INSTANCE; new_handle->pid_maps = NULL; + if (!HAS_SECTIONS(handle)) { + /* Save where we currently are */ + offset = lseek64(handle->fd, 0, SEEK_CUR); - /* Save where we currently are */ - offset = lseek64(handle->fd, 0, SEEK_CUR); - - ret = lseek64(handle->fd, buffer->offset, SEEK_SET); - if (ret < 0) { - tracecmd_warning("could not seek to buffer %s offset %ld\n", - buffer->name, buffer->offset); - goto error; - } - - /* - * read_options_type() is called right after the CPU count so update - * file state accordingly. - */ - new_handle->file_state = TRACECMD_FILE_CPU_COUNT; - ret = read_options_type(new_handle); - if (!ret) - ret = read_cpu_data(new_handle); - if (ret < 0) { - tracecmd_warning("failed to read sub buffer %s\n", buffer->name); - goto error; - } + ret = lseek64(handle->fd, buffer->offset, SEEK_SET); + if (ret == (off64_t)-1) { + tracecmd_warning("could not seek to buffer %s offset %ld\n", + buffer->name, buffer->offset); + goto error; + } + /* + * read_options_type() is called right after the CPU count so update + * file state accordingly. + */ + new_handle->file_state = TRACECMD_FILE_CPU_COUNT; + ret = read_options_type(new_handle); + if (!ret) + ret = read_cpu_data(new_handle); - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret < 0) { - tracecmd_warning("could not seek to back to offset %ld\n", offset); - goto error; + if (ret < 0) { + tracecmd_warning("failed to read sub buffer %s\n", buffer->name); + goto error; + } + ret = lseek64(handle->fd, offset, SEEK_SET); + if (ret < 0) { + tracecmd_warning("could not seek to back to offset %ld\n", offset); + goto error; + } + } else { + if (init_buffer_cpu_data(new_handle, buffer) < 0) + goto error; } return new_handle; From patchwork Wed Jan 19 08:25:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAC17C433F5 for ; Wed, 19 Jan 2022 08:25:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352410AbiASIZg (ORCPT ); Wed, 19 Jan 2022 03:25:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352406AbiASIZf (ORCPT ); Wed, 19 Jan 2022 03:25:35 -0500 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD627C061574 for ; Wed, 19 Jan 2022 00:25:34 -0800 (PST) Received: by mail-ed1-x533.google.com with SMTP id b13so7811652edn.0 for ; Wed, 19 Jan 2022 00:25:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MrWAMP2oQHHiOnTleQUOJe78e7qQgzHgXpbKW6Vk8cU=; b=AVLbp8OMAQj/8ROn0q1KvJeZBMCJ1iBzkWfTOaC80/9uBKL1j4diaD3Mchws5HecIn VAreJSvNlLwVuApZw1I6wQoBC4GL0YMO2qOa4Kfmr3zxmxFWmVVCDwQft+0bwuEYDxZR DV/0bX3loqKOLipMyTlk1Nl+kPVIUrg1E9tgyPLn5/qxzqQxgGg6wwdeuBu5xs2iw2uz hZPrx4Tx6t3QXmyArgG+BgTMzGLu/h9/J4y8g5CBOW8+lki9hK+XbcYFQ1jC5sRWcPlU N9HQEG6g2f5OdOv+mKMr0zLX6xGynPXYIWkvkc9w1ZYLchsSqO/TPIGx99sllqBxragZ zSjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MrWAMP2oQHHiOnTleQUOJe78e7qQgzHgXpbKW6Vk8cU=; b=LyX618sX9f3ypZIngIBu/c5B7yt0sD7H2aoPvDAC9SIPzr2v2Hn/8bENNj1J1uo1jJ m0/uDQhjCYu176x+rYpjrYDNpV4fpP4jT+g16XbULZM/d8BmHLkqKpRxRRcy2TPkTdJ4 d+reAsQAW1/uF7ANKNJfmd6DD4v9s1hignXjfknXgUq4xF5yAH5mcmeDGwoyGLLQ2T3U 25uv4P5+xlGUt5z4dfA8D0/Io3cACn57VFmkEwKea4ev63jvxteVbPfszbwackCVB/Rq bwqZFOsHvqprZ1Sb/ZmQg+CM+v9HqGjoE27AFML6QHLtvFNglzJJNbFF6jr8qPbKXPW+ BZQg== X-Gm-Message-State: AOAM531pafO0gtSxDrldyEZXVW5xtxgeYlQP4b4G9FptTYPhmyfc8k9d wBie6bNVa6wGw4R7zD/jWEjheJTLepo= X-Google-Smtp-Source: ABdhPJwR6a6Rdsur0LRRKAJZuSF75eZsgN59zurOk6w+1/Zoy1/yfU75l0kWwVgsQPBGcfFlDAo28w== X-Received: by 2002:a17:906:3103:: with SMTP id 3mr21271527ejx.289.1642580733465; Wed, 19 Jan 2022 00:25:33 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:33 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 23/25] trace-cmd report: Use the new latency API to read data Date: Wed, 19 Jan 2022 10:25:05 +0200 Message-Id: <20220119082507.245600-24-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading latency trace data, use the new API tracecmd_latency_data_read() It handles reading latency trace data from both version 6 and 7 trace files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-read.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 4b27740a..24177c44 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -951,18 +951,20 @@ void trace_show_data(struct tracecmd_input *handle, struct tep_record *record) printf("\n"); } -static void read_rest(void) +static void read_latency(struct tracecmd_input *handle) { - char buf[BUFSIZ + 1]; + char *buf = NULL; + size_t size = 0; int r; do { - r = read(input_fd, buf, BUFSIZ); - if (r > 0) { - buf[r] = 0; - printf("%s", buf); - } + r = tracecmd_latency_data_read(handle, &buf, &size); + if (r > 0) + printf("%.*s", r, buf); } while (r > 0); + + printf("\n"); + free(buf); } static int @@ -1246,7 +1248,7 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype if (ret > 0) { if (multi_inputs) die("latency traces do not work with multiple inputs"); - read_rest(); + read_latency(handles->handle); return; } From patchwork Wed Jan 19 08:25:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A8D6C433FE for ; Wed, 19 Jan 2022 08:25:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352406AbiASIZg (ORCPT ); Wed, 19 Jan 2022 03:25:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352411AbiASIZg (ORCPT ); Wed, 19 Jan 2022 03:25:36 -0500 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC6B1C06161C for ; Wed, 19 Jan 2022 00:25:35 -0800 (PST) Received: by mail-ed1-x52a.google.com with SMTP id j23so2431513edp.5 for ; Wed, 19 Jan 2022 00:25:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SUbztSKT+SLSn9t/AeyyCMEBQgpxxCttBQHltz2dzwg=; b=KjesvOkDP0Tpm+Ag4r6B+xRHUz6fUoVRxT6QI1dp4jAaWjPnEtmIrTvQzOUrNTcR6i kIeV7SwXuXQhNtlgM0NolD6WLXtWbHhlKOd/XgKC2bsJfBQC/6p39VtyHcNl0qlLvBbK Bd6nie/u2s1695RDLzosscCDe8LLq7zntQ2L6gzHC2AHqGVL3T3DojPtPAL1Lo+6c0TQ 66o7SmiXQM4Fm+0ur2WoiP00ldx6HCsrZf16VwHDRrZWykBkPoL46SjzqyZA20Pwc23g JXU349gT9jDhbNlu3T7KWNtEWmJOfxBDrMocAsSIi6mYvpm8acbikwn0coDFt6tOqJbH nCbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SUbztSKT+SLSn9t/AeyyCMEBQgpxxCttBQHltz2dzwg=; b=ga0vXUW8Gm5mEKKy6/21AZYOUr/2Jm5tO+aw+wbxpp2SQzhB89MlhY7omqNlP7G69h F7JW0Wbtv+2EMFlvKqOyjGNAiSBxKY2JvwHvz0pbfOFXtOxzvvz5iVyTNV6hGYzE6m3r GTK2AsF4BvFGqCb62HfTnrH4zzMi2t40EgHv6Cf65Jv5LKTnjuEVM4qFhaO2Tc7bB6/N IFYPOK/elJFG1wn3GgJ2C0SxUhRAl609X9MJ8ukPzGX9FVUj+PcaUxiLMUzM6n38uAG/ Snl1D5fMZaGu4XSEdsj4pZK7MDCnI8SvY/FdvlUJrCiEuihJeIB1e5GKk6OGOpLJSJU3 dyTA== X-Gm-Message-State: AOAM533t9K+fycyVTDKZ3D940v8FFEn6K7rqCBxLOB4+oX7wnc9DQVqo pXDlenISWkOtRwIUBvpl5Uwu1VbisDo= X-Google-Smtp-Source: ABdhPJzHyVulAqm/1Xzx9zIprWXSO18sKo9htqrffJ1TgL0/Y7zIP0mv12E8AzknxlaHaUwK08PyLQ== X-Received: by 2002:a17:906:14d8:: with SMTP id y24mr3805828ejc.62.1642580734460; Wed, 19 Jan 2022 00:25:34 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:34 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 24/25] trace-cmd: Write buffers metadata in trace files version 6 Date: Wed, 19 Jan 2022 10:25:06 +0200 Message-Id: <20220119082507.245600-25-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a trace file, tracecmd_write_buffer_info() should be called, to write buffers metadata in file version 6 trace files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-listen.c | 3 +++ tracecmd/trace-record.c | 4 ++++ 2 files changed, 7 insertions(+) diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index 45ba1211..874ecd61 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -604,6 +604,9 @@ static int put_together_file(int cpus, int ofd, const char *node, if (write_options) { ret = tracecmd_write_cpus(handle, cpus); + if (ret) + goto out; + ret = tracecmd_write_buffer_info(handle); if (ret) goto out; ret = tracecmd_write_options(handle); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 2dea79dc..8567ae5c 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3729,6 +3729,9 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; ret = tracecmd_write_cpus(network_handle, instance->cpu_count); + if (ret) + goto error; + ret = tracecmd_write_buffer_info(network_handle); if (ret) goto error; ret = tracecmd_write_options(network_handle); @@ -4092,6 +4095,7 @@ static void setup_agent(struct buffer_instance *instance, add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_buffer_info(network_handle); tracecmd_write_options(network_handle); tracecmd_write_meta_strings(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); From patchwork Wed Jan 19 08:25:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12717230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 331E5C433EF for ; Wed, 19 Jan 2022 08:25:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352411AbiASIZh (ORCPT ); Wed, 19 Jan 2022 03:25:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352408AbiASIZh (ORCPT ); Wed, 19 Jan 2022 03:25:37 -0500 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3B9EC061574 for ; Wed, 19 Jan 2022 00:25:36 -0800 (PST) Received: by mail-ed1-x52c.google.com with SMTP id m4so7481745edb.10 for ; Wed, 19 Jan 2022 00:25:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PKE/iqsNAP0FZpe28XX1pe1Bkh82bLkG/AT8vfLqXfM=; b=QMqvzmn1eEQwq9uZ+b3Y/W+CSmBTB8TjrPJ8GEWwsE64PCG2YhXk5KcFkGcU2aw08q RO1ZgsHiR2bRvpYO5cYGCfTgLw26d0waBZJ+FggaJMeLNmC5zRTdqPAIZ7M7JV8knY+S VkrbW2czw53f8TODZotS4fLJWc81OVUe6FEc8sp4Xh+MJmTJGIDGGinWtFjLCcmItEwQ lwjQrsgx0MK6+N4O10+nJcHvaysyY4aVUHKsM23Osqpgt3/C1TIfQ92vVB/T1+LhPT9n u5Q7Mimc95p1XTYbLJrlPJsZ3AmOIoc0iQdTydbSd+2zaFVSP/LEhL51nB0gcbRCt1x6 fWqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PKE/iqsNAP0FZpe28XX1pe1Bkh82bLkG/AT8vfLqXfM=; b=NGETHWgmWW/k90KWcRESHkFI5C/AuTQHL2/mqcGOK353gPbRWXPFpzUfqiLAxl/fjY x4zP2leFvWBeL+LyqfU1mXYmUEPkVWcldrH1hHF4RFgZyUmwlEzlctUy8lWVGJMAhHFt XQKci9xWBzzzN796Ia030MwMt02tfsWI+q1QzgIPvwTGcDQvUuac1YoIE4GJmdRIAL7h 0JPCccwTotxbKIe5xYh1OwV0fMvNIM/d2F94qdlMxytJNmux3fjW5r5BdeVI6lDALtji 8fKQOTqLH/FtmYyrfZ+S3s+t1+8BX/Mm0WkXXaADElR1Jb542BCpKvtWD4y/qy4ebJ6p LegA== X-Gm-Message-State: AOAM533egX2pFcbzVO6Efq1K6wNnrLMfa34LRwMevx9HoXE8vtePjmDx NdiLbGWopiWbMrRJkONsRh3TmJAamh4= X-Google-Smtp-Source: ABdhPJxNxNTo06xCc5k/KpW8rSKUm03AupLGRnby/Kmx+cEFzP6WcBzB5tIY7V7Cke6XxpF5HeMX2w== X-Received: by 2002:a17:906:4fcc:: with SMTP id i12mr23897772ejw.529.1642580735537; Wed, 19 Jan 2022 00:25:35 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id l1sm5011122ejf.44.2022.01.19.00.25.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Jan 2022 00:25:35 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v8 25/25] trace-cmd record: Add new parameter for trace file version Date: Wed, 19 Jan 2022 10:25:07 +0200 Message-Id: <20220119082507.245600-26-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220119082507.245600-1-tz.stoyanov@gmail.com> References: <20220119082507.245600-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org By default, "trace-cmd record" writes in trace file version 6. A new parameter is added, which can be used to set desired version the output trace file. "trace-cmd record --file-version " Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 20 +++++++++++++++++++- tracecmd/trace-usage.c | 2 +- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 8567ae5c..4cf48c86 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3698,6 +3698,8 @@ static struct tracecmd_output *create_net_output(struct common_record_context *c out = tracecmd_output_create(NULL); if (!out) return NULL; + if (ctx->file_version && tracecmd_output_set_version(out, ctx->file_version)) + goto error; if (tracecmd_output_set_msg(out, msg_handle)) goto error; if (tracecmd_output_write_headers(out, listed_events)) @@ -3744,6 +3746,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context network_handle = tracecmd_output_create_fd(msg_handle->fd); if (!network_handle) goto error; + if (tracecmd_output_set_version(network_handle, ctx->file_version)) + goto error; if (tracecmd_output_write_headers(network_handle, listed_events)) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4471,7 +4475,8 @@ static struct tracecmd_output *create_output(struct common_record_context *ctx) out = tracecmd_output_create(ctx->output); if (!out) goto error; - + if (ctx->file_version && tracecmd_output_set_version(out, ctx->file_version)) + goto error; if (tracecmd_output_write_headers(out, listed_events)) goto error; @@ -5780,6 +5785,7 @@ void init_top_instance(void) } enum { + OPT_file_ver = 238, OPT_verbose = 239, OPT_tsc2nsec = 240, OPT_fork = 241, @@ -6219,6 +6225,7 @@ static void parse_record_options(int argc, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, {"poll", no_argument, NULL, OPT_poll}, {"verbose", optional_argument, NULL, OPT_verbose}, + {"file-version", required_argument, NULL, OPT_file_ver}, {NULL, 0, NULL, 0} }; @@ -6644,6 +6651,17 @@ static void parse_record_options(int argc, cmd_check_die(ctx, CMD_set, *(argv+1), "--poll"); recorder_flags |= TRACECMD_RECORD_POLL; break; + case OPT_file_ver: + if (ctx->curr_cmd != CMD_record && ctx->curr_cmd != CMD_record_agent) + die("--file_version has no effect with the command %s\n", + *(argv+1)); + ctx->file_version = atoi(optarg); + if (ctx->file_version < FILE_VERSION_MIN || + ctx->file_version > FILE_VERSION_MAX) + die("Unsupported file version %d, " + "supported versions are from %d to %d", + ctx->file_version, FILE_VERSION_MIN, FILE_VERSION_MAX); + break; case OPT_quiet: case 'q': quiet = true; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 32b38bfd..ac12b066 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -69,7 +69,7 @@ static struct usage_help usage_help[] = { " If 0 is specified, no loop is performed - timestamps offset is calculated only twice," " at the beginnig and at the end of the trace\n" " --poll don't block while reading from the trace buffer\n" - " --verbose 'level' Set the desired log level\n" + " --file-version set the desired trace file version\n" }, { "set",