From patchwork Thu Jul 29 05:08:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9912C432BE for ; Thu, 29 Jul 2021 05:10:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A978061052 for ; Thu, 29 Jul 2021 05:10:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229957AbhG2FKI (ORCPT ); Thu, 29 Jul 2021 01:10:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbhG2FKH (ORCPT ); Thu, 29 Jul 2021 01:10:07 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D29EC061757 for ; Wed, 28 Jul 2021 22:10:04 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id ec13so5879932edb.0 for ; Wed, 28 Jul 2021 22:10:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q/XpTU2XhoHbHAqoA8x/vtqptp62mgjiQRNtFM2Rvqo=; b=KLFswPE55De8xDRScz1m3G054zJpf9P4qxDQQkqwkL573VLgd2WMwoJ+VsOdqLK/nE w/4+rQmJTMgb774cxN/s5Rvv39V9lzAxykgQHiKJ4c4KqZjpd4T5eO3xS0I9Qgni9fdO BHqxnrPzPPOifZjs715C18muUEoTb5oQ1cr8kIUNAKblir64ZDURUfKUY5JWPSoZVeIY 623EeLhGzgL3KNMpxH54hxjDLEOqsAjBnHO2z5CW27zKXouaqJlzoQCbEXGof+To824D KVKfr/8G4kSYCjv4R5oe+c9/7QpSs84T3M0eH+XT0KJAhC65eBLlfINMzeninV9oyLcc brdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q/XpTU2XhoHbHAqoA8x/vtqptp62mgjiQRNtFM2Rvqo=; b=a9l308VWMJLp8b2phLE2n4udz3lHi6Zs2MPbLAlYM2DZV2FcGCU+ZjTjJEKGyTUZn2 1+zkhuilNlzsDhg3JvXVg+NSVy5ZLgJA8FbukSZ0YC2f3TGOtnSHAuqT9wJWdLMJt9sy N2/FHmYXD161kVl7WgE8c+HS/BxSjrln7LhDtJY1iBB9MVbb5Jk7MyDl6YAU/v6qdDw6 lwWYto4A47cqGVTXfKK1KmDzRtL5GIOfPBCohScDtcYag67aV9J/qb1jvx6PkPI9S5h8 gggkBtugo8ImE9Z31FqMpkmk7f1WM5Dlez8l6FzJzQGWAbcXTmvy3DWcEzYJWNKTacsg 2J1Q== X-Gm-Message-State: AOAM532JTMRFgwveZqLVqLrj9vhWQ0LWbH6qz9imeb30caScGp7+38fC UxPWjEF3AcrcaonC36tdeyQ= X-Google-Smtp-Source: ABdhPJyr29cMWg+zeJ0UTjrKoWVNFRJZPGXxd8yrumCbOgksYxHi9axTgNAoVIxKj980FIw0OkUnnQ== X-Received: by 2002:a05:6402:896:: with SMTP id e22mr3997087edy.246.1627535402735; Wed, 28 Jul 2021 22:10:02 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:02 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 01/87] trace-cmd library: Read option id with correct endian Date: Thu, 29 Jul 2021 08:08:33 +0300 Message-Id: <20210729050959.12263-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The id of a trace option is 2 bytes short integer. When reading it from the trace file, use the tep handler associated with the file, if available, to convert the option with the correct endian order. A new helper function is introduced to read and convert 2 byte integer. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index ac57bc4f..0dbcdbdc 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -336,6 +336,18 @@ static char *read_string(struct tracecmd_input *handle) return NULL; } +static int read2(struct tracecmd_input *handle, unsigned short *size) +{ + struct tep_handle *pevent = handle->pevent; + unsigned short data; + + if (do_read_check(handle, &data, 2)) + return -1; + + *size = tep_read_number(pevent, &data, 2); + return 0; +} + static int read4(struct tracecmd_input *handle, unsigned int *size) { struct tep_handle *pevent = handle->pevent; @@ -2660,16 +2672,15 @@ static int handle_options(struct tracecmd_input *handle) handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); for (;;) { - if (do_read_check(handle, &option, 2)) + if (read2(handle, &option)) return -1; if (option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (do_read_check(handle, &size, 4)) + if (read4(handle, &size)) return -1; - size = tep_read_number(handle->pevent, &size, 4); buf = malloc(size); if (!buf) return -ENOMEM; From patchwork Thu Jul 29 05:08:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3A22C4338F for ; Thu, 29 Jul 2021 05:10:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9D15A61019 for ; Thu, 29 Jul 2021 05:10:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233700AbhG2FKJ (ORCPT ); Thu, 29 Jul 2021 01:10:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbhG2FKI (ORCPT ); Thu, 29 Jul 2021 01:10:08 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09E64C061765 for ; Wed, 28 Jul 2021 22:10:05 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id qk33so8384967ejc.12 for ; Wed, 28 Jul 2021 22:10:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rHEDizwZjviRdumQYx/LHW1ieeaoKb8L1up3BfvtlPc=; b=GlRHrB86tPTtt04xkGdqGyNcpHDEE8IM3snaWVu3pbpzM2XoK+ql8n1DqSfnUiruSZ YZnczuhWu7d52NbVAQQlK+hwlk+TuWL7wxDswUoOPlZJTljp5snd1KrDQmVSD3UhboVP XEjMjh3jjk9uq6vgHLjJpCTwUaW7/cQZGGWq81Zj4zV4ItVL5i7s4e3xeiy0aCl72QlY Tc2eXiyjuoGlquz6z0PRkq6nDNF/+T7PNub+jp+KMXGwuw7aQHNP9zQ7SDTO4GM1o8m0 IqXDUDofmzJjSm40KBWdkvu3EYEGbA2DhJHs1iKBpWPLt6ajkdesif+gKNQE3han64DQ j8HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rHEDizwZjviRdumQYx/LHW1ieeaoKb8L1up3BfvtlPc=; b=XiR678I47KHhHnlwm+UKo3tjL1Z/ct4Nz1+xcY7EOBNLtBlraNfWI24TNzWp8PhOvz 5hn1m3ag5GFGFpzI1pa+520xNQA+tzKEk2/7BjWdO5D4VwVaXAxJk8ajDiruGB6eoyrH JZdnxPwHUCAqsbH00rz4uhUuljqeRXrOSsdU77v9pxGJGpdxTfjdka6g/fwtkpbRl7vP JY9nXuX2fKhidBqF/ix/iwJrkPnocE3ouk0okH33RzfHMunuo/wJ8PMCA5pGDHHP5ixb b6SnzEZPGV0S5vJrTV05jZLXfK3hDu2IBOZ0O/6r+eJHBaZvu2lYAai7fwjQaY8WncJu 9M7Q== X-Gm-Message-State: AOAM530/QKdf3VxTTPu3QvSXUb7b1cNdNAhShpQ0Ix87pzMbQpyIzwCv iBWmSWOaU5UPUbYDFTkyzpA= X-Google-Smtp-Source: ABdhPJzpRo5rG+bfYcLhmJ/lCAWKgtZLBEdW34iYKD2Ad+90jnBHrJnAX12UedvsibI8PDNn3on0rw== X-Received: by 2002:a17:907:c08:: with SMTP id ga8mr2923027ejc.318.1627535403726; Wed, 28 Jul 2021 22:10:03 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:03 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 02/87] trace-cmd report: Fix typos in error messages Date: Thu, 29 Jul 2021 08:08:34 +0300 Message-Id: <20210729050959.12263-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Fixed typos in "trace-cmd report" messages that report reading and parsing errors. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-read.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 6f43c1d2..31724b09 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1290,7 +1290,7 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype die("error in reading buffer instance"); new_handle = tracecmd_buffer_instance_handle(handles->handle, i); if (!new_handle) { - warning("could not retreive handle %s", name); + warning("could not retrieve handle %s", name); continue; } add_handle(new_handle, name); @@ -1324,7 +1324,7 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype if (last_record) { int cpu = last_record->cpu; if (cpu >= last_handle->cpus) - die("cpu %d creater than %d\n", cpu, last_handle->cpus); + die("cpu %d greater than %d\n", cpu, last_handle->cpus); if (tscheck && last_handle->last_timestamp[cpu] > last_record->ts) { errno = 0; From patchwork Thu Jul 29 05:08:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12094C4320A for ; Thu, 29 Jul 2021 05:10:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAF5260FED for ; Thu, 29 Jul 2021 05:10:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231329AbhG2FKJ (ORCPT ); Thu, 29 Jul 2021 01:10:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233607AbhG2FKI (ORCPT ); Thu, 29 Jul 2021 01:10:08 -0400 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C2F5C061757 for ; Wed, 28 Jul 2021 22:10:06 -0700 (PDT) Received: by mail-ed1-x534.google.com with SMTP id da26so6429067edb.1 for ; Wed, 28 Jul 2021 22:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Syc19wre6CbFsYuYVxR6wCQV/KN3Z3g3cyWjdHJsI6Y=; b=IEXpJZqtx8/txjTKkDv831jvdcvaJHllokmy+eXvmv3bqQZ0+TNSLMQoEXlh5vz2Gn pONHNXO4iZot4WSomc83GOqXszqxUEQGoo/P5ljme69XG08w0moLF4wTD6DCEGXdKJJT hSJD1CSHTZ0rVUJrMz7KfI3k1LYg+I2ekxLhm5hpJYbEzdtMvzXEZyYvogPSFd0JA1Tz upJYTvAjKe4M2KxBD/ZBT9LiRGFkYvBRMIHvJElVOYe7UdSAnhJp/lAAFGdJqocQuAIv ujcxi62WJbyTCRqqf9KIPlBftYen9QjRLfMpxvwjSkKm1iva5eJvs8IgVBfz5FxPxHsy ubGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Syc19wre6CbFsYuYVxR6wCQV/KN3Z3g3cyWjdHJsI6Y=; b=bZREyDyicCSbVYc/z3oZITzH17ZdL5Sj6Hbok0TIbWBpoqfmQGBjfBPLdryxbjpLHk 1NJwZG/eSDiWIm/irSo+vc4GH+pLLQB+a/6crpDAIZ6GELR7WY2EJzJoCvj6C6XNwtuJ QkhL8D5RfOpzlujDxh3mTxtGEqNETO09QUE5KDaxVxmrH+EWPJppwHKyc5W7jNOfs3C3 jv3vVE5aqe2uJ5IfYVCyiDq6jiFMstLBpgkj1Ub+WK0VrBbnuRAUJPUU1AENvsSxqtjG gpFmHxntk+pSOOSWbn3Z7F00OHUKvKWM+ai+1JW5bsY2sUO9rLaAFsmVEX0LlGBprEQr vuOw== X-Gm-Message-State: AOAM5334fXb3FZ84XD7CqcBd1GYcGY3VpIwi5YMPensmSQhqetP9qJvp BgMORdl6ugOqa7iDCut+JSY= X-Google-Smtp-Source: ABdhPJwkim7LFOb/Jd9cUzuAypFNivcvd1SJzW5tEiTQixmCn6eFVhHUzdJDpYJ20hWgUqEiBzxITw== X-Received: by 2002:aa7:cc83:: with SMTP id p3mr3924256edt.365.1627535404684; Wed, 28 Jul 2021 22:10:04 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:04 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 03/87] tarce-cmd library: Fix version string memory leak Date: Thu, 29 Jul 2021 08:08:35 +0300 Message-Id: <20210729050959.12263-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The version string is allocated when a VERSION option is processed, but is never freed. Free it on input hanlder close. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 0dbcdbdc..9253bc37 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3518,6 +3518,7 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->cpu_data); free(handle->uname); free(handle->trace_clock); + free(handle->version); close(handle->fd); tracecmd_free_hooks(handle->hooks); @@ -3959,6 +3960,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->cpu_data = NULL; new_handle->nr_buffers = 0; new_handle->buffers = NULL; + new_handle->version = NULL; new_handle->ref = 1; if (handle->trace_clock) { new_handle->trace_clock = strdup(handle->trace_clock); From patchwork Thu Jul 29 05:08:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 658FBC4320E for ; Thu, 29 Jul 2021 05:10:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 404F661054 for ; Thu, 29 Jul 2021 05:10:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233607AbhG2FKK (ORCPT ); Thu, 29 Jul 2021 01:10:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233739AbhG2FKJ (ORCPT ); Thu, 29 Jul 2021 01:10:09 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F040CC061757 for ; Wed, 28 Jul 2021 22:10:06 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id nd39so8462272ejc.5 for ; Wed, 28 Jul 2021 22:10:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=otAzU6cU3J4e1+T0g9Y8ReUchZ4/dph0iJyK91g2foI=; b=ZBsUz07gVp6Q8gXB48FWQquoV8Jd+0RBekwtPc/kLKy0uppKYj1v5BepfCfo0Ba1Gy gaMLZZxboa88cw2/ezCeythLCkzLJbq/GTgJ+Ic8Ck5RnQDSIij3eRhsR/uPLtRwOoBY CvIerDApJjlzQKrkK1NfQpfoo/oyJ9K79QDxKcEl+oNGT91TrXiyJWy2CyyZK37/c66y rClLSH5OS0lPM+Qan/pbwFoTn/jVcIUtWcUgEwduZYh5CkzKerBG41kGYlldzQ2zNLhw Z790vRzBEjjC7YoFVPcDKpuO2wmH/5XzMBg3rtmgL22W/EHd+8+GABepfNy5xicJGEpu Ckng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=otAzU6cU3J4e1+T0g9Y8ReUchZ4/dph0iJyK91g2foI=; b=P2A/nkkdKQgrSujWomMvXmHnacf/60efAB9EOXbm1S+v0pJsIoGOcRRMaO4BYyOlZS qaHNnRCue6lkWPmfG+CP58L8F6utgmWjjWkg2MrKbde4C+cXAVQhiLWqcanBO+2VgJ6W HQIAmPosf+xk2DAV/YXwhgFz7FmsxO7m39QAAbHdMFEifACmUOh6H1SFMMfvpCx6bDVV DYorjgYvc5d6kO07YBq1zP4gBc+vb821NgCmhYuU0BEL6tFBhaGDz4n3IAKxgkjY1kvr bbCwG5l7z3mxLxI9DCM77RA0LdqybbOAuxi0jvHfb+rBIPDylDfS6XMLyAYUN+mCESsj ipAA== X-Gm-Message-State: AOAM530THmJ57rMW5IXoN48Bm5diDgFg/9PZYGUwrs6N+etov7dMocFF x1IkIwmKHzYJk9QihN/WEtMRc+P62Ocskg== X-Google-Smtp-Source: ABdhPJw0hAF06eeGbD9Cjq9aOxq1ujjNLMDU/OoX1RTQl0cNaNyScbB+nSbZRbiF3DLRuO4Vy9GuUg== X-Received: by 2002:a17:906:2bd3:: with SMTP id n19mr2960424ejg.232.1627535405630; Wed, 28 Jul 2021 22:10:05 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:05 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 04/87] trace-cmd library: Fixed a memory leak on input handler close Date: Thu, 29 Jul 2021 08:08:36 +0300 Message-Id: <20210729050959.12263-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When an input hanlder to a trace file is closed with tracecmd_close(), the list with buffers is not freed. This leads to a memory leak. Added logic to free that list. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 9253bc37..af11cbc6 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3483,7 +3483,7 @@ void tracecmd_ref(struct tracecmd_input *handle) */ void tracecmd_close(struct tracecmd_input *handle) { - int cpu; + int i; if (!handle) return; @@ -3496,21 +3496,21 @@ void tracecmd_close(struct tracecmd_input *handle) if (--handle->ref) return; - for (cpu = 0; cpu < handle->cpus; cpu++) { + for (i = 0; i < handle->cpus; i++) { /* The tracecmd_peek_data may have cached a record */ - free_next(handle, cpu); - free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) + free_next(handle, i); + free_page(handle, i); + if (handle->cpu_data && handle->cpu_data[i].kbuf) { + kbuffer_free(handle->cpu_data[i].kbuf); + if (handle->cpu_data[i].page_map) + free_page_map(handle->cpu_data[i].page_map); + + if (handle->cpu_data[i].page_cnt) tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + handle->cpu_data[i].page_cnt, i, + show_records(handle->cpu_data[i].pages, + handle->cpu_data[i].nr_pages)); + free(handle->cpu_data[i].pages); } } @@ -3521,6 +3521,11 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->version); close(handle->fd); + for (i = 0; i < handle->nr_buffers; i++) + free(handle->buffers[i].name); + + free(handle->buffers); + tracecmd_free_hooks(handle->hooks); handle->hooks = NULL; From patchwork Thu Jul 29 05:08:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 614B9C432BE for ; Thu, 29 Jul 2021 05:10:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4635B61053 for ; Thu, 29 Jul 2021 05:10:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233782AbhG2FKK (ORCPT ); Thu, 29 Jul 2021 01:10:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41268 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233739AbhG2FKK (ORCPT ); Thu, 29 Jul 2021 01:10:10 -0400 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB2A1C061757 for ; Wed, 28 Jul 2021 22:10:07 -0700 (PDT) Received: by mail-ej1-x62f.google.com with SMTP id ga41so8406945ejc.10 for ; Wed, 28 Jul 2021 22:10:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6ypas6iex836tTwseQtXfnzbSaoAb7AFc6+3qdD4/Bs=; b=FUw0LqPKlmG4rN+P4EO8iuz1HyL9NJ/1yJS7MoGIVDo3ShFEfvmZ4g/VKgrzFjgBUC V9yBA0v+VLconscgcsajr08CVfrf2VsLL3foH03EzB3Tp6er6xy2rSm74Ku0zmsCSz6k QsyIRB8EdyGscPNHgh3DfuMjSY/bAJdMXbtg7TR53H5zB4pnDPmJE5JKvrCmQjCjhYJa mbYD9GR69+NDvAs+8q3437AHvWqvviXsa9P8Ysn0+3hKSLC8AK/LJcKsAmljyC9JR/ku sSvuc3oZE6k42+BkMazIVpTHK0W3AEg/cETA/2UEHKyxJh8782ztUu9uX2TJ6NxiGCDq s9Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6ypas6iex836tTwseQtXfnzbSaoAb7AFc6+3qdD4/Bs=; b=UGwMfnxF6tJUrI3kJiIKEQI2MTQqX4jQggretVNd4/rHWr9CsiIIsNCY9tA8PHRKoS OdRHS5MjDpbLLXk0kueGaKeIECti/fzJPfy21Rf2sumDHKoe0on5KOWbmuPHvzJ3nCvs osMJVgUHGV2BBRKgxvVOeibzykuvyIVGkjyfjBLxNaZAx/C6j+43oQNx0QbzAfWdq935 av775+s1ebN6Ea52ErsHxI2mwU7R2+JvhcyTDl3T874XYoZVh7k9AfFcWbwFxoI8z5J8 AHMdtQwtLDhZXt4u+m+gcG4R29GTM0IvuCfv6rbdSFfc6/1dKyWu/vPE6B0sNOE+B858 oYgA== X-Gm-Message-State: AOAM530vtlnFNbD4zi9yC1/edCd1B6RPtYPswK0ynpruVg2DSo3FwAJj IIFpA2rDFgvkY/L5yE4s6sM= X-Google-Smtp-Source: ABdhPJwGb/ZVJfScGMLLwejKhfSbgkWyI2x0ip1oSvpZnSXAgkds8NLb7z4OoJZrn/4PCGMnwDmfIQ== X-Received: by 2002:a17:906:a195:: with SMTP id s21mr2850361ejy.181.1627535406571; Wed, 28 Jul 2021 22:10:06 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:06 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 05/87] trace-cmd library: Fix possible memory corruption on processing a trace buffer Date: Thu, 29 Jul 2021 08:08:37 +0300 Message-Id: <20210729050959.12263-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace buffer is read from the trace file, a new input handler is duplicated from the top one. Some of these data are poiters and should not be duplicated, as it could lead to a memory corruption on handler close. Added a safety check to ensure requested buffer index is valid. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index af11cbc6..787d6825 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3946,13 +3946,14 @@ struct tracecmd_input * tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) { struct tracecmd_input *new_handle; - struct input_buffer_instance *buffer = &handle->buffers[indx]; + struct input_buffer_instance *buffer; size_t offset; ssize_t ret; if (indx >= handle->nr_buffers) return NULL; + buffer = &handle->buffers[indx]; /* * We make a copy of the current handle, but we substitute * the cpu data with the cpu data for this buffer. @@ -3966,6 +3967,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->nr_buffers = 0; new_handle->buffers = NULL; new_handle->version = NULL; + new_handle->guest = NULL; new_handle->ref = 1; if (handle->trace_clock) { new_handle->trace_clock = strdup(handle->trace_clock); From patchwork Thu Jul 29 05:08:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75E80C4338F for ; Thu, 29 Jul 2021 05:10:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 53FD861053 for ; Thu, 29 Jul 2021 05:10:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233871AbhG2FKN (ORCPT ); Thu, 29 Jul 2021 01:10:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233739AbhG2FKM (ORCPT ); Thu, 29 Jul 2021 01:10:12 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC577C061757 for ; Wed, 28 Jul 2021 22:10:08 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id x90so6371877ede.8 for ; Wed, 28 Jul 2021 22:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JxTSEBNOlrS5WTwoLvfCQN3gMd4AoS1/057XdddjCRU=; b=oMqY01dDCzk+fSRFK8skIYjzLnqrPtPEfmHBB4TLU8aC/oTx7Y3wqofNJW9l8O2Z1o xNK5WSofNOrfX4d2bYMJpeAQbggp3VYXYuERrO6rL89i7UnJf+aaPgFUn8yuLtGwWKCG Ah3Vg9Afy2DJxiyc75vb8yc4D+kOCTy2nohWenCLzLOVsCcbDSCuE5intDH8LwMhA9U5 U2qImlngFfdtLMOlphy5eWzOFyXJF8SNwazFX7YljIL5MCNTp0kWBt6KFXjeUao4DO0s 9PtOsNm7iqvslWDnmxSl3ZDqo2i5LB6Xo97TTL5hfbpFsO/crTSSHlDK4nDdxmMoLiky XP9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JxTSEBNOlrS5WTwoLvfCQN3gMd4AoS1/057XdddjCRU=; b=gydraWNeMgSUInSsGJv4WQK1toZ5AYROvfaubK5Bt2dbxkKYBUrNHtP4GfcOxEWRbb Wr+xI5lOfSeVj7THz8SeEfEweQ2ZUJ+87Jm+OY0/HHXWX+sj0qJsQMXVxpNsGQdcfecc Ka+fsRHkq8StVqBnS8RTLsseOJUWuBc51S2m2qX2FHIo/o6Z/pXbIMTDut1hDnOZWc+I 7YiXrYdtfvY9Lgoe6dPBD7Whx8HIPdq9xowI1MpZGGv4RoBoPQMTeT4RNqAQ0Y2nPOlx kLTb6L+WO99qgeL7LwfINnHntDXwKK6yUgs+/+zJ7xpit7UzdtpL6iNXtmnLO2kyj4d+ cbiA== X-Gm-Message-State: AOAM531pRW8cdqwU2y1gDGU1Wj7T+kGG7k3bWwWbkrkH+iURfnHS2CBs IavPxKQRJxusrx0HLHFJKRc= X-Google-Smtp-Source: ABdhPJwfzQNnUO38nCEVB8tMX4pOcQc6hBG0Bdw4lDP5kXuvUS3AvKXR++BUUrvTatFyyuVb974D9w== X-Received: by 2002:aa7:de92:: with SMTP id j18mr3895929edv.141.1627535407645; Wed, 28 Jul 2021 22:10:07 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:07 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 06/87] trace-cmd library: Add constructor and destructor Date: Thu, 29 Jul 2021 08:08:38 +0300 Message-Id: <20210729050959.12263-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added empty constructor and destructor routines of the trace-cmd library. They will be used to initialize and free compression context of the library. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-util.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b0c98c72..0f49a21a 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -624,3 +624,13 @@ bool tracecmd_is_version_supported(unsigned int version) return true; return false; } + +static void __attribute__ ((constructor)) tracecmd_lib_init(void) +{ + +} + +static void __attribute__((destructor)) tracecmd_lib_free(void) +{ + +} From patchwork Thu Jul 29 05:08:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1E1FC4320A for ; Thu, 29 Jul 2021 05:10:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D44AB61052 for ; Thu, 29 Jul 2021 05:10:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233739AbhG2FKN (ORCPT ); Thu, 29 Jul 2021 01:10:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233790AbhG2FKM (ORCPT ); Thu, 29 Jul 2021 01:10:12 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0C11C061765 for ; Wed, 28 Jul 2021 22:10:09 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id n2so6355912eda.10 for ; Wed, 28 Jul 2021 22:10:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CLSHwkXPM5e7LU+G0SGg5XyFgz4EHgw5nwwxF8BCeVA=; b=Gmf+VCsNQkqxEOKR1micTEcU70CYhmzZKkNeIySu5D8kM6Ef4BZJEEd3cIXgUznGDO EpjNUip6htSHS/mLf7X4U1ggZj7vxm7OZx0+b5NY4mmRDXXePrAdt00FCc1yPGSPDRGj WBJbiwWRlcKLaNLrRaeBw7u8HUD5TmhSqbeZEF/CKveYpNmY76r0LJiXCBttmOkWmBom /v3Aym9LGo7OPDhh1HPnEMsS2EyWN/6X4P8zm+P+/Oz/olszfNXLtfuuIiNntKho0sm0 5XMF8XRHEB9Mnf8wMQkwV7TdT/0rLA533JTXCZdzxe2ZyQgY+dmRITzVvnAnUhi8EsQc GBSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CLSHwkXPM5e7LU+G0SGg5XyFgz4EHgw5nwwxF8BCeVA=; b=JDyebtesSF0dAxEwP72LJZFs0oYi0TpKtSp95WP+ueN7FbaN5MKQIeSNiHfmA6su8y b3m70JjxiVqljdGuBtyrVqlwPeZFfjsmtvNFN65+04eLIuuGv9tqcIGcTd32vFYAhhJv 2KbeBof721b8Glk6fFyPqrqB3NhvLYtSfhExcyWVvoLHLeo3kvOkjEIBSLdC/cvG2lcF hKG8jtL0yubYLZJkkZZoGWMzwsSsC81nEVeaqjJ9IcmMDUfama4GOpAcg1ml1mfJ16Mw DFmRkYlnwPrK1SuXsenSFFSi/WHLeTJDg7wiitcDpkXbHZPd7XoV/2QVJNAlVE5L3IdE NEkg== X-Gm-Message-State: AOAM5305sXlqwdopFi/X8YProq5w6Lb9PPMgGZkKdd/ABt1FZdN6aaSV MyNSCEYPH1ZhkPw68SqSbRCHqgaiGtQSQg== X-Google-Smtp-Source: ABdhPJyHww56p2MVM7tdfkYqVk5cJ5Udqy7/+uIwQrpQku7G3L/cy5HsxXfdqGXoh/RHg1dZTC7LtA== X-Received: by 2002:a05:6402:4246:: with SMTP id g6mr3963969edb.134.1627535408590; Wed, 28 Jul 2021 22:10:08 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:08 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 07/87] trace-cmd library: Add cache functionality to network message handler Date: Thu, 29 Jul 2021 08:08:39 +0300 Message-Id: <20210729050959.12263-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Network message handler is used to send trace metadata through a network socket, instead writing it into a trace file. There are use cases, that could require to do a lseek() on the metadata. It is hard to implement lseek on a network socket, that's why for such use cases a cache to a local file is introduced. Once the metadata is constructed, the local cache is send to the socket. A new library API is used to enable the local cache: tracecmd_msg_handle_cache() The local cahce is flushed on the socket when the tracecmd_msg_finish_sending_data() is called. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 + lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-msg.c | 131 +++++++++++++----- 3 files changed, 103 insertions(+), 34 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 6440084d..68715580 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -339,12 +339,16 @@ enum tracecmd_msg_flags { }; /* for both client and server */ +#define MSG_CACHE_FILE "/tmp/trace_msg_cacheXXXXXX" struct tracecmd_msg_handle { int fd; short cpu_count; short version; /* Current protocol version */ unsigned long flags; bool done; + bool cache; + int cfd; + char cfile[26]; /* strlen(MSG_CACHE_FILE) */ }; struct tracecmd_tsync_protos { @@ -353,6 +357,7 @@ struct tracecmd_tsync_protos { struct tracecmd_msg_handle * tracecmd_msg_handle_alloc(int fd, unsigned long flags); +int tracecmd_msg_handle_cache(struct tracecmd_msg_handle *msg_handle); /* Closes the socket and frees the handle */ void tracecmd_msg_handle_close(struct tracecmd_msg_handle *msg_handle); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 821b5cdb..7691cc05 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,5 +31,6 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-msg.c b/lib/trace-cmd/trace-msg.c index 6667028e..e856fb33 100644 --- a/lib/trace-cmd/trace-msg.c +++ b/lib/trace-cmd/trace-msg.c @@ -154,33 +154,54 @@ static inline int msg_buf_len(struct tracecmd_msg *msg) return ntohl(msg->hdr.size) - MSG_HDR_LEN - ntohl(msg->hdr.cmd_size); } -static int msg_write(int fd, struct tracecmd_msg *msg) +static int __msg_write(int fd, struct tracecmd_msg *msg, bool network) { - int cmd = ntohl(msg->hdr.cmd); int msg_size, data_size; int ret; - - if (cmd < 0 || cmd >= MSG_NR_COMMANDS) - return -EINVAL; - - dprint("msg send: %d (%s) [%d]\n", - cmd, cmd_to_name(cmd), ntohl(msg->hdr.size)); - + int cmd; + + if (network) { + cmd = ntohl(msg->hdr.cmd); + if (cmd < 0 || cmd >= MSG_NR_COMMANDS) + return -EINVAL; + dprint("msg send: %d (%s) [%d]\n", + cmd, cmd_to_name(cmd), ntohl(msg->hdr.size)); + } msg_size = MSG_HDR_LEN + ntohl(msg->hdr.cmd_size); data_size = ntohl(msg->hdr.size) - msg_size; if (data_size < 0) return -EINVAL; - ret = __do_write_check(fd, msg, msg_size); - if (ret < 0) - return ret; - + if (network) { + ret = __do_write_check(fd, msg, msg_size); + if (ret < 0) + return ret; + } if (!data_size) return 0; return __do_write_check(fd, msg->buf, data_size); } +__hidden off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off64_t offset, int whence) +{ + /* lseek works only if the handle is in cache mode, + * cannot seek on a network socket + */ + if (!msg_handle->cache || msg_handle->cfd < 0) + return (off64_t)-1; + return lseek64(msg_handle->cfd, offset, whence); +} + +static int msg_write(struct tracecmd_msg_handle *msg_handle, struct tracecmd_msg *msg) +{ + if (msg_handle->cache && msg_handle->cfd >= 0) + return __msg_write(msg_handle->cfd, msg, false); + + + return __msg_write(msg_handle->fd, msg, true); +} + enum msg_trace_flags { MSG_TRACE_USE_FIFOS = 1 << 0, }; @@ -274,11 +295,11 @@ static void msg_free(struct tracecmd_msg *msg) memset(msg, 0, sizeof(*msg)); } -static int tracecmd_msg_send(int fd, struct tracecmd_msg *msg) +static int tracecmd_msg_send(struct tracecmd_msg_handle *msg_handle, struct tracecmd_msg *msg) { int ret = 0; - ret = msg_write(fd, msg); + ret = msg_write(msg_handle, msg); if (ret < 0) ret = -ECOMM; @@ -287,11 +308,11 @@ static int tracecmd_msg_send(int fd, struct tracecmd_msg *msg) return ret; } -static int msg_send_nofree(int fd, struct tracecmd_msg *msg) +static int msg_send_nofree(struct tracecmd_msg_handle *msg_handle, struct tracecmd_msg *msg) { int ret = 0; - ret = msg_write(fd, msg); + ret = msg_write(msg_handle, msg); if (ret < 0) ret = -ECOMM; @@ -454,7 +475,7 @@ static int tracecmd_msg_send_notsupp(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; tracecmd_msg_init(MSG_NOT_SUPP, &msg); - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } static int handle_unexpected_msg(struct tracecmd_msg_handle *msg_handle, @@ -472,7 +493,6 @@ int tracecmd_msg_send_init_data(struct tracecmd_msg_handle *msg_handle, unsigned int **client_ports) { struct tracecmd_msg msg; - int fd = msg_handle->fd; unsigned int *ports; int i, cpus, ret; char *p, *buf_end; @@ -485,13 +505,13 @@ int tracecmd_msg_send_init_data(struct tracecmd_msg_handle *msg_handle, if (ret < 0) goto out; - ret = tracecmd_msg_send(fd, &msg); + ret = tracecmd_msg_send(msg_handle, &msg); if (ret < 0) goto out; msg_free(&msg); - ret = tracecmd_msg_wait_for_msg(fd, &msg); + ret = tracecmd_msg_wait_for_msg(msg_handle->fd, &msg); if (ret < 0) goto out; @@ -564,12 +584,57 @@ tracecmd_msg_handle_alloc(int fd, unsigned long flags) handle->fd = fd; handle->flags = flags; + handle->cfd = -1; + handle->cache = false; return handle; } +int tracecmd_msg_handle_cache(struct tracecmd_msg_handle *msg_handle) +{ + if (msg_handle->cfd < 0) { + strcpy(msg_handle->cfile, MSG_CACHE_FILE); + msg_handle->cfd = mkstemp(msg_handle->cfile); + if (msg_handle->cfd < 0) + return -1; + } + msg_handle->cache = true; + return 0; +} + +static int flush_cache(struct tracecmd_msg_handle *msg_handle) +{ + char buf[MSG_MAX_DATA_LEN]; + int ret; + int fd; + + if (!msg_handle->cache || msg_handle->cfd < 0) + return 0; + close(msg_handle->cfd); + msg_handle->cfd = -1; + fd = open(msg_handle->cfile, O_RDONLY); + if (fd < 0) + return -1; + do { + ret = read(fd, buf, MSG_MAX_DATA_LEN); + if (ret <= 0) + break; + ret = tracecmd_msg_data_send(msg_handle, buf, ret); + if (ret < 0) + break; + } while (ret >= 0); + + close(fd); + unlink(msg_handle->cfile); + return ret; +} + void tracecmd_msg_handle_close(struct tracecmd_msg_handle *msg_handle) { close(msg_handle->fd); + if (msg_handle->cfd >= 0) { + close(msg_handle->cfd); + unlink(msg_handle->cfile); + } free(msg_handle); } @@ -666,7 +731,7 @@ int tracecmd_msg_send_port_array(struct tracecmd_msg_handle *msg_handle, if (ret < 0) return ret; - ret = tracecmd_msg_send(msg_handle->fd, &msg); + ret = tracecmd_msg_send(msg_handle, &msg); if (ret < 0) return ret; @@ -678,7 +743,7 @@ int tracecmd_msg_send_close_msg(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; tracecmd_msg_init(MSG_CLOSE, &msg); - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } int tracecmd_msg_send_close_resp_msg(struct tracecmd_msg_handle *msg_handle) @@ -686,14 +751,13 @@ int tracecmd_msg_send_close_resp_msg(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; tracecmd_msg_init(MSG_CLOSE_RESP, &msg); - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } int tracecmd_msg_data_send(struct tracecmd_msg_handle *msg_handle, const char *buf, int size) { struct tracecmd_msg msg; - int fd = msg_handle->fd; int n; int ret; int count = 0; @@ -721,7 +785,7 @@ int tracecmd_msg_data_send(struct tracecmd_msg_handle *msg_handle, memcpy(msg.buf, buf + count, n); n = 0; } - ret = msg_write(fd, &msg); + ret = msg_write(msg_handle, &msg); if (ret < 0) break; } @@ -735,8 +799,10 @@ int tracecmd_msg_finish_sending_data(struct tracecmd_msg_handle *msg_handle) struct tracecmd_msg msg; int ret; + flush_cache(msg_handle); + msg_handle->cache = false; tracecmd_msg_init(MSG_FIN_DATA, &msg); - ret = tracecmd_msg_send(msg_handle->fd, &msg); + ret = tracecmd_msg_send(msg_handle, &msg); if (ret < 0) return ret; return 0; @@ -752,10 +818,7 @@ int tracecmd_msg_read_data(struct tracecmd_msg_handle *msg_handle, int ofd) while (!tracecmd_msg_done(msg_handle)) { ret = tracecmd_msg_recv_wait(msg_handle->fd, &msg); if (ret < 0) { - if (ret == -ETIMEDOUT) - tracecmd_warning("Connection timed out\n"); - else - tracecmd_warning("reading client"); + tracecmd_warning("reading client %d (%s)", ret, strerror(ret)); return ret; } @@ -959,7 +1022,7 @@ int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, if (ret < 0) return ret; - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } static int get_trace_req_protos(char *buf, int length, @@ -1151,7 +1214,7 @@ int tracecmd_msg_send_time_sync(struct tracecmd_msg_handle *msg_handle, msg.hdr.size = htonl(ntohl(msg.hdr.size) + payload_size); msg.buf = payload; - return msg_send_nofree(msg_handle->fd, &msg); + return msg_send_nofree(msg_handle, &msg); } /** @@ -1283,7 +1346,7 @@ int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, if (ret < 0) return ret; - return tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle, &msg); } int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, From patchwork Thu Jul 29 05:08:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2ABBCC432BE for ; Thu, 29 Jul 2021 05:10:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1216D61019 for ; Thu, 29 Jul 2021 05:10:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233875AbhG2FKQ (ORCPT ); Thu, 29 Jul 2021 01:10:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233890AbhG2FKP (ORCPT ); Thu, 29 Jul 2021 01:10:15 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 303FBC061765 for ; Wed, 28 Jul 2021 22:10:12 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id r16so6377839edt.7 for ; Wed, 28 Jul 2021 22:10:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CUjjxZskQ2OLnFY5CSizRooo5wZNjwIKk5ks5KtrMOc=; b=AQDTHrcFe2pBWqv4e8fxVzw3pI5OddkYAYjjNb0WHeL2Qd6NxS5hHwjQVEJ4hbB03c MQUyX2Lqm+Em0Rg9uaTfwllUWZa26JQF/eN96RkUl1y4E1uYpgKUas1k8KU4UJWYr3rv 3LXHZzVqMRKljc3XZETcc5v1HWK9JrvNw82iqj+yPPqpSRIM/njk94+HmUqZow5dj8Yf 1ixEmAR0sT6MTc5Fky4ZNKUuBi8oWKiaDjK/snvDVIPKPs2Gga/3Gnh1tJUchPXdrlTx sqVumj2Bj99L3qHB+ex78lOGdxDemRuEcGfm9OhAwn5FQNqCsND+dB7ek5N2w4aIhz5N Lckg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CUjjxZskQ2OLnFY5CSizRooo5wZNjwIKk5ks5KtrMOc=; b=lwJN2YpKUFzN5t7zO7+SWXT7XGHwSLvaz1IoyUPbFBp5OvGLqaQUkczKY46SrE48cH DnDD/T7s08AP/g35cP6uPn7X1eYNSjpjySwMvDWTrYmXZ56OqvOsqfQcfBz0jdpxEHpk 2IY6prycJCOkPas73wsNGvZ4H9VHQ8NbsD5sal+EmbNmpGFqxtqTO1QefEIbTE27kDmj 0jxGY+3D+l8y8KRoREbevRw0NgiFNigbNIpznmct2j7REwwz0VC8/xbFdGGwGta253Yy KzWmUEkEcKbOR84jN3OcaEJN/pNOfsGXnWePc99lU2NXVoPkCBbBBxwjMOVd4vEmknns UYYg== X-Gm-Message-State: AOAM531b04A9WeyoVggg6RyfbCV3jp7L1jzChnRqIdQFKz8izx6aJOns zE8M6G3oRoi+CMJvWCP30qkOj6Zv94frGg== X-Google-Smtp-Source: ABdhPJyegJex9n59+CCCbvUqzqODcJD4LGCWNNF4C4RyB++0PupOcjnM+ny8buiZ5Nr3ECRv4FgUKw== X-Received: by 2002:a05:6402:1601:: with SMTP id f1mr3886910edv.388.1627535409626; Wed, 28 Jul 2021 22:10:09 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:09 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 08/87] trace-cmd library: Add support for compression algorithms Date: Thu, 29 Jul 2021 08:08:40 +0300 Message-Id: <20210729050959.12263-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added infrastructure to trace-cmd library for compression. Introduced various new APIs to work with this new functionality: struct tracecmd_compression tracecmd_compress_init() tracecmd_compress_free() tracecmd_compress_alloc() tracecmd_compress_destroy() tracecmd_compress_block() tracecmd_uncompress_block() tracecmd_compress_reset() tracecmd_compress_read() tracecmd_compress_pread() tracecmd_compress_write() tracecmd_compress_lseek() tracecmd_compress_proto_get_name() tracecmd_compress_is_supported() tracecmd_compress_protos_get() tracecmd_compress_proto_register() tracecmd_compress_copy_from() tracecmd_uncompress_copy_to() tracecmd_uncompress_chunk() tracecmd_load_chunks_info() The compression algorithms are not part of this patch. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 1 + .../include/private/trace-cmd-private.h | 39 + lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-compress.c | 910 ++++++++++++++++++ lib/trace-cmd/trace-util.c | 4 +- 5 files changed, 955 insertions(+), 2 deletions(-) create mode 100644 lib/trace-cmd/trace-compress.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index 17600318..bab4322d 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -25,6 +25,7 @@ ifeq ($(VSOCK_DEFINED), 1) OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif +OBJS += trace-compress.o # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 68715580..72729c57 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -468,6 +468,45 @@ void tracecmd_tsync_free(struct tracecmd_time_sync *tsync); int tracecmd_write_guest_time_shift(struct tracecmd_output *handle, struct tracecmd_time_sync *tsync); +/* --- Compression --- */ +struct tracecmd_compress_chunk { + unsigned int size; + unsigned int zsize; + unsigned long long zoffset; + unsigned long long offset; +}; +struct tracecmd_compression; +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle); +void tracecmd_compress_destroy(struct tracecmd_compression *handle); +int tracecmd_compress_block(struct tracecmd_compression *handle); +int tracecmd_uncompress_block(struct tracecmd_compression *handle); +void tracecmd_compress_reset(struct tracecmd_compression *handle); +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len); +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset); +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size); +off64_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off64_t offset, int whence); +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version); +bool tracecmd_compress_is_supported(const char *name, const char *version); +int tracecmd_compress_protos_get(char ***names, char ***versions); +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)); +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_chunk(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk *chunk, char *data); +int tracecmd_load_chunks_info(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk **chunks_info); /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 7691cc05..2a622003 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,6 +31,9 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +void tracecmd_compress_init(void); +void tracecmd_compress_free(void); + off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c new file mode 100644 index 00000000..5572acd6 --- /dev/null +++ b/lib/trace-cmd/trace-compress.c @@ -0,0 +1,910 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include +#include + +#include "trace-cmd-private.h" +#include "trace-cmd-local.h" + +struct compress_proto { + struct compress_proto *next; + char *proto_name; + char *proto_version; + int weight; + + int (*compress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + int (*uncompress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + unsigned int (*compress_size)(unsigned int bytes); + bool (*is_supported)(const char *name, const char *version); +}; + +static struct compress_proto *proto_list; + +struct tracecmd_compression { + int fd; + unsigned int capacity; + unsigned long pointer; + char *buffer; + struct compress_proto *proto; + struct tep_handle *tep; + struct tracecmd_msg_handle *msg_handle; +}; + +static int read_fd(int fd, char *dst, int len) +{ + size_t size = 0; + int r; + + do { + r = read(fd, dst+size, len); + if (r > 0) { + size += r; + len -= r; + } else + break; + } while (r > 0); + + if (len) + return -1; + return size; +} + +static long long write_fd(int fd, const void *data, size_t size) +{ + long long tot = 0; + long long w; + + do { + w = write(fd, data + tot, size - tot); + tot += w; + + if (!w) + break; + if (w < 0) + return w; + } while (tot != size); + + return tot; +} + +static long long do_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + int ret; + + if (handle->msg_handle) { + ret = tracecmd_msg_data_send(handle->msg_handle, data, size); + if (ret) + return -1; + return size; + } + return write_fd(handle->fd, data, size); +} + +/** + * tracecmd_compress_lseek - Move the read/write pointer into the compression buffer + * @handle: compression handler + * @offset: number of bytes to move the pointer, can be negative or positive + * @whence: the starting position of the pointer movement, + * + * Returns the new file pointer on success, or -1 in case of an error. + */ +off64_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off64_t offset, int whence) +{ + unsigned long p, extend; + char *buf; + + if (!handle || !handle->buffer) + return (off64_t)-1; + + switch (whence) { + case SEEK_CUR: + p = handle->pointer + offset; + break; + case SEEK_END: + p = handle->capacity + offset; + break; + case SEEK_SET: + p = offset; + break; + default: + return (off64_t)-1; + } + + if (p <= handle->capacity) { + handle->pointer = p; + } else { + extend = p - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return (off64_t)-1; + handle->buffer = buf; + handle->capacity += extend; + handle->pointer = p; + } + + return p; +} + +static int compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int s; + + if (handle->pointer + len > handle->capacity) + s = handle->capacity - handle->pointer; + else + s = len; + memcpy(dst, handle->buffer + handle->pointer, s); + + return s; +} + +/** + * tracecmd_compress_pread - pread() on compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * @offset: offset in the buffer of data to be read + * + * Read a @len of data from the compression buffer at given @offset, + * without updating the buffer pointer. + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset) +{ + int ret; + + if (!handle || !handle->buffer || offset > handle->capacity) + return -1; + + ret = tracecmd_compress_lseek(handle, offset, SEEK_SET); + if (ret < 0) + return ret; + return compress_read(handle, dst, len); +} + +/** + * tracecmd_compress_read - read() from compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * + * Read a @len of data from the compression buffer + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int ret; + + if (!handle || !handle->buffer) + return -1; + + ret = compress_read(handle, dst, len); + if (ret > 0) + handle->pointer += ret; + + return ret; +} + +/** + * tracecmd_compress_reset - Reset the compression buffer + * @handle: compression handler + * + * Reset the compression buffer, any data currently in the buffer will be destroyed. + * + */ +void tracecmd_compress_reset(struct tracecmd_compression *handle) +{ + if (!handle) + return; + + free(handle->buffer); + handle->buffer = NULL; + handle->pointer = 0; + handle->capacity = 0; +} + +/** + * tracecmd_uncompress_block - uncompress a memory block + * @handle: compression handler + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_uncompress_block(struct tracecmd_compression *handle) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + char *bytes = NULL; + char buf[4]; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + tracecmd_compress_reset(handle); + + if (read(handle->fd, buf, 4) != 4) + return -1; + s_compressed = tep_read_number(handle->tep, buf, 4); + if (read(handle->fd, buf, 4) != 4) + return -1; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + + handle->buffer = malloc(s_uncompressed); + if (!handle->buffer) + return -1; + bytes = malloc(s_compressed); + if (!bytes) + goto error; + + if (read_fd(handle->fd, bytes, s_compressed) < 0) + goto error; + ret = handle->proto->uncompress_block(bytes, s_compressed, + handle->buffer, &s_uncompressed); + if (ret) + goto error; + free(bytes); + handle->pointer = 0; + handle->capacity = s_uncompressed; + return 0; +error: + tracecmd_compress_reset(handle); + free(bytes); + return -1; +} + +/** + * tracecmd_compress_block - compress a memory block + * @handle: compression handler + * + * Compress the content of the internal memory buffer and write the compressed data in the file + * The tracecmd_compress_write() can be used to write data into the internal memory buffer, before + * calling this API. + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_compress_block(struct tracecmd_compression *handle) +{ + unsigned int size; + char *buf; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->compress_size || !handle->proto->compress_block) + return -1; + + size = handle->proto->compress_size(handle->pointer); + buf = malloc(size); + if (!buf) + return -1; + ret = handle->proto->compress_block(handle->buffer, handle->pointer, buf, &size); + if (ret < 0) + goto out; + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &handle->pointer, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write compressed data */ + ret = do_write(handle, buf, size); + ret = ((ret == size) ? 0 : -1); +out: + tracecmd_compress_reset(handle); + free(buf); + return ret; +} + +/** + * tracecmd_compress_write - write() to compression buffer + * @handle: compression handler + * @data: data to be written + * @size: size of @data + * + * Write @data of @size in the compression buffer + * + * Returns 0 on success, or -1 on failure. + */ +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + char *buf; + int extend; + + if (!handle) + return -1; + + if (handle->capacity < handle->pointer + size) { + extend = (handle->pointer + size) - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return -1; + handle->buffer = buf; + handle->capacity += extend; + } + memcpy(&handle->buffer[handle->pointer], data, size); + handle->pointer += size; + return 0; +} + +/** + * tracecmd_compress_init - initialize the library with available compression algorithms + */ +void tracecmd_compress_init(void) +{ + struct timeval time; + + gettimeofday(&time, NULL); + srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); + +#ifdef HAVE_ZLIB + tracecmd_zlib_init(); +#endif +} + +static struct compress_proto *compress_proto_select(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *selected = NULL; + + while (proto) { + if (!selected || selected->weight > proto->weight) + selected = proto; + proto = proto->next; + } + + return selected; +} + +/** + * tracecmd_compress_alloc - Allocate a new compression context + * @name: name of the compression algorithm, if NULL - auto select the best available algorithm + * @version: version of the compression algorithm, can be NULL + * @fd: file descriptor for reading / writing data + * @tep: tep handler, used to encode the data + * @msg_handle: message handler, use it for reading / writing data instead of @fd + * + * Returns NULL on failure or pointer to allocated compression context. + * The returned context must be freed by tracecmd_compress_destroy() + */ +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle) +{ + struct tracecmd_compression *new; + struct compress_proto *proto; + + if (name) { + proto = proto_list; + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + break; + proto = proto->next; + } + } else { + proto = compress_proto_select(); + } + if (!proto) + return NULL; + + new = calloc(1, sizeof(*new)); + if (!new) + return NULL; + new->fd = fd; + new->tep = tep; + new->msg_handle = msg_handle; + new->proto = proto; + return new; +} + +/** + * tracecmd_compress_destroy - Free a compression context + * @handle: handle to the compression context that will be freed + */ +void tracecmd_compress_destroy(struct tracecmd_compression *handle) +{ + tracecmd_compress_reset(handle); + free(handle); +} + +/** + * tracecmd_compress_is_supported - check if compression algorithm with given name and + * version is supported + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * + * Returns true if the algorithm with given name and version is supported or false if it is not. + */ +bool tracecmd_compress_is_supported(const char *name, const char *version) +{ + struct compress_proto *proto = proto_list; + + if (!name) + return NULL; + + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + return true; + proto = proto->next; + } + return false; +} + +/** + * tracecmd_compress_proto_get_name - get name and version of compression algorithm + * @compress: compression handler. + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version) +{ + if (!compress || !compress->proto) + return -1; + if (name) + *name = compress->proto->proto_name; + if (version) + *version = compress->proto->proto_version; + return 0; +} + +/** + * tracecmd_compress_proto_register - register a new compression algorithm + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * @weight: weight of the compression algorithm, lower is better. + * @compress: compression hook, called to compress a memory block. + * @uncompress: uncompression hook, called to uncompress a memory block. + * @compress_size: hook, called to get the required minimum size of the buffer for compression + * given number of bytes. + * @is_supported: check hook, called to check if compression with given name and version is + * supported by this plugin. + * + * Returns 0 on success, or -1 in case of an error. If algorithm with given name and version is + * already registered, -1 is returned. + */ +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*compress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)) +{ + struct compress_proto *new; + + if (!name || !compress || !uncompress) + return -1; + if (tracecmd_compress_is_supported(name, version)) + return -1; + + new = calloc(1, sizeof(*new)); + if (!new) + return -1; + + new->proto_name = strdup(name); + if (!new->proto_name) + goto error; + new->proto_version = strdup(version); + if (!new->proto_version) + goto error; + new->compress_block = compress; + new->uncompress_block = uncompress; + new->compress_size = compress_size; + new->is_supported = is_supported; + new->weight = weight; + new->next = proto_list; + proto_list = new; + return 0; + +error: + free(new->proto_name); + free(new->proto_version); + free(new); + return -1; +} + +/** + * tracecmd_compress_free - free the library resources, related to available compression algorithms + * + */ +void tracecmd_compress_free(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *del; + + while (proto) { + del = proto; + proto = proto->next; + free(del->proto_name); + free(del->proto_version); + free(del); + } + proto_list = NULL; +} + +/** + * tracecmd_compress_protos_get - get a list of all supported compression algorithms and versions + * @names: return, array with names of all supported compression algorithms + * @versions: return, array with versions of all supported compression algorithms + * + * On success, the size of @names and @versions arrays is returned. Those arrays are allocated by + * the API and must be freed with free() by the caller. Both arrays are with same size, each name + * from @names corresponds to a version from @versions. + * On error -1 is returned and @names and @versions arrays are not allocated. + */ +int tracecmd_compress_protos_get(char ***names, char ***versions) +{ + struct compress_proto *proto = proto_list; + char **n = NULL; + char **v = NULL; + int c, i; + + for (c = 0; proto; proto = proto->next) + c++; + + if (c < 1) + return c; + + n = calloc(c, sizeof(char *)); + if (!n) + goto error; + v = calloc(c, sizeof(char *)); + if (!v) + goto error; + + proto = proto_list; + for (i = 0; i < c && proto; i++) { + n[i] = proto->proto_name; + v[i] = proto->proto_version; + proto = proto->next; + } + + *names = n; + *versions = v; + return c; + +error: + free(n); + free(v); + return -1; +} + +/** + * tracecmd_compress_copy_from - Copy and compress data from a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data to copy from + * @chunk_size: size of one compression chunk + * @read_size: in - max bytes to read from @fd, 0 to read till the EOF + * out - size of the uncompressed data read from @fd + * @write_size: return, size of the compressed data written into @handle + * + * This function reads uncompressed data from given @fd, compresses the data using the @handle + * compression context and writes the compressed data into the fd associated with the @handle. + * The data is compressed on chunks with given @chunk_size size. + * The compressed data is written in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int rchunk = 0; + unsigned int chunks = 0; + unsigned int wsize = 0; + unsigned int rsize = 0; + unsigned int rmax = 0; + unsigned int csize; + unsigned int size; + unsigned int all; + unsigned int r; + off64_t offset; + char *buf_from; + char *buf_to; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->compress_block || !handle->proto->compress_size) + return 0; + if (read_size) + rmax = *read_size; + csize = handle->proto->compress_size(chunk_size); + buf_from = malloc(chunk_size); + if (!buf_from) + return -1; + buf_to = malloc(csize); + if (!buf_to) + return -1; + /* save the initial offset and write 0 chunks */ + offset = lseek64(handle->fd, 0, SEEK_CUR); + write_fd(handle->fd, &chunks, 4); + + do { + all = 0; + if (rmax > 0 && (rmax - rsize) < chunk_size) + rchunk = (rmax - rsize); + else + rchunk = chunk_size; + + do { + r = read(fd, buf_from + all, rchunk - all); + all += r; + + if (r <= 0) + break; + } while (all != rchunk); + + + if (r < 0 || (rmax > 0 && rsize >= rmax)) + break; + rsize += all; + size = csize; + if (all > 0) { + ret = handle->proto->compress_block(buf_from, all, buf_to, &size); + if (ret < 0) { + if (errno == EINTR) + continue; + break; + } + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &all, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write the compressed data */ + ret = write_fd(handle->fd, buf_to, size); + if (ret != size) + break; + /* data + compress header */ + wsize += (size + 8); + chunks++; + } + } while (all > 0); + free(buf_from); + free(buf_to); + if (all) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + endian4 = tep_read_number(handle->tep, &chunks, 4); + /* write chunks count*/ + write_fd(handle->fd, &chunks, 4); + lseek64(handle->fd, offset, SEEK_SET); + if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} + +/** + * tracecmd_load_chunks_info - Read compression chunks information from the file + * @handle: compression handler + * @chunks_info: return, array with compression chunks information + * + * This function reads information of all compression chunks in the current compression block from + * the file and fills that information in a newly allocated array @chunks_info which is returned. + * + * On success count of compression chunks is returned. Array of that count is allocated and + * returned in @chunks_info. Each entry describes one compression chunk. On error -1 is returned. + * In case of success, @chunks_info must be freed by free(). + */ +int tracecmd_load_chunks_info(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk **chunks_info) +{ + struct tracecmd_compress_chunk *chunks = NULL; + unsigned long long size = 0; + unsigned int count = 0; + off64_t offset; + int ret = -1; + char buf[4]; + int i; + + if (!handle) + return -1; + + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (offset == (off64_t)-1) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + count = tep_read_number(handle->tep, buf, 4); + if (!count) { + ret = 0; + goto out; + } + chunks = calloc(count, sizeof(struct tracecmd_compress_chunk)); + if (!chunks) + goto out; + for (i = 0; i < count; i++) { + chunks[i].zoffset = lseek64(handle->fd, 0, SEEK_CUR); + if (chunks[i].zoffset == (off_t)-1) + goto out; + if (read(handle->fd, buf, 4) != 4) + goto out; + chunks[i].zsize = tep_read_number(handle->tep, buf, 4); + chunks[i].offset = size; + if (read(handle->fd, buf, 4) != 4) + goto out; + chunks[i].size = tep_read_number(handle->tep, buf, 4); + size += chunks[i].size; + if (lseek64(handle->fd, chunks[i].zsize, SEEK_CUR) == (off64_t)-1) + goto out; + } + + ret = count; +out: + if (lseek64(handle->fd, offset, SEEK_SET) == (off64_t)-1) + ret = -1; + + if (ret > 0 && chunks_info) + *chunks_info = chunks; + else + free(chunks); + + return ret; +} + +/** + * tracecmd_uncompress_chunk - Uncompress given compression chunk. + * @handle: compression handler + * @chunk: chunk, that will be uncompressed in @data + * @data: Preallocated memory for uncompressed data. Must have enough space to hold + * the uncompressed data + * + * This function uncompresses the chunk described by @chunk and stores the uncompressed data in + * the preallocated memory @data. + * + * On success 0 is returned and the uncompressed data is stored in @data. On error -1 is returned. + */ +int tracecmd_uncompress_chunk(struct tracecmd_compression *handle, + struct tracecmd_compress_chunk *chunk, char *data) +{ + char *bytes_in = NULL; + unsigned int size; + int ret = -1; + + if (!handle || !handle->proto || !handle->proto->uncompress_block || !chunk || !data) + return -1; + + if (lseek64(handle->fd, chunk->zoffset + 8, SEEK_SET) == (off_t)-1) + return -1; + bytes_in = malloc(chunk->zsize); + if (!bytes_in) + return -1; + if (read_fd(handle->fd, bytes_in, chunk->zsize) < 0) + goto out; + size = chunk->size; + if (handle->proto->uncompress_block(bytes_in, chunk->zsize, data, &size)) + goto out; + ret = 0; +out: + free(bytes_in); + return ret; +} + +/** + * tracecmd_uncompress_copy_to - Uncompress data and copy to a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data to copy into + * @read_size: return, size of the compressed data read from @handle + * @write_size: return, size of the uncompressed data written into @fd + * + * This function reads compressed data from the fd, associated with @handle, uncompresses it + * using the @handle compression context and writes the uncompressed data into the fd. + * The compressed data must be in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + unsigned int rsize = 0; + unsigned int wsize = 0; + char *bytes_out = NULL; + char *bytes_in = NULL; + int size_out = 0; + int size_in = 0; + int chunks; + char buf[4]; + char *tmp; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + chunks = tep_read_number(handle->tep, buf, 4); + rsize += 4; + while (chunks) { + if (read(handle->fd, buf, 4) != 4) + break; + s_compressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (read(handle->fd, buf, 4) != 4) + break; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (!bytes_in || size_in < s_compressed) { + tmp = realloc(bytes_in, s_compressed); + if (!tmp) + break; + bytes_in = tmp; + size_in = s_compressed; + } + + if (!bytes_out || size_out < s_uncompressed) { + tmp = realloc(bytes_out, s_uncompressed); + if (!tmp) + break; + bytes_out = tmp; + size_out = s_uncompressed; + } + + if (read_fd(handle->fd, bytes_in, s_compressed) < 0) + break; + rsize += s_compressed; + ret = handle->proto->uncompress_block(bytes_in, s_compressed, + bytes_out, &s_uncompressed); + if (ret) + break; + write_fd(fd, bytes_out, s_uncompressed); + wsize += s_uncompressed; + chunks--; + } + free(bytes_in); + free(bytes_out); + if (chunks) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 0f49a21a..4e1b8427 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -627,10 +627,10 @@ bool tracecmd_is_version_supported(unsigned int version) static void __attribute__ ((constructor)) tracecmd_lib_init(void) { - + tracecmd_compress_init(); } static void __attribute__((destructor)) tracecmd_lib_free(void) { - + tracecmd_compress_free(); } From patchwork Thu Jul 29 05:08:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D895C4338F for ; Thu, 29 Jul 2021 05:10:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2A10661019 for ; Thu, 29 Jul 2021 05:10:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233893AbhG2FKP (ORCPT ); Thu, 29 Jul 2021 01:10:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233790AbhG2FKP (ORCPT ); Thu, 29 Jul 2021 01:10:15 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C6F37C061757 for ; Wed, 28 Jul 2021 22:10:11 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id j2so6361555edp.11 for ; Wed, 28 Jul 2021 22:10:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fv+FJkb2Lkjik897xvnzRSMdiYDW4XmLvXrISE7YIac=; b=pc5QrRZFwIEWVW1vh+5Gtp1IA0zMwFDUqqj0km3XEkq8AXAF6mdYwtn5JiCaVtmTA+ l71nhQlmlHyxoVK5zu8JpuiMo8J3RMnr3R7wEyabbc/4SiKmTjrx5RT4ZTj9T0HXF2zP rsLCoJJ5PF04ZHvtOCzb5R5acRpzvb1VAbqqFctqbPyeKRFvWPh4tFVCi3twPnk1Xaxp MEe6IFe2gyOVb8/uvOhiC+zZcvnyCs60rjcfPYbxQdKTFmLNyc6v5uniJfOwHjlnv4G2 vCE0hmvFprPJP8otACR9SwJnzTG8RFIHpRkRyfszBBtecW6r7Z/7TvIB8H7WZEmQCkrU kamQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fv+FJkb2Lkjik897xvnzRSMdiYDW4XmLvXrISE7YIac=; b=ZVLf63x8nJbgmi2HQNV4xLJVRzYHQXMfMjfracQs2xSwPwG4jimkyFOsfC7t4JNcB5 9hFvlrX/KCO0Af3o5BJfNWPv7k2YhyIvgR7sKMrN30L3ehYghEfFj9jQY3uk37e5YGcY X5VySREf2z1zCUXQIfPchb+AgUNVJd14XJ/5s3IsHeLPBhD4o1m6AtgdRiowIJOMK6o0 bDDZTyMDn53TXD/WHKMr7nwRrK6pXDyPjjKFSpD1a0TXt65X80TjexSBPr1mItqCGsis 44ddLcbEWWptTRf4ryW73/bzckCrQKEQ61U0SUL/jAG+E9qpThBsOmLamlfAyk8LQLQ1 mf4Q== X-Gm-Message-State: AOAM533HmcOrlBLfTEnNXRYNedPxGb/9IIYLV5JgWYHsn0xglU5Z3fOI cXV7L8p1+ShDinVZue+J124= X-Google-Smtp-Source: ABdhPJwPQwxqYhrQ+vuRMfs/VmfesxvrjTpTe1k5QPGQhglpL3G8wjDcRdUL0zTLOeBB9TccOPvvBw== X-Received: by 2002:a05:6402:7da:: with SMTP id u26mr2408473edy.270.1627535410494; Wed, 28 Jul 2021 22:10:10 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:10 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 09/87] trace-cmd list: Show supported compression algorithms Date: Thu, 29 Jul 2021 08:08:41 +0300 Message-Id: <20210729050959.12263-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add new parameter "trace-cmd list -c" to show supported compression algorithms. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd/trace-cmd-list.1.txt | 3 +++ tracecmd/trace-list.c | 26 ++++++++++++++++++++ tracecmd/trace-usage.c | 1 + 3 files changed, 30 insertions(+) diff --git a/Documentation/trace-cmd/trace-cmd-list.1.txt b/Documentation/trace-cmd/trace-cmd-list.1.txt index a5c6b16c..b77e3460 100644 --- a/Documentation/trace-cmd/trace-cmd-list.1.txt +++ b/Documentation/trace-cmd/trace-cmd-list.1.txt @@ -71,6 +71,9 @@ OPTIONS List defined clocks that can be used with trace-cmd record -C. The one in brackets ([]) is the active clock. +*-c*:: + List the available trace file compression algorithms. + SEE ALSO -------- trace-cmd(1), trace-cmd-record(1), trace-cmd-report(1), trace-cmd-start(1), diff --git a/tracecmd/trace-list.c b/tracecmd/trace-list.c index d060c810..900da73b 100644 --- a/tracecmd/trace-list.c +++ b/tracecmd/trace-list.c @@ -549,6 +549,24 @@ static void show_plugins(void) tep_free(pevent); } +static void show_compression(void) +{ + char **versions, **names; + int c, i; + + c = tracecmd_compress_protos_get(&names, &versions); + if (c <= 0) { + printf("No compression algorithms are supported\n"); + return; + } + printf("Supported compression algorithms:\n"); + for (i = 0; i < c; i++) + printf("\t%s, %s\n", names[i], versions[i]); + + free(names); + free(versions); +} + void trace_list(int argc, char **argv) { int events = 0; @@ -562,6 +580,7 @@ void trace_list(int argc, char **argv) int flags = 0; int systems = 0; int show_all = 1; + int compression = 0; int i; const char *arg; const char *funcre = NULL; @@ -626,6 +645,10 @@ void trace_list(int argc, char **argv) systems = 1; show_all = 0; break; + case 'c': + compression = 1; + show_all = 0; + break; case '-': if (strcmp(argv[i], "--debug") == 0) { tracecmd_set_debug(true); @@ -670,6 +693,8 @@ void trace_list(int argc, char **argv) show_clocks(); if (systems) show_systems(); + if (compression) + show_compression(); if (show_all) { printf("event systems:\n"); show_systems(); @@ -679,6 +704,7 @@ void trace_list(int argc, char **argv) show_tracers(); printf("\noptions:\n"); show_options(); + show_compression(); } return; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 32b38bfd..f463465a 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -348,6 +348,7 @@ static struct usage_help usage_help[] = { " -O list plugin options\n" " -B list defined buffer instances\n" " -C list the defined clocks (and active one)\n" + " -c list the supported trace file compression algorithms\n" }, { "restore", From patchwork Thu Jul 29 05:08:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C91A9C4320A for ; Thu, 29 Jul 2021 05:10:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA3BE61053 for ; Thu, 29 Jul 2021 05:10:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233790AbhG2FKQ (ORCPT ); Thu, 29 Jul 2021 01:10:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233875AbhG2FKP (ORCPT ); Thu, 29 Jul 2021 01:10:15 -0400 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7F48C0613C1 for ; Wed, 28 Jul 2021 22:10:12 -0700 (PDT) Received: by mail-ed1-x52b.google.com with SMTP id x14so6360384edr.12 for ; Wed, 28 Jul 2021 22:10:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iYdN1F0j8Ay3XJukVi6oM/G3+VgQFJICm7Q0fdgZNqU=; b=AmpqxE/Rw/fGM9h3zuB7BiOIINIPJUYOnV/ENn+k4R9cyoxTKY5ZLkCbnJs7D3pdRd M3Q7lrdx9kgXpBQp12618ipTuauD4ccK1VqGAfWEfRdSSgU6rRjdD0YWeCbXzICMODZE Zqmh0jYFeQAv7rAugnv3oMMZlT7NnUyg+359OOJGo8eMQCSXZVQFKor/77owqJKOn9T2 f7sKbSU30D0pSKtRG6wBYIuwA9/5zAfJk3ce3SKGfXpiTGlZkoQTq5YR9gZYOgIsQYzN uArX1PFJ9piXLWOcUgeZ2wN2KUAR4WENVyZLVB6yNsm9H2djICkhKhemN7HwckovHbT2 yf1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iYdN1F0j8Ay3XJukVi6oM/G3+VgQFJICm7Q0fdgZNqU=; b=TxICUikp6ZNMOPojIhdaDz2i21SLWTXxd2oV9fwaBJb+BatBt5fq9xmTDaJ4z7PuQa RAqPCsOj1D5o8wNOj9Sjips6raDw7OmvDE8L3cirErwV/4kjpptxFeqnsJXk6SHVXtAP p2VH0bwdtRL57dCKuDszTFIjW4fas0lDVYuGQsah1/Bjws8kAO+9Lgl53NMBLQT/bgSK yqObnbdwToTEojcIRnyAp4UGh3RgcQD/pawyCMkVfAprjazHpNufsCmIl55P9yyoBn/a qWoVMctD16+PckQUAvrwVvbyKzOB531u3hT2GkKxln86XHKypCW8w1LwuhJP4AERf0yi DT+A== X-Gm-Message-State: AOAM5320kgzotQEIRV7XIoI7wu3aInXHE3my5HG5/6dtyRPbdW+BtMu1 oJIICQAn1Yn4Yc7IKO2QwCAIpP43s9rkjg== X-Google-Smtp-Source: ABdhPJxsIzpnWXO6YavAmPYY9y7QQG/8bMUBS68XOXX60M6RL8o5/iNj2iOHqKFn4+Pjoweb/CD7KQ== X-Received: by 2002:a05:6402:516f:: with SMTP id d15mr4089087ede.210.1627535411538; Wed, 28 Jul 2021 22:10:11 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:11 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 10/87] trace-cmd library: Internal helpers for compressing data Date: Thu, 29 Jul 2021 08:08:42 +0300 Message-Id: <20210729050959.12263-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New library internal helper functions are introduced, to add compression functionality to the output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 8 ++++ lib/trace-cmd/trace-output.c | 64 ++++++++++++++++++++++++- 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 2a622003..14c4dd08 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -34,6 +34,14 @@ void tracecmd_info(const char *fmt, ...); void tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_uncompress_block(struct tracecmd_output *handle); +int out_compression_start(struct tracecmd_output *handle); +int out_compression_end(struct tracecmd_output *handle); +void out_compression_reset(struct tracecmd_output *handle); +unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size); + off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a8de107c..552889c6 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -59,6 +59,8 @@ struct tracecmd_output { unsigned long file_state; unsigned long file_version; size_t options_start; + bool do_compress; + struct tracecmd_compression *compress; struct list_head options; struct tracecmd_msg_handle *msg_handle; @@ -77,15 +79,36 @@ struct list_event_system { char *name; }; -static stsize_t -do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) +__hidden long long +do_write_check(struct tracecmd_output *handle, const void *data, long long size) { + if (handle->do_compress) + return tracecmd_compress_write(handle->compress, data, size); + if (handle->msg_handle) return tracecmd_msg_data_send(handle->msg_handle, data, size); return __do_write_check(handle->fd, data, size); } +static inline off64_t do_lseek(struct tracecmd_output *handle, off_t offset, int whence) +{ + if (handle->do_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else if (handle->msg_handle) + return msg_lseek(handle->msg_handle, offset, whence); + else + return lseek64(handle->fd, offset, whence); +} + +static inline int do_preed(struct tracecmd_output *handle, void *dst, int len, off_t offset) +{ + if (handle->do_compress) + return tracecmd_compress_pread(handle->compress, dst, len, offset); + else + return pread(handle->fd, dst, len, offset); +} + static short convert_endian_2(struct tracecmd_output *handle, short val) { if (!handle->pevent) @@ -111,6 +134,43 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +__hidden void out_compression_reset(struct tracecmd_output *handle) +{ + if (!handle->compress) + return; + tracecmd_compress_reset(handle->compress); + handle->do_compress = false; +} + +__hidden int out_uncompress_block(struct tracecmd_output *handle) +{ + int ret = 0; + + if (!handle->compress) + return 0; + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->do_compress = true; + return ret; +} + +__hidden int out_compression_start(struct tracecmd_output *handle) +{ + if (!handle->compress) + return 0; + tracecmd_compress_reset(handle->compress); + handle->do_compress = true; + return 0; +} + +__hidden int out_compression_end(struct tracecmd_output *handle) +{ + if (!handle->compress) + return 0; + handle->do_compress = false; + return tracecmd_compress_block(handle->compress); +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen From patchwork Thu Jul 29 05:08:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8AF3C4338F for ; Thu, 29 Jul 2021 05:10:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8FEB361019 for ; Thu, 29 Jul 2021 05:10:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233890AbhG2FKS (ORCPT ); Thu, 29 Jul 2021 01:10:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233899AbhG2FKR (ORCPT ); Thu, 29 Jul 2021 01:10:17 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8404C061757 for ; Wed, 28 Jul 2021 22:10:13 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id yk17so752187ejb.11 for ; Wed, 28 Jul 2021 22:10:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iqXq1btnwLlu6+e/tvIEz5L9oqCqgH9SaYhQNAigCmA=; b=J5Pov3rE5lLIdVD0GD5SEkcsjncOeKsSd5+ul4ZWlreX+PknuU2+nhJrOOQqUsXYX8 krI0S1FFLXUPLbjwmtQ9OeqcsaxBFIi3Q2wA7kzStDr4DUW7WIb2SdMFLcjnEYOT36Iz mL2IEgUDJlQ2Y1Cr6LdOirA1A9pXpCzSmsW+aGAKn8L3kVNuzqEQAXMm+zXXhp/bUwhF VjcWFqcymEj6JhDKTVFvnAtQ5eL7PV0y2Dn76guyg5+MtRIP8jPPvw8g4onr83Ileoci v2l7EhskvNbo1KhHWV9otRIF35Qf+XQcu9bxsO4z1nIw1/3oNa/g1w5Vj78VZYl3BtQ0 r0Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iqXq1btnwLlu6+e/tvIEz5L9oqCqgH9SaYhQNAigCmA=; b=JNWBHnIpzEOJOh0Dic3TXQfx6J48o+lKYXSCwqzVADWzdMA0ZGVDYzyfpIfvZyOoCt he4x+XGoJ8KbZrd5DMa8QZUJ4KvInthvo7Z0J2n9Q+K7L2hvVxTFBu1Psq7Kuc9/w+W2 CD+MYg6ZMHZ2JdZFWq46RQ6BZc17bjJiknhkWnX3sn5z7XpxbCMvtrxpESSBdQlIPXpG +tZSu3z9iBvGyxtfY3I7XAA6cBpgaPF46Vy5IwJPzdv6vTiVGeNzmlDXTQv7TRaXkUNk myzDQkW1IR7PZ48F8J8/4lWJnB6OAk3eiC8/S/DZvhAeiuSK435CFQntlZwA14olnZiV vT6g== X-Gm-Message-State: AOAM531EAZuJLZshgI7M2wiJfgEAhSAGtnWKB9cLeH7rgOJ4q26IXlbJ PU+ZqXvdaiCOZ3Q9VsTFJ25gqp/ch6q+2w== X-Google-Smtp-Source: ABdhPJxFCH5dzyb8+d958KDnchNIUdtpSxvAU/ZdDiGlOmWCYXAc8OMs5qTmpzwZGYi1QLNmkwZGVw== X-Received: by 2002:a17:906:cc57:: with SMTP id mm23mr2793192ejb.12.1627535412470; Wed, 28 Jul 2021 22:10:12 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:12 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 11/87] trace-cmd library: Internal helpers for uncompressing data Date: Thu, 29 Jul 2021 08:08:43 +0300 Message-Id: <20210729050959.12263-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New library internal helper functions are introduced, to add compression functionality to the input trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 ++ lib/trace-cmd/trace-input.c | 53 +++++++++++++++++++++---- 2 files changed, 49 insertions(+), 8 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 14c4dd08..271c55a7 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -38,6 +38,10 @@ int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); + +void in_uncompress_reset(struct tracecmd_input *handle); +int in_uncompress_block(struct tracecmd_input *handle); + unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, int fd, unsigned long long max, unsigned long long *write_size); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 787d6825..e759285b 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -136,6 +136,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + bool read_compress; + struct tracecmd_compression *compress; + struct host_trace_info host; double ts2secs; char * cpustats; @@ -238,13 +241,13 @@ static const char *show_records(struct page **pages, int nr_pages) static int init_cpu(struct tracecmd_input *handle, int cpu); -static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) +static ssize_t do_read_fd(int fd, void *data, size_t size) { ssize_t tot = 0; ssize_t r; do { - r = read(handle->fd, data + tot, size - tot); + r = read(fd, data + tot, size - tot); tot += r; if (!r) @@ -256,12 +259,28 @@ static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) return tot; } +static inline int do_lseek(struct tracecmd_input *handle, int offset, int whence) +{ + if (handle->read_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek(handle->fd, offset, whence); +} + +static inline ssize_t do_read_compressed(struct tracecmd_input *handle, void *data, size_t size) +{ + if (handle->read_compress) + return tracecmd_compress_read(handle->compress, data, size); + else + return do_read_fd(handle->fd, data, size); +} + static ssize_t do_read_check(struct tracecmd_input *handle, void *data, size_t size) { ssize_t ret; - ret = do_read(handle, data, size); + ret = do_read_compressed(handle, data, size); if (ret < 0) return ret; if (ret != size) @@ -279,10 +298,8 @@ static char *read_string(struct tracecmd_input *handle) ssize_t r; for (;;) { - r = do_read(handle, buf, BUFSIZ); - if (r < 0) - goto fail; - if (!r) + r = do_read_compressed(handle, buf, BUFSIZ); + if (r <= 0) goto fail; for (i = 0; i < r; i++) { @@ -308,7 +325,7 @@ static char *read_string(struct tracecmd_input *handle) } /* move the file descriptor to the end of the string */ - r = lseek(handle->fd, -(r - (i+1)), SEEK_CUR); + r = do_lseek(handle, -(r - (i+1)), SEEK_CUR); if (r < 0) goto fail; @@ -372,6 +389,26 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +__hidden void in_uncompress_reset(struct tracecmd_input *handle) +{ + if (handle->compress) { + handle->read_compress = false; + tracecmd_compress_reset(handle->compress); + } +} + +__hidden int in_uncompress_block(struct tracecmd_input *handle) +{ + int ret = 0; + + if (handle->compress) { + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->read_compress = true; + } + return ret; +} + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; From patchwork Thu Jul 29 05:08:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CD43C43214 for ; Thu, 29 Jul 2021 05:10:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ECA4D61052 for ; Thu, 29 Jul 2021 05:10:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233902AbhG2FKR (ORCPT ); Thu, 29 Jul 2021 01:10:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233890AbhG2FKR (ORCPT ); Thu, 29 Jul 2021 01:10:17 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADBEEC061765 for ; Wed, 28 Jul 2021 22:10:14 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id nd39so8462726ejc.5 for ; Wed, 28 Jul 2021 22:10:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V9e8J9vDPgVv4rZzvSB1N/0tzTS074J4DavFjtJMvLo=; b=uyY0AtkxpJjCtZQA9uRTSWRK/JXxBij2BeaDJsNhg4uYLu/jkto2H2ynV7hVt13Ku/ lh81N53nDGQ43ViE7sZ/2ZdglkXhhrGp3Ui2xrMPrYk/zdJGg3QZJZTTSMD7kSD2qvVv 6G/l7kTYj4SJIyuMuOciuZokxWGayBAot8NhInsK5xcdq0Ebu1dN2T3Y2u0mxQnjaZzo RDcOv/1Ub6QoaM0MlK0nYOcDJu4v3v7jXW71HaBulxJXcO9IJZEPdwag7Kq1lEJ4VA6/ S5g2O7k3ZhxLLIHWzOqBa6vkHM4FSgCXT8EIH7e20ac2fHvWqEYUvtjt5LrEv5cSIf1F +cag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V9e8J9vDPgVv4rZzvSB1N/0tzTS074J4DavFjtJMvLo=; b=cqdx5dE+hpZP+dsg+95vY6LzJExRRoeYSr2tEZEbN55a4zOW/lljydHQumNQ0mIwH/ ksFre4wUT8HGzI3nqKDfbKgPz7XlpSCybEbJ3Wwbjv/lIbLOzQVeLIKDTPX2Mhwl5beH 78qeQCxiZMOt2Z8LD8gJPSRzXhDAIQYUKUup+pcL1gv7zJxy+MJHBzL33lbTJ34YkPKb DYWfmJ6OhaiG4SsnEOsXijRj+9riEQ3inqTQYjfDAnfDEBppvJXASIQF1SN6PDKmstDN z0kwsrhCrs6D+WyX3MskrkiuwnvIZs3S49e2lsxNA7QgwyMMYeeM02d47RnkCN4ciQpl agcA== X-Gm-Message-State: AOAM532C4lEu1cohMgm4r2TTGf4AVDejD1AGdSmD20QibwMzAcX3L6k2 2s5OVyfiqpfIQMqrLE6N0LH+2Uk+Jrjk4w== X-Google-Smtp-Source: ABdhPJx+DhTBIjKfsvcj1k0MFi/+d+BT2JxIplHIuYCH74wJsAgUYh+OAmqVI3xkpF3b2G4SH0J6zA== X-Received: by 2002:a17:907:1b06:: with SMTP id mp6mr3015301ejc.188.1627535413360; Wed, 28 Jul 2021 22:10:13 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:12 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 12/87] trace-cmd library: Define trace file version 7 Date: Thu, 29 Jul 2021 08:08:44 +0300 Message-Id: <20210729050959.12263-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a define for file version 7, but keep the default file version to 6. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 3 +++ lib/trace-cmd/include/trace-cmd-local.h | 7 ------- lib/trace-cmd/trace-output.c | 2 +- lib/trace-cmd/trace-util.c | 2 +- tracecmd/trace-record.c | 2 ++ 5 files changed, 7 insertions(+), 9 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 72729c57..c3b407a0 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -31,6 +31,9 @@ struct tep_plugin_list *trace_load_plugins(struct tep_handle *tep, int flags); int *tracecmd_add_id(int *list, int id, int len); +#define FILE_VERSION_DEFAULT 6 +#define FILE_VERSION_MIN 6 +#define FILE_VERSION_MAX 7 enum { RINGBUF_TYPE_PADDING = 29, RINGBUF_TYPE_TIME_EXTEND = 30, diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 271c55a7..ad3f7e1f 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -14,13 +14,6 @@ void tracecmd_warning(const char *fmt, ...); void tracecmd_critical(const char *fmt, ...); void tracecmd_info(const char *fmt, ...); -/* trace.dat file format version */ -#define FILE_VERSION 6 - -#define _STR(x) #x -#define STR(x) _STR(x) -#define FILE_VERSION_STRING STR(FILE_VERSION) - #ifndef htonll # if __BYTE_ORDER == __LITTLE_ENDIAN #define htonll(x) __bswap_64(x) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 552889c6..ae35c12d 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -976,7 +976,7 @@ static int select_file_version(struct tracecmd_output *handle, if (ihandle) handle->file_version = tracecmd_get_in_file_version(ihandle); else - handle->file_version = FILE_VERSION; + handle->file_version = FILE_VERSION_DEFAULT; return 0; } diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 4e1b8427..00246076 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -620,7 +620,7 @@ unsigned long long tracecmd_generate_traceid(void) bool tracecmd_is_version_supported(unsigned int version) { - if (version <= FILE_VERSION) + if (version >= FILE_VERSION_MIN && version <= FILE_VERSION_MAX) return true; return false; } diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 1767a6c6..9780cedd 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -214,6 +214,7 @@ struct common_record_context { int topt; int run_command; int saved_cmdlines_size; + int file_version; }; static void add_reset_file(const char *file, const char *val, int prio) @@ -5937,6 +5938,7 @@ static void init_common_record_context(struct common_record_context *ctx, ctx->instance = &top_instance; ctx->curr_cmd = curr_cmd; local_cpu_count = tracecmd_count_cpus(); + ctx->file_version = FILE_VERSION_DEFAULT; init_top_instance(); } From patchwork Thu Jul 29 05:08:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DDADC4320A for ; Thu, 29 Jul 2021 05:10:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 892D261052 for ; Thu, 29 Jul 2021 05:10:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229957AbhG2FKU (ORCPT ); Thu, 29 Jul 2021 01:10:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230257AbhG2FKT (ORCPT ); Thu, 29 Jul 2021 01:10:19 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0574DC061757 for ; Wed, 28 Jul 2021 22:10:16 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id nb11so8482531ejc.4 for ; Wed, 28 Jul 2021 22:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bkA0GZ2ckBzgKYqrlrHBm/kGHPZC3u/giUqUO8SvXiQ=; b=kOihxZtsxBgTpcIrklvIH0ZFVu4yEXuz+3PpwgWJ3cVjhNLp3N0euwuLdeCn8k6xd8 wsyrwofKSSKKZeKgghwFh+4Ih5ljg3HJ2d2GImYM6eP8C1TnaAPw2eL2b9GIQBKh8aOC 94QL1RhoKp38L8/t/1z8nhqE1SRS+aZsf7Ks9/1PjvPjW1w5tOtX5ELSNn52mW8iw+Ba r0PoMHu27dWcyD1CCSNsFZiG0hCc9EKTpe69LJofGeNh83Qh64y2CJFhH/llIdmi6ixy hD9Rk2cbHR3f+yZRFp02mB5ZJLQsoZIo+zKhl5TXecGPvt5faYfn4pHBWwRfupaJPGNM Dzaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bkA0GZ2ckBzgKYqrlrHBm/kGHPZC3u/giUqUO8SvXiQ=; b=WkUMTnMcuu6o3z0QLT8bIx0nl4PY345ei5vcqpY5qVch1UX/R2+P3wm9uDCcPgoXjZ Ag1ceg3NpKCvbucMIBd3gx+8Xgn7hvts14HfADZwHquM1Yx3MyAu1hpRJ8urzNHMWFDm Epb+EOV3qgI+567I0EL4rxSDY+9S9rrUwmUZJJPSYueMVfM/zIIpCUhJeACnyIgbLqvF uKOPZO713AsRt6cnkus6Lfg2te3E85N3X1oijBX7EnXBCiZSacZCwzQB+CTxGu0XJ48H hho7fDdbqMt0qRtSe2BPHCZ+sQVk0bbupyTJB7NSQoYxbAZhXBRQcGAtHuvOPlAylHpK eg5A== X-Gm-Message-State: AOAM530M6uDN4GqvT9m87fjmpwH7PBR7LfM5vTKrGS8mkiAIRhqkLYTE x0TMN2KOcTr6Jtr9UoPqxWI= X-Google-Smtp-Source: ABdhPJzrr9o06xyUviLlT/skq9W6J3S5Iybj731L4n3kFuggZaW73nnIAKTIQA3E14IL9WpOGZdEVw== X-Received: by 2002:a17:906:a24c:: with SMTP id bi12mr2884877ejb.530.1627535414406; Wed, 28 Jul 2021 22:10:14 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:13 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 13/87] trace-cmd library: Refactor APIs for creating output handler Date: Thu, 29 Jul 2021 08:08:45 +0300 Message-Id: <20210729050959.12263-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In the trace-cmd library there are various APIs for allocating and initializing output handler to a trace file. The existing APIs are use case oriented, with a lot of parameters. Extending them for new use cases, adding more input parameters, will make the library more complex and not easy to use. Almost all use case oriented APIs for output handler creation are removed and replaced with new flow, which is easier to be extended with new creation parameters. Removed APIs: tracecmd_create_init_fd_msg() tracecmd_create_init_file_glob() tracecmd_create_init_fd_glob() tracecmd_create_init_file_override() New APIs: tracecmd_output_allocate() tracecmd_output_set_msg() tracecmd_output_set_trace_dir() tracecmd_output_set_kallsyms() tracecmd_output_set_from_input() tracecmd_output_set_version() tracecmd_output_write_init() tracecmd_output_write_headers() The new tracecmd_output_allocate() API allocates memory and performs minimalinitialization of an output handler to a trace file. No data is written in the file. The tracecmd_output_set_...() APIs can be used to set various parameters to the newly allocated output handler, that affect the way the data is written into the file. When the output handler is configured for the desired use case, the tracecmd_output_write_init() is used to start writing to the file, it writes initial magic bytes. The tracecmd_output_write_headers() API is used to write the initial headers into the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 29 +- lib/trace-cmd/trace-output.c | 437 ++++++++++++------ tracecmd/trace-record.c | 64 ++- tracecmd/trace-restore.c | 32 +- 4 files changed, 395 insertions(+), 167 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index c3b407a0..16dbed56 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -105,7 +105,8 @@ static inline int tracecmd_host_bigendian(void) /* --- Opening and Reading the trace.dat file --- */ enum tracecmd_file_states { - TRACECMD_FILE_INIT = 1, + TRACECMD_FILE_ALLOCATED = 0, + TRACECMD_FILE_INIT, TRACECMD_FILE_HEADERS, TRACECMD_FILE_FTRACE_EVENTS, TRACECMD_FILE_ALL_EVENTS, @@ -271,20 +272,22 @@ struct tracecmd_event_list { struct tracecmd_option; struct tracecmd_msg_handle; -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); -struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list); +struct tracecmd_output *tracecmd_output_allocate(int fd); +int tracecmd_output_set_msg(struct tracecmd_output *handler, + struct tracecmd_msg_handle *msg_handle); +int tracecmd_output_set_trace_dir(struct tracecmd_output *handler, const char *tracing_dir); +int tracecmd_output_set_kallsyms(struct tracecmd_output *handler, const char *kallsyms); +int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct tracecmd_input *ihandle); +int tracecmd_output_set_version(struct tracecmd_output *handler, int file_version); +int tracecmd_output_write_init(struct tracecmd_output *handler); +int tracecmd_output_write_headers(struct tracecmd_output *handler, + struct tracecmd_event_list *list); + +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + int file_version); struct tracecmd_output *tracecmd_create_init_fd(int fd); -struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list); -struct tracecmd_output * -tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list); + struct tracecmd_output *tracecmd_create_init_file(const char *output_file); -struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, - const char *tracing_dir, - const char *kallsyms); struct tracecmd_option *tracecmd_add_option(struct tracecmd_output *handle, unsigned short id, int size, const void *data); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index ae35c12d..e1373e80 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -31,11 +31,6 @@ typedef unsigned long long tsize_t; typedef long long stsize_t; -static struct tracecmd_event_list all_event_list = { - .next = NULL, - .glob = "all" -}; - struct tracecmd_option { unsigned short id; int size; @@ -54,11 +49,14 @@ struct tracecmd_output { int cpus; struct tep_handle *pevent; char *tracing_dir; + char *kallsyms; + struct tracecmd_event_list *events; int nr_options; bool quiet; unsigned long file_state; unsigned long file_version; size_t options_start; + bool big_endian; bool do_compress; struct tracecmd_compression *compress; @@ -769,7 +767,7 @@ static int read_event_files(struct tracecmd_output *handle, break; } /* all events are listed, use a global glob */ - if (list) + if (!event_list || list) event_list = &all_events; systems = create_event_list(handle, event_list); @@ -839,8 +837,7 @@ err: tracecmd_warning("can't set kptr_restrict"); } -static int read_proc_kallsyms(struct tracecmd_output *handle, - const char *kallsyms) +static int read_proc_kallsyms(struct tracecmd_output *handle) { unsigned int size, check_size, endian4; const char *path = "/proc/kallsyms"; @@ -853,8 +850,8 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, return -1; } - if (kallsyms) - path = kallsyms; + if (handle->kallsyms) + path = handle->kallsyms; ret = stat(path, &st); if (ret < 0) { @@ -970,137 +967,272 @@ out_free: return ret; } -static int select_file_version(struct tracecmd_output *handle, - struct tracecmd_input *ihandle) +/** + * tracecmd_output_allocate - allocate new output handler to a trace file + * @handle: file descriptor to an empty file, it can be -1 if the handler + * will not write to a file + * + * This API only allocates a handler and performs minimal initialization. + * No data is written in the file. + * + * Returns pointer to a newly allocated file descriptor, that can be used + * with other library APIs. In case of an error, NULL is returned. The returned + * handler must be freed with tracecmd_output_close() or tracecmd_output_free() + */ +struct tracecmd_output *tracecmd_output_allocate(int fd) { - if (ihandle) - handle->file_version = tracecmd_get_in_file_version(ihandle); + struct tracecmd_output *handle; + + handle = calloc(1, sizeof(*handle)); + if (!handle) + return NULL; + + handle->fd = fd; + + handle->file_version = FILE_VERSION_DEFAULT; + + handle->page_size = getpagesize(); + + if (tracecmd_host_bigendian()) + handle->big_endian = true; else - handle->file_version = FILE_VERSION_DEFAULT; + handle->big_endian = false; - return 0; + list_head_init(&handle->options); + + handle->file_state = TRACECMD_FILE_ALLOCATED; + + return handle; } -static struct tracecmd_output * -create_file_fd(int fd, struct tracecmd_input *ihandle, - const char *tracing_dir, - const char *kallsyms, - struct tracecmd_event_list *list, - struct tracecmd_msg_handle *msg_handle) +/** + * tracecmd_output_set_msg - associated an output file handler with network message handler + * @handle: output handler to a trace file. + * @msg_handle: network handler, allocated by tracecmd_msg_handle_alloc() + * + * This API associates an output file handler with a network stream. All subsequent API calls + * with this output file handler will send data over the network using the @msg_handle, instead + * of writing to a file. + * This API must be called after the handler file version is set and before + * tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 if the output file handler is not allocated or not in expected state. + */ +int tracecmd_output_set_msg(struct tracecmd_output *handler, struct tracecmd_msg_handle *msg_handle) { - struct tracecmd_output *handle; - struct tep_handle *pevent; - char buf[BUFSIZ]; - int endian4; + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; - handle = malloc(sizeof(*handle)); - if (!handle) - return NULL; - memset(handle, 0, sizeof(*handle)); + handler->msg_handle = msg_handle; + /* Force messages to be cached in a temp file before sending through the socket */ + if (handler->msg_handle && handler->file_version >= 7) + tracecmd_msg_handle_cache(handler->msg_handle); - list_head_init(&handle->options); + return 0; +} - handle->fd = fd; +/** + * tracecmd_output_set_trace_dir - Set a custom tracing dir, instead of system default + * @handle: output handler to a trace file. + * @tracing_dir: full path to a directory with tracing files + * + * This API associates an output file handler with a custom tracing directory, to be used when + * creating the trace file instead of the system default tracing directory. + * This API must be called before tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 if the output file handler is not allocated or not in expected state. + */ +int tracecmd_output_set_trace_dir(struct tracecmd_output *handler, const char *tracing_dir) +{ + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; + + free(handler->tracing_dir); if (tracing_dir) { - handle->tracing_dir = strdup(tracing_dir); - if (!handle->tracing_dir) - goto out_free; - } + handler->tracing_dir = strdup(tracing_dir); + if (!handler->tracing_dir) + return -1; + } else + handler->tracing_dir = NULL; - handle->msg_handle = msg_handle; + return 0; +} - if (select_file_version(handle, ihandle)) - goto out_free; +/** + * tracecmd_output_set_kallsyms - Set a custom kernel symbols file, instead of system default + * @handle: output handler to a trace file. + * @tracing_dir: full path to a file with kernel symbols + * + * This API associates an output file handler with a custom kernel symbols file, to be used when + * creating the trace file instead of the system default kernel symbols file. + * This API must be called before tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 if the output file handler is not allocated or not in expected state. + */ +int tracecmd_output_set_kallsyms(struct tracecmd_output *handler, const char *kallsyms) +{ + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; - buf[0] = 23; - buf[1] = 8; - buf[2] = 68; - memcpy(buf + 3, "tracing", 7); + free(handler->kallsyms); + if (kallsyms) { + handler->kallsyms = strdup(kallsyms); + if (!handler->kallsyms) + return -1; + } else + handler->kallsyms = NULL; - if (do_write_check(handle, buf, 10)) - goto out_free; + return 0; +} - sprintf(buf, "%lu", handle->file_version); - if (do_write_check(handle, buf, strlen(buf) + 1)) - goto out_free; +/** + * tracecmd_output_set_from_input - Inherit parameters from an existing trace file + * @handle: output handler to a trace file. + * @ihandle: input handler to an existing trace file. + * + * This API copies parameters from input handler @ihandle, associated with an existing trace file, + * to the output handler @handle, associated with file that is going to be created. + * These parameters are copied: + * - tep handler + * - page size + * - file endian + * - file version + * - file compression protocol + * This API must be called before tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 if the output file handler is not allocated or not in expected state. + */ +int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct tracecmd_input *ihandle) +{ + if (!handler || !ihandle || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; - /* get endian and page size */ - if (ihandle) { - pevent = tracecmd_get_tep(ihandle); - /* Use the pevent of the ihandle for later writes */ - handle->pevent = tracecmd_get_tep(ihandle); - tep_ref(pevent); - if (tep_is_file_bigendian(pevent)) - buf[0] = 1; - else - buf[0] = 0; - handle->page_size = tracecmd_page_size(ihandle); - } else { - if (tracecmd_host_bigendian()) - buf[0] = 1; - else - buf[0] = 0; - handle->page_size = getpagesize(); - } + /* get endian, page size, file version and compression */ + /* Use the pevent of the ihandle for later writes */ + handler->pevent = tracecmd_get_tep(ihandle); + tep_ref(handler->pevent); + handler->page_size = tracecmd_page_size(ihandle); + handler->file_version = tracecmd_get_in_file_version(ihandle); + if (tep_is_file_bigendian(handler->pevent)) + handler->big_endian = true; + else + handler->big_endian = false; - if (do_write_check(handle, buf, 1)) - goto out_free; - /* save size of long (this may not be what the kernel is) */ - buf[0] = sizeof(long); - if (do_write_check(handle, buf, 1)) - goto out_free; + return 0; +} - endian4 = convert_endian_4(handle, handle->page_size); - if (do_write_check(handle, &endian4, 4)) - goto out_free; - handle->file_state = TRACECMD_FILE_INIT; +/** + * tracecmd_output_set_version - Set file version of the output handler + * @handle: output handler to a trace file. + * @file_version: desired file version + * + * This API must be called before tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 if the output file handler is not allocated or not in expected state. + */ +int tracecmd_output_set_version(struct tracecmd_output *handler, int file_version) +{ + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; + if (file_version < FILE_VERSION_MIN || file_version > FILE_VERSION_MAX) + return -1; + handler->file_version = file_version; + if (handler->file_version < 7) + handler->compress = NULL; + return 0; +} - if (ihandle) - return handle; - if (read_header_files(handle)) - goto out_free; +/** + * tracecmd_output_write_init - Write the initial magics in the trace file + * @handle: output handler to a trace file. + * + * This API must be called after all tracecmd_output_set_...() APIs and before writing anything + * to the trace file. This initial information is written in the file: + * - initial file magic bytes + * - file version + * - data endian + * - long size + * - page size + * - compression header + * + * Returns 0 on success, or -1 if the output file handler is not allocated or not in expected state. + */ +int tracecmd_output_write_init(struct tracecmd_output *handler) +{ + char buf[BUFSIZ]; + int endian4; - if (read_ftrace_files(handle)) - goto out_free; + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; - if (read_event_files(handle, list)) - goto out_free; + buf[0] = 23; + buf[1] = 8; + buf[2] = 68; + memcpy(buf + 3, "tracing", 7); - if (read_proc_kallsyms(handle, kallsyms)) - goto out_free; + if (do_write_check(handler, buf, 10)) + return -1; - if (read_ftrace_printk(handle)) - goto out_free; + sprintf(buf, "%lu", handler->file_version); + if (do_write_check(handler, buf, strlen(buf) + 1)) + return -1; - return handle; + if (handler->big_endian) + buf[0] = 1; + else + buf[0] = 0; + if (do_write_check(handler, buf, 1)) + return -1; - out_free: - tracecmd_output_close(handle); - return NULL; + /* save size of long (this may not be what the kernel is) */ + buf[0] = sizeof(long); + if (do_write_check(handler, buf, 1)) + return -1; + + endian4 = convert_endian_4(handler, handler->page_size); + if (do_write_check(handler, &endian4, 4)) + return -1; + handler->file_state = TRACECMD_FILE_INIT; + return 0; } -static struct tracecmd_output *create_file(const char *output_file, - struct tracecmd_input *ihandle, - const char *tracing_dir, - const char *kallsyms, - struct tracecmd_event_list *list) +/** + * tracecmd_output_write_headers - Write the trace file headers + * @handle: output handler to a trace file. + * @list: desired events that will be included in the trace file. + * It can be NULL for all available events + * + * These headers are written in the file: + * - header files from the tracing directory + * - ftrace events from the tracing directory + * - event file from the tracing directory - all or only the one from @list + * - kernel symbols from the tracing directory + * - kernel printk strings from the tracing directory + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_output_write_headers(struct tracecmd_output *handler, + struct tracecmd_event_list *list) { - struct tracecmd_output *handle; - int fd; - - fd = open(output_file, O_RDWR | O_CREAT | O_TRUNC | O_LARGEFILE, 0644); - if (fd < 0) - return NULL; - - handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL); - if (!handle) { - close(fd); - unlink(output_file); - } + if (!handler || handler->file_state < TRACECMD_FILE_ALLOCATED) + return -1; - return handle; + /* Write init data, if not written yet */ + if (handler->file_state < TRACECMD_FILE_INIT && tracecmd_output_write_init(handler)) + return -1; + if (read_header_files(handler)) + return -1; + if (read_ftrace_files(handler)) + return -1; + if (read_event_files(handler, list)) + return -1; + if (read_proc_kallsyms(handler)) + return -1; + if (read_ftrace_printk(handler)) + return -1; + return 0; } /** @@ -1354,16 +1486,26 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return 0; } -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + int file_version) { struct tracecmd_output *handle; char *path; - int ret; + int fd; - handle = create_file(output_file, NULL, NULL, NULL, &all_event_list); - if (!handle) + fd = open(output_file, O_RDWR | O_CREAT | O_TRUNC | O_LARGEFILE, 0644); + if (fd < 0) return NULL; + handle = tracecmd_output_allocate(fd); + if (!handle) + goto out_free; + if (file_version && tracecmd_output_set_version(handle, file_version)) + goto out_free; + if (tracecmd_output_write_init(handle)) + goto out_free; + if (tracecmd_output_write_headers(handle, NULL)) + goto out_free; /* * Save the command lines; */ @@ -1376,8 +1518,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_options(handle) < 0) goto out_free; - ret = check_out_state(handle, TRACECMD_FILE_CPU_LATENCY); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_LATENCY)) { tracecmd_warning("Cannot write latency data into the file, unexpected state 0x%X", handle->file_state); goto out_free; @@ -1656,39 +1797,38 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) struct tracecmd_output *tracecmd_create_init_fd(int fd) { - return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); -} - -struct tracecmd_output * -tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list) -{ - return create_file_fd(msg_handle->fd, NULL, NULL, NULL, list, msg_handle); -} - -struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list) -{ - return create_file_fd(fd, NULL, NULL, NULL, list, NULL); -} + struct tracecmd_output *out; -struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list) -{ - return create_file(output_file, NULL, NULL, NULL, list); + out = tracecmd_output_allocate(fd); + if (!out) + return NULL; + if (tracecmd_output_write_init(out)) + goto error; + if (tracecmd_output_write_headers(out, NULL)) + goto error; + + return out; +error: + tracecmd_output_close(out); + return NULL; } struct tracecmd_output *tracecmd_create_init_file(const char *output_file) { - return create_file(output_file, NULL, NULL, NULL, &all_event_list); -} + struct tracecmd_output *handle; + int fd; -struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, - const char *tracing_dir, - const char *kallsyms) -{ - return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list); + fd = open(output_file, O_RDWR | O_CREAT | O_TRUNC | O_LARGEFILE, 0644); + if (fd < 0) + return NULL; + handle = tracecmd_create_init_fd(fd); + if (!handle) { + close(fd); + unlink(output_file); + return NULL; + } + + return handle; } /** @@ -1704,11 +1844,18 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) { struct tracecmd_output *handle; + int fd; - handle = create_file(file, ihandle, NULL, NULL, &all_event_list); - if (!handle) + fd = open(file, O_RDWR | O_CREAT | O_TRUNC | O_LARGEFILE, 0644); + if (fd < 0) return NULL; + handle = tracecmd_output_allocate(fd); + if (!handle) + goto out_free; + if (tracecmd_output_set_from_input(handle, ihandle)) + goto out_free; + tracecmd_output_write_init(handle); if (tracecmd_copy_headers(ihandle, handle->fd, 0, 0) < 0) goto out_free; diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 9780cedd..8080d612 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3690,6 +3690,27 @@ again: static void add_options(struct tracecmd_output *handle, struct common_record_context *ctx); +static struct tracecmd_output *create_net_output(struct common_record_context *ctx, + struct tracecmd_msg_handle *msg_handle) +{ + struct tracecmd_output *out; + + out = tracecmd_output_allocate(-1); + if (!out) + return NULL; + if (ctx->file_version && tracecmd_output_set_version(out, ctx->file_version)) + goto error; + if (tracecmd_output_set_msg(out, msg_handle)) + goto error; + if (tracecmd_output_write_headers(out, listed_events)) + goto error; + + return out; +error: + tracecmd_output_close(out); + return NULL; +} + static struct tracecmd_msg_handle * setup_connection(struct buffer_instance *instance, struct common_record_context *ctx) { @@ -3701,7 +3722,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events); + network_handle = create_net_output(ctx, msg_handle); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3719,10 +3740,13 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, - listed_events); + network_handle = tracecmd_output_allocate(msg_handle->fd); if (!network_handle) goto error; + if (tracecmd_output_set_version(network_handle, ctx->file_version)) + goto error; + if (tracecmd_output_write_headers(network_handle, listed_events)) + goto error; tracecmd_set_quiet(network_handle, quiet); } @@ -4068,8 +4092,7 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, - listed_events); + network_handle = create_net_output(ctx, instance->msg_handle); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4438,6 +4461,32 @@ static void write_guest_file(struct buffer_instance *instance) free(temp_files); } +static struct tracecmd_output *create_output(struct common_record_context *ctx) +{ + struct tracecmd_output *out; + int fd; + + fd = open(ctx->output, O_RDWR | O_CREAT | O_TRUNC | O_LARGEFILE, 0644); + if (fd < 0) + return NULL; + + out = tracecmd_output_allocate(fd); + if (!out) + goto error; + if (ctx->file_version && tracecmd_output_set_version(out, ctx->file_version)) + goto error; + if (tracecmd_output_write_headers(out, listed_events)) + goto error; + return out; +error: + if (out) + tracecmd_output_close(out); + else + close(fd); + unlink(ctx->output); + return NULL; +} + static void record_data(struct common_record_context *ctx) { struct tracecmd_option **buffer_options; @@ -4461,7 +4510,8 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count); + handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, + ctx->file_version); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4492,7 +4542,7 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events); + handle = create_output(ctx); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 280a37f0..8d2fcae8 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -22,6 +22,35 @@ #include "trace-local.h" +static struct tracecmd_output *create_output(const char *file, + const char *tracing_dir, const char *kallsyms) +{ + struct tracecmd_output *out; + int fd; + + fd = open(file, O_RDWR | O_CREAT | O_TRUNC | O_LARGEFILE, 0644); + if (fd < 0) + return NULL; + + out = tracecmd_output_allocate(fd); + if (!out) + goto error; + if (tracing_dir && tracecmd_output_set_trace_dir(out, tracing_dir)) + goto error; + if (kallsyms && tracecmd_output_set_kallsyms(out, kallsyms)) + goto error; + if (tracecmd_output_write_headers(out, NULL)) + goto error; + return out; +error: + if (out) + tracecmd_output_close(out); + else + close(fd); + unlink(file); + return NULL; +} + void trace_restore (int argc, char **argv) { struct tracecmd_output *handle; @@ -90,8 +119,7 @@ void trace_restore (int argc, char **argv) usage(argv); } - handle = tracecmd_create_init_file_override(output, tracing_dir, - kallsyms); + handle = create_output(output, tracing_dir, kallsyms); if (!handle) die("Unabled to create output file %s", output); if (tracecmd_write_cmdlines(handle) < 0) From patchwork Thu Jul 29 05:08:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A0C8C4320E for ; Thu, 29 Jul 2021 05:10:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5078261052 for ; Thu, 29 Jul 2021 05:10:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233899AbhG2FKU (ORCPT ); Thu, 29 Jul 2021 01:10:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229957AbhG2FKT (ORCPT ); Thu, 29 Jul 2021 01:10:19 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB4FCC061765 for ; Wed, 28 Jul 2021 22:10:16 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id nb11so8482563ejc.4 for ; Wed, 28 Jul 2021 22:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7W+uYwoE447rYrNNYEJfG9nAps77FgBFYvFHk9rmkZA=; b=NDfMGq6/dBsVk6sGmv6zgKm3Auv59qkubL+cmvtroxOuUNXhOUMBAuSMV9f7yU3WHG Sdt8TrzZygmDmowHoLMB/xx1aXIg6SijADUcF7c1fuLmMC9wtEXBqJvYmlqtdIaZDnvB VBmQV/x6uBc6I/PfADE07sv0RSJkHI/LvFbGSpGg37rzEtUEqJtf6gzGuiNZdkDz3ugw MTQiv68fM1V31qj90AgGvc8pzvLwPgZvzrCAlOKp82pQAZDhriQwkoerD4thTH4RXkgI IRepBpXMGCDbL7VDWNnxtw4Bvd9VVnfAAqt8ndYR6F366Pmw1UPuDtkh7vxLvXo/npTe g1YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7W+uYwoE447rYrNNYEJfG9nAps77FgBFYvFHk9rmkZA=; b=dakT9FCOokGNgtqEBMjTuD7RfqAuxgBeQAHlF0enMKTcwojNCyAlczkLWM245RHlYm Lrnoqi1eDO9xzN2lUqEF6RFLQ1Y+NDtsfast8DGx8d/tkZeaqHJP2PhxyqQ02HxC4bcv XChiOl5/opzR5T0bjLTxfHQDMYY/nG6VIKM/W+UuUxvTm52YO69QrX68CpURT0eFQ1Hz Hiic1peIbz+Fs6+4dozk5akyGSVhBGStoxsVWE+hhiZf0M+9oOs948zLPaZBM2R+dceD 6lDKgJzpRdYIHIh6YzU6ntcKMH4v7q61IcNdlFhRyMUJiojTHHXpaeKaSqkTyagKl13U sTmw== X-Gm-Message-State: AOAM531MePHE2eiYzbUE9NbgIln0wl0md9sRt4cCgE8F81V//yytmD+e L2bQY2/XmhHzpNyWn7I4ZE+WbtCZ7b1mYA== X-Google-Smtp-Source: ABdhPJwEiWOG5TU5Gt2Co2mt2aOHSfLSyM5TbxJgxBhr4XjcidqwgenOfWG2Twr1+uY/56XKYb5qyw== X-Received: by 2002:a17:907:82a6:: with SMTP id mr38mr2821608ejc.218.1627535415347; Wed, 28 Jul 2021 22:10:15 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:14 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 14/87] trace-cmd library: Reuse within the library the function that checks file state. Date: Thu, 29 Jul 2021 08:08:46 +0300 Message-Id: <20210729050959.12263-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Make the function, that checks if the next file state is valid, global for the tracmd-library, so it can be reused. It is important the same check logic to be used in the whole library. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 ++ lib/trace-cmd/trace-input.c | 5 +++ lib/trace-cmd/trace-output.c | 59 +++++++------------------ lib/trace-cmd/trace-util.c | 33 ++++++++++++++ 4 files changed, 59 insertions(+), 42 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index ad3f7e1f..f799ea8d 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -27,6 +27,10 @@ void tracecmd_info(const char *fmt, ...); void tracecmd_compress_init(void); void tracecmd_compress_free(void); +bool check_file_state(unsigned long file_version, int current_state, int new_state); +bool check_in_state(struct tracecmd_input *handle, int new_state); +bool check_out_state(struct tracecmd_output *handle, int new_state); + int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e759285b..8495ce35 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4259,3 +4259,8 @@ int tracecmd_enable_tsync(struct tracecmd_input *handle, bool enable) return 0; } + +__hidden bool check_in_state(struct tracecmd_input *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e1373e80..8e402d35 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -365,33 +365,6 @@ int tracecmd_ftrace_enable(int set) return ret; } -static int check_out_state(struct tracecmd_output *handle, int new_state) -{ - if (!handle) - return -1; - - switch (new_state) { - case TRACECMD_FILE_HEADERS: - case TRACECMD_FILE_FTRACE_EVENTS: - case TRACECMD_FILE_ALL_EVENTS: - case TRACECMD_FILE_KALLSYMS: - case TRACECMD_FILE_PRINTK: - case TRACECMD_FILE_CMD_LINES: - case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_OPTIONS: - if (handle->file_state == (new_state - 1)) - return 0; - break; - case TRACECMD_FILE_CPU_LATENCY: - case TRACECMD_FILE_CPU_FLYRECORD: - if (handle->file_state == TRACECMD_FILE_OPTIONS) - return 0; - break; - } - - return -1; -} - static int read_header_files(struct tracecmd_output *handle) { tsize_t size, check_size, endian8; @@ -400,7 +373,7 @@ static int read_header_files(struct tracecmd_output *handle) int fd; int ret; - if (check_out_state(handle, TRACECMD_FILE_HEADERS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_HEADERS)) { tracecmd_warning("Cannot read header files, unexpected state 0x%X", handle->file_state); return -1; @@ -712,7 +685,7 @@ static int read_ftrace_files(struct tracecmd_output *handle) struct tracecmd_event_list list = { .glob = "ftrace/*" }; int ret; - if (check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS)) { tracecmd_warning("Cannot read ftrace files, unexpected state 0x%X", handle->file_state); return -1; @@ -753,7 +726,7 @@ static int read_event_files(struct tracecmd_output *handle, int endian4; int ret; - if (check_out_state(handle, TRACECMD_FILE_ALL_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_ALL_EVENTS)) { tracecmd_warning("Cannot read event files, unexpected state 0x%X", handle->file_state); return -1; @@ -844,7 +817,7 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) struct stat st; int ret; - if (check_out_state(handle, TRACECMD_FILE_KALLSYMS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_KALLSYMS)) { tracecmd_warning("Cannot read kallsyms, unexpected state 0x%X", handle->file_state); return -1; @@ -889,7 +862,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) char *path; int ret; - if (check_out_state(handle, TRACECMD_FILE_PRINTK) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_PRINTK)) { tracecmd_warning("Cannot read printk, unexpected state 0x%X", handle->file_state); return -1; @@ -1326,11 +1299,10 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CPU_COUNT); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_COUNT)) { tracecmd_warning("Cannot write CPU count into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } cpus = convert_endian_4(handle, cpus); ret = do_write_check(handle, &cpus, 4); @@ -1346,16 +1318,14 @@ int tracecmd_write_options(struct tracecmd_output *handle) unsigned short option; unsigned short endian2; unsigned int endian4; - int ret; /* If already written, ignore */ if (handle->file_state == TRACECMD_FILE_OPTIONS) return 0; - ret = check_out_state(handle, TRACECMD_FILE_OPTIONS); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_OPTIONS)) { tracecmd_warning("Cannot write options into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } if (do_write_check(handle, "options ", 10)) @@ -1473,11 +1443,10 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CMD_LINES); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CMD_LINES)) { tracecmd_warning("Cannot write command lines into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } ret = save_tracing_file_data(handle, "saved_cmdlines"); if (ret < 0) @@ -1868,3 +1837,9 @@ out_free: tracecmd_output_close(handle); return NULL; } + +__hidden bool check_out_state(struct tracecmd_output *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} + diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 00246076..cda125c4 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -634,3 +634,36 @@ static void __attribute__((destructor)) tracecmd_lib_free(void) { tracecmd_compress_free(); } + +__hidden bool check_file_state(unsigned long file_version, int current_state, int new_state) +{ + if (file_version >= 7) { + if (current_state < TRACECMD_FILE_INIT) + return false; + return true; + } + + switch (new_state) { + case TRACECMD_FILE_HEADERS: + case TRACECMD_FILE_FTRACE_EVENTS: + case TRACECMD_FILE_ALL_EVENTS: + case TRACECMD_FILE_KALLSYMS: + case TRACECMD_FILE_PRINTK: + case TRACECMD_FILE_CMD_LINES: + case TRACECMD_FILE_CPU_COUNT: + if (current_state == (new_state - 1)) + return true; + break; + case TRACECMD_FILE_OPTIONS: + if (file_version < 7 && current_state == TRACECMD_FILE_CPU_COUNT) + return true; + break; + case TRACECMD_FILE_CPU_LATENCY: + case TRACECMD_FILE_CPU_FLYRECORD: + if (current_state == TRACECMD_FILE_OPTIONS) + return true; + break; + } + + return false; +} From patchwork Thu Jul 29 05:08:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6938DC4338F for ; Thu, 29 Jul 2021 05:10:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 475B260FED for ; Thu, 29 Jul 2021 05:10:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230257AbhG2FKU (ORCPT ); Thu, 29 Jul 2021 01:10:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233919AbhG2FKU (ORCPT ); Thu, 29 Jul 2021 01:10:20 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3516C061765 for ; Wed, 28 Jul 2021 22:10:17 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id v21so8497454ejg.1 for ; Wed, 28 Jul 2021 22:10:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Estd870NraakHStXcUkj5oDHqTId6qDCNYNSfMNX1Wo=; b=G2a7r4guJ9A9JJJrqhjaNrqahTl06Cv0dMZSzVRb0r80kqBbayp4wbpRhaNgBxKkhM 6otuFIi86uH4exvO4RluFKIYTMXh8sBwIkRJmu0iBOcLgppV179N2Dn3YZc5oik2Iz83 lZUPxWcYKF1caC7gkrWcO9O+mmnG7teRbraO0vovQ1p283D+UL4KU95YLU1xjHs/QibJ lGRXeGK1x9szlz8PztOL6IEA/Gjlxd4fZ4PF8Wo3gyW5OVmdfETNopdkHV0ndgd1lnmz zZ7yf/M05yzHuaMSUNP9fAAIZks0t0uh3LN2kII3GdYdEoryBL5t34lG3Jy8/JsZC+Bn h5Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Estd870NraakHStXcUkj5oDHqTId6qDCNYNSfMNX1Wo=; b=W9E10qE1oStp1sBq5VF0OujwpvDnRx2rVNo8LxoONihjEv6c1vlWgYdnp4c9RQpSot zO4dWqkEHynK0PeQv3J9P6v/RoGs0yPkaBE+q2djuTabz/oqx8bNTub7GBhp7zchHQ0Y 7oBAQyzF4RM9CM915HE1faUFs90bUv3ioOCmd0JaLCFURipLfuS/JSaidBzv2dnTdeLZ fqWYVerQit9hhJKo07IpXgb3zssQan1MNbYM+cz7+xVj93FUM/JhFJn3FOpaNRaZmQLn QagHjiRUfWp96QBAthBCwVENugKBn/bWV+XxAR0DSyeFmEsknaCO9zA88Qj5alHNwBkv OMSA== X-Gm-Message-State: AOAM533V8eojw3gBrTPkJkwaNZ7jjJBzrBs0WIipvsqsYAIZtcWWjQby tSuzq/Ecb6F09MO58+kCxrM= X-Google-Smtp-Source: ABdhPJwMcan+ZHlA8XhGIueQr5BPMEGd7Hif3P30x+zXGlilH1gxtJYA8bihXMB5O7pxSI7wXLfyKA== X-Received: by 2002:a17:906:e241:: with SMTP id gq1mr3036145ejb.87.1627535416405; Wed, 28 Jul 2021 22:10:16 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:15 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 15/87] trace-cmd library: New API to get the version of output handler Date: Thu, 29 Jul 2021 08:08:47 +0300 Message-Id: <20210729050959.12263-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org There are cases where the version of the file, associated with given output handler must be retrieved. Added a new API to get that version: tracecmd_get_out_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-output.c | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 16dbed56..2fb15026 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -316,6 +316,7 @@ int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8e402d35..50a91cf4 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1843,3 +1843,11 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) return check_file_state(handle->file_version, handle->file_state, new_state); } +/** + * tracecmd_get_out_file_version - return the trace.dat file version + * @handle: output handle for the trace.dat file + */ +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle) +{ + return handle->file_version; +} From patchwork Thu Jul 29 05:08:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CDC7C432BE for ; Thu, 29 Jul 2021 05:10:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2658361052 for ; Thu, 29 Jul 2021 05:10:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233923AbhG2FKW (ORCPT ); Thu, 29 Jul 2021 01:10:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233919AbhG2FKV (ORCPT ); Thu, 29 Jul 2021 01:10:21 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B31EDC061757 for ; Wed, 28 Jul 2021 22:10:18 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id x90so6372528ede.8 for ; Wed, 28 Jul 2021 22:10:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6jvd8E3STlarofaCNpnpuoFXWUhcK/XnVX81wSYHSEg=; b=G8hE4WnIEY/6skP44h/a14jcsJrG0NuQZzJz8zIPHdMmR3tEwwAWjRlKYfGWbER/wm 4qoQPvqQnmOIokOCJ4CQhXDPf5ervhEid1Z3WSzjLG8qywKMgBM8DdIJiAhMnpltyVrw tVAKcMD1m/Bp3YvID6/MVdltc9HTDY//EXP1BxBQbYniGjPe5rTkk0TFcug0t1dvmHy+ v5HlV9vrdPcANwRj2vvmFXNxWdVU3o4fq3eStR1hgTgiaCBsKHr5FtlNA3xEMCnMvNPT SwdbIoGTK7sZuNvMc4VZ8EzAs5MvHB0LbLmCyVBK4XkI7R9zt6Fi0FCckpjgr2yepUvw jx+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6jvd8E3STlarofaCNpnpuoFXWUhcK/XnVX81wSYHSEg=; b=RF4zTR0Hk1Xqs4DSA6zmfTMR362d0GMUWW1sCB/d44d1tR/PgSqcoPUSdgjEKzDhd1 ZKuuNza1wqftLadQx13b4+BneHxxbBuEfKkol2dHWF1h+QtY5qV0Vm9fCsK1xLQxqxm9 C16KgFCT1+j7ZaJyDEU9jBQyNF3IwgWUq+sCXXQE34yCc96773pFMyqM3E9Ibgl42ViB gS9Gp/BfK4CG33pFcC6Cl95181Yc70+r/5lT2If6XejVXfPM/XtUvIM40bucZm2IdWoN wH8gFh/YlafXnXCsQWvTLs6HKNHhGDphLz/FlHfGiHW3KrdZ+bHduw1L3g3MXb9YKm+d y/0Q== X-Gm-Message-State: AOAM530geGKIRDP/3ldrUAVfCDi8wkajwZ05kQGNJbfZNOWShKk1MsmT QE8m7ei4/n6z1KvvQ8/MiGI= X-Google-Smtp-Source: ABdhPJxfu7fyY6OwPWKSNZf2f1nzcPM0j8pKTMLv6IwZshvqI+HPwqsVDB2HaB6sWb+woFxq7TvXjA== X-Received: by 2002:a05:6402:1057:: with SMTP id e23mr4070740edu.352.1627535417350; Wed, 28 Jul 2021 22:10:17 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:16 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 16/87] trace-cmd library: Inherit compression algorithm from input file Date: Thu, 29 Jul 2021 08:08:48 +0300 Message-Id: <20210729050959.12263-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace file output handler is allocated, based on given trace file input handler - use the same compression algorithm. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 16 ++++++++++++++++ lib/trace-cmd/trace-output.c | 19 +++++++++++++++++++ 3 files changed, 37 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 2fb15026..455ff680 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -239,6 +239,8 @@ tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); size_t tracecmd_get_options_offset(struct tracecmd_input *handle); +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 8495ce35..c5b4c32e 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4114,6 +4114,22 @@ unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle) return handle->file_version; } +/** + * tracecmd_get_file_compress_proto - get name and version of compression algorithm, + * used to compress the trace file + * @handle: input handle for the trace.dat file + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version) +{ + return tracecmd_compress_proto_get_name(handle->compress, name, version); +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 50a91cf4..42b193dc 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1077,6 +1077,9 @@ int tracecmd_output_set_kallsyms(struct tracecmd_output *handler, const char *ka */ int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct tracecmd_input *ihandle) { + const char *cname = NULL; + const char *cver = NULL; + if (!handler || !ihandle || handler->file_state != TRACECMD_FILE_ALLOCATED) return -1; @@ -1091,6 +1094,14 @@ int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct trace else handler->big_endian = false; + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handler->compress = tracecmd_compress_alloc(cname, cver, handler->fd, + handler->pevent, handler->msg_handle); + if (!handler->compress) + return -1; + if (handler->file_version < 7) + handler->file_version = 7; + } return 0; } @@ -1716,6 +1727,8 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) { struct tracecmd_output *handle = NULL; struct tracecmd_input *ihandle; + const char *cname = NULL; + const char *cver = NULL; int fd2; /* Move the file descriptor to the beginning */ @@ -1754,6 +1767,12 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->options_start = tracecmd_get_options_offset(ihandle); list_head_init(&handle->options); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + goto out_free; + } tracecmd_close(ihandle); return handle; From patchwork Thu Jul 29 05:08:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4A42C4338F for ; Thu, 29 Jul 2021 05:10:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D14FF61052 for ; Thu, 29 Jul 2021 05:10:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233919AbhG2FKX (ORCPT ); Thu, 29 Jul 2021 01:10:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233700AbhG2FKX (ORCPT ); Thu, 29 Jul 2021 01:10:23 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5686C061757 for ; Wed, 28 Jul 2021 22:10:19 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id qk33so8385909ejc.12 for ; Wed, 28 Jul 2021 22:10:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VTibhOmDlgGuWwU/En+DQyC3NtThrVKX7g+NNQCZhR0=; b=WIZRmCU0qzTK5lLYZLFnUO/ZJea6cWtgzQvW2VTvhFPZY/V9oN+5LBm73vdLYkap95 NhXpb4IjXdePI340puWCgb6+sX9kLpO7zwGBTKi5lZRD6WPUcVGfkwP3zT0LYMmbweAZ 9Wsf5JgkQfmzZ3HL2I/EM2B+H9SVw3Z04yP9X5q1Zx1t6Fwh+PTNaef8OOZwfo/vyoRM tq0aER2ZDeizTisct7VA4U2OAYCvkwKqZH1aGVQnpWUbX4QgbMI0YfVoh+ieOGgR30WA oFNCgOYQh5D3VTBsQxLk2YJWfJRTbRkIkdoDOmbtcYyqqEfJPi2CXMqBxwuRcwEb+CBE m2Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VTibhOmDlgGuWwU/En+DQyC3NtThrVKX7g+NNQCZhR0=; b=aRxMUk+g8gsVDwxjyw6f5gKF0wgCXF2epFTBiEdEkTkYiB6/MkdQBOadGYcaQOBprR YUk2/BLkdgLtJ3WzgEsJZNDD2EoZvfWUViYtXbcOkuMnKTPQvMHOnJluRu9Vawr8eWIN sTl87NGFsjjuXehEffoE4cGV4MuvDKiO12WC42AJGCzIA+e77LcXbyX+IiL4hSFmnmMl MCN+HuzJ0vyhlWpJEUnTsTsvklFn/BVxLpcdE0ARANaK+NvvttLYSGXrNdzO8b4hZUhA ZNrA0Uyrli4gTG6CrEVXJLZ5TynoE0EYL4bEvJbZiG8ANTEepaKhwiz758tdkOZJeF39 lg0A== X-Gm-Message-State: AOAM533MAgsKOXIvW1DZ2IhJYOVqAO8Punegrkdg9I+cIsPa1zk4byb/ CkxkWLN0sTiJ7uQgslfEwME= X-Google-Smtp-Source: ABdhPJwR/IVhbNI16YDAOqeYm8uCu9qQwhiGD4g0vpGP4lcSu69MC8Xm2rwFcvJHXK1J1nBcN+3Hsw== X-Received: by 2002:a17:906:1703:: with SMTP id c3mr2927958eje.64.1627535418406; Wed, 28 Jul 2021 22:10:18 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:17 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 17/87] trace-cmd library: New API to configure compression on an output handler Date: Thu, 29 Jul 2021 08:08:49 +0300 Message-Id: <20210729050959.12263-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New API can be used to configure compression algorithm on a output handler to a trace file. tracecmd_output_set_compression() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-output.c | 47 +++++++++++++++++++ 2 files changed, 48 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 455ff680..ee224903 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -281,6 +281,7 @@ int tracecmd_output_set_trace_dir(struct tracecmd_output *handler, const char *t int tracecmd_output_set_kallsyms(struct tracecmd_output *handler, const char *kallsyms); int tracecmd_output_set_from_input(struct tracecmd_output *handler, struct tracecmd_input *ihandle); int tracecmd_output_set_version(struct tracecmd_output *handler, int file_version); +int tracecmd_output_set_compression(struct tracecmd_output *handler, const char *compression); int tracecmd_output_write_init(struct tracecmd_output *handler); int tracecmd_output_write_headers(struct tracecmd_output *handler, struct tracecmd_event_list *list); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 42b193dc..23a10edc 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -219,6 +219,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) free(option); } free(handle->trace_clock); + tracecmd_compress_destroy(handle->compress); free(handle); } @@ -1127,6 +1128,52 @@ int tracecmd_output_set_version(struct tracecmd_output *handler, int file_versio return 0; } +/** + * tracecmd_output_set_compression - Set file compression algorithm of the output handler + * @handle: output handler to a trace file. + * @compression: name of the desired compression algorithm. Can be one of: + * - "none" - do not use compression + * - "all" - use the best available compression algorithm + * - or specific name of the desired compression algorithm + * + * This API must be called before tracecmd_output_write_init(). + * + * Returns 0 on success, or -1 in case of an error: + * - the output file handler is not allocated or not in expected state. + * - the specified compression algorithm is not available + */ +int tracecmd_output_set_compression(struct tracecmd_output *handler, const char *compression) +{ + if (!handler || handler->file_state != TRACECMD_FILE_ALLOCATED) + return -1; + + handler->compress = NULL; + if (compression && strcmp(compression, "none")) { + if (!strcmp(compression, "any")) { + handler->compress = tracecmd_compress_alloc(NULL, NULL, handler->fd, + handler->pevent, + handler->msg_handle); + if (!handler->compress) + tracecmd_warning("No compression algorithms are supported"); + } else { + handler->compress = tracecmd_compress_alloc(compression, NULL, handler->fd, + handler->pevent, + handler->msg_handle); + if (!handler->compress) { + tracecmd_warning("Compression algorithm %s is not supported", + compression); + return -1; + } + } + } + if (handler->compress && handler->file_version < 7) { + handler->file_version = 7; + if (handler->msg_handle) + tracecmd_msg_handle_cache(handler->msg_handle); + } + + return 0; +} /** * tracecmd_output_write_init - Write the initial magics in the trace file From patchwork Thu Jul 29 05:08:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E93A4C432BE for ; Thu, 29 Jul 2021 05:10:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9CC461053 for ; Thu, 29 Jul 2021 05:10:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233925AbhG2FKY (ORCPT ); Thu, 29 Jul 2021 01:10:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233700AbhG2FKY (ORCPT ); Thu, 29 Jul 2021 01:10:24 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A61A5C061765 for ; Wed, 28 Jul 2021 22:10:20 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id go31so8445153ejc.6 for ; Wed, 28 Jul 2021 22:10:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IS4eVyl17/7riV3Ye/VRLCqzkuyENMYHuiXjcl4aEw0=; b=CnGCi/r3btTDa5/DyMXHHepWZMAC4KVPLmJPwTpukzTlA5s1UVsfk/oX5D6FaRxVWk gtMkZ4+NQ61fYck08lO8M0n3vO99oDcm5gEIsQJLlRVGQlPkWJ7sUhTR27XjFvTBPXW0 VyLOdv3odu/rY9bbQhBBkwc/F+ZPcuWnK01ARMF/N875o64aVEayxOanUwfDw3UprCE5 NE1SUFyeINTawaX0XesUN7Td5Y4BxjZ3B4jpJIiztXVN02X/6C3bDD/cIgRCUCZ2+9bn 4zumSyNSvUgkGaQnOzMMrtOrFp7Smv605w8RNJdD5pmZS3Lpq/cgdNw0syu+TCZ0o1k5 z+Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IS4eVyl17/7riV3Ye/VRLCqzkuyENMYHuiXjcl4aEw0=; b=G1+xv2KBufaZNwcqiHi+eAtLYyyo9xYdywk+j2swNGiulyYU/48JqsY0aqqvouJQCW qxNojUl62tTVPYeklLTsGxkgA0jmlubEAyNjTg13xou5by9vE9Vdw+s4iSp6hgq0s60R idljxHwGGZyUwuVSK9bWfCLkriGL0efVXwGJVTNqPMH03kJdHjVplS8dajVor51rmS3s VjWV1km3kiHuLHi7bbS2ydDcgJrp/4A2/yS5ctGHNRAyxa62h9xu5Yq1fiCGMCnfd9Zm hYIpbDBTRWMQSi95ul3h0iQjGjf7QMViC0JAx6T2Ughdc+18wSNHWgLVFJi3kl7Qbyhg BI8A== X-Gm-Message-State: AOAM53382xFsR6fwN/7PHghHxk4UdG3CeCV61Ai+iaJ9WStw6G73a5cL bVhmSB6V6jpzN2WIyV/UjzY= X-Google-Smtp-Source: ABdhPJyoZim5QM4uQfqVzBQlDodsIjj4gbF6M8YJWBNdc046Ldi2iiacBKjt+ibEDu907X4wi6IA5w== X-Received: by 2002:a17:906:c252:: with SMTP id bl18mr2880458ejb.519.1627535419341; Wed, 28 Jul 2021 22:10:19 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:18 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 18/87] trace-cmd record: Add compression to the trace context Date: Thu, 29 Jul 2021 08:08:50 +0300 Message-Id: <20210729050959.12263-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org As the trace-cmd libray supports trace file compression, trace-cmd record command should have a way to configure this functionality. Trace context is extended to hold the compression algorithm, used to compress the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 8080d612..295fe633 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -199,6 +199,7 @@ struct common_record_context { char *date2ts; char *user; const char *clock; + const char *compression; struct tsc_nsec tsc2nsec; int data_flags; int tsync_loop_interval; @@ -3702,6 +3703,12 @@ static struct tracecmd_output *create_net_output(struct common_record_context *c goto error; if (tracecmd_output_set_msg(out, msg_handle)) goto error; + if (ctx->compression) { + if (tracecmd_output_set_compression(out, ctx->compression)) + goto error; + } else if (ctx->file_version >= 7) { + tracecmd_output_set_compression(out, "any"); + } if (tracecmd_output_write_headers(out, listed_events)) goto error; @@ -3745,6 +3752,12 @@ setup_connection(struct buffer_instance *instance, struct common_record_context goto error; if (tracecmd_output_set_version(network_handle, ctx->file_version)) goto error; + if (ctx->compression) { + if (tracecmd_output_set_compression(network_handle, ctx->compression)) + goto error; + } else if (ctx->file_version >= 7) { + tracecmd_output_set_compression(network_handle, "any"); + } if (tracecmd_output_write_headers(network_handle, listed_events)) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4475,6 +4488,12 @@ static struct tracecmd_output *create_output(struct common_record_context *ctx) goto error; if (ctx->file_version && tracecmd_output_set_version(out, ctx->file_version)) goto error; + if (ctx->compression) { + if (tracecmd_output_set_compression(out, ctx->compression)) + goto error; + } else if (ctx->file_version >= 7) { + tracecmd_output_set_compression(out, "any"); + } if (tracecmd_output_write_headers(out, listed_events)) goto error; return out; From patchwork Thu Jul 29 05:08:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 011D0C4338F for ; Thu, 29 Jul 2021 05:10:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBFFE61019 for ; Thu, 29 Jul 2021 05:10:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233927AbhG2FKZ (ORCPT ); Thu, 29 Jul 2021 01:10:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233700AbhG2FKZ (ORCPT ); Thu, 29 Jul 2021 01:10:25 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E995C061757 for ; Wed, 28 Jul 2021 22:10:21 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id b7so6429577edu.3 for ; Wed, 28 Jul 2021 22:10:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oBYEZ1+W0TvAvSuSkDN98dM8L6x2JWzfmxB8PYHD8YI=; b=TLtRye/98JYUIeLwmRy9xx2fl9oYT6O3LGRojJS7AgmJU2/z2ZZ4ckFJ0yECRMFRgg 68pMIjFiJGGWFDZqNBk8DYYtrX5r6eHIAtodgeL6Kq4HM9sVfdRdaKfaIRcfKfYOFfgH xJ654rbm7tPB9r6fFO95rQOiF0CH2flbuUZsXoNr4vv6WuT/Rj0pCbXlzURuG5/+FUZX a1ziIY8bSSdbfLx9Zk6YPFxoiv2wuCSPeuZQTMIV9uJjIR7pwFkT3y50SWXY7YCl+ony QF4X4gJQ+IBufl/e4JChGA+pUpWy85but3MjAcW0nLJ61BTHWUsZ0z6sYtjoz5HsGRdD SwVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oBYEZ1+W0TvAvSuSkDN98dM8L6x2JWzfmxB8PYHD8YI=; b=MfH+HYnr7dAZgvggSYbdkkzTvN6N+fYw3u/MJijEhnmIKxa0ytHyIjLT8XazM2EK38 E1s43q7mmKiei+GcJbc64kFgdV5rzynhYvIg/t41sNxYT0X8Uo99P1fCTIGlR5mbBXra QeapSKcDVJXOpVLj8uDUcVvHuP8Kbq0DJzW67Oyx/kUwfwyNJhxemhgos/at3ZSQps9O AAvrjMdv5jJJg4TANLPLpbGUEUtaai00QuSSFr+0F3fw3jlC0suKEQy5KTIfWaCXADhE quuF+vSSxX6bZ3HQgO+Uj66EaXJ0rxU1rSg8c7viC92zxSi3QyvMaxSj9zmcy31UHe6v 0zjw== X-Gm-Message-State: AOAM531COaE/DGDpJ6r0MVemXa/p1/DwUCZ9g6eVmFudSFRWE3MYdfPD +QyknNx4cN/cI7zcHrBYJFIyfb7cnllWUw== X-Google-Smtp-Source: ABdhPJyiKJOhejQc56thZneZaOdYw+/99utU4o1FAJzBhydvbGZMoeZi7LvNIvBbsrRdtNg34TbaBA== X-Received: by 2002:a05:6402:2714:: with SMTP id y20mr3952907edd.190.1627535420256; Wed, 28 Jul 2021 22:10:20 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:19 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 19/87] trace-cmd library: Write compression header in the trace file Date: Thu, 29 Jul 2021 08:08:51 +0300 Message-Id: <20210729050959.12263-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If there is a compression configured on the output file handler and if the file version is at least 7, write compression header in the file. The compression header is two null terminated strings - name and version of the compression algorithm, used to compress some parts of the file. The header is located after the page size in the file. The new header is mandatory for trace files version 7. If no compression is used, the string "none" is saved as name of the compression algorithm and empty string as compression algorithm version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 23a10edc..d348d6a5 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -941,6 +941,24 @@ out_free: return ret; } +static int write_compression_header(struct tracecmd_output *handle) +{ + const char *name = NULL; + const char *ver = NULL; + int ret; + + ret = tracecmd_compress_proto_get_name(handle->compress, &name, &ver); + if (ret < 0 || !name || !ver) { + name = "none"; + ver = ""; + } + if (do_write_check(handle, name, strlen(name) + 1)) + return -1; + if (do_write_check(handle, ver, strlen(ver) + 1)) + return -1; + return 0; +} + /** * tracecmd_output_allocate - allocate new output handler to a trace file * @handle: file descriptor to an empty file, it can be -1 if the handler @@ -1225,6 +1243,11 @@ int tracecmd_output_write_init(struct tracecmd_output *handler) endian4 = convert_endian_4(handler, handler->page_size); if (do_write_check(handler, &endian4, 4)) return -1; + if (handler->file_version >= 7) { + if (write_compression_header(handler)) + return -1; + } + handler->file_state = TRACECMD_FILE_INIT; return 0; } From patchwork Thu Jul 29 05:08:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B32FC4320A for ; Thu, 29 Jul 2021 05:10:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14A5E61019 for ; Thu, 29 Jul 2021 05:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233700AbhG2FK1 (ORCPT ); Thu, 29 Jul 2021 01:10:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233929AbhG2FK0 (ORCPT ); Thu, 29 Jul 2021 01:10:26 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92603C061765 for ; Wed, 28 Jul 2021 22:10:22 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id gs8so8387283ejc.13 for ; Wed, 28 Jul 2021 22:10:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vD8oZEarNcLf46w6Gd8RWY6YpF3Juu4LYg6zlxfdM6I=; b=fnSa/ranWc5XU+YJ7Ruo0UhO4bG0dFGTIwQM0sSWmShitE7SKvtF9ls+HsvMiCjSak DK/F4jbuN2MBdTT1J/8WaZemL9Hi8sgFBXuaganDdf28QO19RLizIygbZekeeucf17mh 7k+q+USQY15CikjLfJlTdGk1+ahLTnoY1zCIe8laJrXJI1fM0lQCiPskqzU75/6qAMZE eXFgfQGgSH1TIu6m46tn4FR/VqxwlZ5a1xsWdpJOd3q2kcX3WXV4UBlosT16+NP38tAh ZG5l367f4EwcCIVq1YUN9izYtg9tF4NnkbzvFLB9EfuRAcK3VADuEJh44xrwFnR53MD6 0wwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vD8oZEarNcLf46w6Gd8RWY6YpF3Juu4LYg6zlxfdM6I=; b=BL9KDCC5FDiBizH6UjOJNIboxGBeW6eBdF8OlxInuIc/4cQHwuCRezDOtv6TwJCcID 1X34QBjCx4Prw9UtReOWhOZMPNvjMEvIyJD53ejnMvjw+HVGhwp9hHPX9HIltO53Ttoq cThbWYHPIKEymKDILuHIhST1ikROEvCUq6wRcWO74lHErZdrfjK/l6nhwMlfCWn1OOB5 r3LMzhtbEfBzTtjFhtgjOVrCjoU4Ls1X2n9OqmzMpmWSX4kzwk74muGr7xoWYmF0LGCF b1pYzT+jimgFmyTKYQOGJ0dHPM+FpPfc+1qRxWsd/uuH6xs+VzM769TLRtH1FzXIFMjl /9Qg== X-Gm-Message-State: AOAM531gZ/w8nc/6oN0tcHXnLk9/caigS4j4NzO0X/k07Qik7Ke1qux2 NCfMIr+Vma/vSc2WpSL9sEUQ/N4Fkdm54g== X-Google-Smtp-Source: ABdhPJwt1YJUbIZOjr/SQeOzm0TJ9EQHVafT51An14K0IEj6Z6kSsCOYekvWqsedlnGWZDtMISAVAA== X-Received: by 2002:a17:906:1919:: with SMTP id a25mr2273441eje.161.1627535421166; Wed, 28 Jul 2021 22:10:21 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:20 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 20/87] trace-cmd library: Compress part of the trace file Date: Thu, 29 Jul 2021 08:08:52 +0300 Message-Id: <20210729050959.12263-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Compress part of the trace.dat file metadata. If there is compression support, compress these parts of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 130 +++++++++++++++++++++++++---------- 1 file changed, 92 insertions(+), 38 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index d348d6a5..5d3fd58f 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -366,12 +366,12 @@ int tracecmd_ftrace_enable(int set) return ret; } -static int read_header_files(struct tracecmd_output *handle) +static int read_header_files(struct tracecmd_output *handle, bool compress) { tsize_t size, check_size, endian8; struct stat st; char *path; - int fd; + int fd = -1; int ret; if (!check_out_state(handle, TRACECMD_FILE_HEADERS)) { @@ -384,26 +384,30 @@ static int read_header_files(struct tracecmd_output *handle) if (!path) return -1; + if (compress) + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* old style did not show this info, just add zero */ put_tracing_file(path); if (do_write_check(handle, "header_page", 12)) - return -1; + goto out_close; size = 0; if (do_write_check(handle, &size, 8)) - return -1; + goto out_close; if (do_write_check(handle, "header_event", 13)) - return -1; + goto out_close; if (do_write_check(handle, &size, 8)) - return -1; + goto out_close; + if (compress && out_compression_end(handle)) + goto out_close; return 0; } fd = open(path, O_RDONLY); if (fd < 0) { tracecmd_warning("can't read '%s'", path); - return -1; + goto out_close; } /* unfortunately, you can not stat debugfs files for size */ @@ -419,18 +423,18 @@ static int read_header_files(struct tracecmd_output *handle) if (size != check_size) { tracecmd_warning("wrong size for '%s' size=%lld read=%lld", path, size, check_size); errno = EINVAL; - return -1; + goto out_close; } put_tracing_file(path); path = get_tracing_file(handle, "events/header_event"); if (!path) - return -1; + goto out_close; fd = open(path, O_RDONLY); if (fd < 0) { tracecmd_warning("can't read '%s'", path); - return -1; + goto out_close; } size = get_size_fd(fd); @@ -444,16 +448,19 @@ static int read_header_files(struct tracecmd_output *handle) close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s'", path); - return -1; + goto out_close; } put_tracing_file(path); - + if (compress && out_compression_end(handle)) + goto out_close; handle->file_state = TRACECMD_FILE_HEADERS; return 0; out_close: - close(fd); + out_compression_reset(handle); + if (fd >= 0) + close(fd); return -1; } @@ -680,7 +687,7 @@ create_event_list_item(struct tracecmd_output *handle, tracecmd_warning("Insufficient memory"); } -static int read_ftrace_files(struct tracecmd_output *handle) +static int read_ftrace_files(struct tracecmd_output *handle, bool compress) { struct list_event_system *systems = NULL; struct tracecmd_event_list list = { .glob = "ftrace/*" }; @@ -693,9 +700,15 @@ static int read_ftrace_files(struct tracecmd_output *handle) } create_event_list_item(handle, &systems, &list); - + if (compress) + out_compression_start(handle); ret = copy_event_system(handle, systems); - + if (compress) { + if (!ret) + ret = out_compression_end(handle); + else + out_compression_reset(handle); + } free_list_events(systems); handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; @@ -717,7 +730,7 @@ create_event_list(struct tracecmd_output *handle, } static int read_event_files(struct tracecmd_output *handle, - struct tracecmd_event_list *event_list) + struct tracecmd_event_list *event_list, bool compress) { struct list_event_system *systems; struct list_event_system *slist; @@ -748,7 +761,8 @@ static int read_event_files(struct tracecmd_output *handle, for (slist = systems; slist; slist = slist->next) count++; - + if (compress) + out_compression_start(handle); ret = -1; endian4 = convert_endian_4(handle, count); if (do_write_check(handle, &endian4, 4)) @@ -763,9 +777,19 @@ static int read_event_files(struct tracecmd_output *handle, } ret = copy_event_system(handle, slist); } - - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + if (ret) + goto out_free; + if (compress) { + ret = out_compression_end(handle); + if (ret) + goto out_free; + } out_free: + if (!ret) + handle->file_state = TRACECMD_FILE_ALL_EVENTS; + else + out_compression_reset(handle); + free_list_events(systems); return ret; @@ -811,7 +835,7 @@ err: tracecmd_warning("can't set kptr_restrict"); } -static int read_proc_kallsyms(struct tracecmd_output *handle) +static int read_proc_kallsyms(struct tracecmd_output *handle, bool compress) { unsigned int size, check_size, endian4; const char *path = "/proc/kallsyms"; @@ -827,19 +851,21 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) if (handle->kallsyms) path = handle->kallsyms; + if (compress) + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ size = 0; endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; - return 0; + ret = do_write_check(handle, &endian4, 4); + goto out; } size = get_size(path); endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; + ret = do_write_check(handle, &endian4, 4); + if (ret) + goto out; set_proc_kptr_restrict(0); check_size = copy_file(handle, path); @@ -847,16 +873,25 @@ static int read_proc_kallsyms(struct tracecmd_output *handle) errno = EINVAL; tracecmd_warning("error in size of file '%s'", path); set_proc_kptr_restrict(1); - return -1; + ret = -1; + goto out; } set_proc_kptr_restrict(1); - handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + if (compress) { + ret = out_compression_end(handle); + if (ret) + goto out; + } +out: + if (!ret) + handle->file_state = TRACECMD_FILE_KALLSYMS; + else + out_compression_reset(handle); + return ret; } -static int read_ftrace_printk(struct tracecmd_output *handle) +static int read_ftrace_printk(struct tracecmd_output *handle, bool compress) { unsigned int size, check_size, endian4; struct stat st; @@ -873,6 +908,8 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + if (compress) + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -894,11 +931,14 @@ static int read_ftrace_printk(struct tracecmd_output *handle) } out: - handle->file_state = TRACECMD_FILE_PRINTK; put_tracing_file(path); + if (compress && out_compression_end(handle)) + return -1; + handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); + out_compression_reset(handle); return -1; } @@ -1270,21 +1310,25 @@ int tracecmd_output_write_init(struct tracecmd_output *handler) int tracecmd_output_write_headers(struct tracecmd_output *handler, struct tracecmd_event_list *list) { + bool compress = false; + if (!handler || handler->file_state < TRACECMD_FILE_ALLOCATED) return -1; /* Write init data, if not written yet */ if (handler->file_state < TRACECMD_FILE_INIT && tracecmd_output_write_init(handler)) return -1; - if (read_header_files(handler)) + if (handler->compress) + compress = true; + if (read_header_files(handler, compress)) return -1; - if (read_ftrace_files(handler)) + if (read_ftrace_files(handler, compress)) return -1; - if (read_event_files(handler, list)) + if (read_event_files(handler, list, compress)) return -1; - if (read_proc_kallsyms(handler)) + if (read_proc_kallsyms(handler, compress)) return -1; - if (read_ftrace_printk(handler)) + if (read_ftrace_printk(handler, compress)) return -1; return 0; } @@ -1529,9 +1573,19 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) handle->file_state); return -1; } + + if (handle->compress) + out_compression_start(handle); + ret = save_tracing_file_data(handle, "saved_cmdlines"); - if (ret < 0) + if (ret < 0) { + out_compression_reset(handle); return ret; + } + + if (handle->compress && out_compression_end(handle)) + return -1; + handle->file_state = TRACECMD_FILE_CMD_LINES; return 0; } From patchwork Thu Jul 29 05:08:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9D7BC432BE for ; Thu, 29 Jul 2021 05:10:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D21CC61052 for ; Thu, 29 Jul 2021 05:10:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233931AbhG2FK0 (ORCPT ); Thu, 29 Jul 2021 01:10:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233700AbhG2FK0 (ORCPT ); Thu, 29 Jul 2021 01:10:26 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89F8DC061757 for ; Wed, 28 Jul 2021 22:10:23 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id h8so6414377ede.4 for ; Wed, 28 Jul 2021 22:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kD3+Fb2srAq0Ymc8SYIc9EwpBab2zSxSmmk3yxgTfio=; b=PXCTHVrdorkLNJf/dOH621uuS95dFSFojdTmBi13V0xH21QsV5bq4woerywjJY75G/ mlq0bc2lUygR9w6esMMu28a5oQaOFLVHFw8dgspLGnPpfr5A+Ca97uy51LRLCtjqwNgh UmticKISt9TZrp9qoklJ7g5co2gmSW42Eg9mvE6uydi5KuLx8k7S+AtP/pO6woqG3pj1 NzSwaGrRweFdqBavxE9eITqVP0wsGOKomDIvnPxF4AMyPwc3K9x5UeASoS9qcT5Ynyw5 pDuzWg04bzIiVPo/EgmL68hs77+Ev9Jpy4PVbkke0ADWhIkh6v8AQ3WUjVJnznsNc325 0xhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kD3+Fb2srAq0Ymc8SYIc9EwpBab2zSxSmmk3yxgTfio=; b=bB4yiMnUCXZw2+cfPijwhrGejRmABIRHsyw2qbSLtqLj16kwgDUY3XFdGfnXKlmVud e8hEosKPFt4Tp0svU87BnIX7RM/z//WXz+isiWg00WjcyL3KwlabJcS8Y7mkS9Ei4CrO lJJrkKPeQxKg75avnfm83PBIanKwwAVqyoeupt0l70eRQgDkJDfXCjc46K+FIj0khuyJ m2wpGnbVrmg4VqSRWjaI5QnKGv9CIBMKtFwZBQLDXWKzh073/OSffJsykMTFeuX8+lks 4Mt+qrTE4NX+e9VGdgCcgN4CCWff8qyBtflPz+XaBN0/lvOlEbCuhil/xCoCTOgt2QQi wCoQ== X-Gm-Message-State: AOAM532HGgxcEWf/IQmza/s2zJFSiUjkcYRmSbeKJFZhqKsrmn5VCBzi dnmMFSoxev9eKtRMbtHT86W8cjGWfQ7bxA== X-Google-Smtp-Source: ABdhPJwgyPX8z2S1lnV5R9/elacMkhWWjUJJ8Zf58F5zjFp/o3oU+KnN8VXEmatseai7va7qHT+1pA== X-Received: by 2002:a05:6402:781:: with SMTP id d1mr3992984edy.32.1627535422137; Wed, 28 Jul 2021 22:10:22 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:21 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 21/87] trace-cmd library: Add internal helper functon for writing headers before file sections Date: Thu, 29 Jul 2021 08:08:53 +0300 Message-Id: <20210729050959.12263-22-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Introduce headers before each file section, in trace file version 7. The section header has the following format: <2 bytes>, header ID , null terminated ASCII string, description of the header <2 bytes>, section flags: 1: the section is compressed <4 bytes>, size of the section Signed-off-by: Tzvetomir Stoyanov (VMware) --- include/trace-cmd/trace-cmd.h | 5 ++ .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/include/trace-cmd-local.h | 4 ++ lib/trace-cmd/trace-output.c | 70 +++++++++++++++++++ 4 files changed, 80 insertions(+) diff --git a/include/trace-cmd/trace-cmd.h b/include/trace-cmd/trace-cmd.h index 6984db86..6c76b928 100644 --- a/include/trace-cmd/trace-cmd.h +++ b/include/trace-cmd/trace-cmd.h @@ -15,6 +15,11 @@ enum tracecmd_open_flags { TRACECMD_FL_LOAD_NO_PLUGINS = 1 << 0, /* Do not load plugins */ TRACECMD_FL_LOAD_NO_SYSTEM_PLUGINS = 1 << 1, /* Do not load system plugins */ }; + +enum tracecmd_section_flags { + TRACECMD_SEC_FL_COMPRESS = 1 << 0, /* the section is compressed */ +}; + struct tracecmd_input *tracecmd_open_head(const char *file, int flags); struct tracecmd_input *tracecmd_open(const char *file, int flags); struct tracecmd_input *tracecmd_open_fd(int fd, int flags); diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ee224903..77948fbb 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -135,6 +135,7 @@ enum { TRACECMD_OPTION_TIME_SHIFT, TRACECMD_OPTION_GUEST, TRACECMD_OPTION_TSC2NSEC, + TRACECMD_OPTION_MAX, }; enum { diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index f799ea8d..2a23d9e0 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -42,6 +42,10 @@ int in_uncompress_block(struct tracecmd_input *handle); unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, int fd, unsigned long long max, unsigned long long *write_size); +unsigned long long +out_write_section_header(struct tracecmd_output *handle, unsigned short header_id, + char *description, enum tracecmd_section_flags flags, bool option); +int out_update_section_header(struct tracecmd_output *handle, unsigned long long offset); off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 5d3fd58f..74192ea2 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -366,6 +366,76 @@ int tracecmd_ftrace_enable(int set) return ret; } +__hidden unsigned long long +out_write_section_header(struct tracecmd_output *handle, unsigned short header_id, + char *description, enum tracecmd_section_flags flags, bool option) +{ + tsize_t endian8; + tsize_t offset; + short endian2; + int size; + + if (header_id >= TRACECMD_OPTION_MAX) + return -1; + if (handle->file_version < 7) + return 0; + if (!handle->compress) + flags &= ~TRACECMD_SEC_FL_COMPRESS; + offset = do_lseek(handle, 0, SEEK_CUR); + if (option) { + endian8 = convert_endian_8(handle, offset); + if (!tracecmd_add_option(handle, header_id, 8, &endian8)) + return -1; + } + /* Section ID */ + endian2 = convert_endian_2(handle, header_id); + if (do_write_check(handle, &endian2, 2)) + return (off_t)-1; + + /* Section description */ + if (do_write_check(handle, description, strlen(description) + 1)) + return (off_t)-1; + /* Section flags */ + endian2 = convert_endian_2(handle, flags); + if (do_write_check(handle, &endian2, 2)) + return (off_t)-1; + + offset = do_lseek(handle, 0, SEEK_CUR); + size = 0; + /* Reserve for section size */ + if (do_write_check(handle, &size, 4)) + return (off_t)-1; + return offset; +} + +__hidden int out_update_section_header(struct tracecmd_output *handle, unsigned long long offset) +{ + unsigned long long current; + unsigned int endian4; + int size; + + if (handle->file_version < 7 || offset == 0) + return 0; + + current = do_lseek(handle, 0, SEEK_CUR); + /* The real size is the difference between the saved offset and + * the current offset - 4 bytes, the reserved space for the section size. + */ + size = current - offset; + if (size < 4) + return -1; + size -= 4; + if (do_lseek(handle, offset, SEEK_SET) == (off_t)-1) + return -1; + + endian4 = convert_endian_4(handle, size); + if (do_write_check(handle, &endian4, 4)) + return -1; + if (do_lseek(handle, current, SEEK_SET) == (off_t)-1) + return -1; + return 0; +} + static int read_header_files(struct tracecmd_output *handle, bool compress) { tsize_t size, check_size, endian8; From patchwork Thu Jul 29 05:08:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 973AAC43214 for ; Thu, 29 Jul 2021 05:10:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7FC6561052 for ; Thu, 29 Jul 2021 05:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233932AbhG2FK1 (ORCPT ); Thu, 29 Jul 2021 01:10:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233929AbhG2FK1 (ORCPT ); Thu, 29 Jul 2021 01:10:27 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A2E3C061757 for ; Wed, 28 Jul 2021 22:10:24 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id x11so7515726ejj.8 for ; Wed, 28 Jul 2021 22:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KlkRxkg7MkzkBDXkWogFob5bdRiFt8Jp3BlRur7Ub3c=; b=HbyN1cQSDTFpEzyF2bb/gQk/FyZfh3/rA3p5vHUTuC5KXJxrlg1msNVezTwZdDDZUr 1pYsUy4ZXWmfO+BKsXC3ymiKbJ/GwlfQCs7KNxLublKsDJZoYeemj6PZmDebGkrVyjyb d+dfdrGGBlXTFiiSGlu++mw1r445KxEbrQ4etQiZV0Jx5vehRNMMJTe5APIrU7PyeaJS wp16/dvLKFawGeMIjMhHgi3vdfz6UqWe3zM7Mx8ccfwqhyOflTweG8kX6zWumOFP/Zr9 57dJAYBnXNAp7oB+1/6KWQDVu+MrttrdSidpDC4m/mlEqMU98wJ3fZn/mNArjylCRpqq vysw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KlkRxkg7MkzkBDXkWogFob5bdRiFt8Jp3BlRur7Ub3c=; b=eyX76llbMYUtLVXCFyzBcP4aTgfbpolYhTrGZoOEB7tDxKF4nr9OYl+7vlyrKnR0Ua pb2k2MOpMNZFIBQWXhSxQYAAJv2UOoWy4hYEiCIhuNEfpxGO9+I/2q0QK9V/67WJvHm/ kefZDEKKMn2o5ILypmWXDosj64E8JdzwQ9TLpsGqVPAzu4Zl+8HM7mqtGHydWgXYDJiw hoSR/rqO7iOAhGwx1tRQ9Tr3+lezsSmi8gZyRNkMlTdaN8zSW88R6CZHbIuzCk8/NCqh GcrBbifxhzspnwbjj3QhZaJdFz1AsxJJdeuTt1k6XeKgBTKjKOqDvlp0ivofiTKfNcCn l4qA== X-Gm-Message-State: AOAM531pfLWfVDxAmrWE0fxTu/3cG4PuW1SbXuwHy/fHZaVSZvwh741D DRVIgwoP6AWPJsHOVyCsuLw= X-Google-Smtp-Source: ABdhPJxargtwtKhcQVDEGnMrLsOhzmv81Aw1ub1O/735sEbj+N0hpUiykWhLGPTCDeGJzLz+t/2vJw== X-Received: by 2002:a17:906:138d:: with SMTP id f13mr2981980ejc.34.1627535423198; Wed, 28 Jul 2021 22:10:23 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:22 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 22/87] trace-cmd library: Write header before file sections Date: Thu, 29 Jul 2021 08:08:54 +0300 Message-Id: <20210729050959.12263-23-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Headers are added before these file secrtions, in trace file version 7: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name New options are defined for each of these sections, holding the section's offset into the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 6 ++ lib/trace-cmd/trace-output.c | 69 +++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 77948fbb..b193d6de 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -135,6 +135,12 @@ enum { TRACECMD_OPTION_TIME_SHIFT, TRACECMD_OPTION_GUEST, TRACECMD_OPTION_TSC2NSEC, + TRACECMD_OPTION_HEADER_INFO, + TRACECMD_OPTION_FTRACE_EVENTS, + TRACECMD_OPTION_EVENT_FORMATS, + TRACECMD_OPTION_KALLSYMS, + TRACECMD_OPTION_PRINTK, + TRACECMD_OPTION_CMDLINES, TRACECMD_OPTION_MAX, }; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 74192ea2..a5c11eba 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -438,8 +438,10 @@ __hidden int out_update_section_header(struct tracecmd_output *handle, unsigned static int read_header_files(struct tracecmd_output *handle, bool compress) { + enum tracecmd_section_flags flags = 0; tsize_t size, check_size, endian8; struct stat st; + tsize_t offset; char *path; int fd = -1; int ret; @@ -454,6 +456,13 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) if (!path) return -1; + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + offset = out_write_section_header(handle, TRACECMD_OPTION_HEADER_INFO, + "headers", flags, true); + if (offset == (off_t)-1) + return -1; + if (compress) out_compression_start(handle); ret = stat(path, &st); @@ -469,6 +478,8 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) goto out_close; if (do_write_check(handle, &size, 8)) goto out_close; + if (out_update_section_header(handle, offset)) + goto out_close; if (compress && out_compression_end(handle)) goto out_close; return 0; @@ -523,6 +534,8 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) put_tracing_file(path); if (compress && out_compression_end(handle)) goto out_close; + if (out_update_section_header(handle, offset)) + goto out_close; handle->file_state = TRACECMD_FILE_HEADERS; return 0; @@ -759,8 +772,10 @@ create_event_list_item(struct tracecmd_output *handle, static int read_ftrace_files(struct tracecmd_output *handle, bool compress) { + enum tracecmd_section_flags flags = 0; struct list_event_system *systems = NULL; struct tracecmd_event_list list = { .glob = "ftrace/*" }; + tsize_t offset; int ret; if (!check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS)) { @@ -769,6 +784,13 @@ static int read_ftrace_files(struct tracecmd_output *handle, bool compress) return -1; } + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + offset = out_write_section_header(handle, TRACECMD_OPTION_FTRACE_EVENTS, + "ftrace events", flags, true); + if (offset == (off_t)-1) + return -1; + create_event_list_item(handle, &systems, &list); if (compress) out_compression_start(handle); @@ -780,6 +802,10 @@ static int read_ftrace_files(struct tracecmd_output *handle, bool compress) out_compression_reset(handle); } free_list_events(systems); + if (ret) + return ret; + if (out_update_section_header(handle, offset)) + return -1; handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; @@ -802,11 +828,13 @@ create_event_list(struct tracecmd_output *handle, static int read_event_files(struct tracecmd_output *handle, struct tracecmd_event_list *event_list, bool compress) { + enum tracecmd_section_flags flags = 0; struct list_event_system *systems; struct list_event_system *slist; struct tracecmd_event_list *list; struct tracecmd_event_list all_events = { .glob = "*/*" }; int count = 0; + tsize_t offset; int endian4; int ret; @@ -815,6 +843,13 @@ static int read_event_files(struct tracecmd_output *handle, handle->file_state); return -1; } + + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + offset = out_write_section_header(handle, TRACECMD_OPTION_EVENT_FORMATS, + "events format", flags, true); + if (offset == (off64_t)-1) + return -1; /* * If any of the list is the special keyword "all" then * just do all files. @@ -854,6 +889,8 @@ static int read_event_files(struct tracecmd_output *handle, if (ret) goto out_free; } + ret = out_update_section_header(handle, offset); + out_free: if (!ret) handle->file_state = TRACECMD_FILE_ALL_EVENTS; @@ -907,8 +944,10 @@ err: static int read_proc_kallsyms(struct tracecmd_output *handle, bool compress) { + enum tracecmd_section_flags flags = 0; unsigned int size, check_size, endian4; const char *path = "/proc/kallsyms"; + tsize_t offset; struct stat st; int ret; @@ -921,6 +960,13 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, bool compress) if (handle->kallsyms) path = handle->kallsyms; + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + offset = out_write_section_header(handle, TRACECMD_OPTION_KALLSYMS, + "kallsyms", flags, true); + if (offset == (off64_t)-1) + return -1; + if (compress) out_compression_start(handle); ret = stat(path, &st); @@ -953,6 +999,7 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, bool compress) if (ret) goto out; } + ret = out_update_section_header(handle, offset); out: if (!ret) handle->file_state = TRACECMD_FILE_KALLSYMS; @@ -963,7 +1010,9 @@ out: static int read_ftrace_printk(struct tracecmd_output *handle, bool compress) { + enum tracecmd_section_flags flags = 0; unsigned int size, check_size, endian4; + tsize_t offset; struct stat st; char *path; int ret; @@ -978,6 +1027,12 @@ static int read_ftrace_printk(struct tracecmd_output *handle, bool compress) if (!path) return -1; + if (compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + offset = out_write_section_header(handle, TRACECMD_OPTION_PRINTK, "printk", flags, true); + if (offset == (off64_t)-1) + return -1; + if (compress) out_compression_start(handle); ret = stat(path, &st); @@ -1004,6 +1059,8 @@ static int read_ftrace_printk(struct tracecmd_output *handle, bool compress) put_tracing_file(path); if (compress && out_compression_end(handle)) return -1; + if (out_update_section_header(handle, offset)) + return -1; handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: @@ -1636,6 +1693,8 @@ tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, int tracecmd_write_cmdlines(struct tracecmd_output *handle) { + enum tracecmd_section_flags flags = 0; + tsize_t offset; int ret; if (!check_out_state(handle, TRACECMD_FILE_CMD_LINES)) { @@ -1644,6 +1703,13 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return -1; } + if (handle->compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + offset = out_write_section_header(handle, TRACECMD_OPTION_CMDLINES, + "command lines", flags, true); + if (offset == (off_t)-1) + return -1; + if (handle->compress) out_compression_start(handle); @@ -1656,6 +1722,9 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) if (handle->compress && out_compression_end(handle)) return -1; + if (out_update_section_header(handle, offset)) + return -1; + handle->file_state = TRACECMD_FILE_CMD_LINES; return 0; } From patchwork Thu Jul 29 05:08:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50BC5C4338F for ; Thu, 29 Jul 2021 05:10:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 386D461052 for ; Thu, 29 Jul 2021 05:10:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233929AbhG2FKa (ORCPT ); Thu, 29 Jul 2021 01:10:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbhG2FK3 (ORCPT ); Thu, 29 Jul 2021 01:10:29 -0400 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE9C3C061757 for ; Wed, 28 Jul 2021 22:10:25 -0700 (PDT) Received: by mail-ed1-x52d.google.com with SMTP id f13so6341292edq.13 for ; Wed, 28 Jul 2021 22:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6+TeRzZojdNThy+IvprDry+WOveWiv/JX9kWOJms54I=; b=gRm/6Wh7LiqyrwogFu7LQKCoi3DV7OzFyTH//is/9ZMxbvZIpXVrIh5TsP/Nd9o1gp ssoKMEXbOT1l8kOxE/U8IdFYAG5GxFPllR4NsvpQe1cBzalGi6XNdNoCi4L+C5L9PMtK OkKpSwk9Dbv6He+hm1MI7OwIVSd9+XunTtKf1twmMb/TIzTeE3tg4Ek45zM0l0AUHsGE gw4vBEufLjPLyXoER2zlddQwfpeTJ52PMOPG14XQMi34xlTNbnrlfoTnZ+2/bnqqcy0N e6WMKUZ8/CElYTNS+00AkhDshYFHIoe8qo1tkPpKzCdNQ/TFgI029WOnuVx/5spuDbmA nl/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6+TeRzZojdNThy+IvprDry+WOveWiv/JX9kWOJms54I=; b=Ao4JdTrQJg+pw+2M94tG7WvuG6Re+78I/pN+WVn1BpKPgkppSS1DXLXR8BE++7oK/V tuPmatvxB3qt2uWEvEI/v8mmUUIP3eQz3MI3LBotEwL8mQpynQi2QGC1Rw+YjR5szbJk tbFSv/rItlreMQ4cYqM0IzAIqm0VyC+tnlY/cShLv8BbNaG7rgUfzJqy/p96c60lP/tp uDG1l15SzLV9hJhtw23s4hgH96oq0iKwN9rbIA2Lh1LDeTEjcCSJpQQqhwwMt8PhB9Vt yppi5htIWq02TI7d/eM7hmyWaeSxE3mzb575O0c4GTQ4qEO3+pnM80piV3xdFIEa0cuH HF1w== X-Gm-Message-State: AOAM530E+F7cJbtmN4Gj1421rrEwQvS6Lt34ehTd1ZbumsBhBqDhGCZ2 icsqJGQWoqHmUslIMPRKgTIxps+zdcuhqg== X-Google-Smtp-Source: ABdhPJxmYDF3FIBuAw3qduuroic/wIXfLgyHgh2idXvYsTvdBOWuw9GUUH4ZG0M+Lvs5HO7aZ9DfKA== X-Received: by 2002:aa7:c6d2:: with SMTP id b18mr3893158eds.266.1627535424200; Wed, 28 Jul 2021 22:10:24 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:23 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 23/87] trace-cmd library: Refactor the logic for writing trace data in the file Date: Thu, 29 Jul 2021 08:08:55 +0300 Message-Id: <20210729050959.12263-24-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a trace buffer data are written in the trace file, the buffer option in the file metadata is updated with the file offset of the tracing data. Hide this logic into the trace-cmd library. Added new APIs: tracecmd_add_buffer_info() tracecmd_write_buffer_info() Changed APIs: tracecmd_append_buffer_cpu_data() Removed APIs: tracecmd_add_buffer_option() Refactored the internal logic of tracecmd_write_cpu_data() API to be suitable for adding trace data compression. The size and the offset of the trace data is saved in the file right after the data is written. The old logic calculates the size and offset in advance, but when the trace data is compressed it is hard to use that approach. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 10 +- lib/trace-cmd/include/trace-cmd-local.h | 16 + lib/trace-cmd/trace-output.c | 314 ++++++++++++------ tracecmd/trace-listen.c | 2 +- tracecmd/trace-record.c | 18 +- 5 files changed, 240 insertions(+), 120 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index b193d6de..4e718c40 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -305,8 +305,8 @@ struct tracecmd_option * tracecmd_add_option_v(struct tracecmd_output *handle, unsigned short id, const struct iovec *vector, int count); -struct tracecmd_option *tracecmd_add_buffer_option(struct tracecmd_output *handle, - const char *name, int cpus); +int tracecmd_add_buffer_info(struct tracecmd_output *handle, const char *name, int cpus); +int tracecmd_write_buffer_info(struct tracecmd_output *handle); int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); @@ -318,13 +318,11 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file); int tracecmd_write_cpu_data(struct tracecmd_output *handle, - int cpus, char * const *cpu_data_files); + int cpus, char * const *cpu_data_files, const char *buff_name); int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files); - + const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 2a23d9e0..e54ba7e1 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -24,6 +24,14 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +struct data_file_write { + unsigned long long file_size; + unsigned long long write_size; + unsigned long long soffset; + unsigned long long data_offset; + unsigned long long doffset; +}; + void tracecmd_compress_init(void); void tracecmd_compress_free(void); @@ -47,6 +55,14 @@ out_write_section_header(struct tracecmd_output *handle, unsigned short header_i char *description, enum tracecmd_section_flags flags, bool option); int out_update_section_header(struct tracecmd_output *handle, unsigned long long offset); +struct cpu_data_source { + int fd; + int size; + off64_t offset; +}; + +int out_write_cpu_data(struct tracecmd_output *handle, int cpus, + struct cpu_data_source *data, const char *buff_name); off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a5c11eba..6a44a99b 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -39,6 +39,14 @@ struct tracecmd_option { struct list_head list; }; +struct tracecmd_buffer { + int cpus; + void *name; + tsize_t offset; + struct tracecmd_option *option; + struct list_head list; +}; + enum { OUTPUT_FL_SEND_META = (1 << 0), }; @@ -61,6 +69,7 @@ struct tracecmd_output { struct tracecmd_compression *compress; struct list_head options; + struct list_head buffers; struct tracecmd_msg_handle *msg_handle; char *trace_clock; }; @@ -201,6 +210,7 @@ bool tracecmd_get_quiet(struct tracecmd_output *handle) void tracecmd_output_free(struct tracecmd_output *handle) { struct tracecmd_option *option; + struct tracecmd_buffer *buffer; if (!handle) return; @@ -211,6 +221,13 @@ void tracecmd_output_free(struct tracecmd_output *handle) if (handle->pevent) tep_unref(handle->pevent); + while (!list_empty(&handle->buffers)) { + buffer = container_of(handle->buffers.next, + struct tracecmd_buffer, list); + list_del(&buffer->list); + free(buffer->name); + free(buffer); + } while (!list_empty(&handle->options)) { option = container_of(handle->options.next, struct tracecmd_option, list); @@ -1158,6 +1175,7 @@ struct tracecmd_output *tracecmd_output_allocate(int fd) handle->big_endian = false; list_head_init(&handle->options); + list_head_init(&handle->buffers); handle->file_state = TRACECMD_FILE_ALLOCATED; @@ -1661,15 +1679,14 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -struct tracecmd_option * -tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, - int cpus) +static struct tracecmd_option * +add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; int size = 8 + strlen(name) + 1; - buf = malloc(size); + buf = calloc(1, size); if (!buf) { tracecmd_warning("Failed to malloc buffer"); return NULL; @@ -1691,6 +1708,52 @@ tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, return option; } +int tracecmd_add_buffer_info(struct tracecmd_output *handle, const char *name, int cpus) +{ + struct tracecmd_buffer *buf; + + buf = calloc(1, sizeof(struct tracecmd_buffer)); + if (!buf) + return -1; + buf->name = strdup(name); + buf->cpus = cpus; + if (!buf->name) { + free(buf); + return -1; + } + list_add_tail(&buf->list, &handle->buffers); + return 0; +} + +int tracecmd_write_buffer_info(struct tracecmd_output *handle) +{ + struct tracecmd_option *option; + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + option = add_buffer_option(handle, buf->name, buf->cpus); + if (!option) + return -1; + buf->option = option; + } + + return 0; +} + +static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) +{ + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + if (strlen(name) == strlen(buf->name) && !strcmp(name, buf->name)) { + if (!buf->option) + break; + return buf->option->offset; + } + } + return 0; +} + int tracecmd_write_cmdlines(struct tracecmd_output *handle) { enum tracecmd_section_flags flags = 0; @@ -1808,6 +1871,37 @@ out: return ret; } +static int update_buffer_cpu_offset(struct tracecmd_output *handle, + const char *name, tsize_t offset) +{ + tsize_t b_offset; + tsize_t current; + + b_offset = get_buffer_file_offset(handle, name); + if (!b_offset) { + tracecmd_warning("Cannot find description for buffer %s\n", name); + return -1; + } + current = do_lseek(handle, 0, SEEK_CUR); + + /* Go to the option data, where will write the offest */ + if (do_lseek(handle, b_offset, SEEK_SET) == (off64_t)-1) { + tracecmd_warning("could not seek to %lld\n", b_offset); + return -1; + } + + if (do_write_check(handle, &offset, 8)) + return -1; + + /* Go back to end of file */ + if (do_lseek(handle, current, SEEK_SET) == (off64_t)-1) { + tracecmd_warning("could not seek to %lld\n", offset); + return -1; + } + return 0; +} + + static char *get_clock(struct tracecmd_output *handle) { struct tracefs_instance *inst; @@ -1827,120 +1921,158 @@ static char *get_clock(struct tracecmd_output *handle) return handle->trace_clock; } -int tracecmd_write_cpu_data(struct tracecmd_output *handle, - int cpus, char * const *cpu_data_files) + +__hidden int out_write_cpu_data(struct tracecmd_output *handle, + int cpus, struct cpu_data_source *data, const char *buff_name) { - off64_t *offsets = NULL; - unsigned long long *sizes = NULL; - off64_t offset; + struct data_file_write *data_files = NULL; + tsize_t data_offs, offset; unsigned long long endian8; - char *clock = NULL; - off64_t check_size; - char *file; - struct stat st; + unsigned long long read_size; + char *clock; int ret; int i; /* This can be called multiple times (when recording instances) */ ret = handle->file_state == TRACECMD_FILE_CPU_FLYRECORD ? 0 : - check_out_state(handle, TRACECMD_FILE_CPU_FLYRECORD); + check_file_state(handle->file_version, + handle->file_state, + TRACECMD_FILE_CPU_FLYRECORD); if (ret < 0) { tracecmd_warning("Cannot write trace data into the file, unexpected state 0x%X", handle->file_state); goto out_free; } + data_offs = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, "flyrecord", 10)) goto out_free; - offsets = malloc(sizeof(*offsets) * cpus); - if (!offsets) + data_files = calloc(cpus, sizeof(struct data_file_write)); + if (!data_files) goto out_free; - sizes = malloc(sizeof(*sizes) * cpus); - if (!sizes) - goto out_free; - - offset = lseek64(handle->fd, 0, SEEK_CUR); - /* hold any extra data for data */ - offset += cpus * (16); + for (i = 0; i < cpus; i++) { + data_files[i].file_size = data[i].size; + /* Write 0 for trace data offset and size and store offsets of these fields */ + if (handle->file_version < 7) { + endian8 = 0; + data_files[i].doffset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + data_files[i].soffset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + } + } - /* - * Unfortunately, the trace_clock data was placed after the - * cpu data, and wasn't accounted for with the offsets. - * We need to save room for the trace_clock file. This means - * we need to find the size of it before we define the final - * offsets. - */ + update_buffer_cpu_offset(handle, buff_name, data_offs); clock = get_clock(handle); - if (!clock) + if (clock && save_clock(handle, clock)) goto out_free; - /* Save room for storing the size */ - offset += 8; - offset += strlen(clock); - /* 2 bytes for [] around the clock */ - offset += 2; - - /* Page align offset */ - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); for (i = 0; i < cpus; i++) { - file = cpu_data_files[i]; - ret = stat(file, &st); - if (ret < 0) { - tracecmd_warning("can not stat '%s'", file); + data_files[i].data_offset = do_lseek(handle, 0, SEEK_CUR); + /* Page align offset */ + data_files[i].data_offset = (data_files[i].data_offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].data_offset = do_lseek(handle, data_files[i].data_offset, SEEK_SET); + if (data_files[i].data_offset == (off64_t)-1) + goto out_free; + if (!tracecmd_get_quiet(handle)) + fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", + i, (unsigned long long) data_files[i].data_offset); + if (lseek64(data[i].fd, data[i].offset, SEEK_SET) == (off64_t)-1) goto out_free; + if (data[i].size) { + read_size = copy_file_fd(handle, data[i].fd); + if (read_size != data_files[i].file_size) { + errno = EINVAL; + tracecmd_warning("did not match size of %lld to %lld", + read_size, data_files[i].file_size); + goto out_free; + } + } else { + data_files[i].write_size = 0; } - offsets[i] = offset; - sizes[i] = st.st_size; - offset += st.st_size; - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); - endian8 = convert_endian_8(handle, offsets[i]); - if (do_write_check(handle, &endian8, 8)) + /* Write the real CPU data offset in the file */ + if (do_lseek(handle, data_files[i].doffset, SEEK_SET) == (off64_t)-1) goto out_free; - endian8 = convert_endian_8(handle, sizes[i]); + endian8 = convert_endian_8(handle, data_files[i].data_offset); if (do_write_check(handle, &endian8, 8)) goto out_free; - } - - if (save_clock(handle, clock)) - goto out_free; - - for (i = 0; i < cpus; i++) { - if (!tracecmd_get_quiet(handle)) - fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", - i, (unsigned long long) offsets[i]); - offset = lseek64(handle->fd, offsets[i], SEEK_SET); - if (offset == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offsets[i]); + /* Write the real CPU data size in the file */ + if (do_lseek(handle, data_files[i].soffset, SEEK_SET) == (off64_t)-1) goto out_free; - } - check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != sizes[i]) { - errno = EINVAL; - tracecmd_warning("did not match size of %lld to %lld", - check_size, sizes[i]); + endian8 = convert_endian_8(handle, data_files[i].write_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + data_files[i].write_size; + if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) goto out_free; - } if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", - (unsigned long long)check_size); + (unsigned long long)data_files[i].write_size); } - free(offsets); - free(sizes); + if (do_lseek(handle, 0, SEEK_END) == (off64_t)-1) + goto out_free; + free(data_files); handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; return 0; out_free: - free(offsets); - free(sizes); + do_lseek(handle, 0, SEEK_END); + free(data_files); return -1; } +int tracecmd_write_cpu_data(struct tracecmd_output *handle, + int cpus, char * const *cpu_data_files, const char *buff_name) +{ + struct cpu_data_source *data; + struct stat st; + int size = 0; + int ret; + int i; + + data = calloc(cpus, sizeof(struct cpu_data_source)); + if (!data) + return -1; + for (i = 0; i < cpus; i++) + data[i].fd = -1; + for (i = 0; i < cpus; i++) { + ret = stat(cpu_data_files[i], &st); + if (ret < 0) { + tracecmd_warning("can not stat '%s'", cpu_data_files[i]); + break; + } + data[i].fd = open(cpu_data_files[i], O_RDONLY); + if (data[i].fd < 0) { + tracecmd_warning("Can't read '%s'", data[i].fd); + break; + } + + data[i].size = st.st_size; + data[i].offset = 0; + size += st.st_size; + } + + if (i < cpus) + ret = -1; + else + ret = out_write_cpu_data(handle, cpus, data, buff_name); + + for (i = 0; i < cpus; i++) { + if (data[i].fd >= 0) + close(data[i].fd); + } + free(data); + return ret; +} + int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { @@ -1949,41 +2081,20 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, ret = tracecmd_write_cpus(handle, cpus); if (ret) return ret; - + ret = tracecmd_write_buffer_info(handle); + if (ret) + return ret; ret = tracecmd_write_options(handle); if (ret) return ret; - return tracecmd_write_cpu_data(handle, cpus, cpu_data_files); + return tracecmd_write_cpu_data(handle, cpus, cpu_data_files, ""); } int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files) + const char *name, int cpus, char * const *cpu_data_files) { - tsize_t offset; - stsize_t ret; - - offset = lseek64(handle->fd, 0, SEEK_CUR); - - /* Go to the option data, where will write the offest */ - ret = lseek64(handle->fd, option->offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); - return -1; - } - - if (do_write_check(handle, &offset, 8)) - return -1; - - /* Go back to end of file */ - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offset); - return -1; - } - - return tracecmd_write_cpu_data(handle, cpus, cpu_data_files); + return tracecmd_write_cpu_data(handle, cpus, cpu_data_files, name); } struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) @@ -2029,6 +2140,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->file_version = tracecmd_get_in_file_version(ihandle); handle->options_start = tracecmd_get_options_offset(ihandle); list_head_init(&handle->options); + list_head_init(&handle->buffers); if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index 0cb70b7d..d812145b 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -610,7 +610,7 @@ static int put_together_file(int cpus, int ofd, const char *node, if (ret) goto out; } - ret = tracecmd_write_cpu_data(handle, cpus, temp_files); + ret = tracecmd_write_cpu_data(handle, cpus, temp_files, ""); out: tracecmd_output_close(handle); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 295fe633..350d2811 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4187,7 +4187,6 @@ static void touch_file(const char *file) } static void append_buffer(struct tracecmd_output *handle, - struct tracecmd_option *buffer_option, struct buffer_instance *instance, char **temp_files) { @@ -4215,7 +4214,7 @@ static void append_buffer(struct tracecmd_output *handle, touch_file(temp_files[i]); } - tracecmd_append_buffer_cpu_data(handle, buffer_option, + tracecmd_append_buffer_cpu_data(handle, tracefs_instance_get_name(instance->tracefs), cpu_count, temp_files); for (i = 0; i < instance->cpu_count; i++) { @@ -4465,7 +4464,7 @@ static void write_guest_file(struct buffer_instance *instance) die("failed to allocate memory"); } - if (tracecmd_write_cpu_data(handle, cpu_count, temp_files) < 0) + if (tracecmd_write_cpu_data(handle, cpu_count, temp_files, "") < 0) die("failed to write CPU data"); tracecmd_output_close(handle); @@ -4508,7 +4507,6 @@ error: static void record_data(struct common_record_context *ctx) { - struct tracecmd_option **buffer_options; struct tracecmd_output *handle; struct buffer_instance *instance; bool local = false; @@ -4578,9 +4576,6 @@ static void record_data(struct common_record_context *ctx) } if (buffers) { - buffer_options = malloc(sizeof(*buffer_options) * buffers); - if (!buffer_options) - die("Failed to allocate buffer options"); i = 0; for_each_instance(instance) { int cpus = instance->cpu_count != local_cpu_count ? @@ -4588,10 +4583,9 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; - - buffer_options[i++] = tracecmd_add_buffer_option(handle, - tracefs_instance_get_name(instance->tracefs), - cpus); + tracecmd_add_buffer_info(handle, + tracefs_instance_get_name(instance->tracefs), + cpus); add_buffer_stat(handle, instance); } } @@ -4626,7 +4620,7 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; print_stat(instance); - append_buffer(handle, buffer_options[i++], instance, temp_files); + append_buffer(handle, instance, temp_files); } } From patchwork Thu Jul 29 05:08:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30954C432BE for ; Thu, 29 Jul 2021 05:10:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 12E2061052 for ; Thu, 29 Jul 2021 05:10:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233941AbhG2FKb (ORCPT ); Thu, 29 Jul 2021 01:10:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233938AbhG2FKa (ORCPT ); Thu, 29 Jul 2021 01:10:30 -0400 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75461C061765 for ; Wed, 28 Jul 2021 22:10:26 -0700 (PDT) Received: by mail-ed1-x52c.google.com with SMTP id da26so6430009edb.1 for ; Wed, 28 Jul 2021 22:10:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lm1bsc5GF6Gt49CSDosY+JTZ/5fZAVJTfd9TKbeSrcw=; b=kImVaGTZe0PYd3Mu/iLd5hIBhefErK4hu/2L1XBFQegNR67xmpKNGS2qGinMxARaVr uWNQoUcHQfZXJMDAaJU2+LCliK+kNylnrUJIcs4Djmcn5t/A2KF0wOVwXdcC0FoSzcna ORuV4MjJjnOw/SqSfvq04/YjArQG1pD2gjx9P/dQx4fs6jmL3zOb8wnJ8kKHF4XbdK+t cR4egWxbq02FCQaHAnxwwv44BfwP7eB5NX08fLTgEThnN4su6l0BhbUDpU1sDRwXeY/S shQoQpWttHplacIEcCFt9aaD6HWrhA0llMPU9tSQ+QV04aNWXEUxRPn5pZaS/BKS5/2d RxkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lm1bsc5GF6Gt49CSDosY+JTZ/5fZAVJTfd9TKbeSrcw=; b=kl29eVwNjYzK2Gp84AKaUO6o8hBMAZiSBdrc9UTxqx8/d2TRN9qurkMX0VUjlcuIMy LEFk/WA+bQBCIn41Zp3s0WAiyX0C8mc/ZzpdDYEI2EYGUCDQCIao+6Q9P+N7Mp5RDyql LNox9KYhTa2vFN0RzX+rLeg21z6Gs64pfXXK4hMpbSbvvPpG1Yjs2xTqzhhz3sJ2aTtL +Kxd8NVTHIkm6FHOO4qFGjzURVdlHoEjBmwAzfnsOhsqxMhDm5PwI7+Ggy61XaL3AQ48 ivzfyRaXfBNlnO9gvY+MGNBCFnB7JCNzZ2ntW/q6wRTvW+spyJFoKTRZaLf+bWroeL9z pcIA== X-Gm-Message-State: AOAM5316g+JaCi5WubiCazgYfJA2aOkfS8QLPG+hFSAg+//aPaRr70Uj ocqdTkFZcNi9++CqAZU9/XUgzYLxl9HTFg== X-Google-Smtp-Source: ABdhPJzQuYohn8+xcW+zid4T/RLZ9sIpjhYrmipIHoeq2klY8HO2KcGEpkSef5rkbJXXBqmGGrpT+A== X-Received: by 2002:a50:d651:: with SMTP id c17mr2252345edj.69.1627535425136; Wed, 28 Jul 2021 22:10:25 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:24 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 24/87] trace-cmd library: Add local helper function for data compression Date: Thu, 29 Jul 2021 08:08:56 +0300 Message-Id: <20210729050959.12263-25-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The newly added helper functions read data from a file and compress it, before writing into the trace file. The trace data is comressed in chunks, which are page aligned. A new local define is introduced: PAGES_IN_CHUNK which can be used to tune how big a compression chunk is. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 70 ++++++++++++++++++++++++++++++++---- 1 file changed, 64 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 6a44a99b..90625c4e 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -285,18 +285,26 @@ static unsigned long get_size(const char *file) return size; } -static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd) +static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd, unsigned long long max) { + tsize_t rsize = 0; tsize_t size = 0; char buf[BUFSIZ]; stsize_t r; do { - r = read(fd, buf, BUFSIZ); + if (max > 0 && (max - size) < BUFSIZ) + rsize = (max - size); + else + rsize = BUFSIZ; + + r = read(fd, buf, rsize); if (r > 0) { size += r; if (do_write_check(handle, buf, r)) return 0; + if (max > 0 && size >= max) + break; } } while (r > 0); @@ -314,12 +322,62 @@ static tsize_t copy_file(struct tracecmd_output *handle, tracecmd_warning("Can't read '%s'", file); return 0; } - size = copy_file_fd(handle, fd); + size = copy_file_fd(handle, fd, 0); close(fd); return size; } +#define PAGES_IN_CHUNK 10 +__hidden unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size) +{ + unsigned long long rsize = 0; + unsigned long long wsize = 0; + unsigned long long size; + int ret; + + if (handle->compress) { + rsize = max; + ret = tracecmd_compress_copy_from(handle->compress, fd, + PAGES_IN_CHUNK * handle->page_size, + &rsize, &wsize); + if (ret < 0) + return 0; + + size = rsize; + if (write_size) + *write_size = wsize; + } else { + size = copy_file_fd(handle, fd, max); + if (write_size) + *write_size = size; + } + + return size; +} + +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + ret = out_copy_fd_compress(handle, fd, 0, write_size); + if (!ret) + tracecmd_warning("Can't compress '%s'", file); + + close(fd); + return ret; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -516,7 +574,7 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s' size=%lld read=%lld", path, size, check_size); @@ -542,7 +600,7 @@ static int read_header_files(struct tracecmd_output *handle, bool compress) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s'", path); @@ -1984,7 +2042,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (lseek64(data[i].fd, data[i].offset, SEEK_SET) == (off64_t)-1) goto out_free; if (data[i].size) { - read_size = copy_file_fd(handle, data[i].fd); + read_size = copy_file_fd(handle, data[i].fd, data[i].size); if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", From patchwork Thu Jul 29 05:08:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49FDBC4320E for ; Thu, 29 Jul 2021 05:10:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 36D9061052 for ; Thu, 29 Jul 2021 05:10:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233938AbhG2FKb (ORCPT ); Thu, 29 Jul 2021 01:10:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbhG2FKa (ORCPT ); Thu, 29 Jul 2021 01:10:30 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 768E7C061757 for ; Wed, 28 Jul 2021 22:10:27 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id oz16so8446090ejc.7 for ; Wed, 28 Jul 2021 22:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bZV8VezGtTIXq/yNYWV/d3QACsxZ7pN3NcbaApSLeSI=; b=fD0MPuDNLqKvnYGFpteZwd7TXGPsPCQS5cNU4ezAtdjSgra5Xcaa1vxri+Fn5IaG5L 2FfbVUz3/j0MpMY/b6itrSOW17HpMIEqfHLpUOsN+9mu1DnDXgywOy8yxTaB9ShL2/2R dLLqi+aMoU1AHsrR5NIhgYVVXWIEJC6LUCsz/Yx4FQ6U6alNhlE1VOxW09XwbrSo+WUz 7wWfgRqEZGXynQawcIAaOJOrHVUSY4Srn80dXzDyyscmVzaIJuRkCey69SDv4KzGwe2R N98NgfX7WNaDdoGAUeR4eP1jwNCEX+vU73pMAM9jx/XZeeMNVKU+FIfzDYzuDuC1lA0B pdnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bZV8VezGtTIXq/yNYWV/d3QACsxZ7pN3NcbaApSLeSI=; b=P+wvBTV5E1cGT/n8HZHPrgh8pjxJzt2cjkpRjiHQaphNVOIw5xiLVOfwruCeiEMdw4 PhWabZK8kxgZ4aFb7C15hO3LqR7cFteAXn6byrt0XUZvvMPXLe8o17j42ORVKa/M5D56 3al8lN7KUuk6pWafwjqT7swGXhUN68B3Fe4DlgAxlyCkl2RO2m/ID+RLei/9lA6h9uV4 L6S2CGr6xwSOUyQ318U8wj+g+BJXiuqGXZKmX21jucBoTfrQi3eLmvV80NVbXzS9xmjU KUQR/IlsvNSITyR5TrcPVT9bKbbXJAD+vu5s5Sy8rmFv0p3lXnrWQwtv7MgWP9RScz3/ Qonw== X-Gm-Message-State: AOAM530fm0rnV2zJ2tYi/5QAvXHxWov0TmRqEGTiS6tMfhUJntCaUQOr 6WtJMpvn1EvJhDpxK0pUs6kEMGrflDNRzA== X-Google-Smtp-Source: ABdhPJwH1RzqGvf/ZV12rL75mRQTr/vhMIdb+3qamTbs4U6RsRcaU42d5mbX12ShzEp/U3hGJb/PxA== X-Received: by 2002:a17:906:9742:: with SMTP id o2mr2880747ejy.532.1627535426100; Wed, 28 Jul 2021 22:10:26 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:25 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 25/87] trace-cmd library: Compress the trace data Date: Thu, 29 Jul 2021 08:08:57 +0300 Message-Id: <20210729050959.12263-26-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If the output file handler supports compression, use it to compress the flyrecord and latency trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 90625c4e..e232a678 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1895,7 +1895,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (!path) goto out_free; - copy_file(handle, path); + copy_file_compress(handle, path, NULL); put_tracing_file(path); @@ -2042,7 +2042,8 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (lseek64(data[i].fd, data[i].offset, SEEK_SET) == (off64_t)-1) goto out_free; if (data[i].size) { - read_size = copy_file_fd(handle, data[i].fd, data[i].size); + read_size = out_copy_fd_compress(handle, data[i].fd, + data[i].size, &data_files[i].write_size); if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", From patchwork Thu Jul 29 05:08:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF672C4338F for ; Thu, 29 Jul 2021 05:10:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A91AF61052 for ; Thu, 29 Jul 2021 05:10:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233942AbhG2FKc (ORCPT ); Thu, 29 Jul 2021 01:10:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbhG2FKc (ORCPT ); Thu, 29 Jul 2021 01:10:32 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71560C061757 for ; Wed, 28 Jul 2021 22:10:28 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id yk17so753064ejb.11 for ; Wed, 28 Jul 2021 22:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RmoTo/DrBrKEPsjiYGWWfejteUIwXa2uLWnxdU4WRIg=; b=ddeV+4sinrWgohHEkdy6JmtcOp6QjNreREYfuKY83C4en3aqAUa+F82i5H4xg9lV2j +50U9X8lJb4HOpv+VVMvoPJNPwE33vV+LdnX4VeOU/WhLJ0MMBDVLikNZge3tBFaPajs BSO9i3PKYxtJd/eYYjGyGWO3qYY3hnwcIIM2yS/LuB7WqEVfpLuGgr3xKDYPc0l35XVk XC0u74FrSAL0B0tWJNQ34rPuwvyDzMwL/O7Dq5CUJ91UIpdN+ao2otc8FJdrlqyECJX/ 5mPSA1wajk/iuW1jL4CNWZY+HoOWN8U90hMZVsSD8d4g0DqMR2WNOIP3ghUK+Ns8KDWL b2uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RmoTo/DrBrKEPsjiYGWWfejteUIwXa2uLWnxdU4WRIg=; b=D8qXJjsHrFhdvmF9wjqjomUlwIc44ygQ6uQWTV1zZPuV9bJidq59EWEdZUJktYwbGw 0wqk5Q7KCpv00mazIXNjuWCgAgbZWZMWkJaHaC3Ic/ThetCtmDbUTR6SY3wSIuuOEuI8 hrSm568x498oenIK4LdPwkh1pgJ/PnzFm5y9PHrdgPL8Pqn5gOC0wRgKiXFozB7yTm/H V156JOxace6YFVCnAuz8fABjtKcJt7pFzMhAaRa1+c7A4non9WrJLRjnHoGl+RadC/Si ywDZNFoW7rcodaTkrbKY3kevvRmiTRswE9kMWO8+48w63hIRSXeaf1FMKUrrGyVIR1He QMOA== X-Gm-Message-State: AOAM531hKwNcmhm5k+ytOEAJXFgcyvXzWMQDR+z2nZ6ZmYlMJpUwDfJn pnqjR8B97yHbJFhnFR2bUSJaJhf6DVOthQ== X-Google-Smtp-Source: ABdhPJzWuSy4iGCi9lHdGuQ7NtwQ8pZY3GlqEZEg2oNoUEFIzYfEYMMVrz3KphgTnFObY+M2AxNl2w== X-Received: by 2002:a17:906:ed1:: with SMTP id u17mr2840877eji.304.1627535427101; Wed, 28 Jul 2021 22:10:27 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:26 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 26/87] tarce-cmd library: Add multiple options sections in trace file version 7 Date: Thu, 29 Jul 2021 08:08:58 +0300 Message-Id: <20210729050959.12263-27-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Introduced chain of options sections in trace file version 7. Extended the "DONE" option to hold the offset into the file to the next options section. Format of trace file version 7 is extended with a new mandatory field after the compression algorithm header: <8 bytes>, unsigned long long integer - offset into the trace file where the first options section is located. This allows to place this section anywhere in the file. As all other sections have corresponding options, describing their offsets into the trace file, this change makes the structure of trace file version 7 flexible. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 110 ++++++++++++++++++++++++++++++----- 1 file changed, 96 insertions(+), 14 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e232a678..6bb7f4de 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -63,7 +63,7 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; - size_t options_start; + tsize_t options_start; bool big_endian; bool do_compress; struct tracecmd_compression *compress; @@ -1453,6 +1453,7 @@ int tracecmd_output_set_compression(struct tracecmd_output *handler, const char */ int tracecmd_output_write_init(struct tracecmd_output *handler) { + unsigned long long offset; char buf[BUFSIZ]; int endian4; @@ -1489,6 +1490,11 @@ int tracecmd_output_write_init(struct tracecmd_output *handler) if (handler->file_version >= 7) { if (write_compression_header(handler)) return -1; + /* Write 0 as options offset and save its location */ + offset = 0; + handler->options_start = do_lseek(handler, 0, SEEK_CUR); + if (do_write_check(handler, &offset, 8)) + return -1; } handler->file_state = TRACECMD_FILE_INIT; @@ -1563,7 +1569,7 @@ tracecmd_add_option_v(struct tracecmd_output *handle, * We can only add options before tracing data were written. * This may change in the future. */ - if (handle->file_state > TRACECMD_FILE_OPTIONS) + if (handle->file_version < 7 && handle->file_state > TRACECMD_FILE_OPTIONS) return NULL; for (i = 0; i < count; i++) @@ -1576,8 +1582,7 @@ tracecmd_add_option_v(struct tracecmd_output *handle, return NULL; } } - - option = malloc(sizeof(*option)); + option = calloc(1, sizeof(*option)); if (!option) { tracecmd_warning("Could not allocate space for option"); free(data); @@ -1640,7 +1645,7 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) return 0; } -int tracecmd_write_options(struct tracecmd_output *handle) +static int write_options_v6(struct tracecmd_output *handle) { struct tracecmd_option *options; unsigned short option; @@ -1658,7 +1663,7 @@ int tracecmd_write_options(struct tracecmd_output *handle) if (do_write_check(handle, "options ", 10)) return -1; - + handle->options_start = do_lseek(handle, 0, SEEK_CUR); list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) @@ -1669,7 +1674,7 @@ int tracecmd_write_options(struct tracecmd_output *handle) return -1; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) @@ -1682,11 +1687,83 @@ int tracecmd_write_options(struct tracecmd_output *handle) return -1; handle->file_state = TRACECMD_FILE_OPTIONS; + return 0; +} + +static int write_options_v7(struct tracecmd_output *handle) +{ + struct tracecmd_option *options; + unsigned long long endian8; + unsigned short endian2; + unsigned int endian4; + bool new = false; + tsize_t offset; + + /* Check if there are unsaved options */ + list_for_each_entry(options, &handle->options, list) { + if (!options->offset) { + new = true; + break; + } + } + + if (!new) + return 0; + offset = do_lseek(handle, 0, SEEK_CUR); + /* Append to the previous options section, if any */ + if (handle->options_start) { + if (do_lseek(handle, handle->options_start, SEEK_SET) == (off64_t)-1) + return -1; + endian8 = convert_endian_8(handle, offset); + if (do_write_check(handle, &endian8, 8)) + return -1; + if (do_lseek(handle, offset, SEEK_SET) == (off_t)-1) + return -1; + } + offset = out_write_section_header(handle, TRACECMD_OPTION_DONE, "options", 0, false); + if (offset == (off_t)-1) + return -1; + list_for_each_entry(options, &handle->options, list) { + /* Option is already saved, skip it */ + if (options->offset) + continue; + endian2 = convert_endian_2(handle, options->id); + if (do_write_check(handle, &endian2, 2)) + return -1; + endian4 = convert_endian_4(handle, options->size); + if (do_write_check(handle, &endian4, 4)) + return -1; + /* Save the data location */ + options->offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, options->data, options->size)) + return -1; + } + + endian2 = convert_endian_2(handle, TRACECMD_OPTION_DONE); + if (do_write_check(handle, &endian2, 2)) + return -1; + endian4 = convert_endian_4(handle, 8); + if (do_write_check(handle, &endian4, 4)) + return -1; + endian8 = 0; + handle->options_start = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, &endian8, 8)) + return -1; + if (out_update_section_header(handle, offset)) + return -1; return 0; } -int tracecmd_append_options(struct tracecmd_output *handle) +int tracecmd_write_options(struct tracecmd_output *handle) +{ + if (handle->file_version < 7) + return write_options_v6(handle); + return write_options_v7(handle); +} + + +static int append_options_v6(struct tracecmd_output *handle) { struct tracecmd_option *options; unsigned short option; @@ -1701,14 +1778,12 @@ int tracecmd_append_options(struct tracecmd_output *handle) */ if (handle->file_state != TRACECMD_FILE_OPTIONS) return -1; - - if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + if (do_lseek(handle, 0, SEEK_END) == -1) return -1; - offset = lseek64(handle->fd, -2, SEEK_CUR); + offset = do_lseek(handle, -2, SEEK_CUR); if (offset == (off_t)-1) return -1; - - r = pread(handle->fd, &option, 2, offset); + r = do_preed(handle, &option, 2, offset); if (r != 2 || option != TRACECMD_OPTION_DONE) return -1; @@ -1722,7 +1797,7 @@ int tracecmd_append_options(struct tracecmd_output *handle) return -1; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) @@ -1737,6 +1812,13 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } +int tracecmd_append_options(struct tracecmd_output *handle) +{ + if (handle->file_version < 7) + return append_options_v6(handle); + return write_options_v7(handle); +} + static struct tracecmd_option * add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { From patchwork Thu Jul 29 05:08:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BA34C432BE for ; Thu, 29 Jul 2021 05:10:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 34F4C60FED for ; Thu, 29 Jul 2021 05:10:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231329AbhG2FKe (ORCPT ); Thu, 29 Jul 2021 01:10:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41386 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233945AbhG2FKc (ORCPT ); Thu, 29 Jul 2021 01:10:32 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61EFBC061765 for ; Wed, 28 Jul 2021 22:10:29 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id ec13so5881187edb.0 for ; Wed, 28 Jul 2021 22:10:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EIoDKwsrSyrvOhUkkTIHs070KY99lBO+xAo3kDzM5CM=; b=P+mD9sXrIJZqllB7Q0BgWPIYSPsF0N4dcuvrCMzTJvRAmBoE4sA2uV3DtrKj2o7HlL oK2N/+ijlfP/De8k2WOfzOPaEq2QYCf9kl1+QPDVINtLwRvbuBDiC0HlRcNnVXYloIM3 UAeVg/8aX7um37R9GSBh9MpsHvAWWQZHHEqWiVsii+Hmjp9j6y8SxU43JXCMihWZHItl bXbv0fDLXdmtMuPOR3fkvxz++HS79v43dyo8QmHOti+RuJCpktrsJVnEo5xOLJhMQiPf fTvIQhV7tZITCronuOGTbbn/56IalDOQpBMZcLT9vG25k6Vqht+/qa2NcmAAh98PfdQx xyPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EIoDKwsrSyrvOhUkkTIHs070KY99lBO+xAo3kDzM5CM=; b=kd/5FwC+zppGHsSftOp4yXwoHExeucXWrLhd0eZV+7/lNT/7+AVydJvZLLYhv8brhJ wmlbhRf7Uqw39aBcGRH5lVjg6CaO9ftwjFeeggPClYpb9vu1FHczMEQn8NcxRXE2ZHct eG0kw4BDmdEtt54zON6pzmLOsUaSaF4qH1NqXU32kp/dN1WAk6ODxN0aXZMGABKe2ZwM j5sDqxgoVl8J0sdc1bTCx1CoUIpgwx05THAUSMX/4gmit+Mg4FcXFdxV5GrLziAr7gZw lTLjh4G0t4iTYdg0MYD9aPZbeaVFkaEUpswZoYzqSgI+2QW7Qmmb0yDvpZ7LyLzxa6Ci A52g== X-Gm-Message-State: AOAM5318YdbCjS9WJ2cObv/mgvo/velP4ZJBKOEdGRB2t5mwxX9Oq0rM x7qVSuD4jByOUrABrceYQ/k= X-Google-Smtp-Source: ABdhPJwZ5OGy7i1DTmKlneSbpbrjf6f99sV8XAPJg05A1C7Xas4aukL+xJpIj20Z3N8yte5uYlqQ4Q== X-Received: by 2002:a05:6402:10d9:: with SMTP id p25mr3894635edu.51.1627535428072; Wed, 28 Jul 2021 22:10:28 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:27 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 27/87] trace-cmd library: Do not write CPU count section in trace files version 7 Date: Thu, 29 Jul 2021 08:08:59 +0300 Message-Id: <20210729050959.12263-28-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Writing CPU count as a separate section in the trace file is redundant, as there is already an option for that. Use that option in trace files version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 6bb7f4de..d60bd456 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1637,10 +1637,16 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) handle->file_state); return -1; } - cpus = convert_endian_4(handle, cpus); - ret = do_write_check(handle, &cpus, 4); - if (ret < 0) - return ret; + + if (handle->file_version < 7) { + cpus = convert_endian_4(handle, cpus); + ret = do_write_check(handle, &cpus, 4); + if (ret < 0) + return ret; + } else { + tracecmd_add_option(handle, TRACECMD_OPTION_CPUCOUNT, sizeof(int), &cpus); + } + handle->file_state = TRACECMD_FILE_CPU_COUNT; return 0; } From patchwork Thu Jul 29 05:09:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28C85C4320A for ; Thu, 29 Jul 2021 05:10:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1500160FED for ; Thu, 29 Jul 2021 05:10:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233949AbhG2FKh (ORCPT ); Thu, 29 Jul 2021 01:10:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233946AbhG2FKg (ORCPT ); Thu, 29 Jul 2021 01:10:36 -0400 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BB9DC0613C1 for ; Wed, 28 Jul 2021 22:10:30 -0700 (PDT) Received: by mail-ed1-x532.google.com with SMTP id f13so6341447edq.13 for ; Wed, 28 Jul 2021 22:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gW7PgJG7EJV+sB9BOwreDZ9mUjjK2RxrwM5jBYuuzMA=; b=T+L2am4sUXBUjn4M0i5g//Az1Kihjt9rGz2+ryI0Kw+xcHHOogFmN5Z0+5NNKT1dIa NgFDP6A9M280+eGRjDnm/Ks4j8zsB7f8U7LWqmFbnehEgIeB4dt7QCXGUKYnth6uUWdZ i/0Nf4Oa7YxkemAIPVkj5kocUUegJr0pXfRqYZWrhLDrVUpgwU/UnRMtqwCm2M2DZcnf W4e1oR5fF5skGmtvBMf+KJEWdyGPxOdxCinJP9TTBFwHVQg3JwFZmYCqfcSk+PAui3hW dRS7cpA3b2987MjK5r8tVZNeuhVjVr9ovHREUb4Wh7+vyiXgpX/zpMJJCTZ5wqwoiXcy Ivmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gW7PgJG7EJV+sB9BOwreDZ9mUjjK2RxrwM5jBYuuzMA=; b=YSAFkCjZepdTAMUesJfjlfIlvYUlMqZXRh7boq1lO5a2VxA8hlM82pCWNk5GogyWtC RvD0XpHsDj1HGzu5aayhaKRwtFQZX4XL69WZsgqpA3D41BixTH7hVWZO3/HHUBSBWTFN MgN0qSnqH8nmyHFaZ9R7YKS4XQzGr9AUiSUCUgFymo9peQAKmd8JnYDrOKiKoBznWI5Q WQXLe++MQWh+5I+93xvD31S0D9ijL6TQUhtAAhy7PwwQAjc1kS+GY3Hpt4+5nNVFMfrC dOvC5vSNMCyUSMVUfsB1Vs8ospbBVdNX8gBP0dfYbqkZFOD8hH96xa1Ew7k0OJ25G2+F T55Q== X-Gm-Message-State: AOAM533Y45z1gxMYawJN46FMCmj3fyGszS2alcrTVPiVUIQXFbN8oXxw fU1pdvKvXzn7pdfzTTb+vxOSrNax8QD+3w== X-Google-Smtp-Source: ABdhPJy5cRDFTmE3uD/q7qAO0fsySQ0v8ubJ4SpzUun3AscZr4nQnYxBP+sOhcpEMdy6LqMtCFnU1w== X-Received: by 2002:a05:6402:702:: with SMTP id w2mr3901838edx.149.1627535428999; Wed, 28 Jul 2021 22:10:28 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:28 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 28/87] trace-cmd library: Move CPU flyrecord trace metadata into the buffer option, for trace file version 7 Date: Thu, 29 Jul 2021 08:09:00 +0300 Message-Id: <20210729050959.12263-29-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extended the BUFFER trace option in trace file version 7 with CPU flyrecord trace metadata. In the version 6 of the trace file, this metadata is located at several places in the file. Part of the metadata is only for the top trace instance, thus limiting per-instance configuration. Moving all CPU trace related metadata in the BUFFER option simplifies the parsing and makes per-instance configuration more flexible. In the new file structure, the top instance is treated as any other instances. The format of the extended BUFFER option is: - offset of the buffer in the trace file - name of the buffer - trace clock, used in this buffer for events timestamps - count of CPUs with trace data - array, describing each CPU with trace data: - CPU id - offset of CPU trace data in the trace file - size of the recorded CPU trace data Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 171 ++++++++++++++++++++++++++--------- 1 file changed, 127 insertions(+), 44 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index d60bd456..21555d6d 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1826,7 +1826,7 @@ int tracecmd_append_options(struct tracecmd_output *handle) } static struct tracecmd_option * -add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) +add_buffer_option_v6(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; @@ -1876,8 +1876,11 @@ int tracecmd_write_buffer_info(struct tracecmd_output *handle) struct tracecmd_option *option; struct tracecmd_buffer *buf; + if (handle->file_version >= 7) + return 0; + list_for_each_entry(buf, &handle->buffers, list) { - option = add_buffer_option(handle, buf->name, buf->cpus); + option = add_buffer_option_v6(handle, buf->name, buf->cpus); if (!option) return -1; buf->option = option; @@ -1938,6 +1941,98 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return 0; } +static char *get_clock(struct tracecmd_output *handle) +{ + struct tracefs_instance *inst; + + if (handle->trace_clock) + return handle->trace_clock; + + /* + * If no clock is set on this handle, get the trace clock of + * the top instance in the handle's tracing dir + */ + inst = tracefs_instance_alloc(handle->tracing_dir, NULL); + if (!inst) + return NULL; + handle->trace_clock = tracefs_get_clock(inst); + tracefs_instance_free(inst); + return handle->trace_clock; +} + +__hidden struct tracecmd_option * +out_add_buffer_option_v7(struct tracecmd_output *handle, const char *name, + unsigned short id, unsigned long long data_offset, + int cpus, struct data_file_write *cpu_data) +{ + struct tracecmd_option *option; + int i, j = 0, k = 0; + int *cpu_ids = NULL; + struct iovec *vect; + char *clock; + + if (handle->file_version < 7) + return NULL; + + clock = get_clock(handle); + + /* Buffer flyrecord option, v7: + * - trace data offset in the file + * - buffer name + * - buffer clock + * - CPU count + * - for each CPU: + * - CPU id + * - CPU trace data offset in the file + * - CPU trace data size + */ + + /* Buffer latency option, v7: + * - trace data offset in the file + * - buffer name + * - buffer clock + */ + + vect = calloc(5 + (cpus * 3), sizeof(struct iovec)); + if (!vect) + return NULL; + if (cpus) { + cpu_ids = calloc(cpus, sizeof(int)); + if (!cpu_ids) { + free(vect); + return NULL; + } + } + vect[j].iov_base = (void *) &data_offset; + vect[j++].iov_len = 8; + vect[j].iov_base = (void *) name; + vect[j++].iov_len = strlen(name) + 1; + vect[j].iov_base = (void *) clock; + vect[j++].iov_len = strlen(clock) + 1; + if (id == TRACECMD_OPTION_BUFFER) { + vect[j].iov_base = (void *) &k; + vect[j++].iov_len = 4; + for (i = 0; i < cpus; i++) { + if (!cpu_data[i].file_size) + continue; + cpu_ids[i] = i; + vect[j].iov_base = &cpu_ids[i]; + vect[j++].iov_len = 4; + vect[j].iov_base = &cpu_data[i].data_offset; + vect[j++].iov_len = 8; + vect[j].iov_base = &cpu_data[i].write_size; + vect[j++].iov_len = 8; + k++; + } + } + + option = tracecmd_add_option_v(handle, id, vect, j); + free(vect); + free(cpu_ids); + + return option; +} + struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, int file_version) { @@ -2017,7 +2112,7 @@ out: return ret; } -static int update_buffer_cpu_offset(struct tracecmd_output *handle, +static int update_buffer_cpu_offset_v6(struct tracecmd_output *handle, const char *name, tsize_t offset) { tsize_t b_offset; @@ -2047,27 +2142,6 @@ static int update_buffer_cpu_offset(struct tracecmd_output *handle, return 0; } - -static char *get_clock(struct tracecmd_output *handle) -{ - struct tracefs_instance *inst; - - if (handle->trace_clock) - return handle->trace_clock; - - /* - * If no clock is set on this handle, get the trace clock of - * the top instance in the handle's tracing dir - */ - inst = tracefs_instance_alloc(handle->tracing_dir, NULL); - if (!inst) - return NULL; - handle->trace_clock = tracefs_get_clock(inst); - tracefs_instance_free(inst); - return handle->trace_clock; -} - - __hidden int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name) { @@ -2091,7 +2165,7 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, } data_offs = do_lseek(handle, 0, SEEK_CUR); - if (do_write_check(handle, "flyrecord", 10)) + if (handle->file_version < 7 && do_write_check(handle, "flyrecord", 10)) goto out_free; data_files = calloc(cpus, sizeof(struct data_file_write)); @@ -2112,10 +2186,12 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, } } - update_buffer_cpu_offset(handle, buff_name, data_offs); - clock = get_clock(handle); - if (clock && save_clock(handle, clock)) - goto out_free; + if (handle->file_version < 7) { + update_buffer_cpu_offset_v6(handle, buff_name, data_offs); + clock = get_clock(handle); + if (clock && save_clock(handle, clock)) + goto out_free; + } for (i = 0; i < cpus; i++) { data_files[i].data_offset = do_lseek(handle, 0, SEEK_CUR); @@ -2142,26 +2218,33 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, data_files[i].write_size = 0; } - /* Write the real CPU data offset in the file */ - if (do_lseek(handle, data_files[i].doffset, SEEK_SET) == (off64_t)-1) - goto out_free; - endian8 = convert_endian_8(handle, data_files[i].data_offset); - if (do_write_check(handle, &endian8, 8)) - goto out_free; - /* Write the real CPU data size in the file */ - if (do_lseek(handle, data_files[i].soffset, SEEK_SET) == (off64_t)-1) - goto out_free; - endian8 = convert_endian_8(handle, data_files[i].write_size); - if (do_write_check(handle, &endian8, 8)) - goto out_free; - offset = data_files[i].data_offset + data_files[i].write_size; - if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) - goto out_free; + if (handle->file_version < 7) { + /* Write the real CPU data offset in the file */ + if (do_lseek(handle, data_files[i].doffset, SEEK_SET) == (off64_t)-1) + goto out_free; + endian8 = convert_endian_8(handle, data_files[i].data_offset); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + /* Write the real CPU data size in the file */ + if (do_lseek(handle, data_files[i].soffset, SEEK_SET) == (off64_t)-1) + goto out_free; + endian8 = convert_endian_8(handle, data_files[i].write_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + data_files[i].write_size; + if (do_lseek(handle, offset, SEEK_SET) == (off64_t)-1) + goto out_free; + } if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", (unsigned long long)data_files[i].write_size); } + if (handle->file_version >= 7 && + !out_add_buffer_option_v7(handle, buff_name, + TRACECMD_OPTION_BUFFER, data_offs, cpus, data_files)) + goto out_free; + if (do_lseek(handle, 0, SEEK_END) == (off64_t)-1) goto out_free; From patchwork Thu Jul 29 05:09:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDB48C4338F for ; Thu, 29 Jul 2021 05:10:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9FB9061052 for ; Thu, 29 Jul 2021 05:10:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233945AbhG2FKg (ORCPT ); Thu, 29 Jul 2021 01:10:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233607AbhG2FKf (ORCPT ); Thu, 29 Jul 2021 01:10:35 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A89AC061757 for ; Wed, 28 Jul 2021 22:10:31 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id nd39so8463758ejc.5 for ; Wed, 28 Jul 2021 22:10:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+xzGS4lMzgEZCd2cBF9hSNcgr/kC71xtcPquJSbbf3w=; b=l7TK+B7B1fBBW4LBp4vkN8AuT8FzNM/87h25uGbBTWScwWc0F3cuzZr+uYrQ2GMElo fhwIhNmvhCOXKiaoKpFtm8BfhScafQl9QP4sAUACL9glpJrorE/s/yn61fXQDGYM5f4v Du72Icy3wVh/3OGPgtYEDEHcI9zC/Kn7UHUsGXA0spkJ5jmzhl5aaYuixK9UcoDBx3k5 kAatrvoE0fPZ9YspPj9xlya3l77PBZkU4BFo7C3c93Fhmy1xHSUeczjCqZI4fxX/d4xw +ecBqGz2yKa8e1xWbGYdO4M+82HZQVPC9bSST5sE3X8KP/Q8fwk9/kOV2vib/77YmWBA 6mQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+xzGS4lMzgEZCd2cBF9hSNcgr/kC71xtcPquJSbbf3w=; b=McioUiVK8VAN0FO77GXo5cbabL0yxuENO6XmM0xjP/KhNcTryMjcTVWAJBAH3pEPyb Rg/q8P4pn1oX9+0gXH4Mk1uSnim3yG+bcFI2VRfqEnbGe7O9lp3NOTIckRz3sBFQrszU +WFCFBdxdHr+lF75NGmkTIB8S3t/0MuJ1tmRa+sBajdOVumTr4oF2tLjI5BzH8FpLi6E ZLCnGwYXk6WtCpPnGmpj1S8DHl/EqRrAVtl2HBCUQumEJzlDvi+yOYKgd9/D4u4eluu0 U361UGIFW+Fo1CkfKsh/AVAW87kIiEWqBtA49ao+0+PEeedD7qjUABZzv7Ec0BYF2ffn sHyQ== X-Gm-Message-State: AOAM533bERTyEU1HZtX6zXL4DLIdeSB0Eb9fVIHzvWF9z5Mn8dnNwVXU q9OEKaFRs5pwR/aJtjSfsVU= X-Google-Smtp-Source: ABdhPJzH2kbC4LPXxVzmxS1FBMFrqzcTR/2fl72nG2NHpTyRtunjlcJsxoUqtzyx4wfmPv4wYPafuw== X-Received: by 2002:a17:906:b0c5:: with SMTP id bk5mr2804223ejb.428.1627535429947; Wed, 28 Jul 2021 22:10:29 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:29 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 29/87] trace-cmd record: Append trace options after the trace data are written Date: Thu, 29 Jul 2021 08:09:01 +0300 Message-Id: <20210729050959.12263-30-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, the CPU trace metadata is saved in a trace option. As this metadata has file offsets, which depend on the CPU trace data, the option must be written in the file after the CPU trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 350d2811..f8c1c090 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4466,6 +4466,9 @@ static void write_guest_file(struct buffer_instance *instance) if (tracecmd_write_cpu_data(handle, cpu_count, temp_files, "") < 0) die("failed to write CPU data"); + if (tracecmd_get_out_file_version(handle) >= 7) + tracecmd_write_options(handle); + tracecmd_output_close(handle); for (i = 0; i < cpu_count; i++) @@ -4623,7 +4626,8 @@ static void record_data(struct common_record_context *ctx) append_buffer(handle, instance, temp_files); } } - + if (tracecmd_get_out_file_version(handle) >= 7) + tracecmd_write_options(handle); free(temp_files); } if (!handle) From patchwork Thu Jul 29 05:09:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92A6DC43214 for ; Thu, 29 Jul 2021 05:10:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 80F6361054 for ; Thu, 29 Jul 2021 05:10:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233607AbhG2FKh (ORCPT ); Thu, 29 Jul 2021 01:10:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233965AbhG2FKh (ORCPT ); Thu, 29 Jul 2021 01:10:37 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 646EDC061765 for ; Wed, 28 Jul 2021 22:10:32 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id f13so6341529edq.13 for ; Wed, 28 Jul 2021 22:10:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z8XLY6lbSQHAj+YMOU7pBwRn88Cd/3oy3KvyffkBsE4=; b=p8etO9slVlQtl1nMMUcYkJJPKE4Ks/uA/Ttn9ja18rCQNvLN6vMD9SIuikGvBDgwU2 rW93rNWI7WySxW4fKZ+EMTaN79pmxUrGPrJo7C/KWV4QCkMvYFhsKjQVHlvsTA/hiA0O hgD1ARI7NphPPLd2hsZoE85+VBfW7sG2G8LMimSLpFZvNxMTEkSk0o47LNqBx4tud7Tb ZzE6GFS8czwLehQF5e8dK0oHlGgKw95UVjknUL5g6Z5inNO186jlV+cHcTOBMvm1BGJA BOH70lKq8WHXROxnA2DVd3QWQT0rOUjty1FQBvy4DYFZ1KhmEwsJWxU7dKmi/FV0TYDF eA8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z8XLY6lbSQHAj+YMOU7pBwRn88Cd/3oy3KvyffkBsE4=; b=KKAgEtuGOMBw5x+KCnZeP/9TOYV55IKsYnMCI7ZM0qtRryGwkaQE4i8iL7nDOOZLQ7 yNii0lTflHkzsDAE6dnCLvgN/QSRkrKHJ4uekJbAORb7GeYB0lAcspW5zluLCYLmucHZ z/YQOJa+EWlpnbHhs9vDhMi7WniaRRnYxzdY8RCAAyPsAVuzGWh/mjBKkhzp4TmE0bSc PkU+aoei17piXpLEWps7VzxKMuN5Ylm9LafwGdbUEw6otxIrTZRl4MfIkJvbkWOMU1H5 3TQgVPBzUyEV/E8IkUNkYY9v7dRqSoyEGKZWHa4QSk3F+W2odw7shfeFeR1Wtm3OW2w6 Qsvg== X-Gm-Message-State: AOAM530JsLm44JX73FIgZcmM35PWzWftkcgpbBjn99wsiiqlVMUJqulC VXDfGBtKsvzmnbafipdZwOI= X-Google-Smtp-Source: ABdhPJxT3swnCvIZHDJJJOfDRUugZZU3pt0MhnsOxIH7fvOzB/jsmTewMw/wf4pWX8z8TKIdH5FBnw== X-Received: by 2002:aa7:c603:: with SMTP id h3mr3852448edq.165.1627535430882; Wed, 28 Jul 2021 22:10:30 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:30 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 30/87] trace-cmd library: Add section header before flyrecord trace data Date: Thu, 29 Jul 2021 08:09:02 +0300 Message-Id: <20210729050959.12263-31-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, write a section header at the beginning of each section with flyrecord trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 21555d6d..30a6acf4 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2146,10 +2146,12 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name) { struct data_file_write *data_files = NULL; + enum tracecmd_section_flags flags = 0; tsize_t data_offs, offset; unsigned long long endian8; unsigned long long read_size; char *clock; + char *str; int ret; int i; @@ -2168,6 +2170,16 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (handle->file_version < 7 && do_write_check(handle, "flyrecord", 10)) goto out_free; + if (handle->compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + if (asprintf(&str, "buffer flyrecord %s", buff_name) < 1) + goto out_free; + + offset = out_write_section_header(handle, TRACECMD_OPTION_BUFFER, str, flags, false); + free(str); + if (offset == (off_t)-1) + goto out_free; + data_files = calloc(cpus, sizeof(struct data_file_write)); if (!data_files) goto out_free; @@ -2248,6 +2260,9 @@ __hidden int out_write_cpu_data(struct tracecmd_output *handle, if (do_lseek(handle, 0, SEEK_END) == (off64_t)-1) goto out_free; + if (out_update_section_header(handle, offset)) + goto out_free; + free(data_files); handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; From patchwork Thu Jul 29 05:09:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D69CC432BE for ; Thu, 29 Jul 2021 05:10:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5694E61054 for ; Thu, 29 Jul 2021 05:10:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233946AbhG2FKh (ORCPT ); Thu, 29 Jul 2021 01:10:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233607AbhG2FKg (ORCPT ); Thu, 29 Jul 2021 01:10:36 -0400 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64751C0613CF for ; Wed, 28 Jul 2021 22:10:33 -0700 (PDT) Received: by mail-ed1-x52a.google.com with SMTP id b7so6430193edu.3 for ; Wed, 28 Jul 2021 22:10:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eUjZjzWhSkd/cQ2BtYyOOQVWHRzI2U8AQq9A6uGfW30=; b=UR7KMBF3ZtPOPkk7+wcXurLSgP849Mnf6Z5ulLAno6bpo3tQ9m9MK7Yy2nNELOsdWi j22Qab3WbXT3Lg7A/w6LFvout3jHpr9mSwQWPYct4qe48lv98NYw1YzawkO3konA3OwI KhnYn1zKGOEp+6Mksq1skThvBEqCpgpoM5UPiQA9rrHVe2YXT9qIpC3mKM7BEXwXgxlc O3htRV9dYv/Zd4vrXnz/DyLeOi+pxGy0bj728r6zfkHxziewNpHY3csXL6lJTAXILrSt o+q3eB20ZM/uzEMerzq+h6gHLdgxxxW4JOew8zoLHNJRnppkmBjAdUV9fJTaZ2Zzk9K/ 3/Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eUjZjzWhSkd/cQ2BtYyOOQVWHRzI2U8AQq9A6uGfW30=; b=TzExf81y6KcyVMeiBnJjliCaQevkOQzdbZ08DHzXpdoMAp4UdJk8ehqFZxOem6G2rj WT7PVBD3HIzyJib7+AuTvlu17I8Og7Vau0ZKhHKZsvjxzDhzIISpGBfggA9QIhN+fb85 H4SVPRFUBf5SSYuGwF3bP0ktNJ4q8GQwNzoRDzf6ui/ImeAY8gxGBKopdJFirDGuIE1c WRjr1Im0+5zMcaoKLuwX9W3i0YImZyev5rYoK1U5oFpR6aGorY5/l7fxplPGTjuIY3ge w3PMDqy5jsWAs5D7ra9aspFxJlFaZF5LshQr76ZtOi9jurked5HgWlTGec+mIWIkCXIu yi9Q== X-Gm-Message-State: AOAM531CgnlPixobDM/hBpPK46ep59niXc0sYKpZ8D/b906LNcipU0WV xrfrXgRKCJV7go9G5kpIfOs= X-Google-Smtp-Source: ABdhPJwQPyxiwfPiuOhG4LaKcODFCTe3B2mh7wjN7aJ0MJmP6g2wFVs/odnInRY8RTCTOCw4FW57tg== X-Received: by 2002:a05:6402:53:: with SMTP id f19mr3943867edu.200.1627535431984; Wed, 28 Jul 2021 22:10:31 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:31 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 31/87] trace-cmd library: Fit CPU latency trace data in the new trace file version 7 format Date: Thu, 29 Jul 2021 08:09:03 +0300 Message-Id: <20210729050959.12263-32-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 format is based on sections. To fit the latency trace data in this structure, a new section and option for it is defined: BUFFER_LAT It is similar to the BUFFER section which holds the flyrecord data, but has a latency specific design. The BUFFER_LAT section has: - section header, as all other sections - compression of the trace data, optional - corresponding trace option, pointing to the section Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 +- lib/trace-cmd/trace-output.c | 36 ++++++++++++++++--- tracecmd/trace-record.c | 2 +- 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4e718c40..f2868576 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -135,6 +135,7 @@ enum { TRACECMD_OPTION_TIME_SHIFT, TRACECMD_OPTION_GUEST, TRACECMD_OPTION_TSC2NSEC, + TRACECMD_OPTION_BUFFER_LAT, TRACECMD_OPTION_HEADER_INFO, TRACECMD_OPTION_FTRACE_EVENTS, TRACECMD_OPTION_EVENT_FORMATS, @@ -294,7 +295,7 @@ int tracecmd_output_write_headers(struct tracecmd_output *handler, struct tracecmd_event_list *list); struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, - int file_version); + int file_version, const char *compression); struct tracecmd_output *tracecmd_create_init_fd(int fd); struct tracecmd_output *tracecmd_create_init_file(const char *output_file); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 30a6acf4..83c3b9e7 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2034,9 +2034,11 @@ out_add_buffer_option_v7(struct tracecmd_output *handle, const char *name, } struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, - int file_version) + int file_version, const char *compression) { + enum tracecmd_section_flags flags = 0; struct tracecmd_output *handle; + tsize_t offset; char *path; int fd; @@ -2049,6 +2051,12 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in goto out_free; if (file_version && tracecmd_output_set_version(handle, file_version)) goto out_free; + if (compression) { + if (tracecmd_output_set_compression(handle, compression)) + goto out_free; + } else if (file_version >= 7) { + tracecmd_output_set_compression(handle, "any"); + } if (tracecmd_output_write_init(handle)) goto out_free; if (tracecmd_output_write_headers(handle, NULL)) @@ -2061,7 +2069,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_cpus(handle, cpus) < 0) goto out_free; - + if (tracecmd_write_buffer_info(handle) < 0) + goto out_free; if (tracecmd_write_options(handle) < 0) goto out_free; @@ -2071,23 +2080,42 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in goto out_free; } - if (do_write_check(handle, "latency ", 10)) + if (handle->file_version < 7 && do_write_check(handle, "latency ", 10)) goto out_free; path = get_tracing_file(handle, "trace"); if (!path) goto out_free; + offset = do_lseek(handle, 0, SEEK_CUR); + if (handle->file_version >= 7 && + !out_add_buffer_option_v7(handle, "", TRACECMD_OPTION_BUFFER_LAT, offset, 0, NULL)) + goto out_free; + if (handle->compress) + flags |= TRACECMD_SEC_FL_COMPRESS; + + offset = out_write_section_header(handle, TRACECMD_OPTION_BUFFER_LAT, + "buffer latency", flags, false); + copy_file_compress(handle, path, NULL); + if (out_update_section_header(handle, offset)) + goto out_free; put_tracing_file(path); handle->file_state = TRACECMD_FILE_CPU_LATENCY; + if (tracecmd_get_out_file_version(handle) >= 7) + tracecmd_write_options(handle); + return handle; out_free: - tracecmd_output_close(handle); + if (handle) + tracecmd_output_close(handle); + else + close(fd); + unlink(output_file); return NULL; } diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index f8c1c090..2e8db8a4 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4531,7 +4531,7 @@ static void record_data(struct common_record_context *ctx) if (latency) { handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, - ctx->file_version); + ctx->file_version, ctx->compression); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) From patchwork Thu Jul 29 05:09:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD07C43216 for ; Thu, 29 Jul 2021 05:10:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3814160FED for ; Thu, 29 Jul 2021 05:10:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233966AbhG2FKi (ORCPT ); Thu, 29 Jul 2021 01:10:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233965AbhG2FKh (ORCPT ); Thu, 29 Jul 2021 01:10:37 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53DF1C061757 for ; Wed, 28 Jul 2021 22:10:34 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id u12so6434766eds.2 for ; Wed, 28 Jul 2021 22:10:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OcmTwSdFAUVMUYjmZI+PSyTyeZ94oARlcL1CjThkgGo=; b=IpFzmhnSxXsYPP7pII8G4gSbRSV6Ndv/TLZYZ4vxLQxiS39pfDMOdaLmVV4uzWM16I 8tmx06FX1XC7IZrw8GpvN5pA3jm/AkwC6kVW/3aG4Pe+X8rxIyTWOZZ25sSaG00rksdG QgZsA78gBKjiB/bCK98+SxPrQELq3+l9BBcEnKzpRjC+1nalgce3P8MLP5vajVd4HWUG zp8rOpozLN2CmUY710GRAO6NW5VO+FG4Ii9qj5cwW0Sf/bdCIHe2l43hoyYUFfTpsXMU JAVmilQ3gH0iJXUlT3lpyekxgFSCK6UucKVJoSQ7vLyeg+cAEMETRwHBEZiiO8fwMg8j zu7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OcmTwSdFAUVMUYjmZI+PSyTyeZ94oARlcL1CjThkgGo=; b=TbgvZqtWtffLjyj4kWVKMPyshPjQInmlFA7GmbksSjpJ9BOkOnZ4mtMSgxEJwNBN2o hKYjd7Is0JZRVdO0orXLZKYh4cSI9SVmNURuMQ/xCRmtUQRudO1suJ1et+YUt6+secwD qTUquAsXfWM2Sh6LGJBnqhL7Lq24kI8R7h07WcCgsReRLIEhxaZy51WptXOp3p+3eXOe f5VNrcAXO3ruwtw/M0+IID1iqZvlHKqW//umc5TC5WH/363ZFnlJkQd6fnf2IRhrLxwq BoxsuOI6UslapqlIzuWIAhJ86ibf3Q7+88lYu580XyR8AyXtHWhgTOYgqBw63iSrydXb beVg== X-Gm-Message-State: AOAM532LsS7S8Fmy4SVF7E6IsUe6w1G9CRR96xSJrYDfpcOPpr9/qf+F xkW/S35sKebaPT8vAyXhbRo= X-Google-Smtp-Source: ABdhPJyWm38B8uZQrNjX3PN+/i6/fQN5rTcyj1Couw2bwKrQApWwplzMdmq4WxtL4/fLBOjDrETG1A== X-Received: by 2002:a50:f615:: with SMTP id c21mr4058155edn.146.1627535432927; Wed, 28 Jul 2021 22:10:32 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:32 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 32/87] trace-cmd library: Do not write CPUs with empty trace data Date: Thu, 29 Jul 2021 08:09:04 +0300 Message-Id: <20210729050959.12263-33-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 has more flexible design, which allows to write only CPUs with trace data in the file. If a CPU has no recorded trace data, do not put it in the trace file version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 83c3b9e7..87bbcf39 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2335,8 +2335,10 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (i < cpus) ret = -1; - else + else if (size || handle->file_version < 7) /* do not write empty trace buffer in v7 files*/ ret = out_write_cpu_data(handle, cpus, data, buff_name); + else + ret = 0; for (i = 0; i < cpus; i++) { if (data[i].fd >= 0) From patchwork Thu Jul 29 05:09:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E96DDC4338F for ; Thu, 29 Jul 2021 05:10:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D31F461052 for ; Thu, 29 Jul 2021 05:10:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233965AbhG2FKj (ORCPT ); Thu, 29 Jul 2021 01:10:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKi (ORCPT ); Thu, 29 Jul 2021 01:10:38 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28C6DC061765 for ; Wed, 28 Jul 2021 22:10:35 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id nd39so8463978ejc.5 for ; Wed, 28 Jul 2021 22:10:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hHTRLRkVeHPOPLN7JMHBp+dCdTrGhBJFy8HEkuTNVcU=; b=Ast5OoxcIl4tn5IVQkcygMzyrc76scEXYc7sKRVbAUpGvZYQQHXt7HtsIm2ZQHnEZJ oX+tcowZ+DEtJ3tV5XHIhYX73TZhQqqJvUMgY6NXUAMQeGaY3PpSsL7rHHFW6A81M8un 5XWCRjm2b/SK25grFGQ1JT5XVIIiDsJFFd5sGBdpEiwevrpnU/4SVOHFf3JXAH+gI9qD 9oDprQkXFtVp7rUz5MteZn9W+0kfQncEGDHWvd5QOeYztYktCDoTY5y2fyk7Go8k5x+P y7lGV5XwfkRtUVp2UOpGIHj6gHvxuxknsPHCybB+QXE0r3TaJtka49gPLcuO8SNeiFAa J4tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hHTRLRkVeHPOPLN7JMHBp+dCdTrGhBJFy8HEkuTNVcU=; b=AXjqNnfNejI/C7djz9IyaumFS8eLbLnwauvxGMb7jqQsj33aqjpm3ykoXBzFBS/EV8 NNTbTgW/o/HbivdmE9g/q9FiYMQSMqGzV8PfoFuErIOTSvZ/UDC8wabfI2E0ibH78Ume l19VuanvyaRkiHUNClqLRGVP9iIxE8hslPXsqGAT+B9lJ492iQOXTEl5rl6NdfcWRtwd ceMehIu9qnfdlRKnU8DYO7s0o/HPYizjOSrWmhtp/1GhHfDZZ6+kxBlUeo9d+6flBWq2 YQe7CW7weHBE2XkJxmx2hO3Ck2IE0ZSr+xKsGNfWVptaxAcj3K3dhcuDUESm8OBDmFQt n+yA== X-Gm-Message-State: AOAM530b86cOcOQl0QnODDO3R1rlCzh5CUq/h6kIzZfsQRhQVkbCM21R ZOi0zzD4IEJeEHOllY6TIdRNczEiytsVTw== X-Google-Smtp-Source: ABdhPJyZFIgARw4N5AsI1ntY9bKbFfFc0l9tRIyLirH9Q+U7RvR90TZiW9sY5hKwh5q3QWIDsehhjA== X-Received: by 2002:a17:906:2b91:: with SMTP id m17mr2973616ejg.48.1627535433828; Wed, 28 Jul 2021 22:10:33 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:33 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 33/87] trace-cmd library: Add macro to check file state on reading Date: Thu, 29 Jul 2021 08:09:05 +0300 Message-Id: <20210729050959.12263-34-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 has flexible structure. It allows reading almost any part of the file at any time, unlike the version 6 file when reading state must be validated at each step. Added a macro to handle these checks. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index c5b4c32e..6872f6ef 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -166,6 +166,8 @@ struct tracecmd_input { __thread struct tracecmd_input *tracecmd_curr_thread_handle; +#define CHECK_READ_STATE(H, S) ((H)->file_version < 7 && (H)->file_state >= (S)) + static int read_options_type(struct tracecmd_input *handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag) @@ -416,7 +418,7 @@ static int read_header_files(struct tracecmd_input *handle) char *header; char buf[BUFSIZ]; - if (handle->file_state >= TRACECMD_FILE_HEADERS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_HEADERS)) return 0; if (do_read_check(handle, buf, 12)) @@ -622,7 +624,7 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) int unique; int ret; - if (handle->file_state >= TRACECMD_FILE_FTRACE_EVENTS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_FTRACE_EVENTS)) return 0; if (regex) { @@ -693,7 +695,7 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) int unique; int ret; - if (handle->file_state >= TRACECMD_FILE_ALL_EVENTS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_ALL_EVENTS)) return 0; if (regex) { @@ -778,7 +780,7 @@ static int read_proc_kallsyms(struct tracecmd_input *handle) unsigned int size; char *buf; - if (handle->file_state >= TRACECMD_FILE_KALLSYMS) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_KALLSYMS)) return 0; if (read4(handle, &size) < 0) @@ -809,7 +811,7 @@ static int read_ftrace_printk(struct tracecmd_input *handle) unsigned int size; char *buf; - if (handle->file_state >= TRACECMD_FILE_PRINTK) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_PRINTK)) return 0; if (read4(handle, &size) < 0) @@ -855,7 +857,7 @@ static int read_cpus(struct tracecmd_input *handle) { unsigned int cpus; - if (handle->file_state >= TRACECMD_FILE_CPU_COUNT) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_CPU_COUNT)) return 0; if (read4(handle, &cpus) < 0) @@ -2861,7 +2863,7 @@ static int read_options_type(struct tracecmd_input *handle) { char buf[10]; - if (handle->file_state >= TRACECMD_FILE_CPU_LATENCY) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_CPU_LATENCY)) return 0; if (do_read_check(handle, buf, 10)) @@ -3026,7 +3028,7 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) unsigned long long size; char *cmdlines; - if (handle->file_state >= TRACECMD_FILE_CMD_LINES) + if (CHECK_READ_STATE(handle, TRACECMD_FILE_CMD_LINES)) return 0; if (read_data_and_size(handle, &cmdlines, &size) < 0) From patchwork Thu Jul 29 05:09:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79448C432BE for ; Thu, 29 Jul 2021 05:10:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 645FC60FED for ; Thu, 29 Jul 2021 05:10:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233967AbhG2FKk (ORCPT ); Thu, 29 Jul 2021 01:10:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKj (ORCPT ); Thu, 29 Jul 2021 01:10:39 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 329E5C061757 for ; Wed, 28 Jul 2021 22:10:36 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id ga41so8408635ejc.10 for ; Wed, 28 Jul 2021 22:10:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HiWFfARCbjGXEgSBoVkvTrq3/2OvWG8ajOxD5h3Vzy0=; b=A/TUX5zpYpuf0Nv7xERJPiBTd2UPtV3o+RuX2xICLX9eXPwUuOLHQ/eBNDHegqvhXv qkqvdnIBnca5KlSC7+4nn8Qg9+Kvtj6fmTOTr13v7UhJPjVRHQ1RtpENXbHZ7LxU3RDU r5jDRFEXo0AE8WUp3YkL99+Zls+bfL7S8mAN/rJAGCSbwm13ZrFFi7ZLfheWoB1XqJje UOWymShQfve30notDGdYK3dzFe58xUUi9whE3GhCliwTf5PFfYKonf5onakASLpGSYQT wBBvhh1hdv18HnG5wYSuiKhhzYlxsANY0Q+9XsMdJxrDAvSeQQv2h+8RGEL7hgXPcSvO eapg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HiWFfARCbjGXEgSBoVkvTrq3/2OvWG8ajOxD5h3Vzy0=; b=jF8yisSu5FrP0wxcpGOMO4ZzBIXN3XclGWHaNSj3KIvKcadhvgiubzruXb9CA3uG+p sUQWy0a+f0sG/9ahhCUToYZtePXdjZdatRcptE61lmJpO0+CCN0FgZzdTTFvr/t3598m /ktTycQXHVGLDV+zjgcOV2UztaxNsG4x49qdycUth4X51lF5l1QCfH8DxdAKr8nQIWdI fXFlGCDcp1luzPqjIhU/B+ub1nwdv1DH8DXberiDWI/JKncQM3Hu9A+bl0FKN8nygEBC 1JSGcrTGIdVurDTa0koBaNDrgD0eTNbJw6g6nHrAuZdUWyPuG6XEdv9cduFRKxxltuO0 Lmqg== X-Gm-Message-State: AOAM5321s8rqi5wN6X3v+psRhU25Z9NoNFotuDDbF6p79810ZRVryuL0 MCYSFIcTezSlTy7wTaRSXfLLntEBszAexA== X-Google-Smtp-Source: ABdhPJxr+NOsGQ8Ssb6lth0NhPKXO4ojRfgFB54kAB0g0TyDhhDcJGg64RKzSLRT1/Xm6tjDy7oLJg== X-Received: by 2002:a17:906:a445:: with SMTP id cb5mr2999165ejb.320.1627535434861; Wed, 28 Jul 2021 22:10:34 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:34 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 34/87] trace-cmd library: Introduce sections in trace file reading logic Date: Thu, 29 Jul 2021 08:09:06 +0300 Message-Id: <20210729050959.12263-35-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 is based on sections. Added an internal sections database and new helper functions to add, read, open and close file sections. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 71 +++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 6872f6ef..e8eefb5c 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -114,6 +114,14 @@ struct tsc2nsec { unsigned long long offset; }; +struct file_section { + int id; + unsigned long long section_offset; + unsigned long long data_offset; + enum tracecmd_section_flags flags; + struct file_section *next; +}; + struct tracecmd_input { struct tep_handle *pevent; unsigned long file_state; @@ -154,6 +162,7 @@ struct tracecmd_input { struct hook_list *hooks; struct pid_addr_maps *pid_maps; /* file information */ + struct file_section *sections; size_t header_files_start; size_t ftrace_files_start; size_t event_files_start; @@ -411,6 +420,61 @@ __hidden int in_uncompress_block(struct tracecmd_input *handle) return ret; } +static struct file_section *get_section(struct tracecmd_input *handle, int id) +{ + struct file_section *sec = handle->sections; + + while (sec) { + if (sec->id == id) + return sec; + sec = sec->next; + } + return NULL; +} + +static struct file_section *open_section(struct tracecmd_input *handle, int id) +{ + struct file_section *sec = get_section(handle, id); + + if (!sec) + return NULL; + + if (lseek64(handle->fd, sec->data_offset, SEEK_SET) == (off64_t)-1) + return NULL; + if ((sec->flags & TRACECMD_SEC_FL_COMPRESS) && in_uncompress_block(handle)) + return NULL; + return sec; +} + +static void close_section(struct tracecmd_input *handle, struct file_section *sec) +{ + if (sec->flags & TRACECMD_SEC_FL_COMPRESS) + in_uncompress_reset(handle); +} + +static int add_section(struct tracecmd_input *handle, int id, int flags, + unsigned long long soffset, unsigned long long doffset) +{ + struct file_section *sec = get_section(handle, id); + + if (!sec) { + sec = calloc(1, sizeof(struct file_section)); + if (!sec) + return -1; + sec->next = handle->sections; + handle->sections = sec; + } + sec->id = id; + if (soffset) + sec->section_offset = soffset; + if (doffset) + sec->data_offset = doffset; + if (flags > 0) + sec->flags = flags; + return 0; +} + + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -3522,6 +3586,7 @@ void tracecmd_ref(struct tracecmd_input *handle) */ void tracecmd_close(struct tracecmd_input *handle) { + struct file_section *del_sec; int i; if (!handle) @@ -3560,6 +3625,11 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->version); close(handle->fd); + while (handle->sections) { + del_sec = handle->sections; + handle->sections = handle->sections->next; + free(del_sec); + } for (i = 0; i < handle->nr_buffers; i++) free(handle->buffers[i].name); @@ -4006,6 +4076,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->nr_buffers = 0; new_handle->buffers = NULL; new_handle->version = NULL; + new_handle->sections = NULL; new_handle->guest = NULL; new_handle->ref = 1; if (handle->trace_clock) { From patchwork Thu Jul 29 05:09:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDC7FC4320E for ; Thu, 29 Jul 2021 05:10:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D6D1D61054 for ; Thu, 29 Jul 2021 05:10:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233975AbhG2FKk (ORCPT ); Thu, 29 Jul 2021 01:10:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKk (ORCPT ); Thu, 29 Jul 2021 01:10:40 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41A56C061765 for ; Wed, 28 Jul 2021 22:10:37 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id da26so6430476edb.1 for ; Wed, 28 Jul 2021 22:10:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SZGfgbN8GTyZFbiZL50LNL3hWol4el7zjJSKUrxYzbQ=; b=jFOfbzfYVRESLqLtsnQkNZtplfngynPwjNyiuhiSzr3hMUtzEeah9DXIivJfRK/H4D cnA0qXJG5fgnhDfFliaSiYkaJY9xnbwfsjeCslv74rkDq0M7/qiK9TuQ1TkOHN346lKr Nb7frvG4MTCdP+9vlhNWHc8admDTYdWfyU/qljf4sROv87RIq0qSdBGmk8dk+CH6e4Cs ScTjjwo8RnRykJ3nDvQjNqSl2Ut2nZCMauglT/RmocXqcvwGw2EVsvwbp8JM1F+OD/FU 6Oqr080zwvt4lE28Ug6GM3JZCvUdpcaABReXUUuhAwaHUf6zHReoQ3aeVhZmefidbGNE I3yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SZGfgbN8GTyZFbiZL50LNL3hWol4el7zjJSKUrxYzbQ=; b=o1k/Sx6eOWaA+WfpnMuAwEOjNzc6zEiUbID+rcDdw78MY4FY1ajXVv/bRDdTaOh6qY d9RJ42B0HIGL3g1jZVLxnXab5RLKS7q2kNGa64rIDJ9sxGi9vX9Tmo6PVshBZIxxz4xE BgPFUSSjMgEQhVCKwQs0sJgq2lqxt/cTSuMcKIvtlpw+BiniMdi7P+ZtskaagNiCCfrt pkXWLUdM7mII3CZzQVVO7PgcqRGmXhhRaAcqSAimzWtb38hRIy2iL5qD0M7Bq7twzPbH INI6RN7QvR0MMxJRPriK2wLI622Vdkh/n/bguNCieb7twAhyrGsrqXn1nJ5JyjrH6SOI aInQ== X-Gm-Message-State: AOAM530Xa+ez8mptTWY+p22pXT9mH8KPYZujNZWXLI8arm5gRrEgmzWU Lp9ab2QF+w3bECTK2KVMzZc= X-Google-Smtp-Source: ABdhPJxMIfnc4eN09K5z7M8P4lpOKKsNrLfApAujaYAU9YkYLq0d6toLWTLEZ2eHuYrD8iwWtIspIw== X-Received: by 2002:a05:6402:487:: with SMTP id k7mr3819192edv.315.1627535435876; Wed, 28 Jul 2021 22:10:35 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:35 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 35/87] trace-cmd library: Initialize internal sections database on file read Date: Thu, 29 Jul 2021 08:09:07 +0300 Message-Id: <20210729050959.12263-36-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add sections in internal database on file reading and parsing. In trace file version 7, sections are initialized when parsing corresponding trace options. In version 6 files, sections are retrieved on file reading, as there they are in a fixed position file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 55 +++++++++++++++++++++++++++---------- 1 file changed, 40 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e8eefb5c..f5e13bbc 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -165,7 +165,6 @@ struct tracecmd_input { struct file_section *sections; size_t header_files_start; size_t ftrace_files_start; - size_t event_files_start; size_t options_start; size_t total_file_size; @@ -485,6 +484,10 @@ static int read_header_files(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_HEADERS)) return 0; + if (handle->file_version < 7) + add_section(handle, TRACECMD_OPTION_HEADER_INFO, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (do_read_check(handle, buf, 12)) return -1; @@ -528,9 +531,6 @@ static int read_header_files(struct tracecmd_input *handle) free(header); - handle->ftrace_files_start = - lseek64(handle->fd, 0, SEEK_CUR); - handle->file_state = TRACECMD_FILE_HEADERS; return 0; @@ -691,6 +691,10 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) if (CHECK_READ_STATE(handle, TRACECMD_FILE_FTRACE_EVENTS)) return 0; + if (handle->file_version < 7) + add_section(handle, TRACECMD_OPTION_FTRACE_EVENTS, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (regex) { sreg = &spreg; ereg = &epreg; @@ -729,9 +733,6 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) return -1; } - handle->event_files_start = - lseek64(handle->fd, 0, SEEK_CUR); - if (sreg) { regfree(sreg); regfree(ereg); @@ -762,6 +763,10 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) if (CHECK_READ_STATE(handle, TRACECMD_FILE_ALL_EVENTS)) return 0; + if (handle->file_version < 7) + add_section(handle, TRACECMD_OPTION_EVENT_FORMATS, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (regex) { sreg = &spreg; ereg = &epreg; @@ -847,6 +852,11 @@ static int read_proc_kallsyms(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_KALLSYMS)) return 0; + if (handle->file_version < 7) + add_section(handle, TRACECMD_OPTION_KALLSYMS, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + + if (read4(handle, &size) < 0) return -1; if (!size) @@ -878,6 +888,11 @@ static int read_ftrace_printk(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_PRINTK)) return 0; + if (handle->file_version < 7) + add_section(handle, TRACECMD_OPTION_PRINTK, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + + if (read4(handle, &size) < 0) return -1; if (!size) @@ -2909,6 +2924,16 @@ static int handle_options(struct tracecmd_input *handle) handle->tsc_calc.offset = tep_read_number(handle->pevent, buf + 8, 8); break; + case TRACECMD_OPTION_HEADER_INFO: + case TRACECMD_OPTION_FTRACE_EVENTS: + case TRACECMD_OPTION_EVENT_FORMATS: + case TRACECMD_OPTION_KALLSYMS: + case TRACECMD_OPTION_PRINTK: + case TRACECMD_OPTION_CMDLINES: + if (size < 8) + break; + add_section(handle, option, -1, tep_read_number(handle->pevent, buf, 8), 0); + break; default: tracecmd_warning("unknown option %d", option); break; @@ -3095,6 +3120,10 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) if (CHECK_READ_STATE(handle, TRACECMD_FILE_CMD_LINES)) return 0; + if (handle->file_version < 7) + add_section(handle, TRACECMD_OPTION_CMDLINES, 0, 0, + lseek64(handle->fd, 0, SEEK_CUR)); + if (read_data_and_size(handle, &cmdlines, &size) < 0) return -1; cmdlines[size] = 0; @@ -3384,6 +3413,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) struct tracecmd_input *handle; char test[] = TRACECMD_MAGIC; unsigned int page_size; + size_t offset; char *version; char buf[BUFSIZ]; unsigned long ver; @@ -3444,14 +3474,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) read4(handle, &page_size); handle->page_size = page_size; - handle->header_files_start = - lseek64(handle->fd, 0, SEEK_CUR); - - handle->total_file_size = - lseek64(handle->fd, 0, SEEK_END); - - handle->header_files_start = - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + offset = lseek64(handle->fd, 0, SEEK_CUR); + handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); + lseek64(handle->fd, offset, SEEK_SET); handle->file_state = TRACECMD_FILE_INIT; From patchwork Thu Jul 29 05:09:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96B79C4338F for ; Thu, 29 Jul 2021 05:10:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 85B2D61019 for ; Thu, 29 Jul 2021 05:10:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233989AbhG2FKm (ORCPT ); Thu, 29 Jul 2021 01:10:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKm (ORCPT ); Thu, 29 Jul 2021 01:10:42 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50E62C061757 for ; Wed, 28 Jul 2021 22:10:38 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id ga41so8408743ejc.10 for ; Wed, 28 Jul 2021 22:10:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ptPZ/80MdhCrEcenl0jzvNbSbaGJq99hoNeDNg4hlMk=; b=s/7X9LruWAji4MlB7rO7zJxSl86LpDWMGnVys/5klgiWs3xFnqPVFrKArb238aV8ep PZBNm6jz8WEU9DQOi+iNmaRgC8YK8oDcoAwK4XRcyBp6HT6kfvlXhk+NSNuIEq2vvKdA DESZpDfazkplqaSeoqsPR2sMX3K1PJM9ovLtXn/nLPki1VipJwMnMpylt4VxNvMyQh6e g6hRKKBb3EMqMEEZyKHnm3TMIaNKne9CAnbJ7eDs3+wzKxS437OoUzWSbUXEX4qOX2e/ DW3DWZ71EHkXg2KumsWQlumL3WAVdPWLpFXyAomYoqb23/JmNunQ+ClSELd0dNOMRX9Y y6Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ptPZ/80MdhCrEcenl0jzvNbSbaGJq99hoNeDNg4hlMk=; b=oWH9WEEzycRU13gQl+3JQnPoNYO79LRDgdzEDx7uaiUti16HyAiEscfhMbIEhlmB/o +RGTsH1KXddIqkorhd5fLln9RZf+UUciHEG+qMK+Ow8Tr1LfQYJnlPu/mfOFBl0qubDw XgurYIyrqTZR0/Cn9pjoZnPxOwPbBmIbjCp+41R13XF8zqs6pebHVHGGuUGOPwgnh6xB On5WXQDB/DYxPIkap1V7MVZhu+NJGvnL/wGwiMPWczokk3xUKEnJhArVeA9ZVf88OzRK RAU+hiMBSXrHwFJl5K+l5Tov6+fnYQnIjp1hjyl7Q8SjybzWskBVcaUW6184UEemJAPP 0okw== X-Gm-Message-State: AOAM533QUgj4M6DGA7kGmtzHy+m5OuwPKK81MDjJH9c8dXfAHwrpKvZZ ZINZp+QZ55wvjkwSXKnrMUjrbJ2zI1nSmA== X-Google-Smtp-Source: ABdhPJxZ1Ywd8FKh4q5CVP+DMUwoYK8Upy3p3/5q5ibvr6XFit/OGViaU/VE9ZEuTxCChQJeMFUYJA== X-Received: by 2002:a17:906:9246:: with SMTP id c6mr2783848ejx.78.1627535436927; Wed, 28 Jul 2021 22:10:36 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:36 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 36/87] trace-cmd library: Use sections database when reading parts of the trace file Date: Thu, 29 Jul 2021 08:09:08 +0300 Message-Id: <20210729050959.12263-37-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Utilize the internal database with file sections, when reading parts of a trace file. This logic unifies the way trace file version 6 and 7 are processed. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f5e13bbc..bf56c92a 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -163,8 +163,6 @@ struct tracecmd_input { struct pid_addr_maps *pid_maps; /* file information */ struct file_section *sections; - size_t header_files_start; - size_t ftrace_files_start; size_t options_start; size_t total_file_size; @@ -3301,21 +3299,26 @@ int tracecmd_make_pipe(struct tracecmd_input *handle, int cpu, int fd, int cpus) */ void tracecmd_print_events(struct tracecmd_input *handle, const char *regex) { - int ret; + struct file_section *sec; if (!regex) regex = ".*"; - if (!handle->ftrace_files_start) { - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + sec = open_section(handle, TRACECMD_OPTION_HEADER_INFO); + if (sec) { read_header_files(handle); + close_section(handle, sec); + } + sec = open_section(handle, TRACECMD_OPTION_FTRACE_EVENTS); + if (sec) { + read_ftrace_files(handle, regex); + close_section(handle, sec); + } + sec = open_section(handle, TRACECMD_OPTION_EVENT_FORMATS); + if (sec) { + read_event_files(handle, regex); + close_section(handle, sec); } - ret = read_ftrace_files(handle, regex); - if (ret < 0) - return; - - read_event_files(handle, regex); - return; } /* Show the cpu data stats */ @@ -3905,6 +3908,7 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state) { + struct file_section *sec; int ret; if (!start_state) @@ -3920,13 +3924,17 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, if (handle->file_state >= start_state) { /* Set the handle to just before the start state */ - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + sec = open_section(handle, TRACECMD_OPTION_HEADER_INFO); + if (!sec) + return -1; /* Now that the file handle has moved, change its state */ handle->file_state = TRACECMD_FILE_INIT; } - /* Try to bring the input up to the start state - 1 */ ret = tracecmd_read_headers(handle, start_state - 1); + if (sec) + close_section(handle, sec); + if (ret < 0) goto out; From patchwork Thu Jul 29 05:09:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FC76C432BE for ; Thu, 29 Jul 2021 05:10:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2613861052 for ; Thu, 29 Jul 2021 05:10:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233997AbhG2FKn (ORCPT ); Thu, 29 Jul 2021 01:10:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKm (ORCPT ); Thu, 29 Jul 2021 01:10:42 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 266EDC061765 for ; Wed, 28 Jul 2021 22:10:39 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id go31so8446282ejc.6 for ; Wed, 28 Jul 2021 22:10:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rkIWM45qXCxkk1hczFTdWShdbvftacGLnbBLuxGPqLc=; b=OHqN+b5H5hboLDP7eHpevQZpYJx1AmKyNM9heoXMM8dLa68nzh+y54Y8LyhRZkRCMO Pwf3LxSQ9pp+Okyx2KwLdzRmsTOg92LvBdJIQN9Wd/rF5wF7eONwzzlKzgg1eiBs8NVU /G6OKkSd1PKg4eH0kqtd3rY7GfnxbnbBYTCx8iSYdkQL2dxdhoMJGh8W7YW4PI91zZJI h2X2j+ZVdbXbNa2ulp8/NMuB88L0rcQqbAL3u0RD2U5v56gtVT/H9RR8Jd1lAGoLIpFx 73Rjg9SZkEF1AzsUs230xmtL2FMxRCAo9hm2NtmQYFrNx9dfp7oFAo3CiVFA8GrOfLX1 Yr9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rkIWM45qXCxkk1hczFTdWShdbvftacGLnbBLuxGPqLc=; b=N9mXGdRhX1szMEwHTh+8HQFl+QLVG6h6kidIHdjWNKHNJHwSS4dlldoTwFbA8W4sTu 5sUKcoHeqY6+mo4efwXZD0O9NNDuJOT3ebdxv3117n6JjRvxmJk30c8TxM0a01lQCcUY MqM0Go5m7BMq6XMdr5Nl6HC6+HQ+NSwf0k/JSKKZS5huqopG5xiSdSCbc0nJvXZST5Ec SjQBVr54P/O9Sd/uS8p//uO5+hQMAksQrDaZLVaXVHUuVrIWVuiOyxPd+q0bvQRV3/3Y mW/mLgeGi7iCtoyu/npoBszu8wOH/VfYBk65JrfU75Zs5hwPWn63u/xLPF6PSY13tF0e wFvw== X-Gm-Message-State: AOAM533xPVKUxRWdju0UH4WKQHlQphql3yturijJYQds8luc+4cH/Y2f ir0FJ9vwifop5MzIIkdKx90= X-Google-Smtp-Source: ABdhPJweXy2wwTuoLioQa/hch2FDxfB6rIsEao/Ah4iMTg73iJEiPZMiSMGJu2ftZGYcBUppopREzQ== X-Received: by 2002:a17:906:7716:: with SMTP id q22mr2897185ejm.457.1627535437837; Wed, 28 Jul 2021 22:10:37 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:37 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 37/87] trace-cmd library: Set log size to the input tep handler when it is read from the file Date: Thu, 29 Jul 2021 08:09:09 +0300 Message-Id: <20210729050959.12263-38-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Setting the long size to the input tep handler in tracecmd_read_headers() API may be too late, as this tep handler is used to read and parse data from the file before that. The most suitable place for that is tracecmd_alloc_fd() API, right after reading the long size from the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index bf56c92a..0ced15a8 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -976,8 +976,6 @@ int tracecmd_read_headers(struct tracecmd_input *handle, if (ret < 0) return -1; - tep_set_long_size(handle->pevent, handle->long_size); - if (state <= handle->file_state) return 0; @@ -3473,6 +3471,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) do_read_check(handle, buf, 1); handle->long_size = buf[0]; + tep_set_long_size(handle->pevent, handle->long_size); read4(handle, &page_size); handle->page_size = page_size; From patchwork Thu Jul 29 05:09:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39040C4320E for ; Thu, 29 Jul 2021 05:10:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 21B8661052 for ; Thu, 29 Jul 2021 05:10:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233871AbhG2FKo (ORCPT ); Thu, 29 Jul 2021 01:10:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKn (ORCPT ); Thu, 29 Jul 2021 01:10:43 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E9FEC061757 for ; Wed, 28 Jul 2021 22:10:40 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id r16so6379142edt.7 for ; Wed, 28 Jul 2021 22:10:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xWxLdmFaQRTxvlmcJRKFanHtyMsC2W8pFfsQm7TZ+J0=; b=Qm6oB+1UAe2xTpApsVC9TxN5aPslaKlf1Un9rEhCknKM2obrvqmoKJb95/zpqA1DQO XLtbGxP8UuT67WCggw1a6CxIHZ/oW08+h0vSddat+JfjJHODC52e+Q2pbBxqaQLj3eU4 D34wooLB5DUbwv20yEDqToqgpr6Q8hvI2WjkrKS2xqVwoWYze0Pr41ian6baRdvRmtLi e4E5bp5+Oo3NoeJ5IKpqm5FVebaQmoLMRw1MiwPusVUwKtCvD+FJ3lEvFRqlINpNc1L1 nuZ7HUfkXbNRztmkvbjQcQC+i1xJe/MBm7YqwPeBKBsjGlXE/aoqj/fJG02FOZpN1r6B dcZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xWxLdmFaQRTxvlmcJRKFanHtyMsC2W8pFfsQm7TZ+J0=; b=mvRoOsV6elD6vuGoPt4rreYX6JKCIuF4fIBuZKqnrh+o/wa3/WHEB8jApk9qChdQqb pm3g250liY/vrSY7p1wnbT6EGQQFPwaIPYnhyXz0ILGORj5zEGue6HHbPm6YAoVob8CB IoNWMcXY26onRSb7CvwQKj/W2Xq5b4sda4cJqcV+m0AFuCovmxb5e641/2ig1YdCRIvR zFGRYUUb7puvSz0YTScZfa/uAqcWsX2OJx7lcIXkWQu31EwKsUH935J1qp73++DGiN/j bqgdM9MUTTPSM/RiPIn5hwZBga7ZCzH3pRFRbZMHKxAvtkLSFgwHymPqH/6TLBltV24I 8sWg== X-Gm-Message-State: AOAM532Q0HgPJZxya1MjsUDSa90JivAUr6ATt/u4YLrYxxHsAQtqWOxu aXr0SBFAV4KZEGKS029HabA= X-Google-Smtp-Source: ABdhPJzGi1goT2HGvdMG+C/BhB/yY24YW7z6psr4r4xolrjq3HXdoyXoGw6FeW7ZqlVZkQBoCm4Rsw== X-Received: by 2002:a05:6402:896:: with SMTP id e22mr3999428edy.246.1627535438737; Wed, 28 Jul 2021 22:10:38 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:38 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 38/87] trace-cmd library: Fix possible memory leak in read_ftrace_files() Date: Thu, 29 Jul 2021 08:09:10 +0300 Message-Id: <20210729050959.12263-39-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Some error paths in read_ftrace_files() may lead to a memory leak. Improved the error handling of this internal function to avoid it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 0ced15a8..b36df98b 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -720,25 +720,28 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) - return -1; + ret = read4(handle, &count); + if (ret < 0) + goto out; for (i = 0; i < count; i++) { - if (read8(handle, &size) < 0) - return -1; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_ftrace_file(handle, size, print_all, ereg); if (ret < 0) - return -1; + goto out; } + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + ret = 0; +out: if (sreg) { regfree(sreg); regfree(ereg); } - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - - return 0; + return ret; } static int read_event_files(struct tracecmd_input *handle, const char *regex) From patchwork Thu Jul 29 05:09:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17296C432BE for ; Thu, 29 Jul 2021 05:10:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F03A261019 for ; Thu, 29 Jul 2021 05:10:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234005AbhG2FKo (ORCPT ); Thu, 29 Jul 2021 01:10:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKo (ORCPT ); Thu, 29 Jul 2021 01:10:44 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA09AC061765 for ; Wed, 28 Jul 2021 22:10:40 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id jg2so8551390ejc.0 for ; Wed, 28 Jul 2021 22:10:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pDnYY4z8ou6AlEibYEdl+zWXg6GB0ES5kQANFrPKjFc=; b=Eu/6tqgh1vhmmTfECywQqm2QsC6pOzb3E09bsbaSo6jo45lwp8Ays8NGWfvOItLPMq DH8Nk7SfugNuWs5aRrCV8O4f44ncAxoxmWonz2gZUlpgBm6ThG9j3q6FNNN/oISKeQaA TzpTVFxKHap7XcKvm0+xeTjNmCveW1Iojk4IWzunxTHvSVtNQ/T8HzS/k+5mcuaa5Ezq SbwxzuxtY6wDvlmCyXuUABg35FxR9E65DL/1WXi+GM9jepIu/hnl58zBhAImlUDLriX9 K3K3a1CTTgBt7bjr7mdSrWlJx0o7lg7ttY61BVH3H1qCHpvbBRLX6UHKVlJjJ11bNsfd cr+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pDnYY4z8ou6AlEibYEdl+zWXg6GB0ES5kQANFrPKjFc=; b=Rgz01r3fie3+bi5kYRnCSdH1i+JrrSdOexwp6n86Dn6ayOYat8nH9PdC/Nq5dI5Kt1 i65csPKiHCIc96C5dk4vQYmcdWmHqpc/Ildv+pvQ+prXL3r0zXZIGQG/MwZCALFeyNYg QVxUVrWTkKLsmTIwYJlX9jvb6NnPK5LLPdFF64fDr2zelEgaIxrouoTFOwlRU8T78v6t /43/HbVllzTr0kPbbhJljsUFvujMAn2J/EbD9f7QkfhkjZol3XY/YX0a9uFCpJ06zgB9 YepwzNtmtO7PincPodnSazpNpTYNEXpzEsbxi6bH0A7JQz1DG+x9J7j/H2KmkyGS6g96 Zrlg== X-Gm-Message-State: AOAM530E3N+q9tjaF9QqCVbhGsKhmp0OzEc7x+fViGc8Yu/+AGyErkFj PT8hxIbdnhucop84VHqylYU= X-Google-Smtp-Source: ABdhPJwl+oijJBrMR7PwKEgpv5oFcY3Pk2Twsc8cQrXPapAe/rn6hZ9D9AID7sS13aEWNbBk+4KeRQ== X-Received: by 2002:a17:906:c1d0:: with SMTP id bw16mr2964397ejb.146.1627535439617; Wed, 28 Jul 2021 22:10:39 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:39 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 39/87] trace-cmd library: Fix possible memory leak in read_event_files() Date: Thu, 29 Jul 2021 08:09:11 +0300 Message-Id: <20210729050959.12263-40-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Some error paths in read_event_files() may lead to a memory leak. Improved the error handling of this internal function to avoid it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 39 ++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b36df98b..e2e12014 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -747,7 +747,7 @@ out: static int read_event_files(struct tracecmd_input *handle, const char *regex) { unsigned long long size; - char *system; + char *system = NULL; regex_t spreg; regex_t epreg; regex_t *sreg = NULL; @@ -776,13 +776,16 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) return -1; } - if (read4(handle, &systems) < 0) - return -1; + ret = read4(handle, &systems); + if (ret < 0) + goto out; for (i = 0; i < systems; i++) { system = read_string(handle); - if (!system) - return -1; + if (!system) { + ret = -1; + goto out; + } sys_printed = 0; print_all = 0; @@ -809,39 +812,35 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) - goto failed; + ret = read4(handle, &count); + if (ret < 0) + goto out; for (x=0; x < count; x++) { - if (read8(handle, &size) < 0) - goto failed; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_event_file(handle, system, size, print_all, &sys_printed, reg); if (ret < 0) - goto failed; + goto out; } free(system); - } - - if (sreg) { - regfree(sreg); - regfree(ereg); + system = NULL; } handle->file_state = TRACECMD_FILE_ALL_EVENTS; - - return 0; - - failed: + ret = 0; + out: if (sreg) { regfree(sreg); regfree(ereg); } free(system); - return -1; + return ret; } static int read_proc_kallsyms(struct tracecmd_input *handle) From patchwork Thu Jul 29 05:09:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13309C4338F for ; Thu, 29 Jul 2021 05:10:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EE67060FED for ; Thu, 29 Jul 2021 05:10:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234007AbhG2FKp (ORCPT ); Thu, 29 Jul 2021 01:10:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKp (ORCPT ); Thu, 29 Jul 2021 01:10:45 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D190CC061757 for ; Wed, 28 Jul 2021 22:10:41 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id v21so8498961ejg.1 for ; Wed, 28 Jul 2021 22:10:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hpRh7t96oyFAJ+wG6Pr44T7RUZE7tjg8wEUsChKBVYY=; b=bj24AicaB4a6nzoys9n4xiGvWjQRYCHPRa5BF/mMN/jHlMOYFy/kcQ9TI2ivoRS4gw 8eEB/GwppA/e2IfMz/pb84daHwb9Y3cOjNPHDCK3wDtK2LtME4tr2MeLFAk5C+6T77Fw zN791IM+fswjUuxlA6z3+IQzLT6+x3Sd4HYDB0JhFNCrmVv9FhqTeI7b2A1efcCt37KZ ccwuRwdUid+BYyqOu8D4cu53BqnRFO54Z5s2ykAMcOqajjSUQDbEouPQ2rzV/FSOMIio ty1eVt2/7ThdC+Gzcp3P+Wfk+ieOVYyQScNxJnza3RUwHXTMESA+51CQdEJRaSX1DbOY WM2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hpRh7t96oyFAJ+wG6Pr44T7RUZE7tjg8wEUsChKBVYY=; b=n50pfuUHud6rMILxTgv+rUwoNIf0iB/o5zn6Jz55kskavSXJtM6bmh0RNi/08EbHFf 5kyGGncm2fYivqAsMKyLhJgPqy4PN/77yBxzJqBvAQPstHNQ7Foz6642/+nR0niaKG7X CEOG6gO1PQV1UnAi7weg9PzXJ9CxC+UYcAnXpi/CNzm1wjzcxHR88W/hLYcLOXxak0cp 7xiA63s70A02udSkvqSA7qgNLeZnPSMVPhnN0FHqlrSZU09S3oPxdG8Y+Msf3glNHJvr eRqutd+QXbdEDCJv7kE8O5dFUALRvUpJ6hhRFNFRcJLvU2tEThSJhChDztcU/XaR+Ent x8Yg== X-Gm-Message-State: AOAM533iNCC6Pot34DeHwo3zYcHlT4LSp3PKw0pVhvmz/rlwbiett5Ag 1vDgFRB0Ru3FZ3gef5rB0js= X-Google-Smtp-Source: ABdhPJwYZhx1XG1RRJlsQ1gFywo2leT3oELoFW0TTSE1RLcazl6ZtnhRywb85wwFJq26TR0itaq50Q== X-Received: by 2002:a17:906:6d4e:: with SMTP id a14mr2850705ejt.328.1627535440529; Wed, 28 Jul 2021 22:10:40 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:40 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 40/87] trace-cmd library: Fix possible memory leak in read_proc_kallsyms() Date: Thu, 29 Jul 2021 08:09:12 +0300 Message-Id: <20210729050959.12263-41-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Some error paths in read_proc_kallsyms() may lead to a memory leak. Improved the error handling of this internal function to avoid it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 40 ++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e2e12014..99aa33ae 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -845,9 +845,10 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) static int read_proc_kallsyms(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; + struct tep_handle *tep = handle->pevent; unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (CHECK_READ_STATE(handle, TRACECMD_FILE_KALLSYMS)) return 0; @@ -856,28 +857,31 @@ static int read_proc_kallsyms(struct tracecmd_input *handle) add_section(handle, TRACECMD_OPTION_KALLSYMS, 0, 0, lseek64(handle->fd, 0, SEEK_CUR)); - - if (read4(handle, &size) < 0) - return -1; - if (!size) - return 0; /* OK? */ + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_KALLSYMS; + goto out; /* OK? */ + } buf = malloc(size+1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)){ - free(buf); - return -1; + if (!buf) { + ret = -1; + goto out; } - buf[size] = 0; - - tep_parse_kallsyms(pevent, buf); + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; - free(buf); + buf[size] = 0; + tep_parse_kallsyms(tep, buf); handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = 0; +out: + free(buf); + return ret; } static int read_ftrace_printk(struct tracecmd_input *handle) From patchwork Thu Jul 29 05:09:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA331C432BE for ; Thu, 29 Jul 2021 05:10:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9240261053 for ; Thu, 29 Jul 2021 05:10:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234010AbhG2FKq (ORCPT ); Thu, 29 Jul 2021 01:10:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKq (ORCPT ); Thu, 29 Jul 2021 01:10:46 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C337C061765 for ; Wed, 28 Jul 2021 22:10:43 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id da26so6430781edb.1 for ; Wed, 28 Jul 2021 22:10:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lYegybilv5zGZHqg9ZytbAQzubrYWhUWz1c5hZSVIVY=; b=OgYDPEOpN18ZGgZNabV+eo4FyiNTaCxncVrXJZxtZe4QTBo9hGPfj0mml0nnSTgH21 gWhyFrggyqUKW6jd+yyF68iNOPf1EDt5HDk9bdgZNkSdQ+zYHwq2VHAo224DoK5Zzlei Wza6CBDlRBp7trj7QPRk8VfKVvdnYgzlCRVBauy/qgH7SnGFqn7RyybYH7rGN32UJd8M LYZhGxwGDY1hXVtRXMcbtNnYGnfSAVg8eA6/9SRv2/4U+tRce2XedTfY4N0CBTyjCTXI iolBeoqZ1XSwob9JpNm2UeEhe3JuozvFySKtDHj0/H+Pct6F4J6KBfjSvz19xuYT//lj l3qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lYegybilv5zGZHqg9ZytbAQzubrYWhUWz1c5hZSVIVY=; b=B7s4XMNzBSE3dYQAzghwMULjCW5406fWGA/OIuqW9Gk2Ar3zCHtY0q3aF24jDNLoxt 9UqGPx180YVA3bYkKtQBtTJAYJiyfQdUXVmIxmwQ0+vGF7ySkY7Iud1v3cIYj/jh9Qn/ SU+ClbW0WiN9/iVM30ZREHWHOW9QhHzWtV/TMfNf4HG6v4BIg1xe85aDzbLrYuzh47Fv kAVcH8d17RbJyTpwyF3chF7hrbNlhYzslqDcBWcj6SUnPGs6xsSHYSD2omNfyUuURYcK jTKuz9nDeRGPMMk+mP13nUK/GsUulJlAi3YKBF8RDcj7HEbuEVR6yGDdX6jjsnN0ekom aX/Q== X-Gm-Message-State: AOAM531gMX3lnpjgT8wQHhCIYZvwMilctaH/jauFtzFPCwikNSq0qN2F 32mitotdsZkNXJ6tcD5odh8= X-Google-Smtp-Source: ABdhPJwljkE8Z5hIVhK3xWMEWx4IDuwqGRifK3pBnWlkyVAR+X/4QqKf1YYynuWYkLvujM+YM5V3BQ== X-Received: by 2002:a50:f1c7:: with SMTP id y7mr3960555edl.386.1627535441807; Wed, 28 Jul 2021 22:10:41 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:41 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 41/87] trace-cmd library: Fix possible memory leak in read_ftrace_printk() Date: Thu, 29 Jul 2021 08:09:13 +0300 Message-Id: <20210729050959.12263-42-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Some error paths in read_ftrace_printk() may lead to a memory leak. Improved the error handling of this internal function to avoid it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 99aa33ae..5557e772 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -887,7 +887,8 @@ out: static int read_ftrace_printk(struct tracecmd_input *handle) { unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (CHECK_READ_STATE(handle, TRACECMD_FILE_PRINTK)) return 0; @@ -896,29 +897,33 @@ static int read_ftrace_printk(struct tracecmd_input *handle) add_section(handle, TRACECMD_OPTION_PRINTK, 0, 0, lseek64(handle->fd, 0, SEEK_CUR)); - - if (read4(handle, &size) < 0) - return -1; - if (!size) - return 0; /* OK? */ + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_PRINTK; + goto out; /* OK? */ + } buf = malloc(size + 1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)) { - free(buf); - return -1; + if (!buf) { + ret = -1; + goto out; } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; buf[size] = 0; tep_parse_printk_formats(handle->pevent, buf); - free(buf); - handle->file_state = TRACECMD_FILE_PRINTK; + ret = 0; - return 0; +out: + free(buf); + return ret; } static int read_and_parse_cmdlines(struct tracecmd_input *handle); From patchwork Thu Jul 29 05:09:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C9CBC4320A for ; Thu, 29 Jul 2021 05:10:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3272B61053 for ; Thu, 29 Jul 2021 05:10:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234012AbhG2FKr (ORCPT ); Thu, 29 Jul 2021 01:10:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKq (ORCPT ); Thu, 29 Jul 2021 01:10:46 -0400 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1423BC061757 for ; Wed, 28 Jul 2021 22:10:44 -0700 (PDT) Received: by mail-ed1-x534.google.com with SMTP id h8so6415330ede.4 for ; Wed, 28 Jul 2021 22:10:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c3OjbWftgycABFbfOFENpRXx6bmt4MF8AsP7gDBMwBU=; b=bqreP39q7pjSt1Ighothur/1O8nffMhQu6u/dlEe9aniUFOJIJ7tgiMATGV/AD7tU8 6RdQ6bTebalk0JoVQ0sRJbxa/8RQisy9Pw0qYZh5Jq9gCiUi80IZKogMVCNGF/8DAhCP Va8bx59TjD1QykUyz1+y1MIeXUPmc9X56b8OTt1jzQW87cn50CXq6eH7vCRabVlIyyhZ kFlpKSdfgnJ9jDP51rbZZNeM+byisaSB6Katamp7XEuQ+nHbb/OjXMIF2gFoRRslvIkF IGbHJ0VOyYjEBTk0uW7TKjDhxRDWeX6mMOLnCGlXG+N6oRzlrMsxzQcKTHH0xJR6vyy8 9o/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c3OjbWftgycABFbfOFENpRXx6bmt4MF8AsP7gDBMwBU=; b=ujxv0z3Radl6zj6pCmnE4K4x44FN1H4dD5ouTrbrieA3WlfrrehNd/Dwwq0zjlPboz PoMEUVmALbR5qxq9RZehx5xqfXrSNLZfY1QGyc5B0TLSTMNaJBoFi2jVUk+r/7wvZJR0 WaPZTeN0CUDS8A1dM04Hu1vrxocZZq0aRrYFgvr95kqGyfe+vaQh3bW90Tv+Dak5GSzC fvn/bZ+gTfR1auKtS1P2HmNQ89YMe+G0MlL0zfCaVYIjWm1EUJrlEG5X+krRW0sfXFTR C7ubj5p2DpB3u3oT9DUs8Lm/yY9oxWsvwZomHPP1l6gstkhKTB97X6V+Mg8fAnF9zkSf /xJw== X-Gm-Message-State: AOAM5301Vuf2urf7gsJulYhiw4OucBKRH6nCzkPt/+qyOw4bPtvGZPfU jfJAGsBeLsAgIxzjOHIfH18= X-Google-Smtp-Source: ABdhPJxUgSDiWgpqqf0lj2vqwDbtJu47PXNTp/dYtdvGtNgqbaMQnNQ90Y6tpVpP8m9kTNrK6KWC+w== X-Received: by 2002:a05:6402:10d7:: with SMTP id p23mr3975818edu.74.1627535442752; Wed, 28 Jul 2021 22:10:42 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:42 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 42/87] trace-cmd library: Fix possible memory leak in read_and_parse_cmdlines() Date: Thu, 29 Jul 2021 08:09:14 +0300 Message-Id: <20210729050959.12263-43-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Some error paths in read_and_parse_cmdlines() may lead to a memory leak. Improved the error handling of this internal function to avoid it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 5557e772..4324259a 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3122,7 +3122,8 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; unsigned long long size; - char *cmdlines; + char *cmdlines = NULL; + int ret; if (CHECK_READ_STATE(handle, TRACECMD_FILE_CMD_LINES)) return 0; @@ -3131,15 +3132,20 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) add_section(handle, TRACECMD_OPTION_CMDLINES, 0, 0, lseek64(handle->fd, 0, SEEK_CUR)); - if (read_data_and_size(handle, &cmdlines, &size) < 0) - return -1; + ret = read_data_and_size(handle, &cmdlines, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_CMD_LINES; + goto out; + } cmdlines[size] = 0; tep_parse_saved_cmdlines(pevent, cmdlines); - free(cmdlines); - handle->file_state = TRACECMD_FILE_CMD_LINES; - - return 0; + ret = 0; +out: + free(cmdlines); + return ret; } static void extract_trace_clock(struct tracecmd_input *handle, char *line) From patchwork Thu Jul 29 05:09:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24E3CC4338F for ; Thu, 29 Jul 2021 05:10:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0F76461052 for ; Thu, 29 Jul 2021 05:10:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233739AbhG2FKs (ORCPT ); Thu, 29 Jul 2021 01:10:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233782AbhG2FKs (ORCPT ); Thu, 29 Jul 2021 01:10:48 -0400 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E59E2C061757 for ; Wed, 28 Jul 2021 22:10:44 -0700 (PDT) Received: by mail-ed1-x52f.google.com with SMTP id y7so4162475eda.5 for ; Wed, 28 Jul 2021 22:10:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Qk8+Go0Ovg7RA90EQ0E+q9ajl2qchmgE4K3YKj1ocOg=; b=JEz3PCh5NmRuT7q9jHgvnBF0GH6ibp0XTMG4WpBQI0tzEAz9ySCBlryOd3x6JZsEgi BPE7ULZ1OB51HZqlQ5XBGsgdweYV0gpa3lFg1vDG0gHmgiK3qtKM2qAOecl+Y5xvcy9t ILoMnjCF/WgAasAwPmtGJo+QgDnZK0kIpDLyATSk2DsVFnm6oJ1G7Z4I252qhWnH8fdR Jh+iiZyLDLtrYbmTNdyoHDuIfOS5AG/N3f+UYdLMlYBuVeokxHRgqtV6A+I/09DKiSce mxX/tIOBzpOj7LfAdeXGw7q/98EbvFLWl+5Jvb2/tZ8/th3OyJRuuMWjL5tQ8kCJZerI 33Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Qk8+Go0Ovg7RA90EQ0E+q9ajl2qchmgE4K3YKj1ocOg=; b=DFG2ccAd6Ab88c8k8GZ943B6hJKZAHDOdxrae12WpAheGLvdRdNbgjPHR2fECAlfjQ iD0r3gTUmL8Wnp1h4eoBKFDmv1o9+Ha18V65Pyr/55ZUL/KRBEX0ED8pykfBHqfEXwto rIm402JqxQYtrQKp1fSoIA9/kGz97b5pTTCf+Qt2mVYm6Eu/ej8KrXS8DVigepq7b4bV nQYG/sBHNNlkaeJfEr6L0OmBEHYHgCZmBDQsaorGgWlDTgB9XjuNhQREujMuYRy0SUwi nO2hwY79+d/5/FE8zg4ZkJdhsmsGU7y0k6Tnrhc4Tt2W7Ys3GZt1mxkwrc1Q49TlvI4G O6cQ== X-Gm-Message-State: AOAM533R1dm9UoUsGQ2Yz1tHH9ond3lXAYdM2HIr/rcU+mAeLZw4XYrX Inq9sfs0lHT0gqvZt4Hl8r8= X-Google-Smtp-Source: ABdhPJzzyWc8W7lS/NIoLiA8H/TWoB+4pJVG+ci3iVyITgq3VqQvQAVC1b+ii7D30+2N3LxDMGQglg== X-Received: by 2002:a05:6402:1118:: with SMTP id u24mr3925861edv.229.1627535443613; Wed, 28 Jul 2021 22:10:43 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:43 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 43/87] trace-cmd library: Track maximum CPUs count in input handler Date: Thu, 29 Jul 2021 08:09:15 +0300 Message-Id: <20210729050959.12263-44-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, only CPUs with trace data are stored in the file. Each instance may have its own CPU count, depending on collected traces. As the main input handler is used by the top trace instance, the CPU count there is for the top trace instance and may differ with cpu counts of the other instances. Added a new "max_cpu" member of the input handler, that tracks the maximum CPU count of all instances, recorded in the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 4324259a..41f95874 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -133,6 +133,7 @@ struct tracecmd_input { int long_size; int page_size; int page_map_size; + int max_cpu; int cpus; int ref; int nr_buffers; /* buffer instances */ @@ -952,6 +953,7 @@ static int read_cpus(struct tracecmd_input *handle) return -1; handle->cpus = cpus; + handle->max_cpu = cpus; tep_set_cpus(handle->pevent, handle->cpus); handle->file_state = TRACECMD_FILE_CPU_COUNT; @@ -2907,6 +2909,8 @@ static int handle_options(struct tracecmd_input *handle) case TRACECMD_OPTION_CPUCOUNT: cpus = *(int *)buf; handle->cpus = tep_read_number(handle->pevent, &cpus, 4); + handle->max_cpu = handle->cpus; + tep_set_cpus(handle->pevent, handle->cpus); break; case TRACECMD_OPTION_PROCMAPS: if (buf[size-1] == '\0') @@ -4215,7 +4219,7 @@ int tracecmd_page_size(struct tracecmd_input *handle) */ int tracecmd_cpus(struct tracecmd_input *handle) { - return handle->cpus; + return handle->max_cpu; } /** From patchwork Thu Jul 29 05:09:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60DF6C4320A for ; Thu, 29 Jul 2021 05:10:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4B4E761054 for ; Thu, 29 Jul 2021 05:10:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233782AbhG2FKt (ORCPT ); Thu, 29 Jul 2021 01:10:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234016AbhG2FKs (ORCPT ); Thu, 29 Jul 2021 01:10:48 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C84FAC061765 for ; Wed, 28 Jul 2021 22:10:45 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id n2so6357309eda.10 for ; Wed, 28 Jul 2021 22:10:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=W9+QRCCf8HDtczTrLkNWVkYGcyxFDhPU60GJbVA7mRA=; b=XkgRW1QTPuF0bnzLZ2vkwMYZzxGg755PFmuS40QI396mYQ+Qo4Tn31PfStvXMGhIj6 vYcqsAn3OqiWtUemiNZewyDHF46mYYnmQwBcKnyeuggtGTk1OkO3+8IoU8wp1MjY7C4a No1cmoh982zrZpo6es2pgEM5ZaiGmxjjqtxI/iaa+3txrCXNAHVxls3yDh5rmcmvIJG2 nqwYprqSjTzHWRh6G/vx/TtyqYwuQy9IveoF5bKEma7KMZKJt2hu2vIxhLJkXlCpzpm3 Y7XToSXuoIRjbjUPv9G74ysQnXvTMdSV6WSuws1LWKnZ3l05orslR/8jiUq+ONp5281O FoeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W9+QRCCf8HDtczTrLkNWVkYGcyxFDhPU60GJbVA7mRA=; b=YdslvQ8Ye5inzOPgxkKuTToxr1mWdMYF/eh3Z+n+z5s8pvcb25SeJ+m31ydc0a8mCv YaKtqZPQKWsghp2XCXuncYX9pWXri2VUkyL4/u3Hp/xlwwItH1voOCsdHXEqmrQvssl4 kbpNjohwweZGpcIiKIN+LZlxJgWI3iY2a3/RwigXeC87YCYkIBuL8K9MTGBy0KyBozhP LY3xciEJP3BN3rGi1YSqRu0QphVJfSK3jbmv82Wr/r7SwOgODKqp1Tn1lehtMJxhfNVR mPbRvkEiZXQcaGR76iD0u3WSKqgqXZpFrbBB6ym/qRKXzWR6JYE9OR7x3J/loKgGONd9 XO4A== X-Gm-Message-State: AOAM530EvE57FwPv9Th4OV0nBBg/Zqrzr46yqkpaCG2h1b9cQ0gtRWt0 BfruoD1wdwK4fkv+QlMhW2Eff0alqkGiOA== X-Google-Smtp-Source: ABdhPJyY5rIpnSzAaDG5oRs4gTsMqwzEkhSFN7XOmrVw9mMXGgllShrTVI0GLr/BqF9Sej9dSglicA== X-Received: by 2002:a50:fe10:: with SMTP id f16mr3860531edt.85.1627535444497; Wed, 28 Jul 2021 22:10:44 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:44 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 44/87] trace-cmd library: Set input handler default values in allocation function Date: Thu, 29 Jul 2021 08:09:16 +0300 Message-Id: <20210729050959.12263-45-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Set usecs flag by default when the input handler is allocated, it makes more sense than setting it when options are handeled. This clean up is needed for parsing version 7 trace files, where multiple options sections may exist. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 41f95874..23eea0d1 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2794,8 +2794,6 @@ static int handle_options(struct tracecmd_input *handle) int cpus; int ret; - /* By default, use usecs, unless told otherwise */ - handle->flags |= TRACECMD_FL_IN_USECS; handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); for (;;) { @@ -3448,6 +3446,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->fd = fd; handle->ref = 1; + /* By default, use usecs, unless told otherwise */ + handle->flags |= TRACECMD_FL_IN_USECS; + if (do_read_check(handle, buf, 3)) goto failed_read; From patchwork Thu Jul 29 05:09:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB628C432BE for ; Thu, 29 Jul 2021 05:10:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C78B561053 for ; Thu, 29 Jul 2021 05:10:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234020AbhG2FKt (ORCPT ); Thu, 29 Jul 2021 01:10:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234016AbhG2FKt (ORCPT ); Thu, 29 Jul 2021 01:10:49 -0400 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6D9EC061757 for ; Wed, 28 Jul 2021 22:10:46 -0700 (PDT) Received: by mail-ed1-x52d.google.com with SMTP id ec13so5881936edb.0 for ; Wed, 28 Jul 2021 22:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lUO7znpaLChFa2JfkPEHuA13AggMSi78OC3cEBdVLzY=; b=QM1IXU3e46hL0Hw4L4Ay8qouKSJ9rR2y5VbGy1ub96BoGHRfC6qlzNFOK/Z6iC3RJz vp7V46DQKlFhgBC8qbbDBu6ZBAtC094zRKyaguTdVNy/Ji/3Q/t47K26nXA3c5DrZTas uFMsBnYiTE9lhdkos3SCWWb18qTzimOJC2qWcb2Id3QbqUsecrYUDn1shSevSg5Icr6N D0qC3M5B1Uh/83jHofa36h5ebqX4dypX2NKak/oLbFEuc8VTNg0Ma+tfp1wv8Yqwr4ub 61HAou6Pcwq3mku+iH+NXvyeQG1vdEM4a+j7t1flvtCEJQc0U2chhW0jGu+Se8lC3pUV xmoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lUO7znpaLChFa2JfkPEHuA13AggMSi78OC3cEBdVLzY=; b=VxHvOeg+2WmgzH5fdE0RM/6rZFgdHF78X7lyuEEeDjSa6QhhwtdVN1Qk2SEEK7o1Od TV8hSgo+fLdg/eR8sKFX8tW6DpAO3tZrfxFLpOeUho58CO8mfUUqZ4BJYEwjfsU9FMIq kvIjDdL5ga+vSzsed1Dmmuc1IlxozIzXZr3ayVPnR1RajwbD1brsvoPfJx0ljKQEjRne fTX05Q/nvHwt3ehLXUiETDfsqWhNhvEAlrTSWA6nyl1f1XDucTLjTm1kSZqlpdtB9jTZ O2ns+krfuCVcSgIs4VVmJCcvEcU3GKuDyt9vhOhMddz12x6sOXA05JasHG/EVAN90W3N Cw+g== X-Gm-Message-State: AOAM532oVDEGxB4cODuLjgFQebx7AWLGfubtH4yGZK9T2TYzHVmotpR2 gO9W+uZdQK7QDufGWoEQuWQ= X-Google-Smtp-Source: ABdhPJw5b3rgPciDLXnHpEMiz+Zy2KBFe4FuhXJ2NfFpyqAYhi6D+xqYa1y1X2fAD8Cip1fyJKihPw== X-Received: by 2002:aa7:de92:: with SMTP id j18mr3898293edv.141.1627535445386; Wed, 28 Jul 2021 22:10:45 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:44 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 45/87] trace-cmd library: Read headers from trace file version 7 Date: Thu, 29 Jul 2021 08:09:17 +0300 Message-Id: <20210729050959.12263-46-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 has more flexible structure than version 6, headers are not located at fixed position in the file. A new logic is implemented to read and parse the headers from this new format. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 150 ++++++++++++++++++++++++++++++++---- 1 file changed, 136 insertions(+), 14 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 23eea0d1..f447c8d7 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -164,7 +164,8 @@ struct tracecmd_input { struct pid_addr_maps *pid_maps; /* file information */ struct file_section *sections; - size_t options_start; + bool options_init; + unsigned long long options_start; size_t total_file_size; /* For custom profilers. */ @@ -960,19 +961,7 @@ static int read_cpus(struct tracecmd_input *handle) return 0; } -/** - * tracecmd_read_headers - read the header information from trace.dat - * @handle: input handle for the trace.dat file - * @state: The state to read up to or zero to read up to options. - * - * This reads the trace.dat file for various information. Like the - * format of the ring buffer, event formats, ftrace formats, kallsyms - * and printk. This may be called multiple times with different @state - * values, to read partial data at a time. It will always continue - * where it left off. - */ -int tracecmd_read_headers(struct tracecmd_input *handle, - enum tracecmd_file_states state) +static int read_headers_v6(struct tracecmd_input *handle, enum tracecmd_file_states state) { int ret; @@ -1038,6 +1027,139 @@ int tracecmd_read_headers(struct tracecmd_input *handle, return 0; } +static int handle_options(struct tracecmd_input *handle); + +static int read_section_header(struct tracecmd_input *handle, unsigned short *id, + unsigned short *flags, unsigned int *size, char **description) +{ + unsigned short fl; + char *desc = NULL; + unsigned short sec_id; + unsigned int sz; + + if (read2(handle, &sec_id)) + return -1; + desc = read_string(handle); + if (!desc) + return -1; + if (read2(handle, &fl)) + goto error; + if (read4(handle, &sz)) + goto error; + + if (id) + *id = sec_id; + if (flags) + *flags = fl; + if (size) + *size = sz; + if (description) + *description = desc; + else + free(desc); + return 0; + +error: + free(desc); + return -1; +} + +static int handle_section(struct tracecmd_input *handle, struct file_section *section) +{ + unsigned short id, flags; + unsigned int size; + int ret; + + if (lseek64(handle->fd, section->section_offset, SEEK_SET) == (off_t)-1) + return -1; + if (read_section_header(handle, &id, &flags, &size, NULL)) + return -1; + section->flags = flags; + if (id != section->id) + return -1; + + section->data_offset = lseek64(handle->fd, 0, SEEK_CUR); + if ((section->flags & TRACECMD_SEC_FL_COMPRESS) && in_uncompress_block(handle)) + return -1; + + switch (section->id) { + case TRACECMD_OPTION_HEADER_INFO: + ret = read_header_files(handle); + break; + case TRACECMD_OPTION_FTRACE_EVENTS: + ret = read_ftrace_files(handle, NULL); + break; + case TRACECMD_OPTION_EVENT_FORMATS: + ret = read_event_files(handle, NULL); + break; + case TRACECMD_OPTION_KALLSYMS: + ret = read_proc_kallsyms(handle); + break; + case TRACECMD_OPTION_PRINTK: + ret = read_ftrace_printk(handle); + break; + case TRACECMD_OPTION_CMDLINES: + ret = read_and_parse_cmdlines(handle); + break; + default: + ret = 0; + break; + } + + if (section->flags & TRACECMD_SEC_FL_COMPRESS) + in_uncompress_reset(handle); + + return ret; +} + +static int read_headers_v7(struct tracecmd_input *handle) +{ + struct file_section *section; + + if (handle->options_init) + return 0; + + if (!handle->options_start) + return -1; + + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off64_t)-1) { + tracecmd_warning("Filed to goto options offset %lld", handle->options_start); + return -1; + } + + if (handle_options(handle)) + return -1; + + section = handle->sections; + while (section) { + if (handle_section(handle, section)) + return -1; + section = section->next; + } + + handle->options_init = true; + return 0; +} + +/** + * tracecmd_read_headers - read the header information from trace.dat + * @handle: input handle for the trace.dat file + * @state: The state to read up to or zero to read up to options. + * + * This reads the trace.dat file for various information. Like the + * format of the ring buffer, event formats, ftrace formats, kallsyms + * and printk. This may be called multiple times with different @state + * values, to read partial data at a time. It will always continue + * where it left off. + */ +int tracecmd_read_headers(struct tracecmd_input *handle, + enum tracecmd_file_states state) +{ + if (handle->file_version < 7) + return read_headers_v6(handle, state); + return read_headers_v7(handle); +} + static unsigned long long calc_page_offset(struct tracecmd_input *handle, unsigned long long offset) { From patchwork Thu Jul 29 05:09:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC2A9C4338F for ; Thu, 29 Jul 2021 05:10:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9336261019 for ; Thu, 29 Jul 2021 05:10:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234024AbhG2FKv (ORCPT ); Thu, 29 Jul 2021 01:10:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234016AbhG2FKv (ORCPT ); Thu, 29 Jul 2021 01:10:51 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4E01C061757 for ; Wed, 28 Jul 2021 22:10:47 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id p21so6365748edi.9 for ; Wed, 28 Jul 2021 22:10:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gGjxlL/QYZSQ9mqXy+WWO0qfdySoTW4BFaiNTXbOwKM=; b=gXvnSgJbm5E8zBV57fbLo85cExu+ZZkLRVhikmMIY6h22wTm48oJIkDtYhgw6jlLB/ EukvU0pwyTn01Wae02mrtCsz4wskmTrqv6pBHk3iA+5CI0MXuCNOWkcfEYq/WqPPr3MP JiTgZR1mg/yef4vP5U3pWS9adE/aRSjC2m3X54rjf2XIJ3FFwlPJRDeavIIDcESiOaSt I3AqeAhXyL0HouxIoBLga8DkqsO5nmysGR4h9b/3rBH+Murp/Cf1fhj0NJTkBZi1OUi4 M6/rETKbC2CJtYtS7ndUaHMdeOnTvzcdzL6ZSepMgF+EJ6zcFfs1u3jHtXiuRYpFvnci 5/DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gGjxlL/QYZSQ9mqXy+WWO0qfdySoTW4BFaiNTXbOwKM=; b=rXnwg6sG3pBWeoeAZsansWZYRptNlvHYO1S7RsjfiB0Yfny7a4qbCe55Dfz3Q3DsW/ Ox3y84cmBz2apobHThhOdDzlQYNV1dCQwf8jVfXA/cUUY+DA48i8tijbqaHfSfgE9gDn 0l3B3D1EhYNi1OjN47qPlBgBx3p6RVZJU3GNbtG2e8H5spuuKE96qWs+GPcZms71lIC+ MJfdGmBqXD2m6oq/WfFlPoaL3gnpQWV65BcBQ3X/jRLRoCVuyW9EUilW3sjfbRwnteXK 9kXIoBRuMmVzTUXMWtzbUso7lbknYpYZ3tUKPfncmhQBF16AXCp8Qjq8dbVTA8QHQzFn Y6rg== X-Gm-Message-State: AOAM5314mdgEDuZitIp04/aL82ohVS/8xVjDS/n2CfiQM+ic3fcigZwy 27CrE8KVge06VHDvz6saBHw= X-Google-Smtp-Source: ABdhPJxoGKTVxSAqm6/hUYXflWZZclSsCal22g3k5x/UT82Rog1a++QO04T17EwNjWR1a1bQf/KohA== X-Received: by 2002:a05:6402:1601:: with SMTP id f1mr3889210edv.388.1627535446489; Wed, 28 Jul 2021 22:10:46 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:45 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 46/87] tarce-cmd library: Do not use local variables when reading CPU stat option Date: Thu, 29 Jul 2021 08:09:18 +0300 Message-Id: <20210729050959.12263-47-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 can have more than one options section. Assuming that all CPUSTAT options are in a single options section could be wrong, that's why using local variable to track the CPUSTAT size is not correct. Use input handler context instead. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f447c8d7..d8c8b9b0 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -150,6 +150,7 @@ struct tracecmd_input { struct host_trace_info host; double ts2secs; + unsigned int cpustats_size; char * cpustats; char * uname; char * version; @@ -2909,7 +2910,6 @@ static int handle_options(struct tracecmd_input *handle) unsigned short option; unsigned int size; char *cpustats = NULL; - unsigned int cpustats_size = 0; struct input_buffer_instance *buffer; struct hook_list *hook; char *buf; @@ -2987,12 +2987,16 @@ static int handle_options(struct tracecmd_input *handle) break; case TRACECMD_OPTION_CPUSTAT: buf[size-1] = '\n'; - cpustats = realloc(cpustats, cpustats_size + size + 1); - if (!cpustats) - return -ENOMEM; - memcpy(cpustats + cpustats_size, buf, size); - cpustats_size += size; - cpustats[cpustats_size] = 0; + cpustats = realloc(handle->cpustats, + handle->cpustats_size + size + 1); + if (!cpustats) { + ret = -ENOMEM; + return ret; + } + memcpy(cpustats + handle->cpustats_size, buf, size); + handle->cpustats_size += size; + cpustats[handle->cpustats_size] = 0; + handle->cpustats = cpustats; break; case TRACECMD_OPTION_BUFFER: /* A buffer instance is saved at the end of the file */ From patchwork Thu Jul 29 05:09:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2CD1C4320E for ; Thu, 29 Jul 2021 05:10:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D012861052 for ; Thu, 29 Jul 2021 05:10:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234016AbhG2FKv (ORCPT ); Thu, 29 Jul 2021 01:10:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234023AbhG2FKv (ORCPT ); Thu, 29 Jul 2021 01:10:51 -0400 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3D37C061765 for ; Wed, 28 Jul 2021 22:10:48 -0700 (PDT) Received: by mail-ed1-x52a.google.com with SMTP id n2so6357423eda.10 for ; Wed, 28 Jul 2021 22:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CPlQbi1EQ/fkB8hu9TuApFd/+kavhl61vaYmxME6n1Q=; b=AWjlneMDy4aQ3wrAw9VmqVikkys+KpelFh8I4YQyQO3xDvtZs90hDrPqZvSZvxPWlQ ivH3RoYv7h1eBjrXSQUtad0Yx32SIdL39v7B+38mfkOoKfpJF+a7cCWxq9KJs91JA+U6 k7pi+Nevj9IvD61+N6Mp0hsqzJ76p5BaO4i5F4zJlvGM2bO72sbraBbsIQ21AJACt4Hf WdHu1FU6lCM6nU6eCqgU0I9ozyGiXaVDXgx02WwejyQ+27QJUcYJds7nF8KXCicEJLh7 CdRpCjA0FcN+HuRh+JQsik+BiKDDi8FwrY+quZev7ZUps7t2cEAzuWcyBVmisxvmLayn leUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CPlQbi1EQ/fkB8hu9TuApFd/+kavhl61vaYmxME6n1Q=; b=cF4FYI9axKv5l9nDYti/mU9QdJgUhiCB3IJVezRqQOAb84YhcRt01HbcJpyl24UpCg gU+gS9ct0bsdeoqzt9oSRDtSjw6xym3M5tV76o0kNfDvlwupGAEAU9YwHVS8vcOIqpi8 SoT6YiHkJoqYfgLTJLGn10WNaSAu1bKME4qFPuSth5ZIACs+K7SQDUzWV+686sOG6rA5 enGvR7sP4sMx4owbdNvOANH3WmyDdKZj2RdC/w3J2KbYDPZ+wk5dUmKeyF1oSnQIfUzy GVBjAM7j+GjZGvwS5PdUp3sqCyh94Oace2g6f0dLpeJXBb62rWkmkxIDLhObvjq/dcWN i9aw== X-Gm-Message-State: AOAM530jBu5k5P2lzNcM8e4x+qDldX2exPHRC1lPhNMY4e2XGbT7nNjx /GSQW8psTqKjcRZUuf2WCN4= X-Google-Smtp-Source: ABdhPJxfiHIL2AeHM4OcuogSJn4cZcB9DNbLFEVnI1Q5AxupSHaYofqdfIQCFYZEAKjbmjEfGR1j/w== X-Received: by 2002:a05:6402:7da:: with SMTP id u26mr2410812edy.270.1627535447400; Wed, 28 Jul 2021 22:10:47 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:47 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 47/87] trace-cmd library: Read handle header and compression of the option section Date: Thu, 29 Jul 2021 08:09:19 +0300 Message-Id: <20210729050959.12263-48-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, options section has a section header and possible compression of the data. Extend the options handling logic with this new format. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 50 ++++++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index d8c8b9b0..e7f97561 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2909,30 +2909,47 @@ static int handle_options(struct tracecmd_input *handle) long long offset; unsigned short option; unsigned int size; + unsigned short id, flags; char *cpustats = NULL; struct input_buffer_instance *buffer; struct hook_list *hook; + bool comperss = false; char *buf; int cpus; int ret; - handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); - - for (;;) { - if (read2(handle, &option)) + if (handle->file_version < 7) { + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + } else { + if (read_section_header(handle, &id, &flags, NULL, NULL)) return -1; + if (id != TRACECMD_OPTION_DONE) + return -1; + if (flags & TRACECMD_SEC_FL_COMPRESS) + comperss = true; + } - if (option == TRACECMD_OPTION_DONE) + if (comperss && in_uncompress_block(handle)) + return -1; + for (;;) { + ret = read2(handle, &option); + if (ret) + goto out; + if (handle->file_version < 7 && option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (read4(handle, &size)) - return -1; + ret = read4(handle, &size); + if (ret) + goto out; buf = malloc(size); - if (!buf) - return -ENOMEM; - if (do_read_check(handle, buf, size)) - return -1; + if (!buf) { + ret = -ENOMEM; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret) + goto out; switch (option) { case TRACECMD_OPTION_DATE: @@ -2982,7 +2999,7 @@ static int handle_options(struct tracecmd_input *handle) buf + 8, 4); ret = tsync_cpu_offsets_load(handle, buf + 12, size - 12); if (ret < 0) - return ret; + goto out; tracecmd_enable_tsync(handle, true); break; case TRACECMD_OPTION_CPUSTAT: @@ -2991,7 +3008,7 @@ static int handle_options(struct tracecmd_input *handle) handle->cpustats_size + size + 1); if (!cpustats) { ret = -ENOMEM; - return ret; + goto out; } memcpy(cpustats + handle->cpustats_size, buf, size); handle->cpustats_size += size; @@ -3078,9 +3095,12 @@ static int handle_options(struct tracecmd_input *handle) } - handle->cpustats = cpustats; + ret = 0; - return 0; +out: + if (comperss) + in_uncompress_reset(handle); + return ret; } static int read_options_type(struct tracecmd_input *handle) From patchwork Thu Jul 29 05:09:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 374D7C4338F for ; Thu, 29 Jul 2021 05:10:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2261D61053 for ; Thu, 29 Jul 2021 05:10:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234023AbhG2FKx (ORCPT ); Thu, 29 Jul 2021 01:10:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233900AbhG2FKx (ORCPT ); Thu, 29 Jul 2021 01:10:53 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2EF0C061757 for ; Wed, 28 Jul 2021 22:10:49 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id e19so8425788ejs.9 for ; Wed, 28 Jul 2021 22:10:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kABucYEO7dOvb7r+JsA5gtzuFjCQLI9a7krKmnsAwxs=; b=PgRjH3ykVorUaa8pZutryKbRE4KY0HiF5QX4ZeVLTiyq7blj7j6tnBNOeiOqxa47+S 6kearyBPxAFR3L34WFd9VuZCNJ3quWAbOa4Zc4N2xCWERTZHNDXU7dFoxqoyLf7rY5vz lPKLuLd5g6bzMaEwv3+dfWZ/uzL5K6vhDE6B4B1D/NryMyR35Ht9QBZ2M9fDhBoK+O+J g2LKNiJlVHmSrXqDRA6DZRsFJrzyV0H7WjCT+xQ0aH+UrEMP7Luo665ictQ7mND27Oxz PYw/FQ5nqUuQrWm7gX9sjKQ4fhuB8MVIjCL9opyjJtB/IiUlkwoHFiz3P02DTsD0WRBD wvDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kABucYEO7dOvb7r+JsA5gtzuFjCQLI9a7krKmnsAwxs=; b=mRG2NcK9fyCPoFeinfYrtW7bI9crumBaK92+ESn3NkTlw4tleFgD/45N5AIo2qghmH keVC0ix+SSkKxH8z8Cne+CJGx0V0GAdB0LgjpYyBeS7YYssNXqFmVqWhbOoPMS9ZdVXN 4n6oZFb118m+ZAXgeQttoduDkr695fRY80a6ck8J1fpPt/iw6fmpHoF/GR8UgXxXq6d5 wOlWrA6eciUVrKe0QcHG3r8tRkWOP4Mcu4JZaSM6qIHzTgE4UrLsdbuRDMKIG1mTtybf uXe4GCa3+TkBE2GW0rBIV27bKczT1xHxOPh2gI1GMKSqJk1TnGiXDrsAu4o2gGrBuc1g wqvA== X-Gm-Message-State: AOAM5329iPV/n71pd94lJE0Z5+SC2rTJx8EA2EpFqqf1AUF+1g+unyO0 lInHMk+T1QtcvTvAH5JTwXU= X-Google-Smtp-Source: ABdhPJxAjsCqBwl6ze96wYPjk4UYGeDDsdTqoBeLZlOFElwWV7WupP1vyiOFT9T3A31TdnXfZTmTAg== X-Received: by 2002:a17:906:1416:: with SMTP id p22mr2848040ejc.364.1627535448283; Wed, 28 Jul 2021 22:10:48 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:47 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 48/87] trace-cmd library: Read extended BUFFER option Date: Thu, 29 Jul 2021 08:09:20 +0300 Message-Id: <20210729050959.12263-49-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The BUFFER option is extended in trace file version 7. It holds CPU metadata related to the recorded CPU traces. Also, there is a new BUFFER_LAT option for describing the latency trace data. A new logic is implemented for these new options. In trace file version 7, the top buffer is saved as other buffers in the file, no special treatment. But saving the top buffer in the list of buffers in the input handler causes problems. It breaks the legacy logic of trace-cmd library users, which have special logic for trace buffers processing. That's why "top_buffer" member is added in the input handler structure, to hold the top buffer. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 116 ++++++++++++++++++++++++++++++------ 1 file changed, 98 insertions(+), 18 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e7f97561..57ae535a 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -74,9 +74,19 @@ struct cpu_data { int pipe_fd; }; +struct cpu_file_data { + int cpu; + unsigned long long offset; + unsigned long long size; +}; + struct input_buffer_instance { char *name; size_t offset; + char *clock; + bool latency; + int cpus; + struct cpu_file_data *cpu_data; }; struct ts_offset_sample { @@ -155,6 +165,7 @@ struct tracecmd_input { char * uname; char * version; char * trace_clock; + struct input_buffer_instance top_buffer; struct input_buffer_instance *buffers; int parsing_failures; struct guest_trace_info *guest; @@ -2904,6 +2915,80 @@ tracecmd_search_task_map(struct tracecmd_input *handle, return lib; } +#define save_read_number(R, C) \ + do { \ + if ((C) > size) \ + return -1; \ + (R) = tep_read_number(handle->pevent, (data + rsize), (C)); \ + rsize += (C); \ + size -= (C); \ + } while (0) + +#define save_read_string(R) \ + do { \ + if (size < 1) \ + return -1; \ + (R) = strdup(data + rsize); \ + if (!(R)) \ + return -1; \ + rsize += (strlen((R)) + 1); \ + size -= (strlen((R)) + 1); \ + if (size < 0) \ + return -1; \ + } while (0) + +static int handle_buffer_option(struct tracecmd_input *handle, + unsigned short id, char *data, int size) +{ + struct input_buffer_instance *buff; + unsigned long long offset; + int rsize = 0; + char *name; + int i; + + save_read_number(offset, 8); + save_read_string(name); + + if (*name == '\0') { + /* top buffer */ + buff = &handle->top_buffer; + } else { + buff = realloc(handle->buffers, sizeof(*handle->buffers) * (handle->nr_buffers + 1)); + if (!buff) { + free(name); + return -1; + } + handle->buffers = buff; + handle->nr_buffers++; + + buff = &handle->buffers[handle->nr_buffers - 1]; + } + memset(buff, 0, sizeof(struct input_buffer_instance)); + buff->name = name; + buff->offset = offset; + if (handle->file_version < 7) + return 0; + + /* file version >= 7 specific data */ + save_read_string(buff->clock); + if (id == TRACECMD_OPTION_BUFFER) { + save_read_number(buff->cpus, 4); + if (!buff->cpus) + return 0; + buff->cpu_data = calloc(buff->cpus, sizeof(struct cpu_file_data)); + if (!buff->cpu_data) + return -1; + for (i = 0; i < buff->cpus; i++) { + save_read_number(buff->cpu_data[i].cpu, 4); + save_read_number(buff->cpu_data[i].offset, 8); + save_read_number(buff->cpu_data[i].size, 8); + } + } else { + buff->latency = true; + } + return 0; +} + static int handle_options(struct tracecmd_input *handle) { long long offset; @@ -2911,7 +2996,6 @@ static int handle_options(struct tracecmd_input *handle) unsigned int size; unsigned short id, flags; char *cpustats = NULL; - struct input_buffer_instance *buffer; struct hook_list *hook; bool comperss = false; char *buf; @@ -3016,21 +3100,10 @@ static int handle_options(struct tracecmd_input *handle) handle->cpustats = cpustats; break; case TRACECMD_OPTION_BUFFER: - /* A buffer instance is saved at the end of the file */ - handle->nr_buffers++; - handle->buffers = realloc(handle->buffers, - sizeof(*handle->buffers) * handle->nr_buffers); - if (!handle->buffers) - return -ENOMEM; - buffer = &handle->buffers[handle->nr_buffers - 1]; - buffer->name = strdup(buf + 8); - if (!buffer->name) { - free(handle->buffers); - handle->buffers = NULL; - return -ENOMEM; - } - offset = *(unsigned long long *)buf; - buffer->offset = tep_read_number(handle->pevent, &offset, 8); + case TRACECMD_OPTION_BUFFER_LAT: + ret = handle_buffer_option(handle, option, buf, size); + if (ret < 0) + goto out; break; case TRACECMD_OPTION_TRACECLOCK: if (!handle->ts2secs) @@ -3825,9 +3898,15 @@ void tracecmd_close(struct tracecmd_input *handle) handle->sections = handle->sections->next; free(del_sec); } - for (i = 0; i < handle->nr_buffers; i++) - free(handle->buffers[i].name); + free(handle->top_buffer.name); + free(handle->top_buffer.clock); + free(handle->top_buffer.cpu_data); + for (i = 0; i < handle->nr_buffers; i++) { + free(handle->buffers[i].name); + free(handle->buffers[i].clock); + free(handle->buffers[i].cpu_data); + } free(handle->buffers); tracecmd_free_hooks(handle->hooks); @@ -4272,6 +4351,7 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) return NULL; *new_handle = *handle; + memset(&new_handle->top_buffer, 0, sizeof(new_handle->top_buffer)); new_handle->cpu_data = NULL; new_handle->nr_buffers = 0; new_handle->buffers = NULL; From patchwork Thu Jul 29 05:09:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D004C4320A for ; Thu, 29 Jul 2021 05:10:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 79A6360FED for ; Thu, 29 Jul 2021 05:10:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233893AbhG2FKy (ORCPT ); Thu, 29 Jul 2021 01:10:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234027AbhG2FKx (ORCPT ); Thu, 29 Jul 2021 01:10:53 -0400 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BC62C061765 for ; Wed, 28 Jul 2021 22:10:50 -0700 (PDT) Received: by mail-ed1-x52c.google.com with SMTP id p21so6365861edi.9 for ; Wed, 28 Jul 2021 22:10:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=T+XzZoHWC601U8xdKHAVzheaoUYwBuWFTJ3gxmL76WY=; b=rPxuDicPGTAw7JsuLOzWuEZYHo4RHnty/5f+BjMAxcHjFWU9tYygu8v///8HVe5oai 1o7sRORk1MLX8CI0TbdOMeLVZMxOoJcslR3y2ZdoNMmrwE5xn9mNJ+ajWzlQEkVcVfL3 b+vMR8Az/GX9Jw+6J0kxJnxtNY78HvC+qo9s8zu2WyPK6bLWpJgKvxsTdXUDQ1dPk7nl DDjhTgvi71yHHOLC7eFgU5mMyPe9A1jrZNnpHVjDar2Rq+DGOQljYoVhBSK77rElRrzu aM97/0IVV3S5Gt4Wv/skXcR0SFvK0TsL9GXecuVWIRVKqBEo9XHLyDkQmuDr42iYRsWa 8bJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T+XzZoHWC601U8xdKHAVzheaoUYwBuWFTJ3gxmL76WY=; b=rUy0C1GSTd2xL8RIa0xR2u75cInelJ8YZssXXdDCepvqe8jHEoitWv01+BFDroyvmh sXpzRFiTMakiN6lOFUYiEETan2W9xpmnLrYIIFb88fZ1TXF5kMqOxQ7iVglxoE78zyxH B5Qh88RLN2nJQ/lHGzWLCJTJXl/+jXh1dv/3Jt6SqkQWlYhR3Cp48Zdx+jPpmmuUJWHY alIdF4GWdYxNwaG5McEQndTyuxSg4C4Gw8vqXaQNfxilFVHSz9wTXi2Sk8FE7FnLDkrp mR0c8ZXJ31HJoIHZP0wKd8VSk0pRFX5t7JSClZ9E1c+66AbfylcNprsrpoQFj0oIyuwq pebA== X-Gm-Message-State: AOAM533jpyDqGwasu852ewF9kaYpxkhB4t4TVRCAy/3/RizQGiChHI+n MgApmDIYMSN93tKZsXZoiz8= X-Google-Smtp-Source: ABdhPJwDXwFu6FQCq69Y8jNKnVrqPt53PAtSyizYaOeT/IRPjk/LLPXLlymFDRJJHE9SbH57DtcuCw== X-Received: by 2002:aa7:c74b:: with SMTP id c11mr3964320eds.353.1627535449092; Wed, 28 Jul 2021 22:10:49 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:48 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 49/87] trace-cmd library: Handle the extended DONE option Date: Thu, 29 Jul 2021 08:09:21 +0300 Message-Id: <20210729050959.12263-50-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7 the DONE option is extended to store the offset in the file to the next options section. This way a list of options sections can be stored in the file. Added logic to recursively read all option sections from the file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 31 +++++++++++++++++++++++++ lib/trace-cmd/trace-output.c | 2 +- 3 files changed, 33 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index e54ba7e1..0291b8de 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -64,5 +64,6 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name); off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); +unsigned long long get_last_option_offset(struct tracecmd_input *handle); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 57ae535a..9ab20d64 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -178,6 +178,7 @@ struct tracecmd_input { struct file_section *sections; bool options_init; unsigned long long options_start; + unsigned long long options_last_offset; size_t total_file_size; /* For custom profilers. */ @@ -2915,6 +2916,30 @@ tracecmd_search_task_map(struct tracecmd_input *handle, return lib; } +__hidden unsigned long long get_last_option_offset(struct tracecmd_input *handle) +{ + return handle->options_last_offset; +} + +static int handle_option_done(struct tracecmd_input *handle, char *buf, int size) +{ + unsigned long long offset; + + if (size < 8) + return -1; + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (offset >= size) + handle->options_last_offset = offset - size; + offset = tep_read_number(handle->pevent, buf, 8); + if (!offset) + return 0; + + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + + return handle_options(handle); +} + #define save_read_number(R, C) \ do { \ if ((C) > size) \ @@ -3159,6 +3184,12 @@ static int handle_options(struct tracecmd_input *handle) break; add_section(handle, option, -1, tep_read_number(handle->pevent, buf, 8), 0); break; + case TRACECMD_OPTION_DONE: + if (comperss) + in_uncompress_reset(handle); + ret = handle_option_done(handle, buf, size); + free(buf); + return ret; default: tracecmd_warning("unknown option %d", option); break; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 87bbcf39..01c1a4f5 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2413,7 +2413,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); - handle->options_start = tracecmd_get_options_offset(ihandle); + handle->options_start = get_last_option_offset(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Thu Jul 29 05:09:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BD16C432BE for ; Thu, 29 Jul 2021 05:10:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 471AA61053 for ; Thu, 29 Jul 2021 05:10:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234027AbhG2FKz (ORCPT ); Thu, 29 Jul 2021 01:10:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233900AbhG2FKy (ORCPT ); Thu, 29 Jul 2021 01:10:54 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F1C8C061757 for ; Wed, 28 Jul 2021 22:10:51 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id nd39so8464955ejc.5 for ; Wed, 28 Jul 2021 22:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PFUx3VZDL9ywcEd2YoUZx8iPFOxnYwsl8veVsUh+7fw=; b=G9QJbtWjmcKJj2ayUXwpOP06+FKr7vjLGxXtakyXfuBWjoIBA/kulRZRJGEiF5SNHw rIllQ7X8+P6N6EIRTE6IoYvBNqfcyOSUjn7rxMUeLWODAE1fTO6sRkYVjWdV9OHDMUpc /9zUZfxZL68VKRuJdyVAbGXc6hb7fcFfS8AYXV810wKiMOgQHRtL/58hdnn0O/n9Ut10 ZAl/pyiN24gOfs5abPMpUGTRiN1FiPdhXqYX3xIg8cZwGkmnRYZoPp+GBEEtDFHEVQn5 wQoi7dVYvLsh9wXbdMQDbpXezI7tlZG9KbyTqAXx+/IOqlUENsnAaHHjXsl6KmW8ASpI WeWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PFUx3VZDL9ywcEd2YoUZx8iPFOxnYwsl8veVsUh+7fw=; b=W/BIM7naP34W3MAYjurSv5vf0qIQ3iLDaT+lExxLn9Eq5ezoI9gANTxpQvBp3yWXa0 xVXdKq3Xpgiv6RL9HZivh4vOWNWdXmp42MUyyDaBXhKKU8At9Zkgfh/DE0FBhJUO2nV1 nqKImIov0gqruqxec1JGUX8D09WSE4JXbz2sGdFQYlRVoBW4wM3w7Pso+97TtMbmIba9 EvPLVmg6un3pANNdNJZX0dA5Pe5E0mMXrtpRp5tA8A2bDS2qZCTNjBMpY0b1ZAnMXyA0 J+D7YQVHiV1T1f37j3OUEClkuC4vKBHtgSbV52RK7RIqzJQ9hROQ5NAGvPz2yhI9+PqL 0zhQ== X-Gm-Message-State: AOAM5300/whP3okA6y2zC8AOpRgPMO2mvbNS6pcX120eqghZJZPuFjT3 MuGZolVghh6EiOK+rgMcLME= X-Google-Smtp-Source: ABdhPJxqOoczRtXUNa33xVcPuWrDHM9i4Sa7GRsuJ0MJPTGxUQOfQX5Cs3kEeYAoFMo+Egu0RejbOQ== X-Received: by 2002:a17:906:8152:: with SMTP id z18mr2903267ejw.419.1627535450053; Wed, 28 Jul 2021 22:10:50 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:49 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 50/87] trace-cmd library: Read compression header Date: Thu, 29 Jul 2021 08:09:22 +0300 Message-Id: <20210729050959.12263-51-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 introduced new mandatory compression header, storing information about the compression algorithm used to compress the trace file. Added code to read that header and to initialize compression context according to it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 9ab20d64..520d611f 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3684,7 +3684,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) char test[] = TRACECMD_MAGIC; unsigned int page_size; size_t offset; - char *version; + char *version = NULL; + char *zver = NULL; + char *zname = NULL; char buf[BUFSIZ]; unsigned long ver; @@ -3723,6 +3725,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) } handle->file_version = ver; free(version); + version = NULL; if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3752,11 +3755,38 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); lseek64(handle->fd, offset, SEEK_SET); + if (handle->file_version >= 7) { + zname = read_string(handle); + if (!zname) + goto failed_read; + zver = read_string(handle); + if (!zver) + goto failed_read; + if (strcmp(zname, "none")) { + handle->compress = tracecmd_compress_alloc(zname, zver, + handle->fd, + handle->pevent, NULL); + if (!handle->compress) { + tracecmd_warning("Unsupported file compression %s %s", zname, zver); + goto failed_read; + } + } + if (read8(handle, &(handle->options_start))) { + tracecmd_warning("Filed to read the offset of the first option section"); + goto failed_read; + } + free(zname); + free(zver); + } + handle->file_state = TRACECMD_FILE_INIT; return handle; failed_read: + free(version); + free(zname); + free(zver); free(handle); return NULL; @@ -3952,7 +3982,8 @@ void tracecmd_close(struct tracecmd_input *handle) if (handle->flags & TRACECMD_FL_BUFFER_INSTANCE) tracecmd_close(handle->parent); else { - /* Only main handle frees plugins and pevent */ + /* Only main handle frees plugins, pevent and compression context */ + tracecmd_compress_destroy(handle->compress); tep_unload_plugins(handle->plugin_list, handle->pevent); tep_free(handle->pevent); } From patchwork Thu Jul 29 05:09:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AD83C4338F for ; Thu, 29 Jul 2021 05:10:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2517E60FED for ; Thu, 29 Jul 2021 05:10:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234029AbhG2FK4 (ORCPT ); Thu, 29 Jul 2021 01:10:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233900AbhG2FKz (ORCPT ); Thu, 29 Jul 2021 01:10:55 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F07BC061765 for ; Wed, 28 Jul 2021 22:10:52 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id x11so7517322ejj.8 for ; Wed, 28 Jul 2021 22:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GanyT2H7wUj69AtYUc0Y9IEXzrjhWZk0TaTxnSj5h6E=; b=RJGDZ0pc01AwqvmSMQ5FJRm1hspGgF8KeOU0TRWGpdiPARGJautURh03dFSTAGg4MI q5FBW82EEzbXjBdwdCHf2qAHilGYIGip0vNz0JKkgONJXYzf3tRScFvOJbhtfW7p4s3Y ite79SVKuV2qispI8bIa3+6dLtfWzadZ+xlCTwBqUE+vkd+aZLhuvRBGR2hX8h0ArraG 8Fj/OyknIjbugxCGLTxMlObTrrGU4xLn22lQpJTK36IFOv0ISaRAngBgnRmv+DuGqTmK 2d7xDF+iFvkMgxGest2vLw2PA9MvfbGf/gIECGOOCwURSAuktRcxm9NFuZyFmzbjFX01 mQOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GanyT2H7wUj69AtYUc0Y9IEXzrjhWZk0TaTxnSj5h6E=; b=n7yK2VzWBhlfkvx83Z6TF5SnvLNXII7iT30oQYJa2XT9EtGiEF1X/IwURBq0XNlkVt R/FN+N0EtwviQdumS3cT4+eqj5VbeM25z+nHz5fmeGt0oJH1x65PuDT7z0FR0ffnPkcI XMfnGFe2ibIORZcL4f0UPMeOjSIP3PERsDmoMGDbGiQcWK1J+jqB98BEkTD6RhsTyylj 3BitMVXDWLb04uxC3/VdkPL8rYnsngPw64MmlncZNSQii2zYGTlC4stEom5cIPNR/+SI 887KcDpEjKAZ7uVN12cle4DBZaL8zNN1jLCkp7HziXRJ1ZiGRnGZMqLPIIkz46Y6Gj1C 3SUg== X-Gm-Message-State: AOAM533OXjEXpxFF/sr8ycIMCgZDwKx/H4vjXhGUpIQP0mRuuyitSDJx Eb7js1hNcJEui/f579qqy7s= X-Google-Smtp-Source: ABdhPJwt4NIACjHS56fHrEXT/tpojy9vOIyZoszs0zv+EzE1f2Cn1jqCocmwjIn7beIs6ItMS8q6hw== X-Received: by 2002:a17:906:260e:: with SMTP id h14mr2856890ejc.258.1627535450883; Wed, 28 Jul 2021 22:10:50 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:50 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 51/87] trace-cmd library: Extend the input handler with trace data decompression context Date: Thu, 29 Jul 2021 08:09:23 +0300 Message-Id: <20210729050959.12263-52-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The CPU tarce data is compressed in chunks, as chunk's size is multiple trace pages. The input handler is extended with the necessary structures, to control the data decompression. There are two approaches for data decompression, both are supported and can be used in different use cases: - in-memory decompression, page by page. - using a temporary file Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 66 ++++++++++++++++++++++++++++++------- 1 file changed, 54 insertions(+), 12 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 520d611f..6fb63c0f 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -29,6 +29,9 @@ #define COMMIT_MASK ((1 << 27) - 1) +/* force uncompressing in memory */ +#define INMEMORY_DECOMPRESS + /* for debugging read instead of mmap */ static int force_read = 0; @@ -54,6 +57,24 @@ struct page { #endif }; +struct zchunk_cache { + struct list_head list; + struct tracecmd_compress_chunk *chunk; + void *map; + int ref; +}; + +struct cpu_zdata { + /* uncompressed cpu data */ + int fd; + char file[26]; /* strlen(COMPR_TEMP_FILE) */ + unsigned int count; + unsigned int last_chunk; + struct list_head cache; + struct tracecmd_compress_chunk *chunks; +}; + +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +93,7 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + struct cpu_zdata compress; }; struct cpu_file_data { @@ -150,6 +172,8 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + bool read_zpage; /* uncompress pages in memory, do not use tmp files */ + bool cpu_compressed; int file_version; struct cpu_data *cpu_data; long long ts_offset; @@ -3284,7 +3308,7 @@ static int read_cpu_data(struct tracecmd_input *handle) unsigned long long offset; handle->cpu_data[cpu].cpu = cpu; - + handle->cpu_data[cpu].compress.fd = -1; handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; @@ -3701,6 +3725,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; +#ifdef INMEMORY_DECOMPRESS + handle->read_zpage = 1; +#endif if (do_read_check(handle, buf, 3)) goto failed_read; @@ -3915,6 +3942,7 @@ void tracecmd_ref(struct tracecmd_input *handle) */ void tracecmd_close(struct tracecmd_input *handle) { + struct zchunk_cache *cache; struct file_section *del_sec; int i; @@ -3933,17 +3961,31 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, i); free_page(handle, i); - if (handle->cpu_data && handle->cpu_data[i].kbuf) { - kbuffer_free(handle->cpu_data[i].kbuf); - if (handle->cpu_data[i].page_map) - free_page_map(handle->cpu_data[i].page_map); - - if (handle->cpu_data[i].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[i].page_cnt, i, - show_records(handle->cpu_data[i].pages, - handle->cpu_data[i].nr_pages)); - free(handle->cpu_data[i].pages); + if (handle->cpu_data) { + if (handle->cpu_data[i].kbuf) { + kbuffer_free(handle->cpu_data[i].kbuf); + if (handle->cpu_data[i].page_map) + free_page_map(handle->cpu_data[i].page_map); + + if (handle->cpu_data[i].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[i].page_cnt, i, + show_records(handle->cpu_data[i].pages, + handle->cpu_data[i].nr_pages)); + free(handle->cpu_data[i].pages); + } + if (handle->cpu_data[i].compress.fd >= 0) { + close(handle->cpu_data[i].compress.fd); + unlink(handle->cpu_data[i].compress.file); + } + while (!list_empty(&handle->cpu_data[i].compress.cache)) { + cache = container_of(handle->cpu_data[i].compress.cache.next, + struct zchunk_cache, list); + list_del(&cache->list); + free(cache->map); + free(cache); + } + free(handle->cpu_data[i].compress.chunks); } } From patchwork Thu Jul 29 05:09:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74629C432BE for ; Thu, 29 Jul 2021 05:10:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 54BAF60041 for ; Thu, 29 Jul 2021 05:10:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234041AbhG2FK5 (ORCPT ); Thu, 29 Jul 2021 01:10:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233900AbhG2FK4 (ORCPT ); Thu, 29 Jul 2021 01:10:56 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A9DEC061757 for ; Wed, 28 Jul 2021 22:10:53 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id u12so6435546eds.2 for ; Wed, 28 Jul 2021 22:10:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bl0/I845lZrCmZcqjigMiovAvAuzlZ22UJnt/1lm4W8=; b=iCpj6zY1QRsjZ33BAJrfwEe94v6qLwqRHCcjmZTIhObZ6T1Y+tPyoYbs52ZSVg7twX +HueSFV1YcrwM8m/yQiG0zwj5BjWC/OWi/wOs1M9cPum4h65JZG17N3Xj8tgvtT+9kel plosxeSSJJDc/xyhJYYvvbtLKJF8aI+xBq1f3FQVANi8I1z1sHaQILPL7OaEFfL2VwVe DIvMLZTuSvXUrX4DF+GpielHLKPzfSsauGp7nW1N226MV+II/WwFwunW5Goj2UkqGWxi 8mcxrjOyO+8mgdsiioOvfA/XKR3aHhiCOBm4QWK13+xYrDMKdrh9btOAEBcaRuRCqa2O +k1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bl0/I845lZrCmZcqjigMiovAvAuzlZ22UJnt/1lm4W8=; b=isn8BLo1FpXy6ftTdBWo6O75lnj0z3gd2UNauC3WjhjUxtbLyzlxuGaxyvtwWeOaLe JCfi/6LgcFaxAO2jwHjHnya9HSB2s4P2RCHBnIpSHz5mF+hoLurry/o9A0fWWzdKgjvH yoGx1Op25gRX/1d9UJ74Iiz5Zg2SUZDqgfQ7Bs+pqNfq/EZwydyHSm5MaoYJzYjQW1g6 qHdZVRrefKrh+WPWJzrZlHnc9rfzAxxQeD4afrrw+bVrCxOTi01/SQ/5K6wOxQk+3t2w ZTYKsfkfq+QYzlcEuGz+OzNtdynjnPIdQIiamNsUO4GUZn1+KfTn2ElF3lRsp9n72/K/ WQHw== X-Gm-Message-State: AOAM532P/+eWVgH8J6PjHBKbHx2+FnNzXA9w0DB5Wct2vkpzR5PYS3q1 /ZvJXhh8FqWXlDuHq/5wM3I= X-Google-Smtp-Source: ABdhPJwWxHaRFKOzRCMR24q31Rvs6Dg02ZUfJriDwSZ7OtQaonfpQDIbDPE72B/BW4LpJxIHfRFdJQ== X-Received: by 2002:a05:6402:516f:: with SMTP id d15mr4091716ede.210.1627535451826; Wed, 28 Jul 2021 22:10:51 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:51 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 52/87] trace-cmd library: Initialize CPU data decompression logic Date: Thu, 29 Jul 2021 08:09:24 +0300 Message-Id: <20210729050959.12263-53-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org On CPU data initialization stage, initialize decompression context for both in-memory and temporary file decompression logics. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 70 +++++++++++++++++++++++++++++++++++-- 1 file changed, 67 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 6fb63c0f..f3783a98 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -1275,6 +1275,7 @@ static void *allocate_page_map(struct tracecmd_input *handle, off64_t map_offset; void *map; int ret; + int fd; if (handle->read_page) { map = malloc(handle->page_size); @@ -1314,12 +1315,15 @@ static void *allocate_page_map(struct tracecmd_input *handle, map_size -= map_offset + map_size - (cpu_data->file_offset + cpu_data->file_size); + if (cpu_data->compress.fd >= 0) + fd = cpu_data->compress.fd; + else + fd = handle->fd; again: page_map->size = map_size; page_map->offset = map_offset; - page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, - handle->fd, map_offset); + page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, fd, map_offset); if (page_map->map == MAP_FAILED) { /* Try a smaller map */ @@ -2511,16 +2515,76 @@ tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record) /* Not reached */ } -static int init_cpu(struct tracecmd_input *handle, int cpu) +static int init_cpu_zfile(struct tracecmd_input *handle, int cpu) +{ + struct cpu_data *cpu_data; + unsigned long long size; + off64_t offset; + + cpu_data = &handle->cpu_data[cpu]; + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + strcpy(cpu_data->compress.file, COMPR_TEMP_FILE); + cpu_data->compress.fd = mkstemp(cpu_data->compress.file); + if (cpu_data->compress.fd < 0) + return -1; + if (tracecmd_uncompress_copy_to(handle->compress, cpu_data->compress.fd, NULL, &size)) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + cpu_data->offset = 0; + cpu_data->file_offset = 0; + cpu_data->file_size = size; + cpu_data->size = size; + return 0; +} + +static int init_cpu_zpage(struct tracecmd_input *handle, int cpu) { struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + int count; int i; + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + + count = tracecmd_load_chunks_info(handle->compress, &cpu_data->compress.chunks); + if (count < 0) + return -1; + cpu_data->compress.count = count; + cpu_data->compress.last_chunk = 0; + + cpu_data->file_offset = 0; + cpu_data->file_size = 0; + for (i = 0; i < count; i++) + cpu_data->file_size += cpu_data->compress.chunks[i].size; cpu_data->offset = cpu_data->file_offset; cpu_data->size = cpu_data->file_size; + return 0; +} + +static int init_cpu(struct tracecmd_input *handle, int cpu) +{ + struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + int ret; + int i; + + if (handle->cpu_compressed && cpu_data->file_size > 0) { + if (handle->read_zpage) + ret = init_cpu_zpage(handle, cpu); + else + ret = init_cpu_zfile(handle, cpu); + if (ret) + return ret; + } else { + cpu_data->offset = cpu_data->file_offset; + cpu_data->size = cpu_data->file_size; + } cpu_data->timestamp = 0; list_head_init(&cpu_data->page_maps); + list_head_init(&cpu_data->compress.cache); if (!cpu_data->size) { printf("CPU %d is empty\n", cpu); From patchwork Thu Jul 29 05:09:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA19CC4320A for ; Thu, 29 Jul 2021 05:10:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B27C60041 for ; Thu, 29 Jul 2021 05:10:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233900AbhG2FK5 (ORCPT ); Thu, 29 Jul 2021 01:10:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234034AbhG2FK4 (ORCPT ); Thu, 29 Jul 2021 01:10:56 -0400 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29C0BC061765 for ; Wed, 28 Jul 2021 22:10:54 -0700 (PDT) Received: by mail-ed1-x52f.google.com with SMTP id r16so6379671edt.7 for ; Wed, 28 Jul 2021 22:10:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VtQrXLsNum/CuyZmfba2tUWrXV1kO8MN3kVMMvLSCRk=; b=FDfCIMCm3PPUPuIUp3F9oFQwl389NXZ2qzLzCqs68GdPWhXcmOMRY9D1szxbK9kNej A8+DO2pWtZsuOBi/gD4AO1Jjy63027JfQQEeRgOUkxttVCgsImKBP3mt3k+y0hL+CS73 pV2xFEFgFGQIAAe4CKwez7a4eosDrVLcWurPOOPkdTkbeltnI1Oen3ev+kNV74blPjEg D99mTYpFUb3LTbE5idDmcjDJBYZdBvjglfbvGIxw30SnCWcojQJKwpnbkKEVcUyDVse3 7fFuS+CxTM0T3pzuXMksHJ2hCsUXMQSRII/gE9cUwe87JiGnOHkOnBXDqKubI8jil9AT pxzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VtQrXLsNum/CuyZmfba2tUWrXV1kO8MN3kVMMvLSCRk=; b=NoMWxQdPz/lPD6pvO2Al1SW12EWQM/mXi8KNxYwHrMlIp/1GNAV5RcPgZWOFbvH7zY dheWAR3MPjr2n4qnhXxgq7T2lbbJKgD5tqiO0YkqJ6PDmsQMAhGbeibfQlkuO6MDsFJp 9Rh/tcrmjZ8mS/2wSB4dR7prMroKFJA+2wrPK9WY/37wqbpGyqtLPnPgRhIMvmZdwtM1 RIuXOWahUfVJCWCZd/t9UkhccklE1B9Br4cbBEhvPcq30eRChtyE53NHMSS3iwU6JH3M xO04/6rfdNDoq2R49tZsCSe/Tq9KAFnNMuNOUe0EiSzpD3YQyUBWyva8Q9JsAOC0HN37 eptw== X-Gm-Message-State: AOAM533DMPqXpkYYiLWaD1vY7peX4CdkCz30iz0P9oXYMO0UVu8n2mjm XuP8AJaxuQJ6YNkzRKk36AU= X-Google-Smtp-Source: ABdhPJwtybLZWyDjdRbBKockdXf2RAUGYGpGmKblJF7uyRi09lzJU3e4qPrbuzzy5t/z48kDftLIow== X-Received: by 2002:a05:6402:31e2:: with SMTP id dy2mr1135308edb.97.1627535452809; Wed, 28 Jul 2021 22:10:52 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:52 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 53/87] trace-cmd library: Initialize CPU data for reading from version 7 trace files Date: Thu, 29 Jul 2021 08:09:25 +0300 Message-Id: <20210729050959.12263-54-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In version 7 trace files, CPU trace data is written in slightly different way than in version 6 files. Added new CPU data initialization flow, to handle version 7 files: - the top trace instance is saved in the same way as the other trace instances. - per CPU trace metadata is stored in the buffer option. - trace data section has section header. - trace data can be compressed. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 178 +++++++++++++++++++++++++----------- 1 file changed, 124 insertions(+), 54 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f3783a98..3b58cb0a 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3327,34 +3327,18 @@ static int read_options_type(struct tracecmd_input *handle) return 0; } -static int read_cpu_data(struct tracecmd_input *handle) +static int init_cpu_data(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; enum kbuffer_long_size long_size; enum kbuffer_endian endian; - unsigned long long size; unsigned long long max_size = 0; unsigned long long pages; - int cpus; int cpu; - /* - * Check if this is a latency report or not. - */ - if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) - return 1; - /* We expect this to be flyrecord */ if (handle->file_state != TRACECMD_FILE_CPU_FLYRECORD) return -1; - cpus = handle->cpus; - - handle->cpu_data = malloc(sizeof(*handle->cpu_data) * handle->cpus); - if (!handle->cpu_data) - return -1; - memset(handle->cpu_data, 0, sizeof(*handle->cpu_data) * handle->cpus); - if (force_read) handle->read_page = true; @@ -3369,32 +3353,15 @@ static int read_cpu_data(struct tracecmd_input *handle) endian = KBUFFER_ENDIAN_LITTLE; for (cpu = 0; cpu < handle->cpus; cpu++) { - unsigned long long offset; - - handle->cpu_data[cpu].cpu = cpu; handle->cpu_data[cpu].compress.fd = -1; handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; - if (tep_is_old_format(pevent)) + if (tep_is_old_format(handle->pevent)) kbuffer_set_old_format(handle->cpu_data[cpu].kbuf); - read8(handle, &offset); - read8(handle, &size); - - handle->cpu_data[cpu].file_offset = offset; - handle->cpu_data[cpu].file_size = size; - if (size > max_size) - max_size = size; - - if (size && (offset + size > handle->total_file_size)) { - /* this happens if the file got truncated */ - printf("File possibly truncated. " - "Need at least %llu, but file size is %zu.\n", - offset + size, handle->total_file_size); - errno = EINVAL; - goto out_free; - } + if (handle->cpu_data[cpu].file_size > max_size) + max_size = handle->cpu_data[cpu].file_size; } /* Calculate about a meg of pages for buffering */ @@ -3412,6 +3379,102 @@ static int read_cpu_data(struct tracecmd_input *handle) goto out_free; } + return 0; + + out_free: + for ( ; cpu >= 0; cpu--) { + free_page(handle, cpu); + kbuffer_free(handle->cpu_data[cpu].kbuf); + handle->cpu_data[cpu].kbuf = NULL; + } + return -1; +} + +static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buffer_instance *buffer) +{ + unsigned long long offset; + unsigned long long size; + unsigned short id, flags; + int cpu; + + if (handle->cpu_data) + return -1; + + if (lseek64(handle->fd, buffer->offset, SEEK_SET) == (off_t)-1) + return -1; + if (read_section_header(handle, &id, &flags, NULL, NULL)) + return -1; + if (flags & TRACECMD_SEC_FL_COMPRESS) + handle->cpu_compressed = true; + handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; + handle->cpus = buffer->cpus; + if (handle->max_cpu < handle->cpus) + handle->max_cpu = handle->cpus; + + handle->cpu_data = calloc(handle->cpus, sizeof(*handle->cpu_data)); + if (!handle->cpu_data) + return -1; + + for (cpu = 0; cpu < handle->cpus; cpu++) { + handle->cpu_data[cpu].cpu = buffer->cpu_data[cpu].cpu; + offset = buffer->cpu_data[cpu].offset; + size = buffer->cpu_data[cpu].size; + handle->cpu_data[cpu].file_offset = offset; + handle->cpu_data[cpu].file_size = size; + if (size && (offset + size > handle->total_file_size)) { + /* this happens if the file got truncated */ + printf("File possibly truncated. " + "Need at least %llu, but file size is %zu.\n", + offset + size, handle->total_file_size); + errno = EINVAL; + return -1; + } + } + + return init_cpu_data(handle); +} + +static int read_cpu_data(struct tracecmd_input *handle) +{ + unsigned long long size; + int cpus; + int cpu; + + /* + * Check if this is a latency report or not. + */ + if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) + return 1; + + /* We expect this to be flyrecord */ + if (handle->file_state != TRACECMD_FILE_CPU_FLYRECORD) + return -1; + + cpus = handle->cpus; + + handle->cpu_data = malloc(sizeof(*handle->cpu_data) * handle->cpus); + if (!handle->cpu_data) + return -1; + memset(handle->cpu_data, 0, sizeof(*handle->cpu_data) * handle->cpus); + + for (cpu = 0; cpu < handle->cpus; cpu++) { + unsigned long long offset; + + handle->cpu_data[cpu].cpu = cpu; + read8(handle, &offset); + read8(handle, &size); + handle->cpu_data[cpu].file_offset = offset; + handle->cpu_data[cpu].file_size = size; + if (size && (offset + size > handle->total_file_size)) { + /* this happens if the file got truncated */ + printf("File possibly truncated. " + "Need at least %llu, but file size is %zu.\n", + offset + size, handle->total_file_size); + errno = EINVAL; + return -1; + } + } + /* * It is possible that an option changed the number of CPUs. * If that happened, then there's "empty" cpu data saved for @@ -3431,15 +3494,7 @@ static int read_cpu_data(struct tracecmd_input *handle) } } - return 0; - - out_free: - for ( ; cpu >= 0; cpu--) { - free_page(handle, cpu); - kbuffer_free(handle->cpu_data[cpu].kbuf); - handle->cpu_data[cpu].kbuf = NULL; - } - return -1; + return init_cpu_data(handle); } static int read_data_and_size(struct tracecmd_input *handle, @@ -3542,14 +3597,7 @@ static int read_and_parse_trace_clock(struct tracecmd_input *handle, return 0; } -/** - * tracecmd_init_data - prepare reading the data from trace.dat - * @handle: input handle for the trace.dat file - * - * This prepares reading the data from trace.dat. This is called - * after tracecmd_read_headers() and before tracecmd_read_data(). - */ -int tracecmd_init_data(struct tracecmd_input *handle) +static int init_data_v6(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; int ret; @@ -3571,7 +3619,29 @@ int tracecmd_init_data(struct tracecmd_input *handle) tracecmd_parse_trace_clock(handle, clock, 8); } } + return ret; +} + +static int init_data_v7(struct tracecmd_input *handle) +{ + return init_buffer_cpu_data(handle, &handle->top_buffer); +} + +/** + * tracecmd_init_data - prepare reading the data from trace.dat + * @handle: input handle for the trace.dat file + * + * This prepares reading the data from trace.dat. This is called + * after tracecmd_read_headers() and before tracecmd_read_data(). + */ +int tracecmd_init_data(struct tracecmd_input *handle) +{ + int ret; + if (handle->file_version < 7) + ret = init_data_v6(handle); + else + ret = init_data_v7(handle); tracecmd_blk_hack(handle); return ret; From patchwork Thu Jul 29 05:09:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2C9CC4338F for ; Thu, 29 Jul 2021 05:10:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD6496023E for ; Thu, 29 Jul 2021 05:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234058AbhG2FLB (ORCPT ); Thu, 29 Jul 2021 01:11:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234034AbhG2FK6 (ORCPT ); Thu, 29 Jul 2021 01:10:58 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D243C061757 for ; Wed, 28 Jul 2021 22:10:55 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id x11so7517464ejj.8 for ; Wed, 28 Jul 2021 22:10:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JLRedXoW1NUWdO6nkQxVEPJu6H/0wYkRPiho+e2/F3o=; b=klAJS06sDGmF0tcoMvYjKRhL3e0yHAq06DOwllGWQlVomig14K2tLJtO/kw2pkgQQU 1f1aQZWPFpIMv8OCa3EUpsWGmA1snQ4F/XBDhy9te+U2UZpkhxIoYYcnTYsBlHxnyYQ6 SBj574xAjEDq3Lqf9reN0tREdnTuK+IlF2oOl/bfbUWnQKQdDg9/N9u7fkd4jhQ6Y6Zt NFZGVBZ8NDQAuoDqaIsl6P/OqXI114gV/43qB6nkukFgoKhTtpwRRDzJ0HG3OjxNSRxM XN7qtpLuoUW/KvSArWS5TE/DwFEpmtxwLjK2l/IQSBX5ApOwJm9fqWYEQqZMoKZHi75G L8kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JLRedXoW1NUWdO6nkQxVEPJu6H/0wYkRPiho+e2/F3o=; b=fo/hpc89j9yW0JoyWh/FWIJ07WhydpB6L8urMmatoDCUTVb3E7YN84Ak1SfeyiLjlM +giGvslAT2mnkuP1Q3g0eGdkLgDznLX9ygCLGcSUHOqRGZiN+kvRuMSkQFH/Ljmv/Yqq fjXBKW83rPL2GopS4zvM0rvD/1XvQEdz95csS+nLj2X21yDEZizR4JYayQynB52Qiy2/ oyCJJmmq01POaN3+2gd4OVKb8XOh3lsvqiYH+X8A47oaO+1HKtIsXTtRDIQ4GbTqjEn5 kYMvsqJzvHHWkZ6ZVHh8cT5cqSKjp5j64++V6uyHdFcgHadumTar3ezpM2h5pNdQ4rf7 bdyw== X-Gm-Message-State: AOAM532Q6a9HWEg+wer3r8aoFsTm5GzufBE7EqJKOb+wBRNYL3QqAYT+ C6q5HEOw8LRPKtsa1BTkH3w= X-Google-Smtp-Source: ABdhPJwkd05fqxwVaEb0i0KyjgUzuPqyW5CU6ZcviFovWNYFoc7uhnRMVR5W/V4wE/FQvmB9pZF1eQ== X-Received: by 2002:a17:907:c08:: with SMTP id ga8mr2925883ejc.318.1627535453740; Wed, 28 Jul 2021 22:10:53 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:53 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 54/87] trace-cmd library: Add logic for in-memory decompression Date: Thu, 29 Jul 2021 08:09:26 +0300 Message-Id: <20210729050959.12263-55-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org There are two approaches to read compressed trace data: - use a temporary file to decompress entire trace data before reading - use in-memory decompression of requested trace data chunk only In-memory decompression seems to be more efficient, but selecting which approach to use depends in the use case. A compression chunk consists of multiple trace pages, that's why a small cache with uncompressed chunks is implemented. The chunk stays in the cache until there are pages which have reference to it. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 104 ++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 3b58cb0a..4582b9bc 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -1266,6 +1266,105 @@ static void free_page_map(struct page_map *page_map) free(page_map); } +#define CHUNK_CHECK_OFFSET(C, O) ((O) >= (C)->offset && (O) < ((C)->offset + (C)->size)) +static struct tracecmd_compress_chunk *get_zchunk(struct cpu_data *cpu, off64_t offset) +{ + struct cpu_zdata *cpuz = &cpu->compress; + int min, mid, max; + + if (!cpuz->chunks) + return NULL; + if (offset > (cpuz->chunks[cpuz->count - 1].offset + cpuz->chunks[cpuz->count - 1].size)) + return NULL; + + /* check if the requested offset is in the last requested chunk or in the next chunk */ + if (CHUNK_CHECK_OFFSET(cpuz->chunks + cpuz->last_chunk, offset)) + return cpuz->chunks + cpuz->last_chunk; + cpuz->last_chunk++; + if (cpuz->last_chunk < cpuz->count && + CHUNK_CHECK_OFFSET(cpuz->chunks + cpuz->last_chunk, offset)) + return cpuz->chunks + cpuz->last_chunk; + + /* do a binary search to find the chunk holding the given offset */ + min = 0; + max = cpuz->count - 1; + mid = (min + max)/2; + while (min <= max) { + if (offset < cpuz->chunks[mid].offset) + max = mid - 1; + else if (offset > (cpuz->chunks[mid].offset + cpuz->chunks[mid].size)) + min = mid + 1; + else + break; + mid = (min + max)/2; + } + cpuz->last_chunk = mid; + return cpuz->chunks + mid; +} + +static void free_zpage(struct cpu_data *cpu_data, void *map) +{ + struct zchunk_cache *cache; + + list_for_each_entry(cache, &cpu_data->compress.cache, list) { + if (map <= cache->map && map > (cache->map + cache->chunk->size)) + goto found; + } + return; + +found: + cache->ref--; + if (cache->ref) + return; + list_del(&cache->list); + free(cache->map); + free(cache); +} + +static void *read_zpage(struct tracecmd_input *handle, int cpu, off64_t offset) +{ + struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + struct tracecmd_compress_chunk *chunk; + struct zchunk_cache *cache; + void *map = NULL; + int pindex; + int size; + + /* Look in the cache of already loaded chunks */ + list_for_each_entry(cache, &cpu_data->compress.cache, list) { + if (CHUNK_CHECK_OFFSET(cache->chunk, offset)) { + cache->ref++; + goto out; + } + } + + chunk = get_zchunk(cpu_data, offset); + if (!chunk) + return NULL; + size = handle->page_size > chunk->size ? handle->page_size : chunk->size; + map = malloc(size); + if (!map) + return NULL; + if (tracecmd_uncompress_chunk(handle->compress, chunk, map) < 0) + goto error; + + cache = calloc(1, sizeof(struct zchunk_cache)); + if (!cache) + goto error; + cache->ref = 1; + cache->chunk = chunk; + cache->map = map; + list_add(&cache->list, &cpu_data->compress.cache); + + /* a chunk can hold multiple pages, get the requested one */ +out: + pindex = (offset - cache->chunk->offset) / handle->page_size; + return cache->map + (pindex * handle->page_size); +error: + free(map); + return NULL; +} + static void *allocate_page_map(struct tracecmd_input *handle, struct page *page, int cpu, off64_t offset) { @@ -1277,6 +1376,9 @@ static void *allocate_page_map(struct tracecmd_input *handle, int ret; int fd; + if (handle->cpu_compressed && handle->read_zpage) + return read_zpage(handle, cpu, offset); + if (handle->read_page) { map = malloc(handle->page_size); if (!map) @@ -1419,6 +1521,8 @@ static void __free_page(struct tracecmd_input *handle, struct page *page) if (handle->read_page) free(page->map); + else if (handle->read_zpage) + free_zpage(cpu_data, page->map); else free_page_map(page->page_map); From patchwork Thu Jul 29 05:09:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46B1CC4320A for ; Thu, 29 Jul 2021 05:11:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31FFA60EB2 for ; Thu, 29 Jul 2021 05:11:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234034AbhG2FLB (ORCPT ); Thu, 29 Jul 2021 01:11:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234044AbhG2FK6 (ORCPT ); Thu, 29 Jul 2021 01:10:58 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 114A5C061765 for ; Wed, 28 Jul 2021 22:10:56 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id y12so6392402edo.6 for ; Wed, 28 Jul 2021 22:10:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hgl5gMBxItia10lruAV2PkbjEaW+EM2h5ysLJEWNolo=; b=MQ5eM/kankgKcxdAYfzIPpsvMDDi+jZIBMWSKPU9xdji7jVdL8pJvuYHX2vaoLUohw 4hQor4M2s+tnQCu59c0mcOBrB/VCLYQODk7AQtqcuaI8nglnMojh9Q1hkoIz5ySTne12 9hRyKtFnr83q6LKJ21DlBxku8/jGaaX9w+DJ2NApSJ3gJ6xVFXUEwyZNnd6JR+AwGfqx w4a3XGyDHYkAUg1+ofs+ggwLn19JZz4gYWnac4foaE03WJPMdNBZRv203Tie3yUrjpgu BRHz4O2S6DNOI0vhJ1hwpi+MIieV5cFrx1LjCSgnr8OzC53gYmSpqoJ1VhoiWgS2G1/X EXYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hgl5gMBxItia10lruAV2PkbjEaW+EM2h5ysLJEWNolo=; b=fHLOod+Fw/ZRDde+kdnxrERBVoeSlFl0KgDEHJTd38x+PvUOUT3KCc/AmjHV8AOy/z KCLGAdhxMeBOk1cCr0SeL8LwX4aglUzcmC/HyxK6jweIT8lbIb4tG+sg6d7q7tliPB7t zIjvPeSbPq8upRmapV0rZv0dpwhUN1JwUrHuhTQsAdplXBdnhMGZwEqnMTf6mHPh8UvQ D+F39g5EMeknEZ1lw8J1xmgXb+Wdj26psdbrSfA2BnEfk6ciSR8fUOduV1BM51tthzuj zRCZDcFxQAx5nnvlPgEG3JgYoUqYvdtBpyHZsv1eC2+eMEh5k6VIA4PaaJU5Z7Lpw6J1 NJ8g== X-Gm-Message-State: AOAM533suPlyAlpKNle1zlvc2Z4e9Sqv0tYpu8LdCwwFdN2filYdZ/Lt Hdh8k5ioLNYZ5Ab+0Zp89hU= X-Google-Smtp-Source: ABdhPJxZLBiO1JJWZNfMSqfWsXTyazDvMlEScqrQouUwGmcwXsBpKOSOfc/HuM2HgYj5ppeEs9dyQw== X-Received: by 2002:a05:6402:1057:: with SMTP id e23mr4073206edu.352.1627535454729; Wed, 28 Jul 2021 22:10:54 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:54 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 55/87] trace-cmd library: Handle latency trace in version 7 files Date: Thu, 29 Jul 2021 08:09:27 +0300 Message-Id: <20210729050959.12263-56-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Latency trace data is saved the same was as flyrecord buffer data in trace files version 7. There is a BUFFER_LAT option which hols the latency specific trace metadata and points to the secrtion in the file with the trace data. Extended the input hanlder with latency trace context and added new logic to decompress and read the latency data. A new API is added to read latency data: tracecmd_latency_data_read() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 91 ++++++++++++++++++- 2 files changed, 91 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index f2868576..0a3e12cf 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -217,6 +217,8 @@ tracecmd_peek_data_ref(struct tracecmd_input *handle, int cpu) return rec; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, char **buf, size_t *size); + struct tep_record * tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 4582b9bc..b9340160 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -175,6 +175,9 @@ struct tracecmd_input { bool read_zpage; /* uncompress pages in memory, do not use tmp files */ bool cpu_compressed; int file_version; + /* temporary file for uncompressed latency data */ + struct cpu_zdata latz; + struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -3431,6 +3434,56 @@ static int read_options_type(struct tracecmd_input *handle) return 0; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, char **buf, size_t *size) +{ + struct cpu_zdata *zdata = &handle->latz; + void *data; + int rsize; + int fd = -1; + int id; + + if (!handle || !buf || !size) + return -1; + if (handle->file_state != TRACECMD_FILE_CPU_LATENCY) + return -1; + + if (!handle->cpu_compressed) { + fd = handle->fd; + } else if (!handle->read_zpage) { + if (zdata->fd < 0) + return -1; + fd = zdata->fd; + } + + /* Read data from a file */ + if (fd >= 0) { + if (!(*buf)) { + *size = BUFSIZ; + *buf = malloc(*size); + if (!(*buf)) + return -1; + } + return do_read_fd(fd, *buf, *size); + } + + /* Uncompress data in memory */ + if (zdata->last_chunk >= zdata->count) + return 0; + id = zdata->last_chunk; + if (!*buf || *size < zdata->chunks[id].size) { + data = realloc(*buf, zdata->chunks[id].size); + if (!data) + return -1; + *buf = data; + *size = zdata->chunks[id].size; + } + if (tracecmd_uncompress_chunk(handle->compress, &zdata->chunks[id], *buf)) + return -1; + rsize = zdata->chunks[id].size; + zdata->last_chunk++; + return rsize; +} + static int init_cpu_data(struct tracecmd_input *handle) { enum kbuffer_long_size long_size; @@ -3494,6 +3547,32 @@ static int init_cpu_data(struct tracecmd_input *handle) return -1; } +int init_latency_data(struct tracecmd_input *handle) +{ + unsigned long long wsize; + int ret; + + if (!handle->cpu_compressed) + return 0; + + if (handle->read_zpage) { + handle->latz.count = tracecmd_load_chunks_info(handle->compress, &handle->latz.chunks); + if (handle->latz.count < 0) + return -1; + } else { + strcpy(handle->latz.file, COMPR_TEMP_FILE); + handle->latz.fd = mkstemp(handle->latz.file); + if (handle->latz.fd < 0) + return -1; + ret = tracecmd_uncompress_copy_to(handle->compress, handle->latz.fd, NULL, &wsize); + if (ret) + return -1; + lseek64(handle->latz.fd, 0, SEEK_SET); + } + + return 0; +} + static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buffer_instance *buffer) { unsigned long long offset; @@ -3510,6 +3589,10 @@ static int init_buffer_cpu_data(struct tracecmd_input *handle, struct input_buff return -1; if (flags & TRACECMD_SEC_FL_COMPRESS) handle->cpu_compressed = true; + if (buffer->latency) { + handle->file_state = TRACECMD_FILE_CPU_LATENCY; + return init_latency_data(handle) == 0 ? 1 : -1; + } handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; handle->cpus = buffer->cpus; if (handle->max_cpu < handle->cpus) @@ -3959,7 +4042,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->fd = fd; handle->ref = 1; - + handle->latz.fd = -1; /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; @@ -4233,7 +4316,11 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->trace_clock); free(handle->version); close(handle->fd); - + free(handle->latz.chunks); + if (handle->latz.fd >= 0) { + close(handle->latz.fd); + unlink(handle->latz.file); + } while (handle->sections) { del_sec = handle->sections; handle->sections = handle->sections->next; From patchwork Thu Jul 29 05:09:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68026C432BE for ; Thu, 29 Jul 2021 05:11:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 522D460EB2 for ; Thu, 29 Jul 2021 05:11:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234044AbhG2FLC (ORCPT ); Thu, 29 Jul 2021 01:11:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233790AbhG2FLA (ORCPT ); Thu, 29 Jul 2021 01:11:00 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2DEBC0613C1 for ; Wed, 28 Jul 2021 22:10:56 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id ga41so8409769ejc.10 for ; Wed, 28 Jul 2021 22:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ybeGOAZKYXmJFsd3anpx3SK0c5hb8fOH9icHxAb15T0=; b=FVhfzGbf86ycrUQ483b4ZLwyonl+uCQR4qtiNGAgKlNm2CsNpJCzXF07E2Mk6hRt32 adezeRg5xW+zdnEJMl8Q0XfcmesT4YceSdlGvjsmrW/kPDNTzoV8fzDFpIhQILz+X0t4 8Ux7ah+Mh15eupDmPReGlQxKA5OTQcEbKYSpS9naDJ+Hs/tXrsDG4be1mrn48wKgi2OE G29IN0+7rkdaUqOhORCjwN6IUv7SWEDPGk6wBJBXwX08kQ0VSjr7d/YmhFY1w4W0AgP3 7v0dDhCW60UnLv9a2Ayes32epJ+dOiQ9Cbr6wi1/9b622SGzredqlU42jmNyuQu23XX4 6woA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ybeGOAZKYXmJFsd3anpx3SK0c5hb8fOH9icHxAb15T0=; b=Jw8+gilSNJy21cKPcx2ADLS65HBRW4764LYq9XO2CzdSxJh2S68zFRAZ8Dq+oKJ5pO IOkqOsOSFnmsKfKdR3daXgyG64yDZLqclP2SgWPeWTdxdBSLQL1geIMj+njY4Z6TJlLG 9mxrWnOuoqWw62IQ2H0duWA2ySq6kcORzCEcIM9GengXkE0J3Zu7rDNRMeAYiNVXzPtn boGF8y6eSrFbtaCCOfDRCmVyBGzWn9KMg5O/wYhijLZyBqBRUyY2MNgyCs99SWXiVkzT G+/n1Gnk+S59aatzi9naQOG54t+SY01o9a4XpYPOAopSzfAOobWx/bsS1JbkTWzqHGrz 4yKA== X-Gm-Message-State: AOAM531C38yvlAS8GdOfnZb+RmxJdOAVggyT4sAUOR8+TtebGnmedvl9 zfONBffV4a8qNDE6JMq4yqBusXNe5vt+Hg== X-Google-Smtp-Source: ABdhPJzUykfp96uCxvkG5AgN8IWYfas8e/dLKSQsrgIXACkm5F6p3EFo4yFWZNvSxDvbSoripEagcA== X-Received: by 2002:a17:906:2bd3:: with SMTP id n19mr2963201ejg.232.1627535455650; Wed, 28 Jul 2021 22:10:55 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:55 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 56/87] trace-cmd library: Handle buffer trace data init for version 7 files Date: Thu, 29 Jul 2021 08:09:28 +0300 Message-Id: <20210729050959.12263-57-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org CPU data initialization is different for trace files version 6 and 7. When a new input handler to trace buffer is created, initialize the CPU data according to the file version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 55 +++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 26 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b9340160..2b98e6d8 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4809,34 +4809,37 @@ tracecmd_buffer_instance_handle(struct tracecmd_input *handle, int indx) new_handle->flags |= TRACECMD_FL_BUFFER_INSTANCE; new_handle->pid_maps = NULL; + if (handle->file_version < 7) { + /* Save where we currently are */ + offset = lseek64(handle->fd, 0, SEEK_CUR); - /* Save where we currently are */ - offset = lseek64(handle->fd, 0, SEEK_CUR); - - ret = lseek64(handle->fd, buffer->offset, SEEK_SET); - if (ret < 0) { - tracecmd_warning("could not seek to buffer %s offset %ld\n", - buffer->name, buffer->offset); - goto error; - } - - /* - * read_options_type() is called right after the CPU count so update - * file state accordingly. - */ - new_handle->file_state = TRACECMD_FILE_CPU_COUNT; - ret = read_options_type(new_handle); - if (!ret) - ret = read_cpu_data(new_handle); - if (ret < 0) { - tracecmd_warning("failed to read sub buffer %s\n", buffer->name); - goto error; - } + ret = lseek64(handle->fd, buffer->offset, SEEK_SET); + if (ret == (off64_t)-1) { + tracecmd_warning("could not seek to buffer %s offset %ld\n", + buffer->name, buffer->offset); + goto error; + } + /* + * read_options_type() is called right after the CPU count so update + * file state accordingly. + */ + new_handle->file_state = TRACECMD_FILE_CPU_COUNT; + ret = read_options_type(new_handle); + if (!ret) + ret = read_cpu_data(new_handle); - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret < 0) { - tracecmd_warning("could not seek to back to offset %ld\n", offset); - goto error; + if (ret < 0) { + tracecmd_warning("failed to read sub buffer %s\n", buffer->name); + goto error; + } + ret = lseek64(handle->fd, offset, SEEK_SET); + if (ret < 0) { + tracecmd_warning("could not seek to back to offset %ld\n", offset); + goto error; + } + } else { + if (init_buffer_cpu_data(new_handle, buffer) < 0) + goto error; } return new_handle; From patchwork Thu Jul 29 05:09:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A497CC4320E for ; Thu, 29 Jul 2021 05:11:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CF6B60EB2 for ; Thu, 29 Jul 2021 05:11:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234065AbhG2FLC (ORCPT ); Thu, 29 Jul 2021 01:11:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234054AbhG2FLB (ORCPT ); Thu, 29 Jul 2021 01:11:01 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EEB5CC061757 for ; Wed, 28 Jul 2021 22:10:57 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id r16so6379840edt.7 for ; Wed, 28 Jul 2021 22:10:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ho/IUuK1D6V9ESHNX2ltgR4QXMDHcbCK00QO/msZh5A=; b=nEEGF2qgRp+gfdgw+8xN6erSxeNY8I6f92ZK3QHRgqGqswmklzYDb4m5Z1BEEIqNoY D9SMUEbi05iAtrnqkdzhn3Pd6SgdqCUoAMRt9CICK+6x2mTrit6qQbsdH7CpLKshg3mz 5dqdgkGGuTyhwFDIrzMZ4frhTl8MBmVpv8NEND+vqy5bm0BVzkDEZcCqHylLmbEww4Ec 4BLTQ2mDmUlk8XTgucR2W7nsZHxpKKvvD9YxT2RtSQs+F4g1gCWZ3eBS2FpSH4eCOet1 WprguOzS5FKZzzMgYLoQ5irP0D/G4AU9KH1x+yUV6QGhtpenGOCIcKVl8Hon7kBCvfah 6K7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ho/IUuK1D6V9ESHNX2ltgR4QXMDHcbCK00QO/msZh5A=; b=pTaImnEEVj4Zw5BcIs/5y3FrlAfRPzKBLMWz9D6FX0189LnBJfUmAvdEneRSlVSbHP 4xTnb+ZUtkbKWGHhGx1NIN3TM3wWLOaOOXWsa/8xtEsbABVM7PgIA8g0lyRo443tLyQF yql1KQIEH9XEUWH9XR/nuf0kwG2gBExhbtlRA7XB5CV6F/JOB68fGoIV8WxPmLn3pjxS 8Y7XsDqL4taZ3Z7bb+jNaHi898XUXBlTMGvVDyGWRJ7pRF19pOL/tsWGk1W93k02+JdC 5QOLV/SUIHWN5b2sliaQxHqdwTAvAeMcXpDeEW8gqcQ2kRhtpcIzFnT3f6cyks7YD44F 255A== X-Gm-Message-State: AOAM532U3O42rY0p0nJ39wvq7bNlWLJIwivhjiU73H1z8L001EoYHYba d4VmMoVa0CRqE+vzSo4+7i8= X-Google-Smtp-Source: ABdhPJwi+uh0Qbzfe43q/IdRPfFYjeXcxET4Hl5jm+KAaYAu5BoS81TV2yHlHHBgHVZdfV2A1FZSug== X-Received: by 2002:a05:6402:2714:: with SMTP id y20mr3955190edd.190.1627535456562; Wed, 28 Jul 2021 22:10:56 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:56 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 57/87] trace-cmd report: Use the new latency API to read data Date: Thu, 29 Jul 2021 08:09:29 +0300 Message-Id: <20210729050959.12263-58-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading latency trace data, use the new API tracecmd_latency_data_read() It handles reading latency trace data from both version 6 and 7 trace files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-read.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 31724b09..0ec2b2d1 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -948,18 +948,20 @@ void trace_show_data(struct tracecmd_input *handle, struct tep_record *record) printf("\n"); } -static void read_rest(void) +static void read_latency(struct tracecmd_input *handle) { - char buf[BUFSIZ + 1]; + char *buf = NULL; + size_t size = 0; int r; do { - r = read(input_fd, buf, BUFSIZ); - if (r > 0) { - buf[r] = 0; - printf("%s", buf); - } + r = tracecmd_latency_data_read(handle, &buf, &size); + if (r > 0) + printf("%.*s", r, buf); } while (r > 0); + + printf("\n"); + free(buf); } static int @@ -1241,7 +1243,7 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype if (ret > 0) { if (multi_inputs) die("latency traces do not work with multiple inputs"); - read_rest(); + read_latency(handles->handle); return; } From patchwork Thu Jul 29 05:09:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 304A1C43214 for ; Thu, 29 Jul 2021 05:11:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1C08B60C3F for ; Thu, 29 Jul 2021 05:11:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234054AbhG2FLC (ORCPT ); Thu, 29 Jul 2021 01:11:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233790AbhG2FLC (ORCPT ); Thu, 29 Jul 2021 01:11:02 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CED59C061765 for ; Wed, 28 Jul 2021 22:10:58 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id f13so6342718edq.13 for ; Wed, 28 Jul 2021 22:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fH+eB0oQaWGz+OBjpumBHL5m9Ri483L+c2sXZChUFKk=; b=p2YWvPHUgNt1MKON2li1EnGUoTh/a5u4Y59j8l5gxdYuu9F7Of7Olbv1JmvAZl6Aks x25ZN75V6TsV7RPKNz1raQ+dWD7i0LiAi75672F0+m7dkUwjmFUrTwinFZQyNVUVHl1Q EwR+I+OBCE9IGOBxuVUFCZd4BZANUuxzWlnMEN2o7YXzCM9C9b74STz+tmx8q90Y2CRy GAP17XR+WIU0r2YMBCfZNImP6vJPDdkZXzVrQtMOUryW2l5LBCRFbR7nX9uHQOu1dE/V LfXCbwDGcacO3pxLVnYy9zK5OZpArPdCEF3ERRMyNq70TORg9/pRG5S+n4h2MmEjdzAB SI6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fH+eB0oQaWGz+OBjpumBHL5m9Ri483L+c2sXZChUFKk=; b=gdbGHkvfEBIt4jt1Xpbd7V+DUyy+PPVXRP3RFTSp0W8OpdHlYIY65P7FdLvlbyU8sz SxSHIxggXTQeRthJTtSXcioPXd2Lx5qgbi3XVOC+8WJIrhatJNBygAJSnoAe7Iszjqbw UN+vttsp6ILnG6uc337IMPcRkS6rcro99XYN9uOyeXX6o9CQaVmitmgJSgZ02HReaQTc SxahJDHrHxtZx4I/Fm2jF0r0fU6uJpOuLoAgXgVmtSHMGPanpsup7tOhQ30tYmyBQ/BC U5gs8HitkoY6hYjLLbTQUGRyV+Rpi5+xMOcgtpNghcWAfQUz2T0d2ltc2oH13B849Fgb 5Vww== X-Gm-Message-State: AOAM5318xfQz4n2rRb47j8XMMis8AKSgTI1LLhrSEG7dLqljdZuQ3xkT kX9uSMzhWu6jIRN0m+QEB9k= X-Google-Smtp-Source: ABdhPJyVp8BGieMba2+TQjCYOEb3XsDToevfZlFjxBb2GTlZWd/saThM0ptUf4KMJz/8kYqjegbbLg== X-Received: by 2002:a05:6402:781:: with SMTP id d1mr3995251edy.32.1627535457509; Wed, 28 Jul 2021 22:10:57 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:57 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 58/87] trace-cmd report: Close input file handlers on exit Date: Thu, 29 Jul 2021 08:09:30 +0300 Message-Id: <20210729050959.12263-59-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When "trace-cmd report" is interrupted with "ctrl-c", close the input hanlders to opened trace files, to delete any temporary files used when reading the trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-read.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 0ec2b2d1..a36d72e4 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1365,7 +1365,14 @@ struct tracecmd_input *read_trace_header(const char *file, int flags) static void sig_end(int sig) { + struct handle_list *handles; + fprintf(stderr, "trace-cmd: Received SIGINT\n"); + + list_for_each_entry(handles, &handle_list, list) { + tracecmd_close(handles->handle); + } + exit(0); } From patchwork Thu Jul 29 05:09:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D47F3C4338F for ; Thu, 29 Jul 2021 05:11:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BE3E16023E for ; Thu, 29 Jul 2021 05:11:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234074AbhG2FLD (ORCPT ); Thu, 29 Jul 2021 01:11:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233790AbhG2FLD (ORCPT ); Thu, 29 Jul 2021 01:11:03 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2DC3C061757 for ; Wed, 28 Jul 2021 22:10:59 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id y7so4163172eda.5 for ; Wed, 28 Jul 2021 22:10:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=laAbe1GsKsTa0GiJsFPPpEeHWVvQWL3tIhbHAnbGL3M=; b=eJbH8trSlM3uBkfuLb8pPCbpN30UXZ3KbaIbeLJT6svm7GHfQQfkONYZqnuevvlQrE VjuO6lHwJTG4Lfy+zfZzlZWu9JIm357PAKmAtSpHIbQRCv39sny/reWBd2Hl9hLwXxF3 yV8A9rCo1QSBJPlZGtTfHlsDg3UDUxmQDDh7w9GlnimmuIWe3qSbzM+cBQMM7n8b6Wip WBF3sX+PbeAWijcLYknbWQfY6awSPhGdKSCZZIeEoQGXUZ31DoUat5X9VkGpH0ncEDEu ZhO3b7S1xMxJIBdWVFMv/R43HJqqayxfS5Q/yGzr3qpyUd39139VGircFtaIEFayVcPl 5VHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=laAbe1GsKsTa0GiJsFPPpEeHWVvQWL3tIhbHAnbGL3M=; b=YFK4dR8MYjTVYBPZ4rYtaGGZ9I9r4WMmmOmFWIT0l9MMvjvDvWPAhPgH1+y7Efe2cc aq5mVlBkqszMrNSwi1vP6dZy9YWAADDDogk3F1xZrtLM31XFkPwOrVE9r/ixkc1i1Xt0 XcPxh+AR9dT/FJpdH2AlIW6QiMYcqI11DzpZLbQu5KjJpAYeW18fETNEZPBtfQcTZWXf pEcmEJzsbMenrQ5AJEyXD/48tnHVL6LNrDxBTRhQeEv789rr6SgQlTK+VPmG+RmU2Xce IlRjkHZB2o/tDf2urheSzygOIYZ/0I7lZVJiDG0HcY3aNt4s1iiObfo1/DYJLXAytBgr 69Dw== X-Gm-Message-State: AOAM531R85hfiKgbyfKTFes5tLrzozVuKYRvy0MEbVs+wBFdJbdvtJ3i XAFjPpTTxNqyBHCM4P5KfMU= X-Google-Smtp-Source: ABdhPJyqr0sa3KJxY5Uocz30SZvCYnJowfhs2/WzPhkxCAhzNnrNRrdjGbc78RFv67thzOcHa5tzSg== X-Received: by 2002:a05:6402:b8f:: with SMTP id cf15mr3885912edb.286.1627535458485; Wed, 28 Jul 2021 22:10:58 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:58 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 59/87] trace-cmd report: Do not print empty buffer name Date: Thu, 29 Jul 2021 08:09:31 +0300 Message-Id: <20210729050959.12263-60-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, the top buffer is saved with its empty file name, string "". When displaying it, filter those empty strings. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-read.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index a36d72e4..1868a830 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1175,7 +1175,7 @@ static void print_handle_file(struct handle_list *handles) /* Only print file names if more than one file is read */ if (!multi_inputs && !instances) return; - if (handles->file) + if (handles->file && *handles->file != '\0') printf("%*s: ", max_file_size, handles->file); else printf("%*s ", max_file_size, ""); From patchwork Thu Jul 29 05:09:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B970C43216 for ; Thu, 29 Jul 2021 05:11:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0820360EB2 for ; Thu, 29 Jul 2021 05:11:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233790AbhG2FLE (ORCPT ); Thu, 29 Jul 2021 01:11:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234068AbhG2FLD (ORCPT ); Thu, 29 Jul 2021 01:11:03 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C24BEC061765 for ; Wed, 28 Jul 2021 22:11:00 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id u12so6435885eds.2 for ; Wed, 28 Jul 2021 22:11:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xYZjx89pr2ljRxRd4T7dvQBQNHexYzaKEzijEQWYQHA=; b=MYkXsK7d8jvt6RZijHoc98nNV65k/iUCuaWktl4Lx8T4lk4lIDo8NIMR/G32XZfiWF 4wFkN4o+TZzK4up0CsBHiZfX60bN+xm3aP0vKroGyMmO2WJdn4e7dYczJP9Bbi8Va/EA AXb4zoLM3SvmnBdy1bp/N1F632R4e6/7b3ZvmWbKQJOKqcrDr75LqhJo6it3pc+S119E rdwQp0EZ4FkT1RPyS1XINciREgUNxmlAhpJYUHypSvp5K0PUNlV0zybm8GD577G9vXey VKSHShBNzF4Z9E2hliFtaPp19hk8Xbrg9/UVe7hIWWsPKjSzPBJFeE5FGcJd79tpzINz f41A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xYZjx89pr2ljRxRd4T7dvQBQNHexYzaKEzijEQWYQHA=; b=owsBO5EiEYAhNOrCq8ar7s55E+WYvdo2PRnqe0WcuFAGcDtnZt4GxSG9G/rTo8Ak9W /xhHvYKK4zHTuAsoEoyjldsmVBBxvzQ8LdySxwJDRkz95uuRs1svSM3KSUvE5A1DsbQU 6C91vF47YB0i7H1f5G8Rm0EXwlnhKFtBXsGQ751Gz5/Ukjp4NXapSQqPWYiA0SB+xyF1 feFrQA/41WxiFcW7Cc7KbZJ7qfxIwxsGT44Z6M5If904XvGrflFufF3fBHoiTyvBL47f g22gfpib995uZYXPqtGniCBKgXMrqXi8jUpOhkpPbGvCIOwtxKBf6UtNj+u9vEgVBuoY 92eQ== X-Gm-Message-State: AOAM531AI2KomWnP1wwJUYknZcxr+aLM/q9Z2WTScTnbgPN6JjOePLV2 Tp87zKbLOuFdXSQuaWzv3q0K5zF+CVcADA== X-Google-Smtp-Source: ABdhPJyrtfTrutnqZcwJl+YyMQRkMxQUsQ/Btxam7XUJzYpu0f9DagUukI2hf/WpnKEgo8zhcE3LWQ== X-Received: by 2002:aa7:c6d2:: with SMTP id b18mr3895368eds.266.1627535459423; Wed, 28 Jul 2021 22:10:59 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:58 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 60/87] trace-cmd report: Init the top trace instance earlier Date: Thu, 29 Jul 2021 08:09:32 +0300 Message-Id: <20210729050959.12263-61-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The instance must be initialized before calling the tracecmd_cpus() API, as in trace files version 7, CPU count is set when the trace data are initialized. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-read.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 1868a830..cafceffe 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1217,6 +1217,11 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype list_for_each_entry(handles, handle_list, list) { int cpus; + if (!tracecmd_is_buffer_instance(handles->handle)) { + ret = tracecmd_init_data(handles->handle); + if (ret < 0) + die("failed to init data"); + } cpus = tracecmd_cpus(handles->handle); handles->cpus = cpus; handles->last_timestamp = calloc(cpus, sizeof(*handles->last_timestamp)); @@ -1227,9 +1232,6 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype if (tracecmd_is_buffer_instance(handles->handle)) continue; - ret = tracecmd_init_data(handles->handle); - if (ret < 0) - die("failed to init data"); if (align_ts) { ts = tracecmd_get_first_ts(handles->handle); if (first || first_ts > ts) From patchwork Thu Jul 29 05:09:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1C78C4320A for ; Thu, 29 Jul 2021 05:11:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CCDBC60EB2 for ; Thu, 29 Jul 2021 05:11:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234068AbhG2FLE (ORCPT ); Thu, 29 Jul 2021 01:11:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233875AbhG2FLE (ORCPT ); Thu, 29 Jul 2021 01:11:04 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6370C061757 for ; Wed, 28 Jul 2021 22:11:01 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id gn26so8488741ejc.3 for ; Wed, 28 Jul 2021 22:11:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O3MlOCSVZr10henjb/kOmATFSY+Wwfn0tH13upLABCo=; b=W9PwlYMzixtAo31WJnjbDv1un1hxkxjZkNSsFRQ02E/93ZljWW+/OSWUXJHtEwHTfZ LaNbJPfoFsyqP+6AvWyaLLpa2u2qpjnpFtkZ2+mLTEHlnfcsMWLyo3ZvULDrP7Zu7lZa kfG4Q9+C+kEHsGPXtnqnDOf4Hjh5hTOM2nWR5tN0/oq2wbOB8QlBr1QJ4Lg05tLuebnp 3HIruXBCvuRsSY15tLGdsBDodle/NwGoKQQU2rgWo8CnoigxPkEQrXPjptCCwXcG7Hx8 j6lwBSkqLKigH9ptPlXFgdmqfnTnFRusaHjaEKbr2pfCD4HIYTqwar7ubXDxVIhONSTD O+yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O3MlOCSVZr10henjb/kOmATFSY+Wwfn0tH13upLABCo=; b=YQ2coxpJjzdvRAtUDFT0VASpwdsqG3ZLKDToY163Y+qAlAq04D3Ed645hbFEgpYnNE sSHuLrqnqo9Izji4ZY5Fr8BIRLCAxTSGhvpllQvuZyygJWbro4WmJS2oRIvqD9NzN0IB eClsbgXRfKkvAP5nClST+oWQXhtVD9pDrHIpJK3UKhKuk0hhwA9nZ1pRN2yPhZHK7bCi ce8WZJXm8HZek2WgE/MOi+cFHm7Ph0oQMTmLoTkmvEwlLmwXLkUy44jnWsUUcdcJGzYB MzQ+8eFy5bLmFl4T4P/iehNTsOPxNGR0joxUSAbZR7bfEsWe3+GTPvWz08Xp9gQo/V+A cezg== X-Gm-Message-State: AOAM530IPw8hLhhsmGqpq36nfozPOq2Ho0q65LohREv5RfPagObJCLHD J73YgM0XkbGVX490UXePQ5LaiChAjt9JvA== X-Google-Smtp-Source: ABdhPJx6eKPQJjvykaz7c6z8n6Vn+zjTaXc15I3U8LUZGr0ktyJ+qV2ejJkQRjSVpGlkwLcnlaZPsA== X-Received: by 2002:a17:907:1b06:: with SMTP id mp6mr3017944ejc.188.1627535460391; Wed, 28 Jul 2021 22:11:00 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.10.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:10:59 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 61/87] trace-cmd: Call additional APIs when creating trace file Date: Thu, 29 Jul 2021 08:09:33 +0300 Message-Id: <20210729050959.12263-62-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a trace file, two more APIs should be called, compared to the old flow: - tracecmd_write_buffer_info(), to write version 6 buffers metadata in the file. - tracecmd_write_options() after the trace data is written, for version 7 trace files, as the buffer metadata is appended to the options at the end. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-listen.c | 6 ++++++ tracecmd/trace-record.c | 4 ++++ tracecmd/trace-restore.c | 3 ++- tracecmd/trace-split.c | 3 +++ 4 files changed, 15 insertions(+), 1 deletion(-) diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index d812145b..3312a042 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -604,6 +604,9 @@ static int put_together_file(int cpus, int ofd, const char *node, if (write_options) { ret = tracecmd_write_cpus(handle, cpus); + if (ret) + goto out; + ret = tracecmd_write_buffer_info(handle); if (ret) goto out; ret = tracecmd_write_options(handle); @@ -612,6 +615,9 @@ static int put_together_file(int cpus, int ofd, const char *node, } ret = tracecmd_write_cpu_data(handle, cpus, temp_files, ""); + if (!ret && tracecmd_get_out_file_version(handle) >= 7) + tracecmd_write_options(handle); + out: tracecmd_output_close(handle); for (cpu--; cpu >= 0; cpu--) { diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 2e8db8a4..ddab7798 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3738,6 +3738,9 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; ret = tracecmd_write_cpus(network_handle, instance->cpu_count); + if (ret) + goto error; + ret = tracecmd_write_buffer_info(network_handle); if (ret) goto error; ret = tracecmd_write_options(network_handle); @@ -4109,6 +4112,7 @@ static void setup_agent(struct buffer_instance *instance, add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_buffer_info(network_handle); tracecmd_write_options(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); instance->network_handle = network_handle; diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 8d2fcae8..fbda42dd 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -163,6 +163,7 @@ void trace_restore (int argc, char **argv) if (tracecmd_append_cpu_data(handle, args, &argv[first_arg]) < 0) die("failed to append data"); - + if (tracecmd_get_out_file_version(handle) >= 7) + tracecmd_write_options(handle); return; } diff --git a/tracecmd/trace-split.c b/tracecmd/trace-split.c index e4a0c3b3..0768329f 100644 --- a/tracecmd/trace-split.c +++ b/tracecmd/trace-split.c @@ -391,6 +391,9 @@ static double parse_file(struct tracecmd_input *handle, if (tracecmd_append_cpu_data(ohandle, cpus, cpu_list) < 0) die("Failed to append tracing data\n"); + if (tracecmd_get_out_file_version(ohandle) >= 7) + tracecmd_write_options(ohandle); + current = end; for (cpu = 0; cpu < cpus; cpu++) { /* Set the tracecmd cursor to the next set of records */ From patchwork Thu Jul 29 05:09:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09CBDC4338F for ; Thu, 29 Jul 2021 05:11:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E440260C3F for ; Thu, 29 Jul 2021 05:11:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234098AbhG2FLG (ORCPT ); Thu, 29 Jul 2021 01:11:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234079AbhG2FLG (ORCPT ); Thu, 29 Jul 2021 01:11:06 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6D0AC061757 for ; Wed, 28 Jul 2021 22:11:02 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id u12so6435987eds.2 for ; Wed, 28 Jul 2021 22:11:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0e3pzSC/jURaDUPtk817CJ7QmZ7kWKFhZA8bjqTmygo=; b=q4Hnuk5U8Hb5BIXGd4edjad2/9iReZg5gMN12Zqy51aDoo58MlsGywVUvy3m6sKjRn z4E6ZDrm2VonKuV+mwxh2iq3dY7hS6kiE+0QwEpWJa+VylOR8wvIjYCd9c9qpST0aWNZ CLfs95XJkAORy9YdD7rwWAcuRqjytNoPwLB33KByNULCbVfma7vdWk/lF18t+5daGFUg UWQo1OMA5hccp1P87XHFOcok0tyxSiO3Fjz1ekJ0gh1NP9OvsyXPjB/hb0nN/IcjPMYl KNG1X5TrtRjE0BAzVbT4ycsTEjGqyWMHI44uiDyAMPhD6OpR9o19qty1PkYztETSI8Nt WYJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0e3pzSC/jURaDUPtk817CJ7QmZ7kWKFhZA8bjqTmygo=; b=C4C470qQsMVGrIoB4glO9jGX3JT51DEhSixzOlitu6e7O4WJn+tRUL7mwC3qJUEqhe KfC0TMegxBnHudFOE7RrhfLYeBfAqgoQO7/8IaS4NxytyGBmMyfF+E4ZSFqnqKnnR6tr A9+F705bmXztbBYIDBn80ZNtrj6FFr6ZQ3eyUNHHScMtI/Q3XUh3HqTzWBtGCtShDX+4 JaG4xUt+37y5IeUsO3ymBziWD/XzWn/qOqcIZo4BqpzCv3t+7//UAkerdjg+Xm/RSuHJ M0/PcwPH7czSx0GHtPvMrYuT1dBFzkpVAIL2b0fK3fM+Rnh3YJj+/IJcXeU1ADl08qAx YSWA== X-Gm-Message-State: AOAM531Uk52BCfLy5h5ka+Z5WBRcHETqjfXyaTrxOUdeuxRK8OzBFI8s vQEZVUZ7ul0aNMHW8zXW58bPnNdzJdlZcA== X-Google-Smtp-Source: ABdhPJwSrx0N3dCfS5L2Zftn7YtWe5HjxuD7kPZFSvLpvBuYNUpJPMmV2+uVFkpHp1bFVsZ4xlj8hg== X-Received: by 2002:a05:6402:284:: with SMTP id l4mr3845729edv.108.1627535461388; Wed, 28 Jul 2021 22:11:01 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:00 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 62/87] trace-cmd dump: Add helpers for processing trace file version 7 Date: Thu, 29 Jul 2021 08:09:34 +0300 Message-Id: <20210729050959.12263-63-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New global variables are added, that hold trace file context: - trace file version - compression context Also a few helper functions for reading compressed data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 88 ++++++++++++++++++++++++++++++++----------- 1 file changed, 67 insertions(+), 21 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 2334b57e..e150a17d 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -26,6 +26,9 @@ static struct tep_handle *tep; static unsigned int trace_cpus; static int has_clock; +static unsigned long file_version; +static bool read_compress; +static struct tracecmd_compression *compress; enum dump_items { SUMMARY = (1 << 0), @@ -52,46 +55,69 @@ enum dump_items verbosity; tracecmd_plog(fmt, ##__VA_ARGS__); \ } while (0) -static int read_file_string(int fd, char *dst, int len) +static int read_fd(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, 1); + r = read(fd, dst+size, len); if (r > 0) { - size++; - len--; + size += r; + len -= r; } else break; - if (!dst[size - 1]) - break; - } while (r > 0 && len); + } while (r > 0); - if (!size || dst[size - 1]) + if (len) return -1; - return 0; + return size; } -static int read_file_bytes(int fd, char *dst, int len) +static int read_compressed(int fd, char *dst, int len) +{ + + if (read_compress) + return tracecmd_compress_read(compress, dst, len); + return read_fd(fd, dst, len); +} + +static int do_lseek(int fd, int offset, int whence) +{ + if (read_compress) + return tracecmd_compress_lseek(compress, offset, whence); + return lseek64(fd, offset, whence); +} + +static int read_file_string(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, len); + r = read_compressed(fd, dst+size, 1); if (r > 0) { - size += r; - len -= r; + size++; + len--; } else break; - } while (r > 0); + if (!dst[size - 1]) + break; + } while (r > 0 && len); - if (len) + if (!size || dst[size - 1]) return -1; return 0; } +static int read_file_bytes(int fd, char *dst, int len) +{ + int ret; + + ret = read_compressed(fd, dst, len); + return ret < 0 ? ret : 0; +} + static void read_dump_string(int fd, int size, enum dump_items id) { char buf[DUMP_SIZE]; @@ -146,7 +172,6 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; - unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -168,11 +193,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); - ver = strtol(buf, NULL, 10); - if (!ver && errno) + file_version = strtol(buf, NULL, 10); + if (!file_version && errno) die("Invalid file version string %s", buf); - if (!tracecmd_is_version_supported(ver)) - die("Unsupported file version %lu", ver); + if (!tracecmd_is_version_supported(file_version)) + die("Unsupported file version %lu", file_version); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) @@ -234,6 +259,27 @@ static void dump_header_event(int fd) read_dump_string(fd, size, HEAD_EVENT); } +static void uncompress_reset(void) +{ + if (compress && file_version >= 7) { + read_compress = false; + tracecmd_compress_reset(compress); + } +} + +static int uncompress_block(void) +{ + int ret = 0; + + if (compress && file_version >= 7) { + ret = tracecmd_uncompress_block(compress); + if (!ret) + read_compress = true; + + } + return ret; +} + static void dump_ftrace_events_format(int fd) { unsigned long long size; @@ -578,7 +624,7 @@ static void dump_options(int fd) default: do_print(OPTIONS, " %d %d\t[Unknown option, size - skipping]\n", option, size); - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); break; } } From patchwork Thu Jul 29 05:09:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E4D5C4320A for ; Thu, 29 Jul 2021 05:11:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 46F7D60FED for ; Thu, 29 Jul 2021 05:11:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234079AbhG2FLH (ORCPT ); Thu, 29 Jul 2021 01:11:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233875AbhG2FLG (ORCPT ); Thu, 29 Jul 2021 01:11:06 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 937FBC061765 for ; Wed, 28 Jul 2021 22:11:03 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id qk33so8388454ejc.12 for ; Wed, 28 Jul 2021 22:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OVgcQ054TzlseuUkjG0ta+zXAdH5bdxjdFF14khoTQI=; b=fs7KeWEXBvsl+5/x6EcQXLVF2u/pPesDTNOEqPCgLFUig6gazsvLoQpUktcQ/Xyimb 5N2J8QxJDaxZh2JaeeCzwlaKV9WW4ooPvoB0tafjyVsoGYdYv1bJb9XjODa3wfwBxGbf 9wcN63HTEzWiQLPUPzq2ICQXiZHGuu7VWzAJUU9Oj5HWgGoWfgH/jBHhwgS4OJ2k6Jge OPKVYJjXn3XwxuJD0QqhWgusSLEAKbMEG9L1fLkL7VI2sbZUcz477K0hrChXnbMK0sOW HgTUmaiqWo9rF/hmR8KI6W8IakxVc+jeAihe5Kni4YGTxY4T4/fAaANN8xDrCeHXAN4l Z5Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OVgcQ054TzlseuUkjG0ta+zXAdH5bdxjdFF14khoTQI=; b=gRXZzjCgRqwf8LvlAruNtVvW6mDQvDZeMiUnV3TCfRBrCcHZXYOsfq2F1J50FNi0mn SJYwfT8krDpe6fkoZwPK4+LGfJJkjPZp+R877lMSbEGJv8uxhSUITiI5LKilbBEsZMJG UVxTXYFYuM/aLWhh2cKV87pgI0YBPLneBu0LGgPBtsgGXL3vy1SC0bHIVV808fWJoHio seMG6sBF3iRbijA6ugh4iBDuvAqHICw+gQePjWxe6HZavCIKqC+hEI14GpTxZHZja14d 7NPtJ8T2QkiMWk0nxwA5hej8L5pQVkdGBucFx51YyfaZB3BbiM2xxe0rkp6mm/la7BXY jV+w== X-Gm-Message-State: AOAM531/yIlZqwv1xMGuiC5DUBnGEoYXve4v2UJHAq3ETv/KcQFwwtiB whoSOqE23wn3V7FHgVKHw/k= X-Google-Smtp-Source: ABdhPJxaQmmLEHM7ANdGNftErjLIX5rvBjfDobBdrSIlAYMHLa7Cl2ln1mB1JU4/nV02/NmfUHZvSQ== X-Received: by 2002:a17:906:a24c:: with SMTP id bi12mr2887633ejb.530.1627535462284; Wed, 28 Jul 2021 22:11:02 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:01 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 63/87] trace-cmd dump: Print compression header Date: Thu, 29 Jul 2021 08:09:35 +0300 Message-Id: <20210729050959.12263-64-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Dump the compression header of trace file version 7 Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index e150a17d..e6f774bd 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -217,6 +217,29 @@ static void dump_initial_format(int fd) do_print(SUMMARY, "\t\t%d\t[Page size, bytes]\n", val4); } +static void dump_compress(int fd) +{ + char zname[DUMP_SIZE]; + char zver[DUMP_SIZE]; + + if (file_version < 7) + return; + + /* get compression header */ + if (read_file_string(fd, zname, DUMP_SIZE)) + die("no compression header"); + if (read_file_string(fd, zver, DUMP_SIZE)) + die("no compression version"); + do_print((SUMMARY), "\t\t%s\t[Compression algorithm]\n", zname); + do_print((SUMMARY), "\t\t%s\t[Compression version]\n", zver); + + if (strcmp(zname, "none")) { + compress = tracecmd_compress_alloc(zname, zver, fd, tep, NULL); + if (!compress) + die("cannot uncompress the file"); + } +} + static void dump_header_page(int fd) { unsigned long long size; @@ -715,6 +738,7 @@ static void dump_file(const char *file) do_print(SUMMARY, "\n Tracing meta data in file %s:\n", file); dump_initial_format(fd); + dump_compress(fd); dump_header_page(fd); dump_header_event(fd); dump_ftrace_events_format(fd); From patchwork Thu Jul 29 05:09:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97E6CC432BE for ; Thu, 29 Jul 2021 05:11:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 830976023E for ; Thu, 29 Jul 2021 05:11:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233875AbhG2FLH (ORCPT ); Thu, 29 Jul 2021 01:11:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234089AbhG2FLH (ORCPT ); Thu, 29 Jul 2021 01:11:07 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86B22C061757 for ; Wed, 28 Jul 2021 22:11:04 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id n2so6358113eda.10 for ; Wed, 28 Jul 2021 22:11:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zV0CqwA9/wXmtBv9QTZRvsjfkc0zh38RvP8876u1PMw=; b=juPpLBfqxhyo5QiJkI/jEhsbzzpa9XsvsUkgJ+WiccR6RC+Ikh2B7LyszG5QWHamTh 9McwSVQK1+AXNB67IolaE6tR5h3P/IG0lIHQgDMxs3i2JIwMQ/KWIe4kHKbUAYVLPOmt IqiuTXsHc6L6CC2jRrUr1VTdopSSSENaWgqcN27dNXAfHWTMm//5RgbvUcdX1u/SJ6cF EEvo8DD9hTt46iEYOEA+BoHTJAcJ8WFI3rK6lzghMXlQzPM483yH8mVGngKFCf1kh+Pp X80AF9hUJWArTHXw8qQjVXAgH+rdKpLAfPxusAdsb2k5TBSNAtSrI65GDa5oX1qiMsCa gTWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zV0CqwA9/wXmtBv9QTZRvsjfkc0zh38RvP8876u1PMw=; b=tXNCEPrRB6NobZTQdJ5QvD/Uew1KuXsJBYpZmz67DEyEaXOlo2PNJu6NWzTols/xXl 03ouYEttXExACOI5hAE8DC2qL3K/lvoqh/96BlvukUnJ0Mo9J4SQ1eDdbb8JN33y7ux+ 7g7O0bi2UlS48P5e+DRxh6ZSVpbZbdjzFAopNdITwY10VJx2ZxrNHHJflJgFF/XP/98m F/x9mqcvpM1VpuLQgBwQrOq/ldAymc86g7fZjClBwo45yhGOVUxFpfwQHG22LzC4VDRr 3V+JklAXl7jPXM4pRlWmfAuPGO5RDXLNTecsBA+65RN4LzdGq/fExjHHgPI46QJu+RQ7 ms5A== X-Gm-Message-State: AOAM533AQEWBBEYJh0y96S79WcEElV2hknxl2r7l7aWXht0Pi9CWcBTr V8f1gnu4nxfxB55VwvLWMIw= X-Google-Smtp-Source: ABdhPJxvzP6suW157fG0p4641eKb60QvOCfdT+7tCmizTQP8jEGB/eQ4fmNWFahVVQSssG/vjP0hEA== X-Received: by 2002:a50:ab1c:: with SMTP id s28mr3844633edc.214.1627535463228; Wed, 28 Jul 2021 22:11:03 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:02 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 64/87] trace-cmd dump: Add helpers for processing trace file sections Date: Thu, 29 Jul 2021 08:09:36 +0300 Message-Id: <20210729050959.12263-65-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a new local databse for storing sections, found in trace file version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index e6f774bd..010a8188 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -43,8 +43,18 @@ enum dump_items { OPTIONS = (1 << 9), FLYRECORD = (1 << 10), CLOCK = (1 << 11), + SECTIONS = (1 << 12), }; +struct file_section { + int id; + unsigned long long offset; + struct file_section *next; + enum dump_items verbosity; +}; + +static struct file_section *sections; + enum dump_items verbosity; #define DUMP_CHECK(X) ((X) & verbosity) @@ -723,6 +733,17 @@ static void dump_therest(int fd) } } +static void free_sections(void) +{ + struct file_section *del; + + while (sections) { + del = sections; + sections = sections->next; + free(del); + } +} + static void dump_file(const char *file) { int fd; @@ -748,7 +769,7 @@ static void dump_file(const char *file) dump_cmdlines(fd); dump_cpus_count(fd); dump_therest(fd); - + free_sections(); tep_free(tep); tep = NULL; close(fd); From patchwork Thu Jul 29 05:09:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDF92C4338F for ; Thu, 29 Jul 2021 05:11:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D5BB160EB2 for ; Thu, 29 Jul 2021 05:11:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234107AbhG2FLJ (ORCPT ); Thu, 29 Jul 2021 01:11:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41584 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234099AbhG2FLJ (ORCPT ); Thu, 29 Jul 2021 01:11:09 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D684C061757 for ; Wed, 28 Jul 2021 22:11:05 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id y7so4163391eda.5 for ; Wed, 28 Jul 2021 22:11:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n5T1nB9MQRSSRkYUuBZFkjXArsJT3nSCVMgCmD3VoXw=; b=AdLIQGtfWFGtjnNHqc6fiY4FT65BRff+k2XjaJuC5mrbefLYiPo32fOZOMBt5R8NUw u97vH1BiKUCUtGIE7xfJL3MFI3QjIFxib0RROUdw5Df2vE2k3U242ODnCCpJv9eftHHI 4t9pqh+IntvaIawyS35j1tCv8mki9yPIIK2FKXJBGEPBr265AnSlM73/+PTWcCsfcGvw pqVRDjbzTZ5pdoBoCj22/9G3LZNXdyD9yEhiK2moc2NGlHm/LEkk/RA6BxIkfc20+w+i W8fSzFu5FJLsym5waJNssrXGjYHqZi77hBK4TNrUTAQPUSxFzsCArKkW3FYQSnNoyqpK TrSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n5T1nB9MQRSSRkYUuBZFkjXArsJT3nSCVMgCmD3VoXw=; b=krQBYMCMPNXohPVFcprSYWsmVpRqyKIlLVaCTz6FgZ3E9UAXvKZ/9vG2MjCT3qxNzn 8esmt5IYwDLNoZawfSBEMZnSZRDKZluLXjESBpovbqvqg7ms4lkJUS/mfu1aK0XtMk48 EfiAVRc2SLeN72iqd9JSqduM8IVTsgvLH0W466rDd12AG8cesQgVq+s4JGRXkOKrcq3g RO5UysGBsc2aiusKAkSk4B+0NfKdmAenyzoJbGsxLLSQJ3pfpMmSsS6pfSXL6JP9pxvU TwFgrzxq7tc1FjRW2AE6Sqllt7lgi3/n1aHlzKgcU/uFO8aiIhhvMlAeal/h3SGDhSy2 7BNA== X-Gm-Message-State: AOAM531jygpX+GlCOU5Vy6d/wos90b5/7sNpRhs55xzlLUPAeUYZjOgv ZyyrLxF+3oVrP4IQhHfY+t0= X-Google-Smtp-Source: ABdhPJyzQMLwIwpFS0DDTkrPibijtGO/lp8GfKhZdWF2MLFoSOXUABEUkd96Tn3idxmamo248bL3pg== X-Received: by 2002:a05:6402:10d9:: with SMTP id p25mr3896917edu.51.1627535464242; Wed, 28 Jul 2021 22:11:04 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:03 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 65/87] trace-cmd dump: Read recursively all options sections Date: Thu, 29 Jul 2021 08:09:37 +0300 Message-Id: <20210729050959.12263-66-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 supports multiple options sections in the file. The DONE option is extended to hold the offset to the next options section. A new logic for handling the extended DONE option and to read recursively all options sections is added. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 76 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 68 insertions(+), 8 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 010a8188..f801e0b6 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -422,6 +422,30 @@ static void dump_option_string(int fd, int size, char *desc) read_dump_string(fd, size, OPTIONS); } +static void dump_section_header(int fd, enum dump_items v, unsigned short *flags) +{ + unsigned long long offset; + unsigned short fl; + char buf[DUMP_SIZE]; + unsigned short id; + int size; + + offset = lseek64(fd, 0, SEEK_CUR); + if (read_file_number(fd, &id, 2)) + die("cannot read the section id"); + if (read_file_string(fd, buf, DUMP_SIZE)) + die("no section description"); + if (read_file_number(fd, &fl, 2)) + die("cannot read the section flags"); + if (read_file_number(fd, &size, 4)) + die("cannot read section size"); + do_print(v, "\t[Section %d @ %lld: \"%s\", flags 0x%X, %d bytes]\n", + id, offset, buf, fl, size); + + if (flags) + *flags = fl; +} + static void dump_option_buffer(int fd, int size) { unsigned long long offset; @@ -591,25 +615,48 @@ void dump_option_tsc2nsec(int fd, int size) do_print(OPTIONS, "%d %d %llu [multiplier, shift, offset]\n", mult, shift, offset); } -static void dump_options(int fd) +static int dump_options_read(int fd); + +static int dump_option_done(int fd, int size) { + unsigned long long offset; + + do_print(OPTIONS, "\t\t[Option DONE, %d bytes]\n", size); + + if (file_version < 7 || size < 8) + return 0; + if (read_file_number(fd, &offset, 8)) + die("cannot read the next options offset"); + do_print(OPTIONS, "%lld\n", offset); + if (!offset) + return 0; + + if (lseek64(fd, offset, SEEK_SET) == (off_t)-1) + die("cannot goto next options offset %lld", offset); + return dump_options_read(fd); +} + +static int dump_options_read(int fd) +{ + unsigned short flags = 0; unsigned short option; unsigned int size; int count = 0; + if (file_version >= 7) + dump_section_header(fd, OPTIONS, &flags); + if ((flags & TRACECMD_SEC_FL_COMPRESS) && uncompress_block()) + die("cannot uncompress file block"); + for (;;) { if (read_file_number(fd, &option, 2)) die("cannot read the option id"); - if (!option) + if (option == TRACECMD_OPTION_DONE && file_version < 7) break; if (read_file_number(fd, &size, 4)) die("cannot read the option size"); count++; - if (!DUMP_CHECK(OPTIONS) && !DUMP_CHECK(CLOCK) && !DUMP_CHECK(SUMMARY)) { - lseek64(fd, size, SEEK_CUR); - continue; - } switch (option) { case TRACECMD_OPTION_DATE: dump_option_string(fd, size, "DATE"); @@ -621,7 +668,8 @@ static void dump_options(int fd) dump_option_buffer(fd, size); break; case TRACECMD_OPTION_TRACECLOCK: - dump_option_string(fd, size, "TRACECLOCK"); + do_print(OPTIONS, "\t\t[Option TRACECLOCK, %d bytes]\n", size); + read_dump_string(fd, size, OPTIONS | CLOCK); has_clock = 1; break; case TRACECMD_OPTION_UNAME: @@ -654,6 +702,10 @@ static void dump_options(int fd) case TRACECMD_OPTION_TSC2NSEC: dump_option_tsc2nsec(fd, size); break; + case TRACECMD_OPTION_DONE: + uncompress_reset(); + count += dump_option_done(fd, size); + return count; default: do_print(OPTIONS, " %d %d\t[Unknown option, size - skipping]\n", option, size); @@ -661,8 +713,16 @@ static void dump_options(int fd) break; } } - do_print(SUMMARY, "\t[%d options]\n", count); + uncompress_reset(); + return count; +} + +static void dump_options(int fd) +{ + int count; + count = dump_options_read(fd); + do_print(SUMMARY|OPTIONS, "\t[%d options]\n", count); } static void dump_latency(int fd) From patchwork Thu Jul 29 05:09:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 308F6C432BE for ; Thu, 29 Jul 2021 05:11:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1AC5B60FED for ; Thu, 29 Jul 2021 05:11:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234099AbhG2FLK (ORCPT ); Thu, 29 Jul 2021 01:11:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234089AbhG2FLJ (ORCPT ); Thu, 29 Jul 2021 01:11:09 -0400 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DDB1C061765 for ; Wed, 28 Jul 2021 22:11:06 -0700 (PDT) Received: by mail-ed1-x534.google.com with SMTP id j2so6364030edp.11 for ; Wed, 28 Jul 2021 22:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D2407/DndqS7KvrDlWm40krvCPS/afqFfjBRnfcuyBw=; b=rs0QNMjxdEAmjLiQtVTSAxJzh71XwZpw0+g4rtmvIQpojX6M2T57CGWqxzGnAb+zCD PYbzTdREquMe/iUJ8htCjwIOni9BcCJUsLMaY2oJ7hkAfdCEb2AJESAG3Xh0D5CfYguQ 0Xq4o5BT//DxY17EwGkP0xHu21M+RzBBkaGrsADDp4BkscVhgae60gqbQmAplJBmzQyX fecF6aLYvTygUn4WqvkX99qhtZiAOEgF9u4BkdOOqfjAgBznpP9YideA9UnzAtEmS+TP L8Wwd5nC92KYy+2dUjuFq6qPO9lrp28Fkx8rZRCmsZTYduY0lHvEJDrr/cjyLKE6K36c 0x1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D2407/DndqS7KvrDlWm40krvCPS/afqFfjBRnfcuyBw=; b=o8hAtVmXjagdDkLGVfUDf3Dai9BfOrOHo0tw1YEewTfTxvBlWhZ8lXcchTX/CUALW0 Od8CNyu4zFDKvfBklQhyilXaaf/9aB0wjM9MDL9BZ9zOb0u0MJi8LCwjBFYbwDk3tK5j PrQLhKz40x6f3ydbq0gLqUzXwOtSwT3XOu0aPcMboeUU9QpxI6VVJhlpxd/BrWwsPBi9 kAXhcaQ8K/5JXeZcx/LHARxWHMR3X8aWa47TYM1dK2Yc/tRpbiSnGbrlgVA651zjdLti Dfq+bAnjI3KOqImb94Rk2TMAApmnT5xKdNDvnjwd54ZIKxZlHaJcsel+anIg2AG1Zxiu F+8A== X-Gm-Message-State: AOAM530R9X0KA63ppBto7FBe+zFbOwFavviu2bJpEfQpuc9t1y5R/152 gaedFgIGc70QHnTJBLpBqq3Nd6meN8mCsg== X-Google-Smtp-Source: ABdhPJwrDXkBiV9pukkM0dH5rNpqOAQXdD++PJxb+56W6b/9KVA++7bamkLVCL4thxbskXNyzwiIeA== X-Received: by 2002:a05:6402:702:: with SMTP id w2mr3904150edx.149.1627535465158; Wed, 28 Jul 2021 22:11:05 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:04 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 66/87] trace-cmd dump: Read extended BUFFER option Date: Thu, 29 Jul 2021 08:09:38 +0300 Message-Id: <20210729050959.12263-67-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7 the BUFFER option is extended to hold a trace metadata, related to the recorded instance. Also, a new BUFFER_LAT option is added for latency trace data. Implemended logic for reading and printing these extended options. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 64 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 59 insertions(+), 5 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index f801e0b6..63d8bb81 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -446,19 +446,72 @@ static void dump_section_header(int fd, enum dump_items v, unsigned short *flags *flags = fl; } -static void dump_option_buffer(int fd, int size) +static void dump_option_buffer(int fd, unsigned short option, int size) { + unsigned long long total_size = 0; + unsigned long long data_size; + unsigned long long current; unsigned long long offset; + unsigned short flags; + char clock[DUMP_SIZE]; + char name[DUMP_SIZE]; + int cpus = 0; + int id; + int i; if (size < 8) die("broken buffer option with size %d", size); if (read_file_number(fd, &offset, 8)) die("cannot read the offset of the buffer option"); + if (read_file_string(fd, name, DUMP_SIZE)) + die("cannot read the name of the buffer option"); + if (file_version < 7) { + do_print(OPTIONS|FLYRECORD, "\t\t[Option BUFFER, %d bytes]\n", size); + do_print(OPTIONS|FLYRECORD, "%lld [offset]\n", offset); + do_print(OPTIONS|FLYRECORD, "\"%s\" [name]\n", name); + return; + } + + current = lseek64(fd, 0, SEEK_CUR); + if (lseek64(fd, offset, SEEK_SET) == (off_t)-1) + die("cannot goto buffer offset %lld", offset); + + dump_section_header(fd, FLYRECORD, &flags); + + if (lseek64(fd, current, SEEK_SET) == (off_t)-1) + die("cannot go back to buffer option"); + + do_print(OPTIONS|FLYRECORD, "\t\t[Option BUFFER, %d bytes]\n", size); + do_print(OPTIONS|FLYRECORD, "%lld [offset]\n", offset); + do_print(OPTIONS|FLYRECORD, "\"%s\" [name]\n", name); + + if (read_file_string(fd, clock, DUMP_SIZE)) + die("cannot read clock of the buffer option"); + do_print(OPTIONS|FLYRECORD, "\"%s\" [clock]\n", clock); + if (option == TRACECMD_OPTION_BUFFER) { + if (read_file_number(fd, &cpus, 4)) + die("cannot read the cpu count of the buffer option"); + + do_print(OPTIONS|FLYRECORD, "%d [CPUs]:\n", cpus); + for (i = 0; i < cpus; i++) { + if (read_file_number(fd, &id, 4)) + die("cannot read the id of cpu %d from the buffer option", i); + if (read_file_number(fd, &offset, 8)) + die("cannot read the offset of cpu %d from the buffer option", i); + if (read_file_number(fd, &data_size, 8)) + die("cannot read the data size of cpu %d from the buffer option", i); + total_size += data_size; + do_print(OPTIONS|FLYRECORD, " %d %lld\t%lld\t[id, data offset and size]\n", + id, offset, data_size); + } + do_print(SUMMARY, "\t\[buffer \"%s\", \"%s\" clock, " + "%d cpus, %lld bytes flyrecord data]\n", + name, clock, cpus, total_size); + } else { + do_print(SUMMARY, "\t\[buffer \"%s\", \"%s\" clock, latency data]\n", name, clock); + } - do_print(OPTIONS, "\t\t[Option BUFFER, %d bytes]\n", size); - do_print(OPTIONS, "%lld [offset]\n", offset); - read_dump_string(fd, size - 8, OPTIONS); } static void dump_option_int(int fd, int size, char *desc) @@ -665,7 +718,8 @@ static int dump_options_read(int fd) dump_option_string(fd, size, "CPUSTAT"); break; case TRACECMD_OPTION_BUFFER: - dump_option_buffer(fd, size); + case TRACECMD_OPTION_BUFFER_LAT: + dump_option_buffer(fd, option, size); break; case TRACECMD_OPTION_TRACECLOCK: do_print(OPTIONS, "\t\t[Option TRACECLOCK, %d bytes]\n", size); From patchwork Thu Jul 29 05:09:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06E58C4320A for ; Thu, 29 Jul 2021 05:11:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E40D060041 for ; Thu, 29 Jul 2021 05:11:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234089AbhG2FLK (ORCPT ); Thu, 29 Jul 2021 01:11:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234108AbhG2FLK (ORCPT ); Thu, 29 Jul 2021 01:11:10 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7866BC061757 for ; Wed, 28 Jul 2021 22:11:07 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id j2so6364089edp.11 for ; Wed, 28 Jul 2021 22:11:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zvljAPEL24RuPR31f9JTK/flseOCHo3TBjbEyZ3P9rc=; b=VL4bHeoUFz+o5SYpHdBbWKBEVyHQLZ6blK7olGjXqjmyliD08eadkIjovik/dS8RYr m0RxpKzV7XYbLVwvZBCZjf8xhxAFN//23gM0Ox6CUn2W7blhJO4yilYbStxoQgQAhXFY VR/bjcO1vDzfkzGWOAs6QnUgftLjRI0OqqEeGSylFgmVMOF+KSA56sqzEX8ENWpbc3Zp yCudqYgw3VQHRENp0zdvtk6kJu4T5pXe8Q8UqFy2OeJ/d4qZXAEXqJ004LMoS1WjjQrf Pk2eOFE9PgSAxDVE0Z75HsSNzybO5IArHiq1YP/LnuFZ2BjBiCEsdKGG+Tpeq6i5q0Nz AOnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zvljAPEL24RuPR31f9JTK/flseOCHo3TBjbEyZ3P9rc=; b=sw0cpQjfA9RVqIwkAz5v/gQzgDy3Bmk1fShXx8wUj3aMi+MKxbCnWxkJzOUmuvM4iN 7HlFJDUYvhUuGznjK1sCJPjdOLzQalVl0e8Y7tQVIa2yN3EgvEgd1Z3Izr49eJOm3xN0 H8SOWWOTkqsGTI6QmWcngDNc+rm+cxXA6iE4zQhbu/Elj24R0Kt3k3tqg08DpfDdeii/ bLCMPTeCh0bEeDwGAJINKl25OhvSrnhJD+B3p6kRJ8/Z1J329ThBEHq0u50tllilERPx VaUdGtd2+8c9PC+iVEEZECo+uavFhaqpTc+spzrjqgGypI1KLpg3utVs7+dC0U/aKMSp EQzg== X-Gm-Message-State: AOAM533ybmb3MYrlxG8DUux212VgJRCHIqMORpqJErZpFycd7GM7+DJg r1b3r0rIPtIwhE1iNNGAtaA= X-Google-Smtp-Source: ABdhPJyToayfr3/oWMjQStkH00PPzPSMFubjlUOZ8hhUpav6B+KroZb3k6f3xhe0ZEWJ5YihAjKVsw== X-Received: by 2002:aa7:c603:: with SMTP id h3mr3854676edq.165.1627535466117; Wed, 28 Jul 2021 22:11:06 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:05 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 67/87] trace-cmd dump: Dump sections Date: Thu, 29 Jul 2021 08:09:39 +0300 Message-Id: <20210729050959.12263-68-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, headers are stored as sections. This allows their position in the file not to be fixed. Poters to these sections are stored as trace options. Add logic to handle these new options: HEADER_INFO FTRACE_EVENTS EVENT_FORMATS KALLSYM PRINTK CMDLINES Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 63d8bb81..571b6ac2 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -668,6 +668,24 @@ void dump_option_tsc2nsec(int fd, int size) do_print(OPTIONS, "%d %d %llu [multiplier, shift, offset]\n", mult, shift, offset); } +static void dump_option_section(int fd, unsigned int size, + unsigned short id, char *desc, enum dump_items v) +{ + struct file_section *sec; + + sec = calloc(1, sizeof(struct file_section)); + if (!sec) + die("cannot allocate new section"); + sec->next = sections; + sections = sec; + sec->id = id; + sec->verbosity = v; + if (read_file_number(fd, &sec->offset, 8)) + die("cannot read the option %d offset", id); + do_print(OPTIONS, "\t\t[Option %s, %d bytes]\n", desc, size); + do_print(OPTIONS, "%lld\n", sec->offset); +} + static int dump_options_read(int fd); static int dump_option_done(int fd, int size) @@ -756,6 +774,25 @@ static int dump_options_read(int fd) case TRACECMD_OPTION_TSC2NSEC: dump_option_tsc2nsec(fd, size); break; + case TRACECMD_OPTION_HEADER_INFO: + dump_option_section(fd, size, option, "HEADERS", HEAD_PAGE | HEAD_EVENT); + break; + case TRACECMD_OPTION_FTRACE_EVENTS: + dump_option_section(fd, size, option, "FTRACE EVENTS", FTRACE_FORMAT); + break; + case TRACECMD_OPTION_EVENT_FORMATS: + dump_option_section(fd, size, option, + "EVENT FORMATS", EVENT_SYSTEMS | EVENT_FORMAT); + break; + case TRACECMD_OPTION_KALLSYMS: + dump_option_section(fd, size, option, "KALLSYMS", KALLSYMS); + break; + case TRACECMD_OPTION_PRINTK: + dump_option_section(fd, size, option, "PRINTK", TRACE_PRINTK); + break; + case TRACECMD_OPTION_CMDLINES: + dump_option_section(fd, size, option, "CMDLINES", CMDLINES); + break; case TRACECMD_OPTION_DONE: uncompress_reset(); count += dump_option_done(fd, size); From patchwork Thu Jul 29 05:09:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCA40C4338F for ; Thu, 29 Jul 2021 05:11:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B03396023E for ; Thu, 29 Jul 2021 05:11:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234117AbhG2FLM (ORCPT ); Thu, 29 Jul 2021 01:11:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234108AbhG2FLL (ORCPT ); Thu, 29 Jul 2021 01:11:11 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6949FC061757 for ; Wed, 28 Jul 2021 22:11:08 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id e19so8426787ejs.9 for ; Wed, 28 Jul 2021 22:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DZv6utpTkIgjWEWaVEGim6LglWLWmt2byg65wBCXWFc=; b=tsBub1ewL+ZXbZyPHeXyvGSJHG2omqYOgU91cXQuKzDCwAaA1tN8w4jL8bKb/PcSEb YKqiSyzCFYlq2aXMQLHc+PXU0otvCY9Z5xGGdf82jLZLDKSeDcud4fmA2eKeD5n/eAP5 jVoriSpluSY7fhvRu6pyRqomMjyiz/U2ivaDTxYBRYcXjU1BaXbayb/h2wjBJtC6/GeP /XVeQoNhelB/7oFy7CXK7szXYaHy3FqHbX2bACzSLcUK+eA0Q0BqbATRnHE145y/ta5q 1pbrRF+7tW8W1InOCrRrS/NjwBCnHEW2s/RS434ilLSdXpkV8p2hYZEEgwtuNR42vGk5 bPJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DZv6utpTkIgjWEWaVEGim6LglWLWmt2byg65wBCXWFc=; b=J6sTA8/oKVhKT2LCxU1lM6m8IR/VRy5PS2Rk5mEjYmYKIvgl1rrYFrQ9f314GCK9MA 1XarJeL3bYToZ8J3j2C+K9UcRCXAnGoD975Xn/lXmCek5QN865A1gamSV8NxGTLxHxB2 cJ0yxLwKpAidp+99l9y+OZPeG7C/tunvx+bKmIeWzD/cxnHnhYwRmkX4qUFXSM6+kxVr p8NUZRw2WWLAM0BZVQGUFfLEBH4yeSTvwtuoBNvpq4ETF/D8p8ym0NFxrkqsf0oXBjDS OuZ9fnaGo2/cndlbqvN9gjr8FmjhYA0bxDFzGbuR3aKA+vNbT5C+CZ20/RbOsfzJqTG6 XYmQ== X-Gm-Message-State: AOAM530gxJraQUUEvSNAcJi8dwuXZESzIbtEQcJ27wJl20NPqtdntFRv Ho0BCCjDImRjsocizE9cl2loaTKfWqU/Lg== X-Google-Smtp-Source: ABdhPJw8YFvOEAmxFA2miRoucu4Aeg/OYGC+xMR7SOFSNCz2oXt9/8pBufyLxsYNRs5ni49ACAFMCw== X-Received: by 2002:a17:906:e241:: with SMTP id gq1mr3039139ejb.87.1627535467098; Wed, 28 Jul 2021 22:11:07 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:06 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 68/87] trace-cmd dump: Dump trace file version 7 Date: Thu, 29 Jul 2021 08:09:40 +0300 Message-Id: <20210729050959.12263-69-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file version 7 has different structure than version 6. Separates parsing both versions in different flows. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 571b6ac2..9088d2f5 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -884,6 +884,31 @@ static void dump_therest(int fd) } } +static void dump_v6_file(int fd) +{ + dump_header_page(fd); + dump_header_event(fd); + dump_ftrace_events_format(fd); + dump_events_format(fd); + dump_kallsyms(fd); + dump_printk(fd); + dump_cmdlines(fd); + dump_cpus_count(fd); + dump_therest(fd); +} + +static void dump_v7_file(int fd) +{ + long long offset; + + if (read_file_number(fd, &offset, 8)) + die("cannot read offset of the first option section"); + + if (lseek64(fd, offset, SEEK_SET) == (off64_t)-1) + die("cannot goto options offset %lld", offset); + dump_options(fd); +} + static void free_sections(void) { struct file_section *del; @@ -911,15 +936,10 @@ static void dump_file(const char *file) dump_initial_format(fd); dump_compress(fd); - dump_header_page(fd); - dump_header_event(fd); - dump_ftrace_events_format(fd); - dump_events_format(fd); - dump_kallsyms(fd); - dump_printk(fd); - dump_cmdlines(fd); - dump_cpus_count(fd); - dump_therest(fd); + if (file_version < 7) + dump_v6_file(fd); + else + dump_v7_file(fd); free_sections(); tep_free(tep); tep = NULL; From patchwork Thu Jul 29 05:09:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E492C432BE for ; Thu, 29 Jul 2021 05:11:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E47A36023E for ; Thu, 29 Jul 2021 05:11:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234108AbhG2FLM (ORCPT ); Thu, 29 Jul 2021 01:11:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234112AbhG2FLM (ORCPT ); Thu, 29 Jul 2021 01:11:12 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 701D6C061765 for ; Wed, 28 Jul 2021 22:11:09 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id gn26so8489208ejc.3 for ; Wed, 28 Jul 2021 22:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TVer/DT+huronqlCp2uYINmtsbXFUc4Q94qjFRQGwgc=; b=M9p2aSvzFjz7AgVViRgHd15hZqVllAIRBUSENR8gHltTR0P750Knjpai1KceW3jsiQ Eoj0hNAoo5Ct892RqETqHeCExkT2/aJ7TRhpoD3kjBMAAVMnpVkI1MVns38ywc+PTgT/ 78OkLabU+yoiXiLI5+3ij/EObIDFWEwUcCN0Ly9wLigBBWBU7HPOCFGFvN8yThr1WIUT IsU34hCGY9DaFWVR8Rutyw5dA0yW1S0wYovSK3efspLcNzmZDO+mSFN3tXrZAGrf3rGe 1rqSjypoGCBV01D9kRTZdcJ0Q/a9zhQRZ7+lajuUdBY7N2Ap40B7lJ+hoAAV+3V1rHZZ qCZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TVer/DT+huronqlCp2uYINmtsbXFUc4Q94qjFRQGwgc=; b=jjdq2pN0tPJXHWGwukthkV5iitNwu1qMqnY1sVs92ApX4GXqC411P/0nYL+AAYypR1 A06993YgTXttd5YwNGFz3JDbWnr2M2wSlqE42nsVOUOf/wju9DDEGbv0JsXLL9aa3MNt Tpz8h01gknihFW+V/lusvK7bYE9c4tWC6liGLLRl9Gt3PkI8pVu/4Jg5HwDKgYLg1aXu p2PHGW4r7aKSpO6+M7U9I2sWYaqgbTUHRzVbzcXOHtgoi+42X2dL33o1kdk6jphmUzem jES0u0fwknVWJGhpYIgjCxOg8Ole2vI5MXkAsoTZ/ftjgzBc1q/zmy+TLCN0r/Znp5T/ pmgQ== X-Gm-Message-State: AOAM532wyhNcYGmJdd0yiB7FQsyKSTiALkbtWUcHNKCMHutJAImZdR5f UbQjG5P2KV6tWocRajAl40orffkSrrPyxA== X-Google-Smtp-Source: ABdhPJygFbkZ8z/D6YRpjbvlSv0emaotLS77/3ftJvfcQ2EFInaFToLvrkQny/LvOlBt9RMXGE2xVQ== X-Received: by 2002:a17:906:1703:: with SMTP id c3mr2930665eje.64.1627535468131; Wed, 28 Jul 2021 22:11:08 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:07 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 69/87] trace-cmd dump: Dump sections content Date: Thu, 29 Jul 2021 08:09:41 +0300 Message-Id: <20210729050959.12263-70-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org In trace file version 7, dump the content of the headers content, that are stored as sections. Added logic to dump these headers: HEADER_INFO FTRACE_EVENTS EVENT_FORMATS KALLSYM PRINTK CMDLINES Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 9088d2f5..1dcc8c7c 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -686,6 +686,46 @@ static void dump_option_section(int fd, unsigned int size, do_print(OPTIONS, "%lld\n", sec->offset); } +static void dump_sections(int fd) +{ + struct file_section *sec = sections; + unsigned short flags; + + while (sec) { + if (lseek64(fd, sec->offset, SEEK_SET) == (off_t)-1) + die("cannot goto option offset %lld", sec->offset); + + dump_section_header(fd, sec->verbosity, &flags); + + if ((flags & TRACECMD_SEC_FL_COMPRESS) && uncompress_block()) + die("cannot uncompress section block"); + + switch (sec->id) { + case TRACECMD_OPTION_HEADER_INFO: + dump_header_page(fd); + dump_header_event(fd); + break; + case TRACECMD_OPTION_FTRACE_EVENTS: + dump_ftrace_events_format(fd); + break; + case TRACECMD_OPTION_EVENT_FORMATS: + dump_events_format(fd); + break; + case TRACECMD_OPTION_KALLSYMS: + dump_kallsyms(fd); + break; + case TRACECMD_OPTION_PRINTK: + dump_printk(fd); + break; + case TRACECMD_OPTION_CMDLINES: + dump_cmdlines(fd); + break; + } + uncompress_reset(); + sec = sec->next; + } +} + static int dump_options_read(int fd); static int dump_option_done(int fd, int size) @@ -907,6 +947,7 @@ static void dump_v7_file(int fd) if (lseek64(fd, offset, SEEK_SET) == (off64_t)-1) die("cannot goto options offset %lld", offset); dump_options(fd); + dump_sections(fd); } static void free_sections(void) From patchwork Thu Jul 29 05:09:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19484C4320A for ; Thu, 29 Jul 2021 05:11:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 00E1360C3F for ; Thu, 29 Jul 2021 05:11:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233890AbhG2FLO (ORCPT ); Thu, 29 Jul 2021 01:11:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234118AbhG2FLO (ORCPT ); Thu, 29 Jul 2021 01:11:14 -0400 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E1ECC061757 for ; Wed, 28 Jul 2021 22:11:10 -0700 (PDT) Received: by mail-ed1-x52c.google.com with SMTP id ec13so5882968edb.0 for ; Wed, 28 Jul 2021 22:11:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vJmKQ63VIuDUmFCoX0jEknSgKUTSo3zO+2bQdFYAQlE=; b=mSh9yDC+sNImTlZxtLy4e0eQ+JLk7pMxhKbXlqYdxXU4volYglZIfBxYEsTFMW91y/ FGUkBBs5awl7+7OPtvBBOrL8aewWio5oy5Z38H4cU6eQtqI+PFj5RlmhkiRtEvbobU/e LRInjVIIJyz8Y1X9B2RlDsl207mMp58I7D9z19PguJg8Gtctg9p2ScIbJFTedrAd3UAA WPJOwAGlavmDzSo6cF1fHcXkXf00hA+kRHgvQV6L3l0hoOEpHnS/OkhMoile1r7GjP55 UVrHUPsQnPw/zT3N1jnHbHUGF3pbbJk7XbMN7Jn6VshaJthDcqARpQLlXz211OaUoVcC wUdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vJmKQ63VIuDUmFCoX0jEknSgKUTSo3zO+2bQdFYAQlE=; b=gtgObstfhzNdcihtPCRF3ONbBtWhHfuCnX6G/rZPgY9nrayIs3xVmRhKrXwjZq+Npp 0RyTVK4O3e8xaR5mj0pZFCHnNuWVf8xV91u8DSVmPVBlsgfzkGhRAhGo69Dhi1Ng5JOW 4soKl//70I/lmlauho9Dhw9d362RpnWz8uIx7ey++dhn+Z9t92Vn6xFfnjGyU3fP8faV JdTe3sgO5Ac2uph4wq9FeSs6XuLp8lULChg88Zo0DZv6cm8PHl4IUKtwmDeHYumrWhDc +u5f76zhpnVhIRKjYFnQv5gCx+Lp3H89103UBaNjfQ0Sm/+VwLkHqp8ksjTYAk1jnLvW bcBw== X-Gm-Message-State: AOAM531rtE+HpseRSrbmQOwE8ixdAqQebdZ1iRrqjUK3GhJpF9bTxWQb hme3kNgBuyfRe/i+yZZylfZ4Wr75X/y7hw== X-Google-Smtp-Source: ABdhPJwIUgwwDUpwMO+GuFwCWEm+7qukYhjlNzBI5VhhJS1ZCznLX6cvnaV4KOnISLhE2UiA1R+vrw== X-Received: by 2002:a50:f615:: with SMTP id c21mr4060491edn.146.1627535469065; Wed, 28 Jul 2021 22:11:09 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:08 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 70/87] trace-cmd dump: Add new argument --sections Date: Thu, 29 Jul 2021 08:09:42 +0300 Message-Id: <20210729050959.12263-71-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The new "trace-cmd dump --sections" argument walks through the sections from a trace file version 7 and prints their headers. The logic does not rely on the options, describing these sections. It could be useful to dump broken or incomplete trace files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 71 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 68 insertions(+), 3 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 1dcc8c7c..f966212a 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -686,7 +686,7 @@ static void dump_option_section(int fd, unsigned int size, do_print(OPTIONS, "%lld\n", sec->offset); } -static void dump_sections(int fd) +static void dump_sections(int fd, int count) { struct file_section *sec = sections; unsigned short flags; @@ -724,6 +724,7 @@ static void dump_sections(int fd) uncompress_reset(); sec = sec->next; } + do_print(SUMMARY|SECTIONS, "\t[%d sections]\n", count); } static int dump_options_read(int fd); @@ -937,17 +938,76 @@ static void dump_v6_file(int fd) dump_therest(fd); } +static int walk_v7_sections(int fd) +{ + unsigned long long offset, soffset; + unsigned short fl; + char buf[DUMP_SIZE]; + unsigned short id; + int size, csize, rsize; + int count = 0; + + offset = lseek64(fd, 0, SEEK_CUR); + do { + soffset = lseek64(fd, 0, SEEK_CUR); + if (read_file_number(fd, &id, 2)) + break; + if (id >= TRACECMD_OPTION_MAX) + die("Unknown section id %d", id); + if (read_file_string(fd, buf, DUMP_SIZE)) + die("cannot read section description"); + if (read_file_number(fd, &fl, 2)) + die("cannot read section flags"); + if (read_file_number(fd, &size, 4)) + die("cannot read section size"); + if (size <= 4) + die("Section %d (%s) is too small, %d bytes", id, buf, size); + count++; + if (fl & TRACECMD_SEC_FL_COMPRESS) { + if (id == TRACECMD_OPTION_BUFFER || + id == TRACECMD_OPTION_BUFFER_LAT) { + do_print(SECTIONS, + "\t[Section %2d @ %-16lld\t\"%s\", flags 0x%X, " + "%d compressed bytes]\n", + id, soffset, buf, fl, size); + } else { + if (read_file_number(fd, &csize, 4)) + die("cannot read section size"); + if (read_file_number(fd, &rsize, 4)) + die("cannot read section size"); + do_print(SECTIONS, "\t[Section %2d @ %-16lld\t\"%s\", flags 0x%X, " + "%d compressed, %d uncompressed]\n", + id, soffset, buf, fl, csize, rsize); + size -= 8; + } + } else { + do_print(SECTIONS, "\t[Section %2d @ %-16lld\t\"%s\", flags 0x%X, %d bytes]\n", + id, soffset, buf, fl, size); + } + + if (lseek64(fd, size, SEEK_CUR) == (off_t)-1) + break; + } while (1); + + if (lseek64(fd, offset, SEEK_SET) == (off_t)-1) + die("cannot restore the original file location"); + return count; +} + static void dump_v7_file(int fd) { long long offset; + int sections; if (read_file_number(fd, &offset, 8)) die("cannot read offset of the first option section"); - if (lseek64(fd, offset, SEEK_SET) == (off64_t)-1) + sections = walk_v7_sections(fd); + + if (lseek64(fd, offset, SEEK_SET) == (off_t)-1) die("cannot goto options offset %lld", offset); dump_options(fd); - dump_sections(fd); + dump_sections(fd, sections); } static void free_sections(void) @@ -988,6 +1048,7 @@ static void dump_file(const char *file) } enum { + OPT_sections = 241, OPT_verbose = 242, OPT_clock = 243, OPT_all = 244, @@ -1031,6 +1092,7 @@ void trace_dump(int argc, char **argv) {"options", no_argument, NULL, OPT_options}, {"flyrecord", no_argument, NULL, OPT_flyrecord}, {"clock", no_argument, NULL, OPT_clock}, + {"sections", no_argument, NULL, OPT_sections}, {"validate", no_argument, NULL, 'v'}, {"help", no_argument, NULL, '?'}, {"verbose", optional_argument, NULL, OPT_verbose}, @@ -1094,6 +1156,9 @@ void trace_dump(int argc, char **argv) if (trace_set_verbose(optarg) < 0) die("invalid verbose level %s", optarg); break; + case OPT_sections: + verbosity |= SECTIONS; + break; default: usage(argv); } From patchwork Thu Jul 29 05:09:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6F46C4338F for ; Thu, 29 Jul 2021 05:11:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B4CE760C3F for ; Thu, 29 Jul 2021 05:11:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234122AbhG2FLO (ORCPT ); Thu, 29 Jul 2021 01:11:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234112AbhG2FLO (ORCPT ); Thu, 29 Jul 2021 01:11:14 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EAD2C061765 for ; Wed, 28 Jul 2021 22:11:11 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id j2so6364251edp.11 for ; Wed, 28 Jul 2021 22:11:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j2lbA4dZN38akh4DzP6Jp1yJLAlJkBdDhjexsLTpKic=; b=sp+CMudc+51Sf3P/QQF5QJNwATyv8qjhZjt6soRNrWcv8HmeDuTyvalrJ+zfkgwWJM 5uw52waT3EeM17DlKvs+zlx7VJCzuKkkBkVln8PyWTrs0rERSPpSRRZUSal5+oF4wF4j WVAYSXRZ52npNcYT8S6B8Umvi5MLMfBOvodnlOYj+p4G78BKl5wxxrceJWY3ovIK5bEH dYsAS8fOvDs08/pBAskQLQu0lJC+AL4MtRd4x0QYDPKfcbFMucQYCj51ObTH7VPEWWGp jMCIYE1Q0gXPs7p//1NVdcqEkq/1ibH3HU3Wa7lAv4EzVb7wPJsMYaW1kxcR9Wgx0fov CdnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j2lbA4dZN38akh4DzP6Jp1yJLAlJkBdDhjexsLTpKic=; b=XxkZLLLAuuxLsZmdXq7dTVl7wFA251st5j9A8S/tv6TzCr+JqlzQ5IoK8ze0dfLs+M j5BMppa7EZHCmvXORwL4xbTcPkzlL+vjRIEzCnTDfD4ujUlo0k5bxq5i9kvvrrWKegpv GitAYE+jtKGn5k5/JCy2bySxUNSLSwKh/VjqxZaCMXMKvaS1jxFu8C1cv9M0zcM9bc3J ARtdmkOcKFhcLPILHWV8+fvG4pSBJynu0QbitzxDQRAET+3a2zhtpuaBxecNpV5HPffB S7o19audMoayloAGHfYGrBWkIgs8/wiyP4vQ3Zwn3f7YVPeR6fCWuFFkk16h7n+7zGiH HfUw== X-Gm-Message-State: AOAM531HqbQtrVc7pNYREABuYkYyv7IXOOLjZQbC3JFmvnI3+5K5c/kD aIWKOeQ7XqjhNNvbaX7NTzI= X-Google-Smtp-Source: ABdhPJzeizNHVLUTRILob535Iuahn+5USpMhyKPKxlvsGrFhAGKJWVO3HH/fq+U02OwXIcEVdK18cQ== X-Received: by 2002:a05:6402:487:: with SMTP id k7mr3821320edv.315.1627535470019; Wed, 28 Jul 2021 22:11:10 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:09 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 71/87] trace-cmd dump: Align better the output of flyrecord dump Date: Thu, 29 Jul 2021 08:09:43 +0300 Message-Id: <20210729050959.12263-72-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The flyrecord dump prints various file offsets and sizes, that can be huge numbers. Add format arguments to align better the output. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index f966212a..7c390022 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -898,7 +898,7 @@ static void dump_flyrecord(int fd) die("cannot read the cpu %d offset", i); if (read_file_number(fd, &cpu_size, 8)) die("cannot read the cpu %d size", i); - do_print(FLYRECORD, "\t\t %lld %lld\t[offset, size of cpu %d]\n", + do_print(FLYRECORD, "\t %10.lld %10.lld\t[offset, size of cpu %d]\n", cpu_offset, cpu_size, i); } dump_clock(fd); From patchwork Thu Jul 29 05:09:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90610C432BE for ; Thu, 29 Jul 2021 05:11:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 755876023E for ; Thu, 29 Jul 2021 05:11:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234118AbhG2FLQ (ORCPT ); Thu, 29 Jul 2021 01:11:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234112AbhG2FLQ (ORCPT ); Thu, 29 Jul 2021 01:11:16 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66BFCC061757 for ; Wed, 28 Jul 2021 22:11:12 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id y12so6393085edo.6 for ; Wed, 28 Jul 2021 22:11:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=T0zLZZKPmpL25tbLAra5wuwV9NpHnucnTUscDRY9rPc=; b=ri+nAaYHkTrWjkyu0tv/IewltlWlDaLtlA3qODjligNy7yKh4csnjcXrwNVh9glmIh 0vsO69gzhgpYzdlJ/nR1Br3E9PWYlgF0vdeK73ku+SZvn/36NG0pR7XX7O/8KmfvF+AZ vjqMSyP/ReCl1LJNU05AkRuqrQGR7nbJAJqUCxW7G9+vg27AYdaXvyapupau9o3CFxNv EoHGUzNyRroLoHxzqq8GT94+6e6e4pCNgu91gsUFysOUP2+2TJlrILOD03SzxCHKjjWK dREWI46tNY3OG1OrKxO5pQ0VX3sJKAsVLqdDdYHtef7E9EEMAHETy7SQsAJwQao05err nwfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T0zLZZKPmpL25tbLAra5wuwV9NpHnucnTUscDRY9rPc=; b=fndpR8iOxCRuWWBv34/z38monMDVZ9Lo1DF0KK9fechxMvjMbmVacgPbu2LePlMcMV LIcFNwZl7pkXjFMGCwEZqyc596LC8BEuuNIAQGpOXSVh8AglSHiuHYsXgtxZhYaoxEkC oM7HUZzUWv83h6xBIRJRVMn2YsVZMM2PyWz/4cqDUsjHKXBGF3dh07WoHbNVen7/Ypn/ zgz3mk+rSdCNs7+3kAf+KdUCOffqIrVITe7Nvumin/rfk8pGssmw+E8aHehxXQ44WtXU ubBUgovbixjdm1IDF597uteUG4dey4h0xvoZsRHR3+w9r1x5xbkZilexNT2Dun9HbK+O CFuQ== X-Gm-Message-State: AOAM530zY2VySFKNsim4KjfSKYdnyMkTPa0o5PpSxXDwAp0YkP11JZUH D0bOtAQBicV+bgiwPNtuudqWcjTSby2z2A== X-Google-Smtp-Source: ABdhPJxAh9Mxbff2f2S5SR7cPxowYu4VpBiGPoy/6S3rf9UO1Qs6TIpWHfk6FOgYsgvneAiPzwwEOA== X-Received: by 2002:a05:6402:896:: with SMTP id e22mr4001459edy.246.1627535470974; Wed, 28 Jul 2021 22:11:10 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:10 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 72/87] trace-cmd library: Add zlib compression algorithm Date: Thu, 29 Jul 2021 08:09:44 +0300 Message-Id: <20210729050959.12263-73-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added implementation for zlib compression algorithm, using libz libraray, if available. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Makefile | 7 ++ lib/trace-cmd/Makefile | 7 ++ lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-compress-zlib.c | 109 ++++++++++++++++++++++++ tracecmd/Makefile | 4 + 5 files changed, 131 insertions(+) create mode 100644 lib/trace-cmd/trace-compress-zlib.c diff --git a/Makefile b/Makefile index d6fd09d8..2bc325dc 100644 --- a/Makefile +++ b/Makefile @@ -311,6 +311,13 @@ ifeq ($(PERF_DEFINED), 1) CFLAGS += -DPERF endif +ZLIB_INSTALLED := $(shell if (printf "$(pound)include \n void main(){deflateInit(NULL, Z_BEST_COMPRESSION);}" | $(CC) -o /dev/null -x c - -lz >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) +ifeq ($(ZLIB_INSTALLED), 1) +export ZLIB_INSTALLED +CFLAGS += -DHAVE_ZLIB +$(info Have zlib compression support) +endif + CUNIT_INSTALLED := $(shell if (printf "$(pound)include \n void main(){CU_initialize_registry();}" | $(CC) -o /dev/null -x c - -lcunit >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) export CUNIT_INSTALLED diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index bab4322d..1ee6ac4c 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -26,6 +26,9 @@ OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif OBJS += trace-compress.o +ifeq ($(ZLIB_INSTALLED), 1) +OBJS += trace-compress-zlib.o +endif # Additional util objects OBJS += trace-blk-hack.o @@ -47,6 +50,10 @@ $(LIBTRACECMD_STATIC): $(OBJS) LIBS = $(LIBTRACEEVENT_LDLAGS) $(LIBTRACEFS_LDLAGS) -lpthread +ifeq ($(ZLIB_INSTALLED), 1) +LIBS += -lz +endif + $(LIBTRACECMD_SHARED_VERSION): $(LIBTRACECMD_SHARED) @ln -sf $( + * + */ +#include +#include +#include +#include + +#include "trace-cmd-private.h" + +#define __ZLIB_NAME "zlib" +#define __ZLIB_WEIGTH 10 + +static int zlib_compress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + ret = compress2((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes, Z_BEST_COMPRESSION); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_STREAM_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static int zlib_decompress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + ret = uncompress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_DATA_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static unsigned int zlib_compress_bound(unsigned int in_bytes) +{ + return compressBound(in_bytes); +} + +static bool zlib_is_supported(const char *name, const char *version) +{ + const char *zver; + + if (!name) + return false; + if (strlen(name) != strlen(__ZLIB_NAME) || strcmp(name, __ZLIB_NAME)) + return false; + + if (!version) + return true; + + zver = zlibVersion(); + if (!zver) + return false; + + /* Compare the major version number */ + if (atoi(version) <= atoi(zver)) + return true; + + return false; +} + +int tracecmd_zlib_init(void) +{ + return tracecmd_compress_proto_register(__ZLIB_NAME, zlibVersion(), __ZLIB_WEIGTH, + zlib_compress, zlib_decompress, + zlib_compress_bound, zlib_is_supported); +} diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 80c69bbb..b7a23dc4 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -51,6 +51,10 @@ CONFIG_INCLUDES = CONFIG_LIBS = -lrt -lpthread $(TRACE_LIBS) CONFIG_FLAGS = +ifeq ($(ZLIB_INSTALLED), 1) +CONFIG_LIBS += -lz +endif + all: $(TARGETS) $(bdir): From patchwork Thu Jul 29 05:09:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D72AEC4338F for ; Thu, 29 Jul 2021 05:11:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF22F60C3F for ; Thu, 29 Jul 2021 05:11:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234112AbhG2FLR (ORCPT ); Thu, 29 Jul 2021 01:11:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233798AbhG2FLR (ORCPT ); Thu, 29 Jul 2021 01:11:17 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41CF9C061765 for ; Wed, 28 Jul 2021 22:11:13 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id p21so6366820edi.9 for ; Wed, 28 Jul 2021 22:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZeqSXhB1scM/gYsw5n4fAEvLMCdkuPXcrQGTPYDsIC0=; b=p5inOddp/IpPceldirKj99nnEmnSV5GHhUwtiMYwjHW0MFQhoTG+n9COIwqCMmjr4V 4V2nA4jqbCgzxA1a+imX+iPze//KZHXYN1wza6paEEd4GV58u0/yn/TZ1kCnJHZqsVow uj1NZaZ1gE6fxvg27F8IlHvksJtrCNMyClxXasDGX2wxWXC7hCzC+9uP7utBP/ieKdRX veQsb2PjfLuOOnqh4wEi5xGDKb8lie5wp29a5EzIiHtVKPW97P8swCmCbwBAzrx3zlFd 3IJ2ByrJEiGnJqxmIkuazN+7dgZb71wcyEyq6vKODg4EZ8y1tMx3PW2MS4mmTm2/zVEC X0hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZeqSXhB1scM/gYsw5n4fAEvLMCdkuPXcrQGTPYDsIC0=; b=d5v5YINDKhmfDzmb4Mj6J6klUUxphEu7VpcEeslJzAYX2db4Xr6MXA3ptfZQBMNqqZ EN+PTowPfs57FdSUpntQaliOCMt5WDBrMEu3IivDnXpWTLJ1sXGG5f0cqvuyV2QFGg9u s3e7uBfeHWDunXQgFLM9Bu8+Smfp0cluk+k9Mfi+DTIc3uMQkxsl8pb3edvVmNxY9Fu1 1XxcppsMMVmhSuIMn6k6Hnwo6Fm85H117zp+viDHttDIVZAmrjRjCF6jyI9BbqXAKlNp x3O9uTEfrA6WLiFFEF3Z6fSSF6r2HPtuOK+/ZVDWo3sK+di8AZIqpZEkS8Qcgl1zZQhG EoXQ== X-Gm-Message-State: AOAM531b4uXua5HVKooy5U0oOCTtQVxoLxThRuQJ+cbMxxpAZHyJMH9g 9oPEL2VTljfRYfm8HbKVoIA= X-Google-Smtp-Source: ABdhPJwbiI4iUwJyRvikTDExvSZdzLgzuXdFTJ1hMd4VLGctbcS3ewkuc68LZErxCWHC9LuI0SdgaQ== X-Received: by 2002:a50:f1c7:: with SMTP id y7mr3962441edl.386.1627535471899; Wed, 28 Jul 2021 22:11:11 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:11 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 73/87] trace-cmd library: Reuse local function that writes to output handler Date: Thu, 29 Jul 2021 08:09:45 +0300 Message-Id: <20210729050959.12263-74-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Reuse do_write_check() internal helper function in the trace-cmd library. It should be used when writing data to an output handler, instead of writing to a fd, as it handles compression and network contexts. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index f8a335d4..c73b37a6 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -59,6 +59,8 @@ out_write_section_header(struct tracecmd_output *handle, unsigned short header_i char *description, enum tracecmd_section_flags flags, bool option); int out_update_section_header(struct tracecmd_output *handle, unsigned long long offset); +long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); + struct cpu_data_source { int fd; int size; From patchwork Thu Jul 29 05:09:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A221EC432BE for ; Thu, 29 Jul 2021 05:11:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CCB160C3F for ; Thu, 29 Jul 2021 05:11:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234123AbhG2FLS (ORCPT ); Thu, 29 Jul 2021 01:11:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233798AbhG2FLS (ORCPT ); Thu, 29 Jul 2021 01:11:18 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55754C061757 for ; Wed, 28 Jul 2021 22:11:14 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id go31so8448604ejc.6 for ; Wed, 28 Jul 2021 22:11:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BVvrX+9S7ES5E7ACKYpe31zxENBOyA7V53b6GnC/v8o=; b=WbZ51H5/KW4GCmG8eH2G/hxVciY5JJYMuUWxxTguntLeB4YoWyvesY/u2KtMP2Uejp jkBxHTW5jy/MxfkXRchZHplMoDih5Quk9zra3cKGJtiTq393EuIpZDpygKtdwbnlIQ8T f1da5QU5nBAaxf+Xk77r0yMIiIIcZYdn6YK2IJ1b/DF8ozcDGn84qdkHe+imFFfo5sL8 nqr0IJhhdKu/7K04TXSbtHa+lu/gowBSaxwgYnrCVGU1gS6SkyNwMztxES6Mu0l6FKzA PjiWmB0PdPu6e8V20DjXRnj9JzwCHzXDciBzgmAr60GbPlITVLp3yY60xm+/JmZh+TE6 izPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BVvrX+9S7ES5E7ACKYpe31zxENBOyA7V53b6GnC/v8o=; b=fm91VE7k4om2pQKLVd1d3XsR9Jh8f2PvCPJ6CXEdoLFpj7b2JivEPtiMWchG23/KHK Rx25szNEEyEhPMxJz3PtjaAQ4BbfUBLLpmo9qUP2AAMVr7H16t/UyYDVMPz3Ro/ShJXs 5w/uQRpSZAkB38kMMyuyhC2t+Ldi41HSPz/BZGE55s9GbN7Ctr3tDIb8bl0d4b1Mwozb GdZeX00nstTFDPIVlQ5NAkxc5xLaoHnlD9lCgeiL6SS6UVNrPkJ24ZaDsTJ30ODLRGps oyPWY12p9b9blW3YxodpsMwt0UFcSoFqNoGGkCAGvUIQMDYLSWoDYnJwUyB1BtEcJhAy mylA== X-Gm-Message-State: AOAM532pCFmPqKX9Jf/UDiuYGVqyTtbUiARw/1/RrUs4GW+EjvgQph8j 2af1D3PZlINhri8Y6x8lFbh+lNvEU6AnLg== X-Google-Smtp-Source: ABdhPJxVKZM689jCxCH06o8GezJtmegJ65Ho33ygztfzYDH39anQJwgGPQNFHlnJ3JizKsPKBPrZYA== X-Received: by 2002:a17:907:9604:: with SMTP id gb4mr2928927ejc.142.1627535472874; Wed, 28 Jul 2021 22:11:12 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:12 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 74/87] trace-cmd library: Use output handler when copying data from input file Date: Thu, 29 Jul 2021 08:09:46 +0300 Message-Id: <20210729050959.12263-75-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When copying data between two trace files, use input and output handlers, instead of raw file descriptors. Handlers take care of file version and compression context. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 +- lib/trace-cmd/trace-input.c | 145 +++++++++--------- lib/trace-cmd/trace-output.c | 2 +- 3 files changed, 78 insertions(+), 72 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 0a3e12cf..05a2ea01 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -179,7 +179,8 @@ int tracecmd_get_parsing_failures(struct tracecmd_input *handle); int tracecmd_long_size(struct tracecmd_input *handle); int tracecmd_page_size(struct tracecmd_input *handle); int tracecmd_cpus(struct tracecmd_input *handle); -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 2b98e6d8..2d2750ad 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4357,44 +4357,47 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle); } -static int read_copy_size8(struct tracecmd_input *handle, int fd, unsigned long long *size) +static int read_copy_size8(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, unsigned long long *size) { /* read size */ - if (do_read_check(handle, size, 8)) + if (do_read_check(in_handle, size, 8)) return -1; - if (__do_write_check(fd, size, 8)) + if (do_write_check(out_handle, size, 8)) return -1; - *size = tep_read_number(handle->pevent, size, 8); + *size = tep_read_number(in_handle->pevent, size, 8); return 0; } -static int read_copy_size4(struct tracecmd_input *handle, int fd, unsigned int *size) +static int read_copy_size4(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, + unsigned int *size) { /* read size */ - if (do_read_check(handle, size, 4)) + if (do_read_check(in_handle, size, 4)) return -1; - if (__do_write_check(fd, size, 4)) + if (do_write_check(out_handle, size, 4)) return -1; - *size = tep_read_number(handle->pevent, size, 4); + *size = tep_read_number(in_handle->pevent, size, 4); return 0; } -static int read_copy_data(struct tracecmd_input *handle, - unsigned long long size, int fd) +static int read_copy_data(struct tracecmd_input *in_handle, + unsigned long long size, + struct tracecmd_output *out_handle) { char *buf; buf = malloc(size); if (!buf) return -1; - if (do_read_check(handle, buf, size)) + if (do_read_check(in_handle, buf, size)) goto failed_read; - if (__do_write_check(fd, buf, size)) + if (do_write_check(out_handle, buf, size)) goto failed_read; free(buf); @@ -4406,65 +4409,66 @@ static int read_copy_data(struct tracecmd_input *handle, return -1; } -static int copy_header_files(struct tracecmd_input *handle, int fd) +static int copy_header_files(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_HEADERS - 1) + if (in_handle->file_state != TRACECMD_FILE_HEADERS - 1) return -1; /* "header_page" */ - if (read_copy_data(handle, 12, fd) < 0) + if (read_copy_data(in_handle, 12, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; /* "header_event" */ - if (read_copy_data(handle, 13, fd) < 0) + if (read_copy_data(in_handle, 13, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_HEADERS; + in_handle->file_state = TRACECMD_FILE_HEADERS; return 0; } -static int copy_ftrace_files(struct tracecmd_input *handle, int fd) +static int copy_ftrace_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; unsigned int count; unsigned int i; - if (handle->file_state != TRACECMD_FILE_FTRACE_EVENTS - 1) + if (in_handle->file_state != TRACECMD_FILE_FTRACE_EVENTS - 1) return -1; - if (read_copy_size4(handle, fd, &count) < 0) + if (read_copy_size4(in_handle, out_handle, &count) < 0) return -1; for (i = 0; i < count; i++) { - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; } - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + in_handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; return 0; } -static int copy_event_files(struct tracecmd_input *handle, int fd) +static int copy_event_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; char *system; @@ -4472,103 +4476,103 @@ static int copy_event_files(struct tracecmd_input *handle, int fd) unsigned int count; unsigned int i,x; - if (handle->file_state != TRACECMD_FILE_ALL_EVENTS - 1) + if (in_handle->file_state != TRACECMD_FILE_ALL_EVENTS - 1) return -1; - if (read_copy_size4(handle, fd, &systems) < 0) + if (read_copy_size4(in_handle, out_handle, &systems) < 0) return -1; for (i = 0; i < systems; i++) { - system = read_string(handle); + system = read_string(in_handle); if (!system) return -1; - if (__do_write_check(fd, system, strlen(system) + 1)) { + if (do_write_check(out_handle, system, strlen(system) + 1)) { free(system); return -1; } free(system); - if (read_copy_size4(handle, fd, &count) < 0) + if (read_copy_size4(in_handle, out_handle, &count) < 0) return -1; for (x=0; x < count; x++) { - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; } } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + in_handle->file_state = TRACECMD_FILE_ALL_EVENTS; return 0; } -static int copy_proc_kallsyms(struct tracecmd_input *handle, int fd) +static int copy_proc_kallsyms(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_KALLSYMS - 1) + if (in_handle->file_state != TRACECMD_FILE_KALLSYMS - 1) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (read_copy_size4(in_handle, out_handle, &size) < 0) return -1; if (!size) return 0; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_KALLSYMS; + in_handle->file_state = TRACECMD_FILE_KALLSYMS; return 0; } -static int copy_ftrace_printk(struct tracecmd_input *handle, int fd) +static int copy_ftrace_printk(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_PRINTK - 1) + if (in_handle->file_state != TRACECMD_FILE_PRINTK - 1) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (read_copy_size4(in_handle, out_handle, &size) < 0) return -1; if (!size) return 0; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_PRINTK; + in_handle->file_state = TRACECMD_FILE_PRINTK; return 0; } -static int copy_command_lines(struct tracecmd_input *handle, int fd) +static int copy_command_lines(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_CMD_LINES - 1) + if (!check_out_state(out_handle, TRACECMD_FILE_CMD_LINES)) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; if (!size) return 0; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_CMD_LINES; + in_handle->file_state = TRACECMD_FILE_CMD_LINES; return 0; } /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor - * @handle: input handle for the trace.dat file to copy from. - * @fd: The file descriptor to copy to. + * @in_handle: input handle for the trace.dat file to copy from. + * @out_handle: output handle to the trace.dat file to copy to. * @start_state: The file state to start copying from (zero for the beginnig) * @end_state: The file state to stop at (zero for up to cmdlines) * @@ -4579,7 +4583,8 @@ static int copy_command_lines(struct tracecmd_input *handle, int fd) * NOTE: The input handle is also modified, and ends at the end * state as well. */ -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state) { @@ -4597,71 +4602,71 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, if (end_state < TRACECMD_FILE_HEADERS) return 0; - if (handle->file_state >= start_state) { + if (in_handle->file_state >= start_state) { /* Set the handle to just before the start state */ - sec = open_section(handle, TRACECMD_OPTION_HEADER_INFO); + sec = open_section(in_handle, TRACECMD_OPTION_HEADER_INFO); if (!sec) return -1; /* Now that the file handle has moved, change its state */ - handle->file_state = TRACECMD_FILE_INIT; + in_handle->file_state = TRACECMD_FILE_INIT; } /* Try to bring the input up to the start state - 1 */ - ret = tracecmd_read_headers(handle, start_state - 1); + ret = tracecmd_read_headers(in_handle, start_state - 1); if (sec) - close_section(handle, sec); + close_section(in_handle, sec); if (ret < 0) goto out; switch (start_state) { case TRACECMD_FILE_HEADERS: - ret = copy_header_files(handle, fd); + ret = copy_header_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_FTRACE_EVENTS: /* handle's state is now updating with the copies */ - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_files(handle, fd); + ret = copy_ftrace_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_ALL_EVENTS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_event_files(handle, fd); + ret = copy_event_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_KALLSYMS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_proc_kallsyms(handle, fd); + ret = copy_proc_kallsyms(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_PRINTK: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_printk(handle, fd); + ret = copy_ftrace_printk(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_CMD_LINES: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_command_lines(handle, fd); + ret = copy_command_lines(in_handle, out_handle); default: break; } diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 01c1a4f5..8499ab6f 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2494,7 +2494,7 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, if (tracecmd_output_set_from_input(handle, ihandle)) goto out_free; tracecmd_output_write_init(handle); - if (tracecmd_copy_headers(ihandle, handle->fd, 0, 0) < 0) + if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) goto out_free; handle->file_state = tracecmd_get_file_state(ihandle); From patchwork Thu Jul 29 05:09:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22761C4320A for ; Thu, 29 Jul 2021 05:11:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FFA56023E for ; Thu, 29 Jul 2021 05:11:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234130AbhG2FLT (ORCPT ); Thu, 29 Jul 2021 01:11:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233798AbhG2FLS (ORCPT ); Thu, 29 Jul 2021 01:11:18 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22444C0613C1 for ; Wed, 28 Jul 2021 22:11:15 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id nd39so8466509ejc.5 for ; Wed, 28 Jul 2021 22:11:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hvBfWVcxz5c7nYZDb5JZXKqUOClgX+o4XpZ1B0dYn1g=; b=P49aqXMKKw3cUhrcM3egUcLh+Zz6ejU+GYXHKrEBd+snTfmUJpw0yclXpjC07m4qjD uTGwAwIyg9bYUaE//kVGkftIiH7d/W3upjGgyjYntLbFJshttduqqf9jlaFa2CqU7Z3/ PyALFlSmJGdNbuGOVFIkoaqcQJOkFDC5ueHqpPLMymn1+1RNOGJYw3oqLEpSoDahZOyR cALiYL881aKeohLeAcxmIBqo0QsN7IOdelCAQePzaNnkuqr222RTfR+yWmsNXjv3/wBu Vkvy3BmkALWgIK1VNhawAM8hv0WfO2RVsqyNtta0u6UIv2Y5FZFkBr8kIi9dc03NfY/4 UiTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hvBfWVcxz5c7nYZDb5JZXKqUOClgX+o4XpZ1B0dYn1g=; b=jiHo/dBoy9HKqBgNUXk8nHoelax2XNqf5D9JPPkLFaVmZQGPiFvGjNRyRmnaFYAPjg aemQDSUnfWKctGUZ032ZHZUOhdd+M88vNvyK9DmWAdGgG4HjDNMlhEMHbU2dF+e2kO2q LG2pbwVNHkkFv/jPs0pnp7VQuJu/6RvN6SN5uE+OA4+uhPymmxFAz9anTCudKmvhfZOw BD2taZpkt4moE65cL6vNV6G1sPVZtxaUz8HTtBQWPpqczlzetbTjyNdlXzfh6/BUeW+D T3GIP48STNGejtMgcml7twtI+kIUrevJVhstdbmYNTo5y5DbCGUqwkLM7QQ2VwDbK5xz 2cZw== X-Gm-Message-State: AOAM533qlAOFkySdpvsX/vc/r9uZmx2JisM/UQkhDk+5G3c58tU+uaGi 0wNnnc7TkGupmM8KZ7uJlFLdsxZTcYvSrg== X-Google-Smtp-Source: ABdhPJxm863y+Ji6i6a5rZ1s/L8ymeCFRg3v2axsUx1O/Vz8WFtWSWLgw7uf932xrEZJSjbgTt2PaA== X-Received: by 2002:a17:906:404:: with SMTP id d4mr2802334eja.449.1627535473757; Wed, 28 Jul 2021 22:11:13 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:13 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 75/87] trace-cmd library: Handle version 7 files when copying headers between files Date: Thu, 29 Jul 2021 08:09:47 +0300 Message-Id: <20210729050959.12263-76-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When copying header data between trace files, handle the file versions. Internal library functions for copying these headers are fixed to work with trace file version 7: copy_header_files copy_ftrace_files copy_event_files copy_proc_kallsyms copy_command_lines Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 191 ++++++++++++++++++++---- lib/trace-cmd/trace-output.c | 5 + 3 files changed, 167 insertions(+), 30 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index c73b37a6..60520b07 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -51,6 +51,7 @@ void out_compression_reset(struct tracecmd_output *handle); void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); +void out_set_file_state(struct tracecmd_output *handle, int new_state); unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, int fd, unsigned long long max, unsigned long long *write_size); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 2d2750ad..5ceb9756 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4412,161 +4412,292 @@ static int read_copy_data(struct tracecmd_input *in_handle, static int copy_header_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { + struct file_section *sec; + unsigned long long offset; unsigned long long size; - if (in_handle->file_state != TRACECMD_FILE_HEADERS - 1) + if (!check_out_state(out_handle, TRACECMD_FILE_HEADERS)) + return -1; + + sec = open_section(in_handle, TRACECMD_OPTION_HEADER_INFO); + if (!sec) return -1; + offset = out_write_section_header(out_handle, TRACECMD_OPTION_HEADER_INFO, + "headers", TRACECMD_SEC_FL_COMPRESS, true); + + out_compression_start(out_handle); /* "header_page" */ if (read_copy_data(in_handle, 12, out_handle) < 0) - return -1; + goto error; if (read_copy_size8(in_handle, out_handle, &size) < 0) - return -1; + goto error; if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; + goto error; /* "header_event" */ if (read_copy_data(in_handle, 13, out_handle) < 0) - return -1; + goto error; if (read_copy_size8(in_handle, out_handle, &size) < 0) - return -1; + goto error; if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; + goto error; in_handle->file_state = TRACECMD_FILE_HEADERS; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); + close_section(in_handle, sec); + + if (out_update_section_header(out_handle, offset)) + goto error; return 0; +error: + out_compression_reset(out_handle); + close_section(in_handle, sec); + return -1; } static int copy_ftrace_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { + struct file_section *sec; + unsigned long long offset; unsigned long long size; unsigned int count; unsigned int i; - if (in_handle->file_state != TRACECMD_FILE_FTRACE_EVENTS - 1) + if (!check_out_state(out_handle, TRACECMD_FILE_FTRACE_EVENTS)) return -1; - if (read_copy_size4(in_handle, out_handle, &count) < 0) + sec = open_section(in_handle, TRACECMD_OPTION_FTRACE_EVENTS); + if (!sec) return -1; + offset = out_write_section_header(out_handle, TRACECMD_OPTION_FTRACE_EVENTS, + "ftrace events", TRACECMD_SEC_FL_COMPRESS, true); + + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (i = 0; i < count; i++) { if (read_copy_size8(in_handle, out_handle, &size) < 0) - return -1; + goto error; if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; + goto error; } in_handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); + + close_section(in_handle, sec); + + if (out_update_section_header(out_handle, offset)) + goto error; return 0; +error: + out_compression_reset(out_handle); + close_section(in_handle, sec); + return -1; } static int copy_event_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { + struct file_section *sec; + unsigned long long offset; unsigned long long size; char *system; unsigned int systems; unsigned int count; unsigned int i,x; - if (in_handle->file_state != TRACECMD_FILE_ALL_EVENTS - 1) + if (!check_out_state(out_handle, TRACECMD_FILE_ALL_EVENTS)) return -1; - if (read_copy_size4(in_handle, out_handle, &systems) < 0) + sec = open_section(in_handle, TRACECMD_OPTION_EVENT_FORMATS); + if (!sec) return -1; + offset = out_write_section_header(out_handle, TRACECMD_OPTION_EVENT_FORMATS, + "events format", TRACECMD_SEC_FL_COMPRESS, true); + + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &systems) < 0) + goto error; for (i = 0; i < systems; i++) { system = read_string(in_handle); if (!system) - return -1; + goto error; if (do_write_check(out_handle, system, strlen(system) + 1)) { free(system); - return -1; + goto error; } free(system); if (read_copy_size4(in_handle, out_handle, &count) < 0) - return -1; + goto error; for (x=0; x < count; x++) { if (read_copy_size8(in_handle, out_handle, &size) < 0) - return -1; + goto error; if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; + goto error; } } in_handle->file_state = TRACECMD_FILE_ALL_EVENTS; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); + + close_section(in_handle, sec); + + if (out_update_section_header(out_handle, offset)) + goto error; return 0; +error: + out_compression_reset(out_handle); + close_section(in_handle, sec); + return -1; } static int copy_proc_kallsyms(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { + struct file_section *sec; + unsigned long long offset; unsigned int size; - if (in_handle->file_state != TRACECMD_FILE_KALLSYMS - 1) + if (!check_out_state(out_handle, TRACECMD_FILE_KALLSYMS)) return -1; + sec = open_section(in_handle, TRACECMD_OPTION_KALLSYMS); + if (!sec) + return -1; + offset = out_write_section_header(out_handle, TRACECMD_OPTION_KALLSYMS, + "kallsyms", TRACECMD_SEC_FL_COMPRESS, true); + out_compression_start(out_handle); if (read_copy_size4(in_handle, out_handle, &size) < 0) - return -1; + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; - + goto error; +out: in_handle->file_state = TRACECMD_FILE_KALLSYMS; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); + + close_section(in_handle, sec); + + if (out_update_section_header(out_handle, offset)) + goto error; return 0; +error: + out_compression_reset(out_handle); + close_section(in_handle, sec); + return -1; } static int copy_ftrace_printk(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { + struct file_section *sec; + unsigned long long offset; unsigned int size; - if (in_handle->file_state != TRACECMD_FILE_PRINTK - 1) + if (!check_out_state(out_handle, TRACECMD_FILE_PRINTK)) return -1; - if (read_copy_size4(in_handle, out_handle, &size) < 0) + sec = open_section(in_handle, TRACECMD_OPTION_PRINTK); + if (!sec) return -1; + offset = out_write_section_header(out_handle, TRACECMD_OPTION_PRINTK, + "printk", TRACECMD_SEC_FL_COMPRESS, true); + + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; + goto error; +out: in_handle->file_state = TRACECMD_FILE_PRINTK; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); + + close_section(in_handle, sec); + + if (out_update_section_header(out_handle, offset)) + goto error; return 0; +error: + out_compression_reset(out_handle); + close_section(in_handle, sec); + return -1; } static int copy_command_lines(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { + struct file_section *sec; + unsigned long long offset; unsigned long long size; if (!check_out_state(out_handle, TRACECMD_FILE_CMD_LINES)) return -1; - if (read_copy_size8(in_handle, out_handle, &size) < 0) + sec = open_section(in_handle, TRACECMD_OPTION_CMDLINES); + if (!sec) return -1; + offset = out_write_section_header(out_handle, TRACECMD_OPTION_CMDLINES, + "command lines", TRACECMD_SEC_FL_COMPRESS, true); + + out_compression_start(out_handle); + + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ if (read_copy_data(in_handle, size, out_handle) < 0) - return -1; + goto error; +out: in_handle->file_state = TRACECMD_FILE_CMD_LINES; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); + + close_section(in_handle, sec); + + if (out_update_section_header(out_handle, offset)) + goto error; return 0; +error: + out_compression_reset(out_handle); + close_section(in_handle, sec); + return -1; } /** diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8499ab6f..9972185a 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2507,6 +2507,11 @@ out_free: return NULL; } +__hidden void out_set_file_state(struct tracecmd_output *handle, int new_state) +{ + handle->file_state = new_state; +} + __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); From patchwork Thu Jul 29 05:09:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E53EC4338F for ; Thu, 29 Jul 2021 05:11:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 042BE60C3F for ; Thu, 29 Jul 2021 05:11:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234137AbhG2FLU (ORCPT ); Thu, 29 Jul 2021 01:11:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233798AbhG2FLU (ORCPT ); Thu, 29 Jul 2021 01:11:20 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E474C061757 for ; Wed, 28 Jul 2021 22:11:16 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id gs8so8390493ejc.13 for ; Wed, 28 Jul 2021 22:11:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4NTNKjtb81y+MvZ+Ru9YbE9e0/w1hBWYDLtZxkoh4fM=; b=XoDL7c5+Xd5JpnqKwPtEBVPbm8LN6B8w0+vGBP1XDKn1jQSkLEi1Q7MzRYhiPVaavU 1hiAJ4uskilQoWy0pKYXLBi4hD0wqo/4qEA/7P3d9Y+Q0JZQt+sb3r0if7+GLxuFHi2z iRhKJ31qyjDKuTFaxKkKkD9tjJffiJnWJGiw18nNsYR4hbk4Fo63j/G0ObiwZZ9hEOPS RSGmBDuUnh2SZq/uVMbTXULJTDB0V0KqhvIBrYpesMNYEpfvhu5sQ+L7HM5TWfPg504e Sgob4PiOq1DcayfTJZjuGAliz6oIDEAraWbZjwrNGzjSjoKQE4R94e6PAjePiX7ZEAU6 iB8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4NTNKjtb81y+MvZ+Ru9YbE9e0/w1hBWYDLtZxkoh4fM=; b=ocvSQZI6JctvU8yjGsBdAmtRJDqVjtEKKepfItqtG4FHj7VINTohJtyd0Gk6AcnrKH VqIDPJvIa+fF6qs7XuLK3YMk1I0cuPJ/aQ9URT5aY97rJiSAppEb5WdxyC7RF3yhXoVk f8C6/vy25NSZjqJ9fkg9+5XPWKB8SvePI7CJNgNergV0TbAsdFY0UkkxnwPPoYiE91Ql cZADxQQqFfpr7JSdRsu0SWu/PmlzTaAGARAd031zeRrRqFsjsS1mZ2YGb6omlrg/EJwd OqZS48UWxmtAyHBAyY0E50Gd6oUJr+Osr+fir6LA8Rx+kv6TC598VXyZUSupM3S6VZtN bduA== X-Gm-Message-State: AOAM532PkHbtIKR2KQcMXQ+pE8hKF+ac/zDCj3sagORUTTdQGFKj+mus 29iG3deLS2JbdKFRVT/1hgMl1DG5Xo/jxw== X-Google-Smtp-Source: ABdhPJxp4mckGsIJQ4EDMaMQM55bBlt4odQaRYT3ylhuZQG9Q2OMMnvwrNpWM90VKALE+Rb5CUptlg== X-Received: by 2002:a17:906:9742:: with SMTP id o2mr2883450ejy.532.1627535474724; Wed, 28 Jul 2021 22:11:14 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:14 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 76/87] tarce-cmd library: Copy CPU count between trace files Date: Thu, 29 Jul 2021 08:09:48 +0300 Message-Id: <20210729050959.12263-77-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The tracecmd_copy_headers() API should be able to copy CPU count also, as it is part of the headers. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 40 +++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 5ceb9756..f4533f7e 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4700,6 +4700,33 @@ error: return -1; } +static int copy_cpu_count(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) +{ + unsigned int cpus; + + if (!check_in_state(in_handle, TRACECMD_FILE_CPU_COUNT) || + !check_out_state(out_handle, TRACECMD_FILE_CPU_COUNT)) + return -1; + + if (in_handle->file_version < 7) { + if (read4(in_handle, &cpus)) + return -1; + } else { + cpus = in_handle->max_cpu; + } + + if (tracecmd_get_out_file_version(out_handle) < 7) { + cpus = tep_read_number(in_handle->pevent, &cpus, 4); + if (do_write_check(out_handle, &cpus, 4)) + return -1; + } + + in_handle->file_state = TRACECMD_FILE_CPU_COUNT; + out_set_file_state(out_handle, in_handle->file_state); + + return 0; +} + /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor * @in_handle: input handle for the trace.dat file to copy from. @@ -4798,6 +4825,19 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, return 0; ret = copy_command_lines(in_handle, out_handle); + if (ret < 0) + goto out; + + /* fallthrough */ + case TRACECMD_FILE_CPU_COUNT: + if (end_state <= in_handle->file_state) + return 0; + + ret = copy_cpu_count(in_handle, out_handle); + if (ret < 0) + goto out; + + /* fallthrough */ default: break; } From patchwork Thu Jul 29 05:09:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BBCAC4320A for ; Thu, 29 Jul 2021 05:11:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 362806023E for ; Thu, 29 Jul 2021 05:11:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233798AbhG2FLV (ORCPT ); Thu, 29 Jul 2021 01:11:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233899AbhG2FLU (ORCPT ); Thu, 29 Jul 2021 01:11:20 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E7E0C061765 for ; Wed, 28 Jul 2021 22:11:17 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id o5so8485528ejy.2 for ; Wed, 28 Jul 2021 22:11:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xq+3L4+TFICrMplTofG0MACMJbsB95F3lY2de2dQMBY=; b=s8fRTaNVB4+0DYvj9E0tCiNTlPTW3GzmD6Bz0ghAcXGb2oIHSPnlY8m/SsB89fm4+f eyj0bOUB+4GJp2h/g8UONxebEioKjbJkWfKco+Z8+TFkWxhCy/SiessAylxOYrShq2JV ikE1QDoa5GaLWf1udK6KdXKgHRymLyHcXALfXbcUlhCUFbwKagdLzx5EmjTjkH+wbOHW Gf4i94PKsGkaQRD/TtBzQ+ullcARmeMrsfWbQ0Y3ocQ8YybayiWWFfBvsy1+9nuHF8OU nXPey4tEeIb7/7hMJkxDM7lkdQ8ILR+SKayYVn2ujzKIbqjhWfihrf4uQOZhOJimr/mF t8pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xq+3L4+TFICrMplTofG0MACMJbsB95F3lY2de2dQMBY=; b=Tx3RZ5G/skEiIOmPzMm0IZ08N1zVKjCGlhRQf7sTZqX+z80xvLj6scVPJBxMVbkq1o 5CnSPtXL1rOQrLT6OdhJ6h4MVV7/OnH++ih4MLwoFocxrVOKczpgHXAlsPcYzse+hmnk nIfVw7LqDho4m34ZIXDMq4KuV3a2wrXXrC6xRE+kTI/IcN1klX6A6ab994UhEmoGGlku WFyrCaQpa/bPwPFPzkP8QL88g8JGlLsMLEPAe/6L0E20t0QPHPd6ZdmbJsBP0ugtkFG6 ZwRN4RQiO5rm4sn6Nq/ULel/Y0q4W6M1hNjjmg3TtiEWo7FnBi0RCEwQES2ckDXCMKWX kg9w== X-Gm-Message-State: AOAM531wAvDsiWYBi/HdArZa6/j8V84idRKtdJoUQ5eQbabndCwSJfnp 0wkTzNB659neKg2EPlPoshI= X-Google-Smtp-Source: ABdhPJzAl56f4+nelniGeYXGgHqk0l+M+3OQ/7U3zSmd/y5/lRfmcm6HO3lWl5yeROJO+UyjS8yjpQ== X-Received: by 2002:a17:906:ed1:: with SMTP id u17mr2843501eji.304.1627535475781; Wed, 28 Jul 2021 22:11:15 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:15 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 77/87] tarce-cmd library: New API to copy buffer description between trace files Date: Thu, 29 Jul 2021 08:09:49 +0300 Message-Id: <20210729050959.12263-78-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A new library API is introduced, to copy buffers description between trace files: tracecmd_copy_buffer_descr() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 13 +++++++++++++ 2 files changed, 15 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 05a2ea01..58c17fe5 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -183,6 +183,8 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); +int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f4533f7e..76be1a02 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4846,6 +4846,19 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, return ret < 0 ? -1 : 0; } +int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int i; + + if (tracecmd_get_out_file_version(out_handle) >= 7) + return 0; + + for (i = 0; i < in_handle->nr_buffers; i++) + tracecmd_add_buffer_info(out_handle, in_handle->buffers[i].name, 0); + return tracecmd_write_buffer_info(out_handle); +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file From patchwork Thu Jul 29 05:09:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0C3DC432BE for ; Thu, 29 Jul 2021 05:11:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D74B160C3F for ; Thu, 29 Jul 2021 05:11:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234144AbhG2FLV (ORCPT ); Thu, 29 Jul 2021 01:11:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233899AbhG2FLV (ORCPT ); Thu, 29 Jul 2021 01:11:21 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CC12C0613C1 for ; Wed, 28 Jul 2021 22:11:18 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id jg2so8553956ejc.0 for ; Wed, 28 Jul 2021 22:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KVB4SAjVlM/gzPx3D53WvWLD2sCICy7MCFgyiI9jskU=; b=ajjvZg2MUXfn0TgFo+4ofwQUlsci7M6ljvonDvfa5bKuuhvMxfMdH5+74MSwJ2V7tI /Qtvb00xbu2n3IPp3v7NmNJ4FsX3T3XAzZ63LeqySs25qXzAxkX9gARj6JCHzgbqUyb4 CBR3VqzMAg3YIGoLijP8w5gfdDG4LlEDo4zPyklWFYuGEyZ0KKRQBUhRD6iqxYCgfw17 boa9qPlWNEHqezWtJcwBB/usQLWnZ9+SHQJ0+FPYdK2Ntc9//CyiHIDodH8KdmojHmF0 wVzmpv1W/RUfuzrTMEuf45Z2vUb4PxOmFOrj941t7/2V3Uh8DxEFMNvu+rs4jUwBCoUd QAYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KVB4SAjVlM/gzPx3D53WvWLD2sCICy7MCFgyiI9jskU=; b=kbiffh+UCqQ+h4huiwhZIfgR3ead97rfAcOyrXNIdcNGkr2kvZ4d0jEsiVtEKhD+T8 CkGhjDfBjfD8LzA/Uchw5UsiGpUc2l7CrkTEVwp8g8FgD9Xrvfz3DYDkLbwPwT2yhwTw 3Qhsqqvdb1vIO2VKSYIQBzEXMYp+w7hDa+JY0Bwz4gi7IGxG1p1eUp+Zus3rh7fXgcgI fXsj/2Oz/NQneysQO1Pou0Oec/CUpjtbTpIni8ZoG/1v4bifdRZ/jB++a0VwmD6cILD9 u65WNGIpV3R+TULjxdzZVJTpIcXWU7JSo/ND1rB9WVMtOs3XuR/ZNlj/lw8tQ3+u25Gj k5zw== X-Gm-Message-State: AOAM533Y+euL8rm/cz0HlblpiRoWamqKuCD5Q6xwQG57jitQ3MTSO4f3 IO8ckUu+q0dTZ6LWR1r17A7Z69bNwazqkA== X-Google-Smtp-Source: ABdhPJwoMeL5m1UJw4QbQzyQLJaLL82YusiljT69uKK4pFNdIitL9R5Kyd7s8ijdy1KYGNAXlMKSxA== X-Received: by 2002:a17:906:b0c5:: with SMTP id bk5mr2806754ejb.428.1627535476761; Wed, 28 Jul 2021 22:11:16 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:16 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 78/87] tarce-cmd library: New API to copy options between trace files Date: Thu, 29 Jul 2021 08:09:50 +0300 Message-Id: <20210729050959.12263-79-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A new library API is introduced, to copy options between trace files: tracecmd_copy_options() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 + lib/trace-cmd/include/trace-cmd-local.h | 2 + lib/trace-cmd/trace-input.c | 140 ++++++++++++++++++ lib/trace-cmd/trace-output.c | 29 ++++ 4 files changed, 174 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 58c17fe5..8b730d1a 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -185,6 +185,8 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, enum tracecmd_file_states end_state); int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); +int tracecmd_copy_options(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); @@ -331,6 +333,7 @@ int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle); +unsigned long long tracecmd_get_out_file_offset(struct tracecmd_output *handle); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 60520b07..eb9d6de4 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -52,6 +52,8 @@ void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); void out_set_file_state(struct tracecmd_output *handle, int new_state); +int out_save_options_offset(struct tracecmd_output *handle, + unsigned long long start); unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, int fd, unsigned long long max, unsigned long long *write_size); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 76be1a02..be8c3e7c 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4859,6 +4859,146 @@ int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, return tracecmd_write_buffer_info(out_handle); } +static int copy_options_recursive(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + unsigned short id, flags = 0; + unsigned short option, en2; + unsigned long long next; + unsigned int size, en4; + bool skip; + + for (;;) { + if (do_read_check(in_handle, &option, 2)) + return -1; + en2 = tep_read_number(in_handle->pevent, &option, 2); + + if (en2 == TRACECMD_OPTION_DONE) { + if (in_handle->file_version < 7) + return 0; + } + /* next 4 bytes is the size of the option */ + if (do_read_check(in_handle, &size, 4)) + return -1; + en4 = tep_read_number(in_handle->pevent, &size, 4); + if (en2 == TRACECMD_OPTION_DONE) { + /* option done v7 */ + if (en4 < 8) + return -1; + if (read8(in_handle, &next)) + return -1; + if (!next) + break; + if (do_lseek(in_handle, next, SEEK_SET) == (off64_t)-1) + return -1; + if (read_section_header(in_handle, &id, &flags, NULL, NULL)) + return -1; + if (id != TRACECMD_OPTION_DONE) + return -1; + if (flags & TRACECMD_SEC_FL_COMPRESS && in_uncompress_block(in_handle)) + return -1; + return copy_options_recursive(in_handle, out_handle); + } + /* Do not copy these, as they have file specific offsets */ + switch (en2) { + case TRACECMD_OPTION_BUFFER: + case TRACECMD_OPTION_BUFFER_LAT: + case TRACECMD_OPTION_HEADER_INFO: + case TRACECMD_OPTION_FTRACE_EVENTS: + case TRACECMD_OPTION_EVENT_FORMATS: + case TRACECMD_OPTION_KALLSYMS: + case TRACECMD_OPTION_PRINTK: + case TRACECMD_OPTION_CMDLINES: + skip = true; + break; + default: + skip = false; + break; + } + if (skip) { + do_lseek(in_handle, en4, SEEK_CUR); + continue; + } + if (do_write_check(out_handle, &option, 2)) + return -1; + if (do_write_check(out_handle, &size, 4)) + return -1; + if (read_copy_data(in_handle, en4, out_handle)) + return -1; + } + + return 0; +} + +static int copy_options(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) +{ + unsigned long long offset, start; + unsigned short id, en2, flags = 0; + int tmp; + + if (in_handle->file_version >= 7) { + if (read_section_header(in_handle, &id, &flags, NULL, NULL)) + return -1; + if (id != TRACECMD_OPTION_DONE) + return -1; + if (flags & TRACECMD_SEC_FL_COMPRESS && in_uncompress_block(in_handle)) + return -1; + } + start = tracecmd_get_out_file_offset(out_handle); + if (tracecmd_get_out_file_version(out_handle) < 7) { + if (do_write_check(out_handle, "options ", 10)) + return -1; + } + + offset = out_write_section_header(out_handle, TRACECMD_OPTION_DONE, "options", 0, false); + + if (copy_options_recursive(in_handle, out_handle)) + goto error; + + id = TRACECMD_OPTION_DONE; + en2 = tep_read_number(in_handle->pevent, &id, 2); + if (do_write_check(out_handle, &en2, 2)) + goto error; + if (tracecmd_get_out_file_version(out_handle) < 7) { + out_save_options_offset(out_handle, start); + } else { + tmp = 8; + if (do_write_check(out_handle, &tmp, 4)) + goto error; + out_save_options_offset(out_handle, start); + start = 0; + if (do_write_check(out_handle, &start, 8)) + goto error; + } + out_update_section_header(out_handle, offset); + if (flags & TRACECMD_SEC_FL_COMPRESS) + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_OPTIONS; + out_set_file_state(out_handle, in_handle->file_state); + /* Append local options */ + return tracecmd_append_options(out_handle); + +error: + if (flags & TRACECMD_SEC_FL_COMPRESS) + in_uncompress_reset(in_handle); + return 0; +} + +int tracecmd_copy_options(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + if (!check_out_state(out_handle, TRACECMD_FILE_OPTIONS)) + return -1; + + if (!in_handle->options_start) + return 0; + if (lseek64(in_handle->fd, in_handle->options_start, SEEK_SET) == (off64_t)-1) + return -1; + if (copy_options(in_handle, out_handle) < 0) + return -1; + return 0; +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9972185a..b02fad2f 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2517,6 +2517,30 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) return check_file_state(handle->file_version, handle->file_state, new_state); } +__hidden int out_save_options_offset(struct tracecmd_output *handle, unsigned long long start) +{ + unsigned long long new, en8; + + if (handle->file_version < 7) { + handle->options_start = start; + } else { + /* Append to the previous options section, if any */ + if (!handle->options_start) + return -1; + new = do_lseek(handle, 0, SEEK_CUR); + if (do_lseek(handle, handle->options_start, SEEK_SET) == (off64_t)-1) + return -1; + + en8 = convert_endian_8(handle, start); + if (do_write_check(handle, &en8, 8)) + return -1; + handle->options_start = new; + if (do_lseek(handle, new, SEEK_SET) == (off64_t)-1) + return -1; + } + return 0; +} + /** * tracecmd_get_out_file_version - return the trace.dat file version * @handle: output handle for the trace.dat file @@ -2525,3 +2549,8 @@ unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle) { return handle->file_version; } + +unsigned long long tracecmd_get_out_file_offset(struct tracecmd_output *handle) +{ + return do_lseek(handle, 0, SEEK_CUR); +} From patchwork Thu Jul 29 05:09:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A7A8C4338F for ; Thu, 29 Jul 2021 05:11:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6358660EB2 for ; Thu, 29 Jul 2021 05:11:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234154AbhG2FLX (ORCPT ); Thu, 29 Jul 2021 01:11:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233899AbhG2FLW (ORCPT ); Thu, 29 Jul 2021 01:11:22 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 163BFC061757 for ; Wed, 28 Jul 2021 22:11:19 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id oz16so8449416ejc.7 for ; Wed, 28 Jul 2021 22:11:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uTYqHseuPERYC/C28h6E//clEl8KOpg7LII4T5/pF2E=; b=UQTibcXagQUa46Bv+DOuBAJQsCaGJn8arKZukEqHlBWOlHCXwsEdydwpzk3S3oe4pu ibtwrd8AANJSFrbehp9DLpnoarVSRgeIzeWh/sfzCCtdEcHEdFj93ed/OjZl9JJLf15d GHQzzggO6eEdsy/hFBq7YyEcTejm9e0xuVFkAdL5zZar0Lg33wVIUVdj0Jjj0EKJC7Lu KjKVsdiHK6FGNjY/273p92X0ClSpupq2ihQ5SYclsltMHaBMHJnwJuMjI2b3rfz35mBm dIWabZ3aByYzKU5GsxZ7KTIi3YTMa8RvZ5HYokib0xWMhtRYNodQO65S263lp4OpRCjS xmsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uTYqHseuPERYC/C28h6E//clEl8KOpg7LII4T5/pF2E=; b=MOk9KUcO59USwnJM9E0SNqoZOYoos/cUd8VvrQPZ5I4Y7eI9VlsMcgqtecIl12Ez9A Lm/nfEb30yUMMG6CMlWhNC7wZQCBepBbJob/xB3icMULEPZpNaBcVPtVai6Z4ctpetdP R+q+pgd3n8aG9i8an/HzQXupj8oxJsIgPo8XdWxcpqqLqHTuXtapFMPvqNdqWFdKtB4u TS+UwTfwXwp/ZnuMqSLa4l4av4LznWkpKzMJWCF9un2ZH1FvMLcojy9nTKfZRy0UjPlc pOOD2edm4lHzDM5iSHlZgDRaOnCdgFrIzglMcokZhnY9vnF7H+uKCwLN7p1DPzYq+XT2 63pA== X-Gm-Message-State: AOAM5331KHFKXMOmHyRXvR52xA4qZhMgzSDnZpduFaunmHlxmYJovIMB 8ubQp9gg5M7e3xxMqN207Ww= X-Google-Smtp-Source: ABdhPJy/2+memsxR17OGolQx7BlbsFXssuOhtVcf8egmPPSNiwRBVdGjLQ9reTfn21BnU72QNeeaNQ== X-Received: by 2002:a17:906:2b91:: with SMTP id m17mr2976106ejg.48.1627535477682; Wed, 28 Jul 2021 22:11:17 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:17 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 79/87] tarce-cmd library: New API to copy trace data between trace files Date: Thu, 29 Jul 2021 08:09:51 +0300 Message-Id: <20210729050959.12263-80-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A new library API is introduced, to copy trace data between trace files: tracecmd_copy_trace_data() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/include/trace-cmd-local.h | 6 + lib/trace-cmd/trace-input.c | 166 ++++++++++++++++++ lib/trace-cmd/trace-output.c | 38 ++++ 4 files changed, 212 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 8b730d1a..3c3c98aa 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -187,6 +187,8 @@ int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); int tracecmd_copy_options(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); +int tracecmd_copy_trace_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index eb9d6de4..79c09fca 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -64,6 +64,11 @@ int out_update_section_header(struct tracecmd_output *handle, unsigned long long long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); +struct tracecmd_option * +out_add_buffer_option_v7(struct tracecmd_output *handle, const char *name, + unsigned short id, unsigned long long data_offset, + int cpus, struct data_file_write *cpu_data); + struct cpu_data_source { int fd; int size; @@ -72,6 +77,7 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name); +int out_write_emty_cpu_data(struct tracecmd_output *handle, int cpus); off64_t msg_lseek(struct tracecmd_msg_handle *msg_handle, off_t offset, int whence); unsigned long long get_last_option_offset(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index be8c3e7c..b66d876f 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4999,6 +4999,172 @@ int tracecmd_copy_options(struct tracecmd_input *in_handle, return 0; } +static int copy_trace_latency(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, const char *buf_name) +{ + unsigned long long wsize; + unsigned long long offset; + int fd; + + if (tracecmd_get_out_file_version(out_handle) < 7 && + do_write_check(out_handle, "latency ", 10)) + return -1; + + offset = tracecmd_get_out_file_offset(out_handle); + + if (tracecmd_get_out_file_version(out_handle) >= 7 && + !out_add_buffer_option_v7(out_handle, buf_name, TRACECMD_OPTION_BUFFER_LAT, offset, 0, NULL)) + return -1; + + offset = out_write_section_header(out_handle, TRACECMD_OPTION_BUFFER_LAT, + "buffer latency", TRACECMD_SEC_FL_COMPRESS, false); + + if (in_handle->latz.fd >= 0) + fd = in_handle->latz.fd; + else + fd = in_handle->fd; + + if (!out_copy_fd_compress(out_handle, fd, 0, &wsize)) + return -1; + if (out_update_section_header(out_handle, offset)) + return -1; + + out_set_file_state(out_handle, TRACECMD_FILE_CPU_LATENCY); + return 0; +} + +static int copy_trace_flyrecord_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, const char *buff_name) +{ + struct cpu_data_source *data; + int total_size = 0; + int ret; + int i; + + data = calloc(in_handle->cpus, sizeof(struct cpu_data_source)); + if (!data) + return -1; + for (i = 0; i < in_handle->cpus; i++) { + data[i].size = in_handle->cpu_data[i].file_size; + total_size += data[i].size; + if (in_handle->cpu_data[i].compress.fd >= 0) { + data[i].fd = in_handle->cpu_data[i].compress.fd; + data[i].offset = 0; + } else { + data[i].fd = in_handle->fd; + data[i].offset = in_handle->cpu_data[i].file_offset; + } + } + if (total_size || tracecmd_get_out_file_version(out_handle) < 7) + ret = out_write_cpu_data(out_handle, in_handle->cpus, data, buff_name); + else + ret = 0; + free(data); + + return ret; +} + +static int copy_flyrecord_buffer(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, int index) +{ + struct tracecmd_input *instance; + const char *name; + int ret; + + name = tracecmd_buffer_instance_name(in_handle, index); + if (!name) + return -1; + + instance = tracecmd_buffer_instance_handle(in_handle, index); + if (!instance) + return -1; + if (!tracecmd_get_quiet(out_handle) && *name) + fprintf(stderr, "\nBuffer: %s\n\n", name); + + if (in_handle->buffers[index].latency) + ret = copy_trace_latency(in_handle, out_handle, name); + else + ret = copy_trace_flyrecord_data(instance, out_handle, name); + tracecmd_close(instance); + + return ret; +} + +static int copy_trace_data_from_v6(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + char buf[10]; + int ret; + int i; + + if (do_read_check(in_handle, buf, 10)) + return -1; + + if (strncmp(buf, "latency", 7) == 0) + in_handle->file_state = TRACECMD_FILE_CPU_LATENCY; + else if (strncmp(buf, "flyrecord", 9) == 0) + in_handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; + + tracecmd_init_data(in_handle); + + if (in_handle->file_state == TRACECMD_FILE_CPU_LATENCY) + return copy_trace_latency(in_handle, out_handle, ""); + + /* top instance */ + ret = copy_trace_flyrecord_data(in_handle, out_handle, ""); + if (ret) + return ret; + + for (i = 0; i < in_handle->nr_buffers; i++) + copy_flyrecord_buffer(in_handle, out_handle, i); + return 0; +} + +static int copy_trace_data_from_v7(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int ret; + int i; + + /* Force using temporary files for trace data decompression */ + in_handle->read_zpage = false; + tracecmd_init_data(in_handle); + + if (tracecmd_get_out_file_version(out_handle) < 7) { + /* copy top buffer in v6 files */ + if (in_handle->top_buffer.latency) + ret = copy_trace_latency(in_handle, out_handle, in_handle->top_buffer.name); + else if (in_handle->top_buffer.cpus) + ret = copy_trace_flyrecord_data(in_handle, out_handle, + in_handle->top_buffer.name); + else + ret = out_write_emty_cpu_data(out_handle, in_handle->max_cpu); + if (ret) + return ret; + } + + for (i = 0; i < in_handle->nr_buffers; i++) + copy_flyrecord_buffer(in_handle, out_handle, i); + + return 0; +} + +__hidden int tracecmd_copy_trace_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int ret; + + if (!check_in_state(in_handle, TRACECMD_FILE_CPU_FLYRECORD) || + !check_out_state(out_handle, TRACECMD_FILE_CPU_FLYRECORD)) + return -1; + tracecmd_set_out_clock(out_handle, in_handle->trace_clock); + if (in_handle->file_version < 7) + ret = copy_trace_data_from_v6(in_handle, out_handle); + else + ret = copy_trace_data_from_v7(in_handle, out_handle); + return ret; +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index b02fad2f..b718770d 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2170,6 +2170,44 @@ static int update_buffer_cpu_offset_v6(struct tracecmd_output *handle, return 0; } +__hidden int out_write_emty_cpu_data(struct tracecmd_output *handle, int cpus) +{ + unsigned long long zero = 0; + char *clock; + int ret; + int i; + + if (handle->file_version >= 7) + return 0; + + ret = handle->file_state == TRACECMD_FILE_CPU_FLYRECORD ? 0 : + check_file_state(handle->file_version, + handle->file_state, + TRACECMD_FILE_CPU_FLYRECORD); + if (ret < 0) { + tracecmd_warning("Cannot write trace data into the file, unexpected state 0x%X", + handle->file_state); + return ret; + } + + if (do_write_check(handle, "flyrecord", 10)) + return -1; + + for (i = 0; i < cpus; i++) { + /* Write 0 for trace data offset and size */ + if (do_write_check(handle, &zero, 8)) + return -1; + if (do_write_check(handle, &zero, 8)) + return -1; + } + clock = get_clock(handle); + if (clock && save_clock(handle, clock)) + return -1; + handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; + return 0; +} + + __hidden int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data, const char *buff_name) { From patchwork Thu Jul 29 05:09:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA228C4320E for ; Thu, 29 Jul 2021 05:11:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B254960C3F for ; Thu, 29 Jul 2021 05:11:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233899AbhG2FLX (ORCPT ); Thu, 29 Jul 2021 01:11:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234153AbhG2FLX (ORCPT ); Thu, 29 Jul 2021 01:11:23 -0400 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DACE7C061765 for ; Wed, 28 Jul 2021 22:11:19 -0700 (PDT) Received: by mail-ed1-x532.google.com with SMTP id y12so6393340edo.6 for ; Wed, 28 Jul 2021 22:11:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v48F3F+r6Sfjv4Egks+xj7J/KD8GnfqPDTRoDi9ghUE=; b=HKj+lua4VhkKoSW7Fl3s+i7yNpz8fIn9B1btBykb8F+D47g1tK6UTThQ4MQ7yeOJdv 9Y0+7xwuDOSuJbW18ZJqR9vBPxWUvKysa+8ogZajiNaqzsEKnRhHZ9N7WnXuViyBJ3mb eGw2Wf2Pt5KMCeGB8qPr+QCKlhVXrgE73kU4hi6yt7Pw5rph4iY8WNhb6SBIVu3uF2w5 dbjM9Ux806yo9SF1w60DTpYa1MDfCm3b7lz7vtRWXXAAr7yzCcG7C3iATux37ae+9/3/ RVlx75q+3vwYlVrUdZnSS6pnNPXmMtH1zyUaA4SV4QRHqEwHX0SUcjwZkoRYOEfp6PYY QE7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v48F3F+r6Sfjv4Egks+xj7J/KD8GnfqPDTRoDi9ghUE=; b=h1SYhpK6XF7Ne3kxo5DbRDZ0sPctyNKa9WkJngISeGknrU724v/iG27D3FJmFClWdk BOn4dlJX47z5wrmX1brJKamYHq4sCwS857S0ZQoNCMt6eLu7izlgA2RfYWKSnHKEcy1o oIJ9glDBRmqh5QuzaCjFm4u/ssjnuecuS/iAaEPDvlVpZRhScofMOMKV6OAjUAis4vGp Q2gjqTYhOVjI/+cFPNtbftMGENiTF22U+r4kvWY6jY0X8zAhMDIcQXQUdjBKx2YbKJp6 3+VvAVHp5lQkaujqo/8IY6FT0xq1w3egc41PP2rnNpEfr3AZo8KAybiVVCapOoeDunBk Y90g== X-Gm-Message-State: AOAM531lv4DanSxWXk9eG3kqOCBk1qBTsAojkbhZ5kWqoinHMrbS/Gkp XU5NWtS8A99dkDzB23IdD4Q= X-Google-Smtp-Source: ABdhPJy86mmzeil+UO0E+WD8Pf4rBWTQtnY81BeGKheOwCfwVh5wPfKST0I0x/5YOArnpnoWMy9n4Q== X-Received: by 2002:a05:6402:10d7:: with SMTP id p23mr3978035edu.74.1627535478499; Wed, 28 Jul 2021 22:11:18 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:18 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 80/87] trace-cmd library: Extend tracecmd_copy() API Date: Thu, 29 Jul 2021 08:09:52 +0300 Message-Id: <20210729050959.12263-81-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The tracecmd_copy() API is used to copy data between trace files. Extended it to support copying data bwteen files from different versions. Added also functionality to copy all sections between the files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 ++- lib/trace-cmd/trace-output.c | 38 +++++++++++++++---- tracecmd/trace-restore.c | 2 +- tracecmd/trace-split.c | 2 +- 4 files changed, 36 insertions(+), 11 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 3c3c98aa..fd7b17e1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -324,8 +324,9 @@ int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file); +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, + enum tracecmd_file_states state, int file_version, + const char *compression); int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files, const char *buff_name); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index b718770d..e45369d3 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2511,14 +2511,22 @@ struct tracecmd_output *tracecmd_create_init_file(const char *output_file) * tracecmd_copy - copy the headers of one trace.dat file for another * @ihandle: input handle of the trace.dat file to copy * @file: the trace.dat file to create + * @state: what data will be copied from the source handle + * @compression: compression of the output file, can be one of: + * NULL - inherit compression from the input file + * "any" - compress the output file with the best available algorithm + * "none" - do not compress the output file + * algorithm_name - compress the output file with specified algorithm * * Reads the header information and creates a new trace data file * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, + enum tracecmd_file_states state, int file_version, + const char *compression) { + enum tracecmd_file_states fstate; struct tracecmd_output *handle; int fd; @@ -2531,17 +2539,33 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, goto out_free; if (tracecmd_output_set_from_input(handle, ihandle)) goto out_free; + if (file_version >= FILE_VERSION_MIN) + tracecmd_output_set_version(handle, file_version); + if (compression && tracecmd_output_set_compression(handle, compression)) + goto out_free; tracecmd_output_write_init(handle); - if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) + fstate = state > TRACECMD_FILE_CPU_COUNT ? TRACECMD_FILE_CPU_COUNT : state; + if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; - - handle->file_state = tracecmd_get_file_state(ihandle); - + if (tracecmd_copy_buffer_descr(ihandle, handle) < 0) + goto out_free; + if (state >= TRACECMD_FILE_OPTIONS && + tracecmd_copy_options(ihandle, handle) < 0) + goto out_free; + if (state >= TRACECMD_FILE_CPU_LATENCY && + tracecmd_copy_trace_data(ihandle, handle) < 0) + goto out_free; + if (handle->file_version >= 7) + tracecmd_write_options(handle); /* The file is all ready to have cpu data attached */ return handle; out_free: - tracecmd_output_close(handle); + if (handle) + tracecmd_output_close(handle); + else + close(fd); + unlink(file); return NULL; } diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index fbda42dd..eedf6b49 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -153,7 +153,7 @@ void trace_restore (int argc, char **argv) if (tracecmd_read_headers(ihandle, TRACECMD_FILE_CMD_LINES) < 0) die("error reading file %s headers", input); - handle = tracecmd_copy(ihandle, output); + handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES, 0, NULL); tracecmd_close(ihandle); } else handle = tracecmd_create_init_file(output); diff --git a/tracecmd/trace-split.c b/tracecmd/trace-split.c index 0768329f..3759539c 100644 --- a/tracecmd/trace-split.c +++ b/tracecmd/trace-split.c @@ -348,7 +348,7 @@ static double parse_file(struct tracecmd_input *handle, dir = dirname(output); base = basename(output); - ohandle = tracecmd_copy(handle, output_file); + ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES, 0, NULL); cpus = tracecmd_cpus(handle); cpu_data = malloc(sizeof(*cpu_data) * cpus); From patchwork Thu Jul 29 05:09:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A71CC4320A for ; Thu, 29 Jul 2021 05:11:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B9B360041 for ; Thu, 29 Jul 2021 05:11:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234155AbhG2FLY (ORCPT ); Thu, 29 Jul 2021 01:11:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234153AbhG2FLX (ORCPT ); Thu, 29 Jul 2021 01:11:23 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9F59C061757 for ; Wed, 28 Jul 2021 22:11:20 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id o5so8485719ejy.2 for ; Wed, 28 Jul 2021 22:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oaOQsPQWt+4/gfHtzVQ0CEx87Z5rT8r8+pQ4gqvBav0=; b=p2aBW1+1DGkBDUhJLoWHaXEdlI4iFTU5N6dl1HwYm3Q6CEEs7rBzZ2AXTQxWcSJJOp 5PO1gJXpuxD5MhvwU+7lam9JiUkYBA/7TttRKv7X6T4rxCC1pw+292DA60+DoffVCTYO N7JOUjhScdQI/isOCiy82xQ1PtKaBsQKg8KUdgS8522VlZ5bw1D35Fjhge/ogixgDqgI dnOqqOqlCWet5hXMw5eXWGod1SOnhk8xPvMTCM9qxTGRCvx87qCoQ1vBZCGQJWKxSWRH K6c96DvJCFt2fJiKkGHWSfyXoWLGYu/mY5MeSrv5GRbY0V2AcoDHPWuZpebM2hcc1aII tI6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oaOQsPQWt+4/gfHtzVQ0CEx87Z5rT8r8+pQ4gqvBav0=; b=M+fc9snqzw/MiLViwPHeidg0yH8bs2jWL+twroWhd7GzCl74dD2QYSje3W4WL8eaPi yUMOQ4FEnmmAGqmJmdmX1jleIW3ujzyfFdzFrQoJJd314g4ZlWW3ckhrZc5HoOF/q49e pTfkIFmstvxsPdCWLa6c0cdJ9uKDDqfNekrFDoxdQdFkSvXJwvHnIckcP62lfzGy/+h3 ZcvZoVO8yZKyLYCKnEi7WuIb4NbDk7AfRtEtvE1kJshzsn9zEICitpCHvfHHwoFHmZlP HMRBMYQtovHqH1M+z0qY7v8D1ZHnozLHhpn7oarusgA2ALS3Ttjpfp0KNFoD8ikKBqBz KQkg== X-Gm-Message-State: AOAM531fAHT1to1Y4JLCyo/YLVVDoPd6BAGvVzxe9rPBFigJ0oPZnn9P 8GkIAAK9T14mfEODMRXqiaU= X-Google-Smtp-Source: ABdhPJzVdhu6O2Vc+iAiTOXelMVI2n2bqoLPs1FU2ZRGP588R+D3cLkKzob+ghehRTknMyI7n2e/OA== X-Received: by 2002:a17:906:a445:: with SMTP id cb5mr3001761ejb.320.1627535479496; Wed, 28 Jul 2021 22:11:19 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:19 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 81/87] trace-cmd library: Set correct CPU to the record, retrieved with tracecmd_peek_data Date: Thu, 29 Jul 2021 08:09:53 +0300 Message-Id: <20210729050959.12263-82-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace files version 7 stores only CPUs with trace data in the file, empty CPUs are ommited. Fixed tracecmd_peek_data() to handle that case, set the CPU in the record - do not use CPU indfex, but CPU id. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b66d876f..df2e999c 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2403,7 +2403,7 @@ read_again: record->ts = handle->cpu_data[cpu].timestamp; record->size = kbuffer_event_size(kbuf); - record->cpu = cpu; + record->cpu = handle->cpu_data[cpu].cpu; record->data = data; record->offset = handle->cpu_data[cpu].offset + index; record->missed_events = kbuffer_missed_events(kbuf); From patchwork Thu Jul 29 05:09:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 376D2C43214 for ; Thu, 29 Jul 2021 05:11:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F8076023E for ; Thu, 29 Jul 2021 05:11:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234158AbhG2FLY (ORCPT ); Thu, 29 Jul 2021 01:11:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234153AbhG2FLY (ORCPT ); Thu, 29 Jul 2021 01:11:24 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7DCCC061757 for ; Wed, 28 Jul 2021 22:11:21 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id r16so6380895edt.7 for ; Wed, 28 Jul 2021 22:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MWZzrv494veUtpk3Vcs7uK4zsL4FcQwiH8pTZuD9Qgk=; b=llp70+579HXdcI80uYAnXV67CNfxOR9ey+ZMDcyd5iIS+b40triMTDKGlP0WVlzuuN LoD5wI1+RBbjykb4j93XHSoh2vSLFsHJUFLx2Zp3tmchGMls0dQphAdpOt4KZILVFDoW bUwxClE053xFcXruCGlsoOGlwERKryqRrOOOvsIJvmDNiFzZdGhxA7CscEvXKbkuP8RS OwZyAg5i9Ob9qxHp2POHtrK3IH/fvVjz3iiaWc+ugNYGliZ24OFlnk2+7mZp6zsFoNqL VpgPXSk4EANUErWwcCYggAXepQDsLkaT450KHqqEoxPGEbUsoUkObehg2MJgLBgNMW1S mRaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MWZzrv494veUtpk3Vcs7uK4zsL4FcQwiH8pTZuD9Qgk=; b=aoHtgS+djyYDZxvCUfC0t94vAZwkhpaLJRqu110wk2dyKgEln8hUWJPADm9jSAtLEk qlayc3tnHoztFl/gNGTVM8bagGej20pVntN36Nz81U+8DgfNo6qEIb+C2RWfMdCqDfl9 r4iT7qG8/r05fD+LqT3PiasCpiqwRxUHxi3u/4slMFaxuFTAUC3ta7l/FkeWxsDdiz2h sr9uHp+iodQdy4aBPvTIvrEzuSGL16xbdPRc3T9Ki18FLh3CVHvOtDFq1dfalpHkpRB1 EJwf/nhT5Xid03ddSvRQc4kfmQO85LYOTDtJbRn89aKLsS5I3kndZ1mQ/N8N8egrqlFW BU/g== X-Gm-Message-State: AOAM530gtWNGV+eggUf9KxUPFwTN8xaO+RcZPHfohuzwRgsVcumLg9OJ hPRpRJLq9v06jJiP+w8+V1p7jlTiC0bNew== X-Google-Smtp-Source: ABdhPJxWKfQO9sgut4Fvp7T/EJezf1qVDsHPYWHNEaiHAWKhVOEV4i0xT97P1/oJ7r0mXB8f100sFQ== X-Received: by 2002:a05:6402:1118:: with SMTP id u24mr3928099edv.229.1627535480400; Wed, 28 Jul 2021 22:11:20 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:19 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 82/87] trace-cmd: Add new subcommand "convert" Date: Thu, 29 Jul 2021 08:09:54 +0300 Message-Id: <20210729050959.12263-83-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org There is a need to convert trace files between different versions. A new "trace-cmd convert" subcommand is introduced, which reads and input trace file and copies to an output trace file. Both files can be from different versions. trace-cmd convert -i -o --file-version --compression Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/Makefile | 1 + tracecmd/include/trace-local.h | 2 + tracecmd/trace-cmd.c | 1 + tracecmd/trace-convert.c | 106 +++++++++++++++++++++++++++++++++ tracecmd/trace-usage.c | 13 ++++ 5 files changed, 123 insertions(+) create mode 100644 tracecmd/trace-convert.c diff --git a/tracecmd/Makefile b/tracecmd/Makefile index b7a23dc4..56742f0a 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -36,6 +36,7 @@ TRACE_CMD_OBJS += trace-usage.o TRACE_CMD_OBJS += trace-dump.o TRACE_CMD_OBJS += trace-clear.o TRACE_CMD_OBJS += trace-vm.o +TRACE_CMD_OBJS += trace-convert.o ifeq ($(VSOCK_DEFINED), 1) TRACE_CMD_OBJS += trace-agent.o diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index e9a0aea8..13dab44c 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -110,6 +110,8 @@ void trace_usage(int argc, char **argv); void trace_dump(int argc, char **argv); +void trace_convert(int argc, char **argv); + int trace_record_agent(struct tracecmd_msg_handle *msg_handle, int cpus, int *fds, int argc, char **argv, bool use_fifos, diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 9fc126e4..a83a8d0b 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -133,6 +133,7 @@ struct command commands[] = { {"list", trace_list}, {"help", trace_usage}, {"dump", trace_dump}, + {"convert", trace_convert}, {"-h", trace_usage}, }; diff --git a/tracecmd/trace-convert.c b/tracecmd/trace-convert.c new file mode 100644 index 00000000..2484d529 --- /dev/null +++ b/tracecmd/trace-convert.c @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov + */ +#include +#include +#include +#include +#include + +#include "trace-local.h" +#include "trace-cmd.h" +#include "trace-cmd-private.h" + +static void convert_file(const char *in, const char *out, int file_version, char *compr) +{ + struct tracecmd_input *ihandle; + struct tracecmd_output *ohandle; + + ihandle = tracecmd_open_head(in, 0); + if (!ihandle) + die("error reading %s", in); + ohandle = tracecmd_copy(ihandle, out, TRACECMD_FILE_CPU_FLYRECORD, file_version, compr); + if (!ohandle) + die("error writing %s", out); + tracecmd_output_close(ohandle); + tracecmd_close(ihandle); +} + +enum { + OPT_file_version = 254, + OPT_comporession = 255, +}; + +void trace_convert(int argc, char **argv) +{ + char *input_file = NULL; + char *output_file = NULL; + char *compression = NULL; + int file_version = FILE_VERSION_DEFAULT; + int c; + + if (argc < 2) + usage(argv); + + if (strcmp(argv[1], "convert") != 0) + usage(argv); + for (;;) { + int option_index = 0; + static struct option long_options[] = { + {"compression", required_argument, NULL, OPT_comporession}, + {"file-version", required_argument, NULL, OPT_file_version}, + {"help", no_argument, NULL, '?'}, + {NULL, 0, NULL, 0} + }; + + c = getopt_long (argc-1, argv+1, "+hi:o:", long_options, &option_index); + if (c == -1) + break; + switch (c) { + case 'i': + if (input_file) + die("Only one input file is supported, %s already set", + input_file); + input_file = optarg; + break; + case 'o': + if (output_file) + die("Only one output file is supported, %s already set", + output_file); + output_file = optarg; + break; + case OPT_comporession: + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + compression = optarg; + break; + case OPT_file_version: + file_version = atoi(optarg); + if (file_version < FILE_VERSION_MIN || file_version > FILE_VERSION_MAX) + die("Unsupported file version %d, " + "supported versions are from %d to %d", + file_version, FILE_VERSION_MIN, FILE_VERSION_MAX); + + break; + case 'h': + case '?': + default: + usage(argv); + } + } + + if ((argc - optind) >= 2) { + if (output_file) + usage(argv); + output_file = argv[optind + 1]; + } + + if (!input_file) + input_file = DEFAULT_INPUT_FILE; + if (!output_file) + usage(argv); + + convert_file(input_file, output_file, file_version, compression); +} diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index f463465a..9eb13ecb 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -407,6 +407,19 @@ static struct usage_help usage_help[] = { " -h, --help show usage information\n" " --verbose 'level' Set the desired log level\n" }, + { + "convert", + "convert trace file to different version", + " %s convert [options]\n" + " -i input file, default is trace.dat\n" + " -o output file, mandatory parameter.\n" + " The output file can be specified also as last argument of the command\n" + " --file-version set the desired trace file version\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { NULL, NULL, NULL } From patchwork Thu Jul 29 05:09:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54583C4338F for ; Thu, 29 Jul 2021 05:11:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 32EA560C3F for ; Thu, 29 Jul 2021 05:11:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234161AbhG2FLZ (ORCPT ); Thu, 29 Jul 2021 01:11:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234153AbhG2FLZ (ORCPT ); Thu, 29 Jul 2021 01:11:25 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6670C061757 for ; Wed, 28 Jul 2021 22:11:22 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id f13so6343908edq.13 for ; Wed, 28 Jul 2021 22:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mbrlD10Y1hBDYOmV/4CQJbgk1L4szY/IgjNe1kinDHA=; b=LHHvfVkLWptQ1EQ+z45WlJ8jr1fQ0b0LRTCphFffQzuTL9BVgb+7XHtckdZEquVhnU htOKNg382fKdShlCyEaOt/VxuAo2eqkXeQLvAxHZ8FDOr6hDx7wvTBJKqBJXbp0ViwMj DM5bMQcvHyfGvfkOTf6d1kDjyeac5ecybnmicIf9OhzEs0gPcC+Qrfa2JMI7SqHXRNDx fH2yOmanONYJrKvY4DRAQdVU/O1EcZstlIxiUNYHZvPIL+FaTWSi9dfGkle6NAWcX5gI ZfbtV9OcUtpJyPlm8WXSOdgOZ61m5KsqrDoXhmAo0t4vedw3tUfryjq4TL81IkDB+wHw vvOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mbrlD10Y1hBDYOmV/4CQJbgk1L4szY/IgjNe1kinDHA=; b=e1BJk9nHi1tQTZA+tj5u7kNsrCOZ7U4c90XdGqOTCB1NDl/KDfOdmAo04jqrc065hx coyUCUvqmX+aH3ogG/g0DgvI6rt9h8G5dFTAbwNErXgPs/ZrOVYaBZUFHQO9sHEkYXpm yzOEJNVeeTSn1FO40o+/i95FdlqxMeXoUz2izIrqIZp+m+J20oLuZKeQ5hjNPC/dZTWJ vKw8bPeOGsLfXKC9iwSVnKzMofw6aYAp7H26rMhIPS02IiKR5LM+ZjdH3JutxNuwcjW+ tpZ4rwOqUq0WXDH5X45f27im/InB3p9M5vYZB3RP7ARMb0eDu7VJBrn4K37CyNloPwnh zoJw== X-Gm-Message-State: AOAM530vW1mQtyxGPSp/eZx01htGfwk0asSlMQhTPztTY5C/eUEmQix4 NeUN3uWKxDpdO86oJ/tIA60= X-Google-Smtp-Source: ABdhPJwYedaWLpuAFgwE88DSK29FuMNIlbmLOYtUgn1p8Bvi0mjGf2/VNnbhkXTu9DZE/k4y/99WMA== X-Received: by 2002:a50:fe10:: with SMTP id f16mr3862817edt.85.1627535481343; Wed, 28 Jul 2021 22:11:21 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:20 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 83/87] trace-cmd report: Add new parameters for version 7 trace files Date: Thu, 29 Jul 2021 08:09:55 +0300 Message-Id: <20210729050959.12263-84-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org By default, "trace-cmd report" writes in trace file version 7 and no trace file compression. A new parameters are added, which can be used to set desired verrsion and compression of the output trace file. "trace-cmd report --file-version --compression " Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 28 ++++++++++++++++++++++++++++ tracecmd/trace-usage.c | 7 ++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index ddab7798..7013aee3 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -5810,6 +5810,8 @@ void init_top_instance(void) } enum { + OPT_file_ver = 237, + OPT_comporession = 238, OPT_verbose = 239, OPT_tsc2nsec = 240, OPT_fork = 241, @@ -6249,6 +6251,8 @@ static void parse_record_options(int argc, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, {"poll", no_argument, NULL, OPT_poll}, {"verbose", optional_argument, NULL, OPT_verbose}, + {"compression", required_argument, NULL, OPT_comporession}, + {"file-version", required_argument, NULL, OPT_file_ver}, {NULL, 0, NULL, 0} }; @@ -6674,6 +6678,30 @@ static void parse_record_options(int argc, cmd_check_die(ctx, CMD_set, *(argv+1), "--poll"); recorder_flags |= TRACECMD_RECORD_POLL; break; + case OPT_comporession: + cmd_check_die(ctx, CMD_start, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_set, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_extract, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_stream, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_profile, *(argv+1), "--compression"); + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + ctx->compression = strdup(optarg); + break; + case OPT_file_ver: + cmd_check_die(ctx, CMD_start, *(argv+1), "--file_version"); + cmd_check_die(ctx, CMD_set, *(argv+1), "--file_version"); + cmd_check_die(ctx, CMD_extract, *(argv+1), "--file_version"); + cmd_check_die(ctx, CMD_stream, *(argv+1), "--file_version"); + cmd_check_die(ctx, CMD_profile, *(argv+1), "--file_version"); + ctx->file_version = atoi(optarg); + if (ctx->file_version < FILE_VERSION_MIN || + ctx->file_version > FILE_VERSION_MAX) + die("Unsupported file version %d, " + "supported versions are from %d to %d", + ctx->file_version, FILE_VERSION_MIN, FILE_VERSION_MAX); + break; case OPT_quiet: case 'q': quiet = true; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 9eb13ecb..ec0af46f 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -69,7 +69,12 @@ static struct usage_help usage_help[] = { " If 0 is specified, no loop is performed - timestamps offset is calculated only twice," " at the beginnig and at the end of the trace\n" " --poll don't block while reading from the trace buffer\n" - " --verbose 'level' Set the desired log level\n" + " --file-version set the desired trace file version\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { "set", From patchwork Thu Jul 29 05:09:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91FEAC4338F for ; Thu, 29 Jul 2021 05:11:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7C44A60EB2 for ; Thu, 29 Jul 2021 05:11:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234163AbhG2FL2 (ORCPT ); Thu, 29 Jul 2021 01:11:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234153AbhG2FL1 (ORCPT ); Thu, 29 Jul 2021 01:11:27 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 995D9C061757 for ; Wed, 28 Jul 2021 22:11:23 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id gn26so8489994ejc.3 for ; Wed, 28 Jul 2021 22:11:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bDE5H6ouGRCAEZ9YfNrgle8dTCkLnHF61NMoDExWoeY=; b=ES5MtIL5vwNF5TaBt3d4wKZpa/mdKfW1ElpmRXJ4vwZXznbucNY1gyAWqWnrWG+NQo DTbkCOWaqOePJMSlPKDXWsLunttQ9nwoB0GX9B+BFwW8wz2fv2RTvissCaFryQHwmdlu NBbOPg77slDiu05nHqy8wzlS9iyW/GLVMaGDCQnGYQ97Xq99Z20Ej955kI0n4qrvScUC zgcZbW7mgeCu5jxu0M8Wh8Cv+sjb86dztXOak26DY3Q19Lbh7GQqjp7+c+J+eCeuXzng fvOKHwqExbflbSrEzWMJrHEFQ+4AQVub5SHPCUWQTSleb1wdTjS3unVsiWd2qESwLihM acNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bDE5H6ouGRCAEZ9YfNrgle8dTCkLnHF61NMoDExWoeY=; b=WDShIEFWDPd9KAXJ+jcgqPPR+HRDC+cvVl2lY42WWnx+h7tfZ2cDfOxgQaHrv3fYby SoEAeidBzxmD363/06OHJmBvV9OF6dcbrMF0PSZiioKsVzYNTnlqPgFnCjryFYCxKyv7 9kZyKNx339r/nke1q3jJjwWUiTp4AVM/HJMtm12jlKBUyGQCU3MMiMyEofDkSkNiU+qn VXAsg2Cfv99Mml9MYF0yxsuKljxI3wpfIwgXhqmukCNIfxOm9xTQ22wNfzQOpHtvZnQb 1iOV8FZFnFh66yjeB3wX1Z924Gw+bdYVnnFEubRztCvMunKxgrlPr0h5QyOmRcOoU51l 68pw== X-Gm-Message-State: AOAM5338pXM2m1GM6Jc4aEixF4U0CTIOOMbHqcGi6YdFUk773YBH0NwF k7l4dY2YyV17YlsGcQ/ezl4UAfkfAWGouQ== X-Google-Smtp-Source: ABdhPJx9qoMQ0GelWilN5gcI4Vl+0/s5H9WB9/csADOht0s7CJXhmLvm0Tz2Bg/+v/awPqKEn917ow== X-Received: by 2002:a17:906:9246:: with SMTP id c6mr2786307ejx.78.1627535482296; Wed, 28 Jul 2021 22:11:22 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:21 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 84/87] trace-cmd: Update bash completion Date: Thu, 29 Jul 2021 08:09:56 +0300 Message-Id: <20210729050959.12263-85-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added convert subcommand and its arguments to bash completion logic. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-cmd.bash | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/tracecmd/trace-cmd.bash b/tracecmd/trace-cmd.bash index b01c7a07..6639c143 100644 --- a/tracecmd/trace-cmd.bash +++ b/tracecmd/trace-cmd.bash @@ -64,6 +64,13 @@ plugin_options() COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) } +compression_param() +{ + local opts=$(trace-cmd list -c | grep -v 'Supported' | cut -d "," -f1) + opts+=" any none " + COMPREPLY=( $(compgen -W "${opts}") ) +} + __trace_cmd_list_complete() { local prev=$1 @@ -181,6 +188,9 @@ __trace_cmd_record_complete() cmd_options record "$cur" fi ;; + --compression) + compression_param + ;; *) # stream start and profile do not show all options cmd_options record "$cur" @@ -222,6 +232,29 @@ __trace_cmd_dump_complete() esac } +__trace_cmd_convert_complete() +{ + local prev=$1 + local cur=$2 + shift 2 + local words=("$@") + + case "$prev" in + -i) + __show_files + ;; + -o) + __show_files + ;; + --compression) + compression_param + ;; + *) + cmd_options convert "$cur" + ;; + esac +} + __show_command_options() { local command="$1" @@ -298,6 +331,10 @@ _trace_cmd_complete() __trace_cmd_dump_complete "${prev}" "${cur}" ${words[@]} return 0 ;; + convert) + __trace_cmd_convert_complete "${prev}" "${cur}" ${words[@]} + return 0 + ;; *) __show_command_options "$w" "${prev}" "${cur}" ;; From patchwork Thu Jul 29 05:09:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 692BEC4338F for ; Thu, 29 Jul 2021 05:11:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5252361019 for ; Thu, 29 Jul 2021 05:11:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234181AbhG2FLj (ORCPT ); Thu, 29 Jul 2021 01:11:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234164AbhG2FL2 (ORCPT ); Thu, 29 Jul 2021 01:11:28 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D78DC061765 for ; Wed, 28 Jul 2021 22:11:24 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id ga41so8411848ejc.10 for ; Wed, 28 Jul 2021 22:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jFqYVxkdWFkFLbm8eow9LgOG4BZkciuN5sl8VyVpIDY=; b=YJ89dvfovicjPPo+VKBTtHrfP+REkZJDEbmujh7Fj84RDKXQasH9raDgR/TQYYPLIQ BQAHhZR331sMJ9S64CN+ItHALTAThb/EmD78bZod4NQBGgDrD5BzFVBJl2zFvAdpWrvd 34gIj3DQ6nygfvXIx1aoDwYqhcHC+noHTdtqN1/L2mOwcrsKEJCTPejanDGupE5wVcmD MCkNe49SjULvYJJG+PhxX3gQNfI0lzeCOnCsWou0i7rxsPqr9CmcwUJe9sackLjbppBe IPjxPf2Vmrnp4SuMs2jsyuCAo+N5/p8KDGPDlTB1ve5MN4o7fKVGU9bOGnmJ4mUI1T7U SfRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jFqYVxkdWFkFLbm8eow9LgOG4BZkciuN5sl8VyVpIDY=; b=rI+7TqmoUdkTAasCWXvHwFZGd9ud87CprynPkuDRDv4l5VI+VJzfBd7ecQNRzuXGne 9yIOv6sjqObdFk38Bzb3Pl8fChZfUWvwdxDSqDU06XfrBQuPE0bXemLDk5CtpSPG6YtO fglJqwDxUgRjMLIDEmMxyO6UQd5Yti5fVa+21YKm9P2pNiC9mDHJpJt3yJ/bpid48Llk cuQUFE3HWOV6IsW1aABIc5zqfoCOJ0S55xksoItcXvKho2JIdFMQE4+n7w/BJDrccynh 0uzNc9e9AM9Ts20QhjW1kLqCj64KOY7EatFL5AKu3nV6miwd+NFv0ngtIv8aQaXYkCeM ISsg== X-Gm-Message-State: AOAM533fEzg1bjAQGFf3T7DAtNpP1MT8+eO8oRazaW34/YwTB6OudJeW XAYi82xkZmrEigHxlE+LTVo= X-Google-Smtp-Source: ABdhPJxnvs90NvZINxw7tf/JL4ym5GZqWNHmXuusP2jyNj7KfPAs7LcMKPsfqzNnUiswSDRpLJNOXQ== X-Received: by 2002:a17:906:7716:: with SMTP id q22mr2899697ejm.457.1627535483186; Wed, 28 Jul 2021 22:11:23 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:22 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 85/87] tarce-cmd: Man page for "trace-cmd convert" Date: Thu, 29 Jul 2021 08:09:57 +0300 Message-Id: <20210729050959.12263-86-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Documented new "trace-cmnd convert" subcommand. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../trace-cmd/trace-cmd-convert.1.txt | 65 +++++++++++++++++++ Documentation/trace-cmd/trace-cmd.1.txt | 4 +- 2 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 Documentation/trace-cmd/trace-cmd-convert.1.txt diff --git a/Documentation/trace-cmd/trace-cmd-convert.1.txt b/Documentation/trace-cmd/trace-cmd-convert.1.txt new file mode 100644 index 00000000..7c13cf3d --- /dev/null +++ b/Documentation/trace-cmd/trace-cmd-convert.1.txt @@ -0,0 +1,65 @@ +TRACE-CMD-CONVERT(1) +=================== + +NAME +---- +trace-cmd-convert - convert trace files + +SYNOPSIS +-------- +*trace-cmd convert* ['OPTIONS'] ['output-file'] + +DESCRIPTION +----------- +The trace-cmd(1) convert command converts trace file. It reads the input file and copies the data +into an output file. The output file may be in different format, depending on the command line +arguments. + +OPTIONS +------- +*-i* 'input-file':: + By default, trace-cmd convert will read the file 'trace.dat'. But the *-i* + option open up the given 'input-file' instead. + +*-o* 'out-file':: + The name of the output file, this parameter is mandatory. Note, the output file may also be + specified as the last item on the command line. + +*--file-version*:: + Desired version of the output file. Supported versions are 6 or 7. + +*--compression*:: + Compression of the trace output file, one of these strings can be passed: + + 'any' - auto select the best available compression algorithm + + 'none' - do not compress the trace file + + 'name' - the name of the desired compression algorithms. Available algorithms can be listed with + trace-cmd list -c + +*--help*:: + Print usage information. + +EXAMPLES +-------- + +# trace-cmd convert --compression any trace_compress.dat + +SEE ALSO +-------- +trace-cmd(1), trace-cmd.dat(1) + +AUTHOR +------ +*Steven Rostedt* , author of *trace-cmd*. +*Tzvetomir Stoyanov* , author of this man page. + +RESOURCES +--------- +https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/ + +COPYING +------- +Copyright \(C) 2021 VMware. Free use of this software is granted under +the terms of the GNU Public License (GPL). diff --git a/Documentation/trace-cmd/trace-cmd.1.txt b/Documentation/trace-cmd/trace-cmd.1.txt index b330b4fe..27c6e973 100644 --- a/Documentation/trace-cmd/trace-cmd.1.txt +++ b/Documentation/trace-cmd/trace-cmd.1.txt @@ -58,6 +58,8 @@ COMMANDS check-events - parse format strings for all trace events and return whether all formats are parseable + convert - convert trace files + OPTIONS ------- @@ -70,7 +72,7 @@ SEE ALSO -------- trace-cmd-record(1), trace-cmd-report(1), trace-cmd-hist(1), trace-cmd-start(1), trace-cmd-stop(1), trace-cmd-extract(1), trace-cmd-reset(1), -trace-cmd-restore(1), trace-cmd-stack(1), +trace-cmd-restore(1), trace-cmd-stack(1), trace-cmd-convert(1), trace-cmd-split(1), trace-cmd-list(1), trace-cmd-listen(1), trace-cmd.dat(5), trace-cmd-check-events(1) trace-cmd-stat(1) From patchwork Thu Jul 29 05:09:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DF6EC4338F for ; Thu, 29 Jul 2021 05:11:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6042E61053 for ; Thu, 29 Jul 2021 05:11:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234153AbhG2FLd (ORCPT ); Thu, 29 Jul 2021 01:11:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234181AbhG2FL2 (ORCPT ); Thu, 29 Jul 2021 01:11:28 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2E78C061757 for ; Wed, 28 Jul 2021 22:11:25 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id x11so7519097ejj.8 for ; Wed, 28 Jul 2021 22:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7/tzADKHCs+G6aHZyQMgF666aPdEc8iV9Wpqr5trpyE=; b=tyXkNNzPNq2tIoa0ZM3XQrhNSaeuNONmWk6cT5j2ojM5iHo/lwkQGJSQloLzY1FqDN giYV+xUKz4NRYCmzyz8WWzPfMVl6O7uf3TP11d9eothQH31G6EKzf2MKV6ZscMFkgo51 7hphsc1nhl7bChCHCep158koWpS+w14JUMTodXLIRAykfrIbKD9+1CIQzo06U6PcHEQv sYq5UWbkya79PzyOHtBoDm0RCp5Ac0hvPwrwFdW0cKfNfXt7ACsUAWJqbJlBeuaapnfd 5sLI+Ihjkw79J7SJC7KE4iDagt73bh7hDgxR+tBuG/cFMuvO5ZpqxowURcx6lWPUg7d/ jlhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7/tzADKHCs+G6aHZyQMgF666aPdEc8iV9Wpqr5trpyE=; b=bXuLNzNsJpGv2yZwIgZRnxspXydzkzcbqikcN2ei3HFdZWnEjGzEu9+klz4oKHuMNh 79+lxQtQ226W1og0o6HMx94Tz7auyRjS3pXrtI+ZpwBDWNyUeHnd4KRkHaqA52AQILUq nGAJ2DGY75kFdQ9h6dQV6MFxQk+lj9A4pCneJFbsDJZOCGFrZG8A9AdvothtOezH7TJC 9gzNFHnYvrGZ/pAlxK1qlVzY5Id+X3LCD4DezjODP5A1nmspXOzieQAdI32No4bUGQ7a vMyKdrZSVXm5zi2VnpHphdkN6dm2NYbVxatY/6JkqdHMEIBV7SZBzSpzVsIxnL1GA6DA 6pOw== X-Gm-Message-State: AOAM531IogXnVb5Oxn8w7SDFR9P8/X8yRyX8UGNrwQ5mIDbBsmomrYai yS/knD55WL5BTpASuvh3Izo= X-Google-Smtp-Source: ABdhPJzIQnC1rkOFIS0zQBIN0A1PDw4zL3aw7tnFtuUfvlEfeUgPyR5kZ/oLK6g45VfyniOj2lEibw== X-Received: by 2002:a17:906:6d4e:: with SMTP id a14mr2853150ejt.328.1627535484332; Wed, 28 Jul 2021 22:11:24 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:23 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 86/87] tarce-cmd: Update record man page Date: Thu, 29 Jul 2021 08:09:58 +0300 Message-Id: <20210729050959.12263-87-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Documented new arguments of the "trace-cmd record" subcommand: --compression --file-version Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd/trace-cmd-record.1.txt | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/Documentation/trace-cmd/trace-cmd-record.1.txt b/Documentation/trace-cmd/trace-cmd-record.1.txt index 96c27108..b3620474 100644 --- a/Documentation/trace-cmd/trace-cmd-record.1.txt +++ b/Documentation/trace-cmd/trace-cmd-record.1.txt @@ -377,6 +377,19 @@ OPTIONS trace-cmd record --verbose=warning +*--file-version*:: + Desired version of the output file. Supported versions are 6 or 7. + +*--compression*:: + Compression of the trace output file, one of these strings can be passed: + + 'any' - auto select the best available compression algorithm + + 'none' - do not compress the trace file + + 'name' - the name of the desired compression algorithms. Available algorithms can be listed with + trace-cmd list -c + EXAMPLES -------- From patchwork Thu Jul 29 05:09:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12407787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC405C4320A for ; Thu, 29 Jul 2021 05:11:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 977B261052 for ; Thu, 29 Jul 2021 05:11:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234164AbhG2FLl (ORCPT ); Thu, 29 Jul 2021 01:11:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230257AbhG2FLb (ORCPT ); Thu, 29 Jul 2021 01:11:31 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5734C0613C1 for ; Wed, 28 Jul 2021 22:11:26 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id go31so8449479ejc.6 for ; Wed, 28 Jul 2021 22:11:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E5tACfbD6gZxN3/AxsrkK5neSrDnowEsGM+c8JMiHmU=; b=W/bx3p+IjM58ZkJD1JAMf4C67qMY/ycF9n5YBUQlqu8HkyA5T/hICS325D8iCWpwBZ SXVsHTsTldbyp7IMsitGUp+/ueeKA/ieiiQcD/1VqyYRjGdOv+1AI9PMZQlIn53NBA1L NqVcSss9s3MrRIIA//M5bcYMEXHOWhGK4upeeCKnzhkQ4TooR/JQa8+rM2ZrrFb3rPft dRIMb6sWUQzt1PVhZIAn8eCpw2/Cv/d01PW+vZvD1MCNqhpJNJulPTVGl3IsN/aDqcgD LpfAMANLvz+Wr618+SP7mnZzD9UNdVcP8kmT9DA3RzZY9Ed9hGVwnrSVxGMQhIoF3Tcg fG9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E5tACfbD6gZxN3/AxsrkK5neSrDnowEsGM+c8JMiHmU=; b=o5jnj2rZBdoqgX238yNxU8DJnM5i4JAWhUsX8KJaYNFIfCU9zQBt86AsSefxATfug6 47suluY8O5+v5cjVvO1rSvjp2yrbtOg0WIeF62Cb6hc3w0xCSjUeGnr2E9PnOO4vOCAv rITAanHmeknNgldItx4SOi3cDyZZMQ2+B89Tsh43rdiwi0BRld68v9rygeeRO+9Ht56O aO+VxqpbWsfcu4/jWQ1y4QPPYuR6RBSf4n7HKovyUqSX89GwB3RgLYT8X9CNufk7M8Te o4WAxr1UU/A4Sr13jOOPCcZ3XeadVXqkFpk0arGvBE2rvbra02z09W6FEkWESrtraFXb Wnkw== X-Gm-Message-State: AOAM532gVpu91MiCpxkr3+eFZ67QVVR2INcqMXj6eUdGoMo0ChAneBOE mxOLO5mTgRJATVSL6yTfYRU= X-Google-Smtp-Source: ABdhPJxpvuCr2PKKiy0wLdjlGmTwMFSmfYkn8peuS+2OTLIiM/tR18QlTV4SjdxPpI7Q2K3NPa5LGg== X-Received: by 2002:a17:906:14c8:: with SMTP id y8mr2856376ejc.475.1627535485283; Wed, 28 Jul 2021 22:11:25 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id s20sm530615eji.116.2021.07.28.22.11.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jul 2021 22:11:24 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v2 87/87] trace-cmd: Document trace file version 7 Date: Thu, 29 Jul 2021 08:09:59 +0300 Message-Id: <20210729050959.12263-88-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210729050959.12263-1-tz.stoyanov@gmail.com> References: <20210729050959.12263-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Trace file versions 6 and 7 have a lot of differences. Added new man page describing the version 7 and renamed the existing trace-cmd.dat page for version 6 only. Signed-off-by: Tzvetomir Stoyanov (VMware) --- ...e-cmd.dat.5.txt => trace-cmd.dat.v6.5.txt} | 8 +- .../trace-cmd/trace-cmd.dat.v7.5.txt | 442 ++++++++++++++++++ 2 files changed, 446 insertions(+), 4 deletions(-) rename Documentation/trace-cmd/{trace-cmd.dat.5.txt => trace-cmd.dat.v6.5.txt} (98%) create mode 100644 Documentation/trace-cmd/trace-cmd.dat.v7.5.txt diff --git a/Documentation/trace-cmd/trace-cmd.dat.5.txt b/Documentation/trace-cmd/trace-cmd.dat.v6.5.txt similarity index 98% rename from Documentation/trace-cmd/trace-cmd.dat.5.txt rename to Documentation/trace-cmd/trace-cmd.dat.v6.5.txt index 8d285353..b412bfc7 100644 --- a/Documentation/trace-cmd/trace-cmd.dat.5.txt +++ b/Documentation/trace-cmd/trace-cmd.dat.v6.5.txt @@ -1,9 +1,9 @@ -TRACE-CMD.DAT(5) -================ +TRACE-CMD.DAT.v6(5) +=================== NAME ---- -trace-cmd.dat - trace-cmd file format +trace-cmd.dat.v6 - trace-cmd version 6 file format SYNOPSIS -------- @@ -30,7 +30,7 @@ INITIAL FORMAT "tracing" The next set of characters contain a null '\0' terminated string - that contains the version of the file (for example): + that contains the version of the file: "6\0" diff --git a/Documentation/trace-cmd/trace-cmd.dat.v7.5.txt b/Documentation/trace-cmd/trace-cmd.dat.v7.5.txt new file mode 100644 index 00000000..36dae643 --- /dev/null +++ b/Documentation/trace-cmd/trace-cmd.dat.v7.5.txt @@ -0,0 +1,442 @@ +TRACE-CMD.DAT.v7(5) +=================== + +NAME +---- +trace-cmd.dat.v7 - trace-cmd version 7 file format + +SYNOPSIS +-------- +*trace-cmd.dat* ignore + +DESCRIPTION +----------- +The trace-cmd(1) utility produces a "trace.dat" file. The file may also +be named anything depending if the user specifies a different output name, +but it must have a certain binary format. The file is used +by trace-cmd to save kernel traces into it and be able to extract +the trace from it at a later point (see *trace-cmd-report(1)*). + + +INITIAL FORMAT +-------------- + + The first three bytes contain the magic value: + + 0x17 0x08 0x44 + + The next 7 bytes contain the characters: + + "tracing" + + The next set of characters contain a null '\0' terminated string + that contains the version of the file: + + "7\0" + + The next 1 byte contains the flags for the file endianess: + + 0 = little endian + 1 = big endian + + The next byte contains the number of bytes per "long" value: + + 4 - 32-bit long values + 8 - 64-bit long values + + Note: This is the long size of the target's userspace. Not the + kernel space size. + + [ Now all numbers are written in file defined endianess. ] + + The next 4 bytes are a 32-bit word that defines what the traced + host machine page size was. + + The compression algorithm header is written next: + "name\0version\0" + where "name" and "version" are strings, name and version of the + compression algorithm used to compress the trace file. If the name + is "none", the data in the file is not compressed. + + The next 8 bytes are 64-bit integer, the offset within the file where + the first OPTIONS section is located. + + The rest of the file consists of different sections. The only mandatory + is the first OPTIONS section, all others are optional. The location and + the order of the sections is not strict. Each section starts with a header: + +FORMAT OF THE SECTION HEADER +---------------------------- + <2 bytes> unsigned short integer, ID of the section. + a null terminated ASCII string, description of the section. + <2 bytes> unsigned short integer, section flags: + 1 = the section is compressed. + <4 bytes> unsigned integer, size of the section in the file. + + If the section is compressed, the above is the compressed size. + The section must be uncompressed on reading. The described format of + the sections refers to the uncompressed data. + +COMPRESSION FORMAT OF THE FILE SECTIONS +--------------------------------------- + + Some of the sections in the file may be compressed with the compression algorithm, + specified in the compression algorithm header. Compressed sections have a compression + header, written after the section header and right before the compressed data: + <4 bytes> unsigned int, size of compressed data in this section. + <4 bytes> unsigned int, size of uncompressed data. + binary compressed data, with the specified size. + +COMPRESSION FORMAT OF THE TRACE DATA +------------------------------------ + + There are two special sections, BUFFER FLYRECORD and BUFFER LATENCY, containing + trace data. These sections may be compressed with the compression algorithm, specified + in the compression header. Usually the size of these sections is huge, that's why its + compression format is different from the other sections. The trace data is compressed + in chunks The size of one chunk is specified in the file creation time. The format + of compressed trace data is: + <4 bytes> unsigned int, count of chunks. + Follows the compressed chunks of given count. For each chunk: + <4 bytes> unsigned int, size of compressed data in this chunk. + <4 bytes> unsigned int, size of uncompressed data, aligned with the trace page size. + binary compressed data, with the specified size. + These chunks must be uncompressed on reading. The described format of + trace data refers to the uncompressed data. + +OPTIONS SECTION +--------------- + + Section ID: 0 + + This is the the only mandatory section in the file. There can be multiple + options sections, the first one is located at the offset specified right + after the compression algorithm header. The section consists of multiple + trace options, each option has the following format: + <2 bytes> unsigned short integer, ID of the option. + <4 bytes> unsigned integer, size of the option's data. + bytes of the size specified above, data of the option. + + + Options, supported by the trace file version 7: + + DONE: id 0, size 8 + This option indicates the end of the options section, it is written + always as last option. The DONE option data is: + <8 bytes> long long unsigned integer, offset in the trace file where + the next options section is located. If this offset is 0, then there + are no more options sections. + + DATE: id 1, size vary + The DATE option data is a null terminated ASCII string, which represents + the time difference between trace events timestamps and the Generic Time + of Day of the system. + + CPUSTAT: id 2, size vary + The CPUSTAT option data is a null terminated ASCII string, the content of the + "per_cpu/cpu/stats" file from the trace directory. There is a CPUSTAT option + for each CPU. + + BUFFER: id 3, size vary + The BUFFER option describes the flyrecord trace data saved in the file, collected + from one trace instance. There is BUFFER option for each trace instance. The format + of the BUFFER data is: + <8 bytes> long long unsigned integer, offset in the trace file where the + BUFFER FLYRECORD section is located, containing flyrecord trace data. + a null terminated ASCII string, name of the trace instance. Empty string "" + is saved as name of the top instance. + a null terminated ASCII string, trace clock used for events timestamps in + this trace instance. + <4 bytes> unsigned integer, count of the CPUs with trace data. + For each CPU of the above count: + <4 bytes> unsigned integer, ID of the CPU. + <8 bytes> long long unsigned integer, offset in the trace file where the trace data + for this CPU is located. + <8 bytes> long long unsigned integer, size of the trace data for this CPU. + + TRACECLOCK: id 4, size vary + The TRACECLOCK option data is a null terminated ASCII string, the content of the + "trace_clock" file from the trace directory. + + UNAME: id 5, size vary + The UNAME option data is a null terminated ASCII string, identifying the system where + the trace data is collected. The string is retrieved by the uname() system call. + + HOOK: id 6, size vary + The HOOK option data is a null terminated ASCII string, describing event hooks: custom + event matching to connect any two events together. + + OFFSET: id 7, size vary + The OFFSET option data is a null terminated ASCII string, representing a fixed time that + is added to each event timestamp on reading. + + CPUCOUNT: id 8, size 4 + The CPUCOUNT option data is: + <4 bytes> unsigned integer, number of CPUs in the system. + + VERSION: id 9, size vary + The VERSION option data is a null terminated ASCII string, representing the version of + the trace-cmd application, used to collect these trace logs. + + PROCMAPS: id 10, size vary + The PROCMAPS option data is a null terminated ASCII string, representing the memory map + of each traced filtered process. The format of the string is, for each filtered process: + \n + \n + ... + separate line for each library, used by this process + ... + ... + + TRACEID: id 11, size 8 + The TRACEID option data is a unique identifier of this tracing session: + <8 bytes> long long unsigned integer, trace session identifier. + + TIME_SHIFT: id 12, size vary + The TIME_SHIFT option stores time synchronization information, collected during host and guest + tracing session. Usually it is saved in the guest trace file. This information is used to + synchronize guest with host events timestamps, when displaying all files from this tracing + session. The format of the TIME_SHIFT option data is: + <8 bytes> long long unsigned integer, trace identifier of the peer (usually the host). + <4 bytes> unsigned integer, flags specific to the time synchronization protocol, used in this + trace session. + <4 bytes> unsigned integer, number of traced CPUs. For each CPU, timestamps corrections + are recorded: + <4 bytes> unsigned integer, count of the recorded timestamps corrections for this CPU. + , times when the corrections are calculated + , corrections offsets + , corrections scaling ratio + + GUEST: id 13, size vary + The GUEST option stores information about traced guests in this tracing session. Usually it is + saved in the host trace file. There is a separate GUEST option for each traced guest. + The information is used when displaying all files from this tracing session. The format of + the GUEST option data is: + a null terminated ASCII string, name of the guest. + <8 bytes> long long unsigned integer, trace identifier of the guest for this session. + <4 bytes> unsigned integer, number of guest's CPUs. For each CPU: + <4 bytes> unsigned integer, ID of the CPU. + <4 bytes> unsigned integer, PID of the host task, emulating this guest CPU. + + TSC2NSEC: id 14, size 16 + The TSC2NSEC option stores information, used to convert TSC events timestamps to nanoseconds. + The format of the TSC2NSEC option data is: + <4 bytes> unsigned integer, time multiplier. + <4 bytes> unsigned integer, time shift. + <8 bytes> unsigned long long integer, time offset. + + BUFFER_LAT: id 15, size + The BUFFER_LAT option describes the latency trace data saved in the file. The format + of the BUFFER_LAT data is: + <8 bytes> long long unsigned integer, offset in the trace file where the + BUFFER LATENCY section is located, containing latency trace data. + a null terminated ASCII string, name of the trace instance. Empty string "" + is saved as name of the top instance. + a null terminated ASCII string, trace clock used for events timestamps in + this trace instance. + + HEADER_INFO: id 15, size 8 + The HEADER_INFO option data is: + <8 bytes> long long unsigned integer, offset into the trace file where the HEADER INFO + section is located + + FTRACE_EVENTS: id 16, size 8 + The FTRACE_EVENTS option data is: + <8 bytes> long long unsigned integer, offset into the trace file where the + FTRACE EVENT FORMATS section is located. + + EVENT_FORMATS: id 17, size 8 + The EVENT_FORMATS option data is: + <8 bytes> long long unsigned integer, offset into the trace file where the EVENT FORMATS + section is located. + + KALLSYMS: id 18, size 8 + The KALLSYMS option data is: + <8 bytes> long long unsigned integer, offset into the trace file where the KALLSYMS + section is located. + + PRINTK: id 19, size 8 + The PRINTK option data is: + <8 bytes> long long unsigned integer, offset into the trace file where the TRACE_PRINTK + section is located. + + CMDLINES: id 20, size 8 + The CMDLINES option data is: + <8 bytes> long long unsigned integer, offset into the trace file where the + SAVED COMMAND LINES section is located. + +HEADER INFO SECTION +------------------- + + Section ID: 16 + + The first 12 bytes of the section, after the section header, contain the string: + + "header_page\0" + + The next 8 bytes are a 64-bit word containing the size of the + page header information stored next. + + The next set of data is of the size read from the previous 8 bytes, + and contains the data retrieved from debugfs/tracing/events/header_page. + + Note: The size of the second field \fBcommit\fR contains the target + kernel long size. For example: + + field: local_t commit; offset:8; \fBsize:8;\fR signed:1; + + shows the kernel has a 64-bit long. + + The next 13 bytes contain the string: + + "header_event\0" + + The next 8 bytes are a 64-bit word containing the size of the + event header information stored next. + + The next set of data is of the size read from the previous 8 bytes + and contains the data retrieved from debugfs/tracing/events/header_event. + + This data allows the trace-cmd tool to know if the ring buffer format + of the kernel made any changes. + +FTRACE EVENT FORMATS SECTION +---------------------------- + + Section ID: 17 + + Directly after the section header comes the information about + the Ftrace specific events. These are the events used by the Ftrace plugins + and are not enabled by the event tracing. + + The next 4 bytes contain a 32-bit word of the number of Ftrace event + format files that are stored in the file. + + For the number of times defined by the previous 4 bytes is the + following: + + 8 bytes for the size of the Ftrace event format file. + + The Ftrace event format file copied from the target machine: + debugfs/tracing/events/ftrace//format + +EVENT FORMATS SECTION +--------------------- + + Section ID: 18 + + Directly after the section header comes the information about + the event layout. + + The next 4 bytes are a 32-bit word containing the number of + event systems that are stored in the file. These are the + directories in debugfs/tracing/events excluding the \fBftrace\fR + directory. + + For the number of times defined by the previous 4 bytes is the + following: + + A null-terminated string containing the system name. + + 4 bytes containing a 32-bit word containing the number + of events within the system. + + For the number of times defined in the previous 4 bytes is the + following: + + 8 bytes for the size of the event format file. + + The event format file copied from the target machine: + debugfs/tracing/events///format + +KALLSYMS SECTION +---------------- + + Section ID: 19 + + Directly after the section header comes the information of the mapping + of function addresses to the function names. + + The next 4 bytes are a 32-bit word containing the size of the + data holding the function mappings. + + The next set of data is of the size defined by the previous 4 bytes + and contains the information from the target machine's file: + /proc/kallsyms + + +TRACE_PRINTK SECTION +-------------------- + + Section ID: 20 + + If a developer used trace_printk() within the kernel, it may + store the format string outside the ring buffer. + This information can be found in: + debugfs/tracing/printk_formats + + The next 4 bytes are a 32-bit word containing the size of the + data holding the printk formats. + + The next set of data is of the size defined by the previous 4 bytes + and contains the information from debugfs/tracing/printk_formats. + + +SAVED COMMAND LINES SECTION +--------------------------- + + Section ID: 21 + + Directly after the section header comes the information mapping + a PID to a process name. + + The next 8 bytes contain a 64-bit word that holds the size of the + data mapping the PID to a process name. + + The next set of data is of the size defined by the previous 8 bytes + and contains the information from debugfs/tracing/saved_cmdlines. + + +BUFFER FLYRECORD SECTION +------------------------ + + This section contains flyrecord tracing data, collected in one trace instance. + The data is saved per CPU. Each BUFFER FLYRECORD section has a corresponding BUFFER + option, containing information about saved CPU's trace data. Padding is placed between + the section header and the CPU data, placing the CPU data at a page aligned (target page) + position in the file. + + This data is copied directly from the Ftrace ring buffer and is of the + same format as the ring buffer specified by the event header files + loaded in the header format file. + + The trace-cmd tool will try to \fBmmap(2)\fR the data page by page with the + target's page size if possible. If it fails to mmap, it will just read the + data instead. + +BUFFER LATENCY SECTION +------------------------ + + This section contains latency tracing data, ASCII text taken from the + target's debugfs/tracing/trace file. + +SEE ALSO +-------- +trace-cmd(1), trace-cmd-record(1), trace-cmd-report(1), trace-cmd-start(1), +trace-cmd-stop(1), trace-cmd-extract(1), trace-cmd-reset(1), +trace-cmd-split(1), trace-cmd-list(1), trace-cmd-listen(1), +trace-cmd.dat(5) + +AUTHOR +------ +Written by Steven Rostedt, + +RESOURCES +--------- +https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/ + +COPYING +------- +Copyright \(C) 2010 Red Hat, Inc. Free use of this software is granted under +the terms of the GNU Public License (GPL). +