From patchwork Thu Jun 10 11:33:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1560C48BD1 for ; Thu, 10 Jun 2021 11:35:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9D5A261406 for ; Thu, 10 Jun 2021 11:35:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230447AbhFJLhq (ORCPT ); Thu, 10 Jun 2021 07:37:46 -0400 Received: from mail-wr1-f42.google.com ([209.85.221.42]:41787 "EHLO mail-wr1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230400AbhFJLhi (ORCPT ); Thu, 10 Jun 2021 07:37:38 -0400 Received: by mail-wr1-f42.google.com with SMTP id o3so1903044wri.8 for ; Thu, 10 Jun 2021 04:35:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SL302fTyFfaFnXlErdYp1JrAPy3ZPN35AL5S03E1of4=; b=Puz9pVBcmVgloVl3X0jDBoqce+pByemJkeQvKnfJi+6I09vxXWeYS8hkhIg5ILXwNo PVVJsy3sIx1oD5TCcoagb8lELKBqBbzP0JZEU/5UX1m69of2WiqhX5R5mGMBb/02iJM0 CW48Mg9mcIJf9jarz1mFhPRjsQ94ZWfLKP5zzMUwUi+N8hEEXl9740+3TNuG5kwxAU2V bcyr/Pp+3Xg3nR31NXbEJzvb6K1F2OJLp+KkJYVFsZl9WbRk4rBAD/oy67WtiN/XW1Nq //YEW3fQ65w034fvQ4rL5fE/Kn+ADvX0+5lvGsNcWgPZXT80fg1aFLodJJ5A/zKhpcN/ GJng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SL302fTyFfaFnXlErdYp1JrAPy3ZPN35AL5S03E1of4=; b=degBIJerounnvH7XqQdlKYn5Rkey3fvqm+eXtN8z2qMixi7F8IgPK8PHUhIP6rReI8 6+Dz+pmPhCcUYn5j/gO/zvKc/enbcfypoOLPPqRa7e91k2Si7Xmz7sptfFeM9BYi5Sby Bcv74bAkmES2DFsFQEgVXu0iezbC1yQJJFupm4Zm7yaPZGBaNZWNo2yB2KaNfxwh0tg2 j49HY4OnY0eewOcKqt88SO92Z3J5vm9fqYhNwotj2IHr78vzexuMyh1+oLj81x4cG8If P+sS/HHZzCOQinhQpDfUmi5jSFGrpoNyFfLOC0M6FVgSe3tczM/tJV3H+1InHI5WEY7A IPcg== X-Gm-Message-State: AOAM531pyJfAQIXA+xFZGwxQ1oKeWFpXod4FXm+FplUlLo/wIjAdvbQt zV9/QnrI9rBIFjEIDwr5G6x65tWU5dEsMQ== X-Google-Smtp-Source: ABdhPJwjAdbNAaXr5aNU8SpqQ7JG/yP+BvBlmbJDVS9deJdMclPItoaZlMUo0f+AR3lymCEkwuDXmA== X-Received: by 2002:a05:6000:18f:: with SMTP id p15mr4990565wrx.102.1623324869770; Thu, 10 Jun 2021 04:34:29 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:29 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 01/40] trace-cmd library: Remove unused private APIs for creating trace files Date: Thu, 10 Jun 2021 14:33:47 +0300 Message-Id: <20210610113426.257931-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Remove these unused private trace-cmd library APIs: tracecmd_create_file() tracecmd_create_file_glob() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 6 ---- lib/trace-cmd/trace-output.c | 29 ------------------- 2 files changed, 35 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 5c2ab4ca..4bd36baa 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -263,12 +263,6 @@ struct tracecmd_option; struct tracecmd_msg_handle; struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files); -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *event_globs); struct tracecmd_output * tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a4a1eecc..9231c319 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1614,35 +1614,6 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *list) -{ - struct tracecmd_output *handle; - - handle = create_file(output_file, NULL, NULL, NULL, list); - if (!handle) - return NULL; - - if (tracecmd_write_cmdlines(handle)) - return NULL; - - if (tracecmd_append_cpu_data(handle, cpus, cpu_data_files) < 0) { - tracecmd_output_close(handle); - return NULL; - } - - return handle; -} - -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files) -{ - return tracecmd_create_file_glob(output_file, cpus, - cpu_data_files, &all_event_list); -} - struct tracecmd_output *tracecmd_create_init_fd(int fd) { return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); From patchwork Thu Jun 10 11:33:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9099C48BD1 for ; Thu, 10 Jun 2021 11:35:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8ED8613FF for ; Thu, 10 Jun 2021 11:35:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230413AbhFJLha (ORCPT ); Thu, 10 Jun 2021 07:37:30 -0400 Received: from mail-wr1-f52.google.com ([209.85.221.52]:39501 "EHLO mail-wr1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230117AbhFJLh1 (ORCPT ); Thu, 10 Jun 2021 07:37:27 -0400 Received: by mail-wr1-f52.google.com with SMTP id l2so1907980wrw.6 for ; Thu, 10 Jun 2021 04:35:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ul0cYskXBCYyoanWYB1IZdsjLcqgyRalazjxs5buo98=; b=mB52Zaj4kFscXcxls6227hHw7ebbq9Adfpb2LylvlR2G1viNhuGGtiF+5pYQexDPrZ fjEWbAkJkIi+1PhAah/D0VvAd212UWWqfzjoTjSo9hoTm0wLYDLUEs86t9mfPDQSyNWm LLPWV1fQzH38drnLthUTqTlqUiNChuBWShZTZ1HUNlg5UwPIfRpoBbDtpzdglP6yhkW/ 5yTvTVhjLMoNPFsj8rItvmidIAOey8CVcvJOYRnYK3IoRxXfOY9boJwLIPFF6e/Ym5vJ vksgZ29Tsqj76uO6hTIe/SJsI2p9o9oJeF80+bWgYqQHDGtTOo/ZYMZrVbdXbcmbMKM2 cVRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ul0cYskXBCYyoanWYB1IZdsjLcqgyRalazjxs5buo98=; b=iKjAQexfZ1W10qTUqdpKywCtsujnghKDNtkFbG6pSSv/wRz5YqTMSle9qzfGyDqnK1 qWwO9Xs0jVvoU8A8YAZFFFfhQcyn2tNNktqkKf5B/WEas0kiuJZDSlsJyjZRUMrWc3GO 29tAUcG1uBkyfHPOyk0t4qy/JI/gRJHXvhX09KddwUoHuOSv03VzTnsMYi8oybEV2twQ o9JGVqNzLXQYo5WVGSTp/Ka7LnY6+TEoevyYTbI2x/V+BbCMtw1rkJbXkSd4wZdASCe5 IBC8CUf6hf7OCflHAFuyETdqn5U7EBexVeUqoL3Gfkmtm/hqcrCUgGfqOoX7VStpYg9z zJkA== X-Gm-Message-State: AOAM533K9tQCbGdxrKihXpJzhGGMKPQxX62TleeO8WiYkGAH+kzV6Ym1 H9NS30GsVSxbYWSyvP26epo= X-Google-Smtp-Source: ABdhPJxw91iJ+vyt817tPv/UyfLTPUHCtBDCrjd+TFTWyXuFccSa8gHmMji+3mq4jPPtIse224k6Zg== X-Received: by 2002:adf:ee85:: with SMTP id b5mr4890793wro.95.1623324870780; Thu, 10 Jun 2021 04:34:30 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:30 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 02/40] trace-cmd library: Remove unused API tracecmd_update_option Date: Thu, 10 Jun 2021 14:33:48 +0300 Message-Id: <20210610113426.257931-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Removed not used trace-cmd library API for updating the options section of already written trace file: tracecmd_update_option() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 -- lib/trace-cmd/trace-output.c | 40 ------------------- 2 files changed, 43 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4bd36baa..49539432 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -290,9 +290,6 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9231c319..c0aa58cf 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1229,46 +1229,6 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data) -{ - tsize_t offset; - stsize_t ret; - - if (size > option->size) { - tracecmd_warning("Can't update option with more data than allocated"); - return -1; - } - - if (handle->file_state < TRACECMD_FILE_OPTIONS) { - /* Hasn't been written yet. Just update current pointer */ - option->size = size; - memcpy(option->data, data, size); - return 0; - } - - /* Save current offset */ - offset = lseek64(handle->fd, 0, SEEK_CUR); - - ret = lseek64(handle->fd, option->offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); - return -1; - } - - if (do_write_check(handle, data, size)) - return -1; - - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offset); - return -1; - } - - return 0; -} - struct tracecmd_option * tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) From patchwork Thu Jun 10 11:33:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CB8FC47094 for ; Thu, 10 Jun 2021 11:35:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6794061406 for ; Thu, 10 Jun 2021 11:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229983AbhFJLhs (ORCPT ); Thu, 10 Jun 2021 07:37:48 -0400 Received: from mail-wr1-f53.google.com ([209.85.221.53]:37689 "EHLO mail-wr1-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230396AbhFJLhl (ORCPT ); Thu, 10 Jun 2021 07:37:41 -0400 Received: by mail-wr1-f53.google.com with SMTP id i94so1915553wri.4 for ; Thu, 10 Jun 2021 04:35:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RiwZC1yBs8SQ9z1BwUSlU9j5di5yu+Mv55pgJ5/bATY=; b=fT1pr+fmf9fOX14E7zGu8I80lb5GZs55WkvEO/wKbQDuMfj3FwTLQu0M6Pizhxd7Uz GVjNbB4miIFJjEwQaRQ1P+kbzW5AEb97EXfgBCyvQZcblKcNH3Vh57UqbhUC9G7JlVQf 1Te8QNAgv6bB82lXavBoH7jDry1P1CaJsTH5ffWji5LeRBLyZ3fA1cgs4pQ3PxlmeLLT jbLpqiqtDDoAijuFYS3IUmdDcjfwkCKVISdZpVc+BdF1AI2fiZsmdWhxN00S0K57Ei8U Ob2P+TKh7XitU7SQUoh7qEavGWBLq9paeVj//EVn+lUuyUjljOhLIfavO5eAp8XEZ+IW VkaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RiwZC1yBs8SQ9z1BwUSlU9j5di5yu+Mv55pgJ5/bATY=; b=iJnWvqtyMsOmIFKn2ZVmh91OuZlbhhh9EFHvxLSEUWNuC6krhwFH8UQuf3aAPfsEPp fSf84d35mdLahzQtxOo8R10Q8AY+G84wOjZKFDcHF1sKGUvBifzb8gO7llzPox1LTT+g LtiJXUtDonjbZwmhfhHqSMp/3FsuUUt7VkruRye83RPyRQqTBW+6MGNqZUaps28x4Sjq Z4C8GTwwVhcy4vu3JDG8vhtF9ebi0cgbyAT+ahxPPkBZjwQHa48B+R943H2kKT9udsLD CADy68t90ixrXQb1BljRSJy4K8CuU+Uih6belPTJLD0GTBFX0iXfpbpKopI071N5kpXz rd1g== X-Gm-Message-State: AOAM5318bpvjN7sK54KvU+tCaZHuvXw38u6oRGcY4n7A78pUD/NjkZPy Ihd2DzeW9W53IhYFgmj4h8drzZiyZA4jTg== X-Google-Smtp-Source: ABdhPJzRivtwfLGkmepixYhe1bTqgjVepCQJi4joGhzKtMbNxNV0Rhy7P+HucfUBCp20ETfRz8AOaA== X-Received: by 2002:a5d:4e4d:: with SMTP id r13mr5028339wrt.218.1623324871693; Thu, 10 Jun 2021 04:34:31 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:31 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 03/40] trace-cmd: Check if file version is supported Date: Thu, 10 Jun 2021 14:33:49 +0300 Message-Id: <20210610113426.257931-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace file, version of the file is ignored. This could case problems when bumping the version number because of changes in in the structure of the file. The old code should detect unsupported file version and should not try to read it. A new trace-cmd library API is added to check if version is supported: tracecmd_is_version_supported() Checks are added in the code to ensure not trying to read trace file from unsupported version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 10 ++++++++++ lib/trace-cmd/trace-util.c | 7 +++++++ tracecmd/trace-dump.c | 7 +++++++ 4 files changed, 26 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 49539432..6fc18938 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -42,6 +42,8 @@ void tracecmd_record_ref(struct tep_record *record); void tracecmd_set_debug(bool set_debug); bool tracecmd_get_debug(void); +bool tracecmd_is_version_supported(unsigned int version); + struct tracecmd_output; struct tracecmd_recorder; struct hook_list; diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 5ee69b14..97ad0a5d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -131,6 +131,7 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + int file_version; struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -3269,6 +3270,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) unsigned int page_size; char *version; char buf[BUFSIZ]; + unsigned long ver; handle = malloc(sizeof(*handle)); if (!handle) @@ -3293,6 +3295,14 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (!version) goto failed_read; tracecmd_info("version = %s\n", version); + ver = strtol(version, NULL, 10); + if (!ver && errno) + goto failed_read; + if (!tracecmd_is_version_supported(ver)) { + tracecmd_warning("Unsupported file version %lu", ver); + goto failed_read; + } + handle->file_version = ver; free(version); if (do_read_check(handle, buf, 1)) diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b65f9dec..b0c98c72 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -617,3 +617,10 @@ unsigned long long tracecmd_generate_traceid(void) free(str); return hash; } + +bool tracecmd_is_version_supported(unsigned int version) +{ + if (version <= FILE_VERSION) + return true; + return false; +} diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 98425b98..03cc82b4 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "trace-local.h" @@ -145,6 +146,7 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; + unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -166,6 +168,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); + ver = strtol(buf, NULL, 10); + if (!ver && errno) + die("Invalid file version string %s", buf); + if (!tracecmd_is_version_supported(ver)) + die("Unsupported file version %lu", ver); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) From patchwork Thu Jun 10 11:33:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3EB2C47094 for ; Thu, 10 Jun 2021 11:34:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7FD0613F1 for ; Thu, 10 Jun 2021 11:34:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230262AbhFJLgc (ORCPT ); Thu, 10 Jun 2021 07:36:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbhFJLga (ORCPT ); Thu, 10 Jun 2021 07:36:30 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24DE0C061574 for ; Thu, 10 Jun 2021 04:34:34 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id l2so1908070wrw.6 for ; Thu, 10 Jun 2021 04:34:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K/wi6U7hzPtl4la0xaLhqgtGsmlwcZ2Gfg6Lcb4Uo4I=; b=pfuIuVzIAq6K10dKCNX4YwSKNvbNuB0ark9ziTLS7Vn/5U7wrYMHLdwou5sRZNA0ov 75qShZh5K367O4xFNMjswUV3LzoML57qvxvrrK1JFtLdzxPfVIKRv5evwoGVpKADvcCH NnFEtYt7GeB1SWYwLd7rgrFbTdAFHSRij0YjmoZTLl5ZkM0e6IL5iGVxaXiW/KjMvBO1 9nj6P3jWsiWxbxcnX6ObucP13dqOJD6ANsuI8Xev/Y8kaSZ442SMo/0P5EhgWd95UcW7 U8HIbxd2xj75aBi/aPUNtmaPKUu9z0LvO3g2IOqJlfBf7K4ACa6oKOfoVG1eZGSKMsBC RENw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K/wi6U7hzPtl4la0xaLhqgtGsmlwcZ2Gfg6Lcb4Uo4I=; b=VWZue6+xrq6Y9Pu5e+PDL/NjWZlLuCslCPSd1yMsbmONLIt04cqtKcDVOUpQjqYrTY z0wffFA6bAIMzoj73LWTQiueUmbisngdAgZxzDaE+dT63IzSvMyu/BnUS/L+g23rYGNN FEvg/s83qmf/ZbjHmu6fjlJpUV+BKJ2WUrACW6H7BJ3ansIWiUBHVT0FPLwDWF5JGc7V 5lgcSiFcwpC+mOCaxxwivSNDpFtyuYnH0Ae4A2ChryeYx2mfb47ufzjjhuD3tvSspueo YF1hwQLpgaIFpcbVvxClWVM2jBeBdZ4uS2GsgaGBZ2pj5IdorUP+DhsADyZTuCIHP50d s0bA== X-Gm-Message-State: AOAM532aCFeSLTUKGtpAlj+8OQJ4kFCk9XwGcwtLhtXOIdvKXuuCFutn aRfKScdom8gjw8vES+EU9bbYMIpGJB1FHg== X-Google-Smtp-Source: ABdhPJwE7AlPB16508cEOa25NnkELS5O+XJbtQR7s6TV9cKQN8iMOtGbSCRWKwNULNxT3LXQBQ3QPg== X-Received: by 2002:a5d:4bc9:: with SMTP id l9mr4995162wrt.97.1623324872720; Thu, 10 Jun 2021 04:34:32 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:32 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 04/40] trace-cmd library: Add new API to get file version of input handler Date: Thu, 10 Jun 2021 14:33:50 +0300 Message-Id: <20210610113426.257931-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added an API to get the version of the trace file, associated with given input file handler. tracecmd_get_in_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 6fc18938..01b12c47 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -232,6 +232,8 @@ int tracecmd_set_cursor(struct tracecmd_input *handle, unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); +unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); + int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); tracecmd_show_data_func diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 97ad0a5d..46619ac7 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4046,6 +4046,15 @@ struct tep_handle *tracecmd_get_tep(struct tracecmd_input *handle) return handle->pevent; } +/** + * tracecmd_get_in_file_version - return the trace.dat file version + * @handle: input handle for the trace.dat file + */ +unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle) +{ + return handle->file_version; +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Thu Jun 10 11:33:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3FEAC48BD1 for ; Thu, 10 Jun 2021 11:34:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BD91E613E9 for ; Thu, 10 Jun 2021 11:34:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230130AbhFJLge (ORCPT ); Thu, 10 Jun 2021 07:36:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230264AbhFJLgd (ORCPT ); Thu, 10 Jun 2021 07:36:33 -0400 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29E7DC061760 for ; Thu, 10 Jun 2021 04:34:35 -0700 (PDT) Received: by mail-wm1-x333.google.com with SMTP id f16-20020a05600c1550b02901b00c1be4abso6336616wmg.2 for ; Thu, 10 Jun 2021 04:34:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sP2SzL+jTkso72skt+AgZYS5f7e9OZMAJYuOulQ00BI=; b=MkM/QzdyEQaIqHBEa2JxNnb8/9+OTZBWOjyX7MPa7JEiqL/yo8JwS4vv+5SgcbYW1p RxlOV7d6CPiQOYxoBjGM0a5dy+exxwvbtud7NRiQFiFfIWBVME5w5X5xcuPIEUp3LvKc hkORyPdfCjgV7vCZBw3k+abGt6uj7HIfUVo0gjOxqUlXBRf5k433B1oDmFJSbeLU8SFV qpLnVyJqGZbe8jPWNknWZW4ltooV/tFOySFkzEa9FvGexCP41+l5sYsVQxbOaqd3LejN umDck633wA0qWGW15ppSJ3QcgHUpSumh2+Ks5bIMua4DkWziYrhbRstNMatfbeDC6gIG IQwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sP2SzL+jTkso72skt+AgZYS5f7e9OZMAJYuOulQ00BI=; b=RGaIqA2Y2GtB3z3niEb0a1pHGutpzG1g5GCH+Iv6G7cLg3mJzFBRHXS0m4tIvHJNTU U8S4SL2TU9Wg7QRjH+GoTvlNsKlCH3Ugaq1OIBvLZiQWR4WHH9XPdc8tWw7bFPz45Q8X Y57qbfUkY6qu7gatyYgHe9lenhTCsOgzkWAzDNkczbi1zxf2cnb4UYPj5YUwzgoZLh7i B7KM25D995t6jN7l8ScX8z4rPGHW4xf+RBRltc1WM+gud7GFebjif7CTjDFjx13qu/1Y wekrvM91ihHWIi1KjmBX+oLF4t0XxrIEeZO8D7dsz8MlnkskgTCl7+0pm8Gvio51GA7F Wj6w== X-Gm-Message-State: AOAM5333UJtDcu3Pw07JlnQHOe6NSqDScwWAhS6ebKCO8/VR7kUkXES5 prT4xLY/kRrCGErpvrKD4rE= X-Google-Smtp-Source: ABdhPJyUATnxeAgxRTwtmJWyjnPOdmM+DZkvzOiDcl3lJ3SKmHHa371yUKPdPFN92zRBWlaiNUuhYw== X-Received: by 2002:a7b:c20a:: with SMTP id x10mr4501350wmi.141.1623324873785; Thu, 10 Jun 2021 04:34:33 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:33 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 05/40] trace-cmd library: Select the file version when writing trace file Date: Thu, 10 Jun 2021 14:33:51 +0300 Message-Id: <20210610113426.257931-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new output handler to trace file is allocated, select the proper file version. If this output handler is based on an existing input trace file handler, inherit the trace file version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c0aa58cf..bacda23b 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -57,6 +57,7 @@ struct tracecmd_output { int nr_options; bool quiet; unsigned long file_state; + unsigned long file_version; struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -907,6 +908,17 @@ out_free: return ret; } +static int select_file_version(struct tracecmd_output *handle, + struct tracecmd_input *ihandle) +{ + if (ihandle) + handle->file_version = tracecmd_get_in_file_version(ihandle); + else + handle->file_version = FILE_VERSION; + + return 0; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -933,6 +945,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; + if (select_file_version(handle, ihandle)) + goto out_free; + list_head_init(&handle->options); buf[0] = 23; @@ -943,7 +958,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, if (do_write_check(handle, buf, 10)) goto out_free; - if (do_write_check(handle, FILE_VERSION_STRING, strlen(FILE_VERSION_STRING) + 1)) + sprintf(buf, "%lu", handle->file_version); + if (do_write_check(handle, buf, strlen(buf) + 1)) goto out_free; /* get endian and page size */ @@ -1562,6 +1578,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->pevent = tracecmd_get_tep(ihandle); tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); + handle->file_version = tracecmd_get_in_file_version(ihandle); list_head_init(&handle->options); tracecmd_close(ihandle); From patchwork Thu Jun 10 11:33:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6FAAC47094 for ; Thu, 10 Jun 2021 11:34:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B082A613FE for ; Thu, 10 Jun 2021 11:34:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230103AbhFJLgo (ORCPT ); Thu, 10 Jun 2021 07:36:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbhFJLgo (ORCPT ); Thu, 10 Jun 2021 07:36:44 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAA8DC061574 for ; Thu, 10 Jun 2021 04:34:36 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id v206-20020a1cded70000b02901a586d3fa23so6198518wmg.4 for ; Thu, 10 Jun 2021 04:34:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2G0CILf23KT9a1hgIdunlhYcwh2NDqtDLL7z1ixvlIw=; b=RSSME0LnG6yecone1S4rHimrInP7gyMWdWxxIBTd+N5ITTo9fZW/nTKTh9aasUil34 aMtRu1WRtQuGiRu1yi6QhWHYM9wlSNyGFmkMx3ZsJznx3v2WBqSxBKN95ZvHZtDvbU76 /j7BdMfRl2SxbDiblfuFj6YqbRO6CTyMELluHIvWNoCTLUqK72rFcZUY51WBPR5DoMzC 0y5XdqrZ4FyRhukGx2PRVp9T0yaHDRJIpC5SqsMxJFN4NdYYH/YM7Uqdo7f2iV4kT9Yi Fb3DUcuymeq842jgxwVMGs6Sg3+gB+QLNcNqvEXV43IqQSTerap5SOIHHxU6PFbN1S3L BMjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2G0CILf23KT9a1hgIdunlhYcwh2NDqtDLL7z1ixvlIw=; b=dLTB9/JWaSKr/5wwwr0uIFJXLWQ4w33QO44/szJJDSkOqztovJvtQwqZ1JIXTmDf5a i+jbPFPr5UP1k+HT+vERjUhxB4k1NPlJ+ZK1z4Dek517JPfyMGcj6Fv0ZPc9ntMEwfUV 0jlum0cnrgbRIxtL6XYZVpuiHYWA7F327KornRRZ/sVRtONjN3ZmzAdIbnC5H6HAmXfA NrmAv3Owt0ylXVY/KM5oo0rqiLl16ncfD/qSiSHS0ba+sI9R8vvN2YCwZyN9JBhXaLjK +Q5znUBNN0LRVnSNUJyCaVF8T8XZ1ozWnyYT1Q4cA+6DJOR6tXv8bSnLXQ0m91JYSYju IpWw== X-Gm-Message-State: AOAM533fOa+uZ8Q2VsR02Kk1YC+Z5WB2SJHz0OdFNPS0Dnnj9xZ1PV8v xvUnXxnJGViAitC9bNgBxEQ= X-Google-Smtp-Source: ABdhPJyuk9jgH75bp5a5JGPvrHz7Ms8BCJvui8UtpWnmNf79EvUWaKMc/xi05n4saKyalnN+1x+j6Q== X-Received: by 2002:a7b:ca44:: with SMTP id m4mr4580435wml.52.1623324874700; Thu, 10 Jun 2021 04:34:34 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:34 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 06/40] trace-cmd: Add APIs for library initialization and free Date: Thu, 10 Jun 2021 14:33:52 +0300 Message-Id: <20210610113426.257931-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The trace-cmd library has no APIs for initialization and free of the whole library. Added these new APIs: tracecmd_lib_init() tracecmd_lib_free() and call them in trace-cmd main function. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 3 +++ lib/trace-cmd/trace-util.c | 9 +++++++++ tracecmd/trace-cmd.c | 11 ++++++++--- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 01b12c47..be8b3c48 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -29,6 +29,9 @@ struct tep_plugin_list *trace_load_plugins(struct tep_handle *tep, int flags); +int tracecmd_lib_init(void); +void tracecmd_lib_free(void); + int *tracecmd_add_id(int *list, int id, int len); enum { diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b0c98c72..61054ad2 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -624,3 +624,12 @@ bool tracecmd_is_version_supported(unsigned int version) return true; return false; } + +int tracecmd_lib_init(void) +{ + return 0; +} + +void tracecmd_lib_free(void) +{ +} diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 00cdaa37..71c8f6d6 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -142,15 +142,20 @@ int main (int argc, char **argv) if (argc < 2) trace_usage(argc, argv); + tracecmd_lib_init(); + for (i = 0; i < ARRAY_SIZE(commands); ++i) { if (strcmp(argv[1], commands[i].name) == 0 ){ commands[i].run(argc, argv); - goto out; + break; } } + tracecmd_lib_free(); + /* No valid command found, show help */ - trace_usage(argc, argv); -out: + if (i == ARRAY_SIZE(commands)) + trace_usage(argc, argv); + exit(0); } From patchwork Thu Jun 10 11:33:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF9E4C48BD1 for ; Thu, 10 Jun 2021 11:35:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A70F561407 for ; Thu, 10 Jun 2021 11:35:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230393AbhFJLhf (ORCPT ); Thu, 10 Jun 2021 07:37:35 -0400 Received: from mail-wm1-f41.google.com ([209.85.128.41]:51054 "EHLO mail-wm1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230425AbhFJLhc (ORCPT ); Thu, 10 Jun 2021 07:37:32 -0400 Received: by mail-wm1-f41.google.com with SMTP id d184so5995919wmd.0 for ; Thu, 10 Jun 2021 04:35:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8fWiNKWPG8N8nDALWm+HC00JAxzKmwhDrSmza+sbb4g=; b=QFzcf5vMiWQG0tREINFRf8y0QOrqsV9LDHA5mc+0O4nfWBcqqpnf1gXKBa0edk+Zq+ 6GQz5lfzNdRQOtK54zN2NyCXQ0kUXnF8RudrPApXo3qzfHmEgyPWIa6l3UuN7hfhsi20 CpauBIEF7HQm6psehZ+eByKF+OlX4HvsRxrB6FhdXPK5GYAZmg8UEBUeq5YMkHkzNQSJ /N0cMVc/So++VaB7aqiIMq9ZKAp2irHh9eHUC+1NOThg9Ug/Meo6KYTOetjV8vzWDbO1 bMNzJLTxjYB8IW8PiMS6z1dFJHO03liqR0ujZtzyurzbjnI+bG5glbgaGyg4yv4bKhOD enqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8fWiNKWPG8N8nDALWm+HC00JAxzKmwhDrSmza+sbb4g=; b=IZvDzndTtItc20DK1J9zKoYJOCPcFW4tZxjOJKiVq9O0AG7mOAkTb8kxFUx5PeFJc3 S0jvxGNRt8eL6gWT7dMC6AUaCcCYWGKVWJjkPUzKNQO2gRtJ+6WcRbgf4e5OGlVy7t43 tXRkwliWYiZRQRGRi1ONs92YBqDv4tde2vh5mG4o5WfD2vewViUXbb1njz0XUJHUdK5r /NYQn1CWuahBGkd+GlrSpLsmPmSTx8n8dSiHnIR8AYMc98tZhDeGEiQr1ssPW3oOYa+E TrzNhdpLitVUEFjDv/YxAIT2Zf0xU1F788guG0iSjMoVGlVz5FwQQ+Vxsegd7mI5SP10 ndfA== X-Gm-Message-State: AOAM5300uTcoWGm6pJI6Ar9HXoZoYMBeiwmwTS5CDhEq/O6KG2CgHTeN gOEYObuSlo4Z0Al9ifnt5XM= X-Google-Smtp-Source: ABdhPJw5LKIr1YCfAo4LOYvZd5coMpIC5sjKGnvoppSk3bUAIP3EwTYNzMA66MsYNXC9uIACjMWetQ== X-Received: by 2002:a1c:f70b:: with SMTP id v11mr4629757wmh.186.1623324875632; Thu, 10 Jun 2021 04:34:35 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:35 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 07/40] trace-cmd library: Add support for compression algorithms Date: Thu, 10 Jun 2021 14:33:53 +0300 Message-Id: <20210610113426.257931-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added infrastructure to trace-cmd library for compression. Introduced various new APIs to work with this new functionality: struct tracecmd_compression tracecmd_compress_init() tracecmd_compress_free() tracecmd_compress_alloc() tracecmd_compress_destroy() tracecmd_compress_block() tracecmd_uncompress_block() tracecmd_compress_reset() tracecmd_compress_read() tracecmd_compress_pread() tracecmd_compress_write() tracecmd_compress_lseek() tracecmd_compress_proto_get_name() tracecmd_compress_is_supported() tracecmd_compress_protos_get() tracecmd_compress_proto_register() The compression algorithms are not part of this patch. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 1 + .../include/private/trace-cmd-private.h | 26 + lib/trace-cmd/include/trace-cmd-local.h | 2 + lib/trace-cmd/trace-compress.c | 588 ++++++++++++++++++ lib/trace-cmd/trace-util.c | 3 + 5 files changed, 620 insertions(+) create mode 100644 lib/trace-cmd/trace-compress.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index 17600318..bab4322d 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -25,6 +25,7 @@ ifeq ($(VSOCK_DEFINED), 1) OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif +OBJS += trace-compress.o # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index be8b3c48..ec534d51 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -464,6 +464,32 @@ void tracecmd_tsync_free(struct tracecmd_time_sync *tsync); int tracecmd_write_guest_time_shift(struct tracecmd_output *handle, struct tracecmd_time_sync *tsync); +/* --- Compression --- */ +struct tracecmd_compression; +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle); +void tracecmd_compress_destroy(struct tracecmd_compression *handle); +int tracecmd_compress_block(struct tracecmd_compression *handle); +int tracecmd_uncompress_block(struct tracecmd_compression *handle); +void tracecmd_compress_reset(struct tracecmd_compression *handle); +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len); +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset); +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size); +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence); +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version); +bool tracecmd_compress_is_supported(const char *name, const char *version); +int tracecmd_compress_protos_get(char ***names, char ***versions); +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)); + /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 821b5cdb..041582c6 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,5 +31,7 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +int tracecmd_compress_init(void); +void tracecmd_compress_free(void); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c new file mode 100644 index 00000000..ff742226 --- /dev/null +++ b/lib/trace-cmd/trace-compress.c @@ -0,0 +1,588 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include +#include + +#include "trace-cmd-private.h" +#include "trace-cmd-local.h" + +struct compress_proto { + struct compress_proto *next; + char *proto_name; + char *proto_version; + int weight; + + int (*comress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + int (*uncompress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + unsigned int (*comress_size)(unsigned int bytes); + bool (*is_supported)(const char *name, const char *version); +}; + +static struct compress_proto *proto_list; + +struct tracecmd_compression { + int fd; + unsigned int capacity; + unsigned long pointer; + char *buffer; + struct compress_proto *proto; + struct tep_handle *tep; + struct tracecmd_msg_handle *msg_handle; +}; + +static int read_fd(int fd, char *dst, int len) +{ + size_t size = 0; + int r; + + do { + r = read(fd, dst+size, len); + if (r > 0) { + size += r; + len -= r; + } else + break; + } while (r > 0); + + if (len) + return -1; + return size; +} + +static long long write_fd(int fd, const void *data, size_t size) +{ + long long tot = 0; + long long w; + + do { + w = write(fd, data + tot, size - tot); + tot += w; + + if (!w) + break; + if (w < 0) + return w; + } while (tot != size); + + return tot; +} + +static long long do_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + int ret; + + if (handle->msg_handle) { + ret = tracecmd_msg_data_send(handle->msg_handle, data, size); + if (ret) + return -1; + return size; + } + return write_fd(handle->fd, data, size); +} + +/** + * tracecmd_compress_lseek - Move the read/write pointer into the compression buffer + * @handle: compression handler + * @offset: number of bytes to move the pointer, can be negative or positive + * @whence: the starting position of the pointer movement, + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns the new file pointer on success, or -1 in case of an error. + */ +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence) +{ + unsigned long p, extend; + char *buf; + + if (!handle || !handle->buffer) + return (off_t)-1; + + switch (whence) { + case SEEK_CUR: + p = handle->pointer + offset; + break; + case SEEK_END: + p = handle->capacity + offset; + break; + case SEEK_SET: + p = offset; + break; + default: + return (off_t)-1; + } + + if (p <= handle->capacity) { + handle->pointer = p; + } else { + extend = p - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return (off_t)-1; + handle->buffer = buf; + handle->capacity += extend; + handle->pointer = p; + } + + return p; +} + +static int compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int s; + + if (handle->pointer + len > handle->capacity) + s = handle->capacity - handle->pointer; + else + s = len; + memcpy(dst, handle->buffer + handle->pointer, s); + + return s; +} + +/** + * tracecmd_compress_pread - pread() on compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * @offset: offset in the buffer of data to be read + * + * Read a @len of data from the compression buffer at given @offset, + * without updating the buffer pointer. + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset) +{ + int ret; + + if (!handle || !handle->buffer || offset > handle->capacity) + return -1; + + ret = tracecmd_compress_lseek(handle, offset, SEEK_SET); + if (ret < 0) + return ret; + return compress_read(handle, dst, len); +} + +/** + * tracecmd_compress_read - read() from compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * + * Read a @len of data from the compression buffer + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int ret; + + if (!handle || !handle->buffer) + return -1; + + ret = compress_read(handle, dst, len); + if (ret > 0) + handle->pointer += ret; + + return ret; +} + +/** + * tracecmd_compress_reset - Reset the compression buffer + * @handle: compression handler + * + * Reset the compression buffer, any data currently in the buffer will be destroyed. + * + */ +void tracecmd_compress_reset(struct tracecmd_compression *handle) +{ + if (handle) { + free(handle->buffer); + handle->buffer = NULL; + handle->pointer = 0; + handle->capacity = 0; + } +} + +/** + * tracecmd_uncompress_block - uncompress a memory block + * @handle: compression handler + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_uncompress_block(struct tracecmd_compression *handle) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + char buf[4]; + char *bytes = NULL; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + tracecmd_compress_reset(handle); + + if (read(handle->fd, buf, 4) != 4) + return -1; + s_compressed = tep_read_number(handle->tep, buf, 4); + if (read(handle->fd, buf, 4) != 4) + return -1; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + + handle->buffer = malloc(s_uncompressed); + if (!handle->buffer) + return -1; + bytes = malloc(s_compressed); + if (!bytes) + goto error; + + if (read_fd(handle->fd, bytes, s_compressed) < 0) + goto error; + ret = handle->proto->uncompress_block(bytes, s_compressed, + handle->buffer, &s_uncompressed); + if (ret) + goto error; + free(bytes); + handle->pointer = 0; + handle->capacity = s_uncompressed; + return 0; +error: + tracecmd_compress_reset(handle); + free(bytes); + return -1; +} + +/** + * tracecmd_compress_block - compress a memory block + * @handle: compression handler + * + * Compress the content of the internal memory buffer and write the compressed data in the file + * The tracecmd_compress_write() can be used to write data into the internal memory buffer, before + * calling this API. + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_compress_block(struct tracecmd_compression *handle) +{ + unsigned int size; + char *buf; + int endian4; + int ret; + + size = handle->proto->comress_size(handle->pointer); + buf = malloc(size); + if (!buf) + return -1; + ret = handle->proto->comress_block(handle->buffer, handle->pointer, buf, &size); + if (ret < 0) + goto out; + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &handle->pointer, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write the uncompressed data */ + ret = do_write(handle, buf, size); + ret = ((ret == size) ? 0 : -1); +out: + tracecmd_compress_reset(handle); + free(buf); + return ret; +} + +/** + * tracecmd_compress_write - write() to compression buffer + * @handle: compression handler + * @data: data to be written + * @size: size of @data + * + * Write @data of @size in the compression buffer + * + * Returns 0 on success, or -1 on failure. + */ +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + char *buf; + int extend; + + if (handle->capacity < handle->pointer + size) { + extend = (handle->pointer + size) - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return -1; + handle->buffer = buf; + handle->capacity += extend; + } + memcpy(&handle->buffer[handle->pointer], data, size); + handle->pointer += size; + return 0; +} + +/** + * tracecmd_compress_init - initialize the library with available compression algorithms + * + * Returns 0. If no compression algorithms are available, a warning is printed. + */ +int tracecmd_compress_init(void) +{ + struct timeval time; + + gettimeofday(&time, NULL); + srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); + + return 0; +} + +static struct compress_proto *compress_proto_select(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *selected = NULL; + + while (proto) { + if (!selected || selected->weight > proto->weight) + selected = proto; + proto = proto->next; + } + + return selected; +} + +/** + * tracecmd_compress_alloc - Allocate a new compression context + * @name: name of the compression algorithm + * @version: version of the compression algorithm + * @fd: file descriptor for reading / writing data + * @tep: tep handler, used to encode the data + * @msg_handle: message handler, use it for reading / writing data instead of @fd + * + * Returns NULL on failure or pointer to allocated compression context. + * The returned context must be freed by tracecmd_compress_destroy() + */ +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle) +{ + struct compress_proto *proto; + struct tracecmd_compression *new; + + if (name) { + proto = proto_list; + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + break; + proto = proto->next; + } + } else { + proto = compress_proto_select(); + } + if (!proto) + return NULL; + + new = calloc(1, sizeof(*new)); + if (!new) + return NULL; + new->fd = fd; + new->tep = tep; + new->msg_handle = msg_handle; + new->proto = proto; + return new; +} + +/** + * tracecmd_compress_destroy - Free a compression context + * @handle: handle to the compression context that will be freed + */ +void tracecmd_compress_destroy(struct tracecmd_compression *handle) +{ + tracecmd_compress_reset(handle); + free(handle); +} + +/** + * tracecmd_compress_is_supported - check if compression algorithm with given name and + * version is supported + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * + * Returns true if the algorithm with given name and version is supported or false if it is not. + */ +bool tracecmd_compress_is_supported(const char *name, const char *version) +{ + struct compress_proto *proto = proto_list; + + if (!name) + return NULL; + + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + return true; + proto = proto->next; + } + return false; +} + +/** + * tracecmd_compress_proto_get_name - get name and version of compression algorithm + * @compress: compression handler. + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version) +{ + if (!compress || !compress->proto) + return -1; + if (name) + *name = compress->proto->proto_name; + if (version) + *version = compress->proto->proto_version; + return 0; +} + +/** + * tracecmd_compress_proto_register - register a new compression algorithm + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * @weight: weight of the compression algorithm, lower is better. + * @compress: compression hook, called to compress a memory block. + * @uncompress: uncompression hook, called to uncompress a memory block. + * @comress_size: hook, called to get the required minimum size of the buffer for compression + * given number of bytes. + * @is_supported: check hook, called to check if compression with given name and version is + * supported by this plugin. + * + * Returns 0 on success, or -1 in case of an error. If algorithm with given name and version is + * already registered, -1 is returned. + */ +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)) +{ + struct compress_proto *new; + + if (!name || !compress || !uncompress) + return -1; + if (tracecmd_compress_is_supported(name, version)) + return -1; + + new = calloc(1, sizeof(*new)); + if (!new) + return -1; + + new->proto_name = strdup(name); + if (!new->proto_name) + goto error; + new->proto_version = strdup(version); + if (!new->proto_version) + goto error; + new->comress_block = compress; + new->uncompress_block = uncompress; + new->comress_size = comress_size; + new->is_supported = is_supported; + new->weight = weight; + new->next = proto_list; + proto_list = new; + return 0; + +error: + free(new->proto_name); + free(new->proto_version); + free(new); + return -1; +} + +/** + * tracecmd_compress_free - free the library resources, related to available compression algorithms + * + */ +void tracecmd_compress_free(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *del; + + while (proto) { + del = proto; + proto = proto->next; + free(del->proto_name); + free(del->proto_version); + free(del); + } + proto_list = NULL; +} + +/** + * tracecmd_compress_protos_get - get a list of all supported compression algorithms and versions + * @names: return, array with names of all supported compression algorithms + * @versions: return, array with versions of all supported compression algorithms + * + * On success, the size of @names and @versions arrays is returned. Those arrays are allocated by + * the API and must be freed with free() by the caller. Both arrays are with same size, each name + * from @names corresponds to a version from @versions. + * On error -1 is returned and @names and @versions arrays are not allocated. + */ +int tracecmd_compress_protos_get(char ***names, char ***versions) +{ + struct compress_proto *proto = proto_list; + char **n = NULL; + char **v = NULL; + int c, i; + + for (c = 0; proto; proto = proto->next) + c++; + + if (c < 1) + return c; + + n = calloc(c, sizeof(char *)); + if (!n) + goto error; + v = calloc(c, sizeof(char *)); + if (!v) + goto error; + + proto = proto_list; + for (i = 0; i < c && proto; i++) { + n[i] = proto->proto_name; + v[i] = proto->proto_version; + proto = proto->next; + } + + *names = n; + *versions = v; + return c; + +error: + free(n); + free(v); + return -1; +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 61054ad2..136260c6 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -627,9 +627,12 @@ bool tracecmd_is_version_supported(unsigned int version) int tracecmd_lib_init(void) { + + tracecmd_compress_init(); return 0; } void tracecmd_lib_free(void) { + tracecmd_compress_free(); } From patchwork Thu Jun 10 11:33:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED966C48BE5 for ; Thu, 10 Jun 2021 11:35:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9D7A61407 for ; Thu, 10 Jun 2021 11:35:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230402AbhFJLhg (ORCPT ); Thu, 10 Jun 2021 07:37:36 -0400 Received: from mail-wr1-f52.google.com ([209.85.221.52]:36488 "EHLO mail-wr1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230400AbhFJLhd (ORCPT ); Thu, 10 Jun 2021 07:37:33 -0400 Received: by mail-wr1-f52.google.com with SMTP id e11so1926592wrg.3 for ; Thu, 10 Jun 2021 04:35:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3KFLzzFKc9m2UFTOpsVh7cbeeh2hvHNTp1bJlNMgn90=; b=IfXUuzeMNAroZ4dTXClLoMygDvMlZnwlkh/WlToZdviFhdElw38iTPRNBIEyzFEoSR kUZ8ukG8lE6/RFPihTmrQrXTAhI4et+/GEsQwO0FNLu/NSpGNZMI+9cLkO1omZps7Aan 3zSMgW0e8mTPF0iyySeeQLqXv1RhBjIB7CM3RR3Tem3gfxFFb95/PAxOvP0sTJr3AUOX SwRGosq09HKu2OzvmE2OOMzzyIf1KauBKFTPq5BkwQATxWIfnMTMVzJdoq9Qx1b+ed8W LBlbHzx/d1BLEJ/qLWRoZovPpdY7noIh7ZlVs0+bsAuCDMBc3DhUCUR6CkESOsXqWfDU YLBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3KFLzzFKc9m2UFTOpsVh7cbeeh2hvHNTp1bJlNMgn90=; b=HooGA80CsOR0AuxflkP+qxCxcqQP4m4on2ZLlFXd1TJpNuAYNmpAhOdSE8iLQnpwjh Xn/p1en6Cp9HpVt7kp4LW1PqqBtUvTLcdZJbmXjM88zAeeJz8R17/5P0xhGpW1dSniTp VsAtit3w2N2V/vMuA/UUCVhmKwDHtTFSUy4t7WsGoxfrLTIkEx3CXaFv5wajkeKHhkbZ Ec6nSZCJ/zFu6Lh86IYkUObwLDiSkRZhdEzqTv6P67xlyZ1MQTsCiYjMK+dIHfwZE+1/ WC4Aef/KklBj+mZu4BZgjUhlUASSh6gtlAbk+52DalzetZEY1TTC4iMzIbEOnztUgtGA JXUg== X-Gm-Message-State: AOAM532uJ0E1QhlmJVBeKXqvqxaqBcCZ2UqfjYjd3qjmAp9cE2YGBoHa jghpcfhk8oXQzKtsV5j//kgdq7nodj+rHQ== X-Google-Smtp-Source: ABdhPJwQsfCUWsWa+ygExLm9yqsvc7p3JcNdIqpB4oOYDMf2d5E2V/ygP9cvr0fGQ0xNcabsCUyfJw== X-Received: by 2002:a05:6000:2ce:: with SMTP id o14mr5000448wry.145.1623324876548; Thu, 10 Jun 2021 04:34:36 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:36 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 08/40] trace-cmd list: Show supported compression algorithms Date: Thu, 10 Jun 2021 14:33:54 +0300 Message-Id: <20210610113426.257931-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add new parameter "trace-cmd list -c" to show supported compression algorithms. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-list.c | 26 ++++++++++++++++++++++++++ tracecmd/trace-usage.c | 1 + 2 files changed, 27 insertions(+) diff --git a/tracecmd/trace-list.c b/tracecmd/trace-list.c index d060c810..900da73b 100644 --- a/tracecmd/trace-list.c +++ b/tracecmd/trace-list.c @@ -549,6 +549,24 @@ static void show_plugins(void) tep_free(pevent); } +static void show_compression(void) +{ + char **versions, **names; + int c, i; + + c = tracecmd_compress_protos_get(&names, &versions); + if (c <= 0) { + printf("No compression algorithms are supported\n"); + return; + } + printf("Supported compression algorithms:\n"); + for (i = 0; i < c; i++) + printf("\t%s, %s\n", names[i], versions[i]); + + free(names); + free(versions); +} + void trace_list(int argc, char **argv) { int events = 0; @@ -562,6 +580,7 @@ void trace_list(int argc, char **argv) int flags = 0; int systems = 0; int show_all = 1; + int compression = 0; int i; const char *arg; const char *funcre = NULL; @@ -626,6 +645,10 @@ void trace_list(int argc, char **argv) systems = 1; show_all = 0; break; + case 'c': + compression = 1; + show_all = 0; + break; case '-': if (strcmp(argv[i], "--debug") == 0) { tracecmd_set_debug(true); @@ -670,6 +693,8 @@ void trace_list(int argc, char **argv) show_clocks(); if (systems) show_systems(); + if (compression) + show_compression(); if (show_all) { printf("event systems:\n"); show_systems(); @@ -679,6 +704,7 @@ void trace_list(int argc, char **argv) show_tracers(); printf("\noptions:\n"); show_options(); + show_compression(); } return; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index c0033ce8..cd8ca391 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -331,6 +331,7 @@ static struct usage_help usage_help[] = { " -O list plugin options\n" " -B list defined buffer instances\n" " -C list the defined clocks (and active one)\n" + " -c list the supported trace file compression algorithms\n" }, { "restore", From patchwork Thu Jun 10 11:33:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B113C47094 for ; Thu, 10 Jun 2021 11:34:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 11465613FF for ; Thu, 10 Jun 2021 11:34:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230248AbhFJLgf (ORCPT ); Thu, 10 Jun 2021 07:36:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbhFJLge (ORCPT ); Thu, 10 Jun 2021 07:36:34 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFCEFC061760 for ; Thu, 10 Jun 2021 04:34:38 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id l9so5982107wms.1 for ; Thu, 10 Jun 2021 04:34:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cp2xjlaw4uvJnn+MOUAkFZQ+eOSnxvlwG7WEJsSGUdQ=; b=ia+PU4Jl/xaJBPkueBMLwIqAcAEbJ+97n3rNhWOqBU9iII/w9dPmBTvEc1jmS/Orp2 bvdndHomUW2zc+WUWffTrUYgHFhqK6VtP2zw4l/7Kgtku3gR+WEWNw+BmFlsP2/7R91n p+TLYRphGWt9sBF7xQL06vqIQigRyNQhqOGiK8qzVMh23gVGua14U1Y+r0dTWCkLrmRj gYAVBRouYzlihkQk62EOxtIXp7TYKU6ZCXEhjyYW9Pad4uwh6EtNdTGMUcqQ10ZjI2Vk BwZhmEghaSen3T76j7e6/mqs82rIw0Cr+JILcEhi6r8ALkVgUUcdc4EenTvZpt4BBdnE CzXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cp2xjlaw4uvJnn+MOUAkFZQ+eOSnxvlwG7WEJsSGUdQ=; b=MXdIZ2QRETOZFQL1zp8kGd1hNErRl6rFZkvQN9lvuwzMZEWIKv+XPiUgtQjkCy6Xei k0sFEaT2hnc74dtUfjQaDbsuFOrsxQ0be/tf0zkyR83t2DotSLzmJlSSXyNNEtR5tuti sS++JBGh6dpXd3HfItRCVdbHRLy34uGkGF75LnsACRwULcOmjIS2mw4tNRz/ihKpfAIw LsOIKwD4Htz4uaBqU1vyrJ+FTywFhZL60KsO64ZikpSMc3qOS9F3d7PixGRNbSKgoWew l7Ec6/m7494bWmQrRpVKHY7Tv6o/c91eIduYhNNrPIn4wt+UUpQcKN2nOCs1q5xnzxQ2 CttA== X-Gm-Message-State: AOAM533etQCSv1v9NxYF45/polO4TAjVEJiweaIvxiGKGq8LF8FIRRFy saj/VzQ6n1cj459l+tUgTlI= X-Google-Smtp-Source: ABdhPJytbUnYMNmkNoLBbB6gNuoGWZRRMMSufvGg8PRKpXEtnsIhz85WfLZcW3CqEE3zl1Wz6634JA== X-Received: by 2002:a05:600c:4b88:: with SMTP id e8mr4484355wmp.107.1623324877496; Thu, 10 Jun 2021 04:34:37 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:37 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 09/40] trace-cmd library: Bump the trace file version to 7 Date: Thu, 10 Jun 2021 14:33:55 +0300 Message-Id: <20210610113426.257931-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Adding a compression of the trace.dat file will change its structure. These changes are not backward compatible, the old trace-cmd binaries will not be able to read compressed trace files. Bumping the version to 7 will prevent old trace-cmd to read such files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 041582c6..3332572b 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -15,7 +15,7 @@ void tracecmd_critical(const char *fmt, ...); void tracecmd_info(const char *fmt, ...); /* trace.dat file format version */ -#define FILE_VERSION 6 +#define FILE_VERSION 7 #define _STR(x) #x #define STR(x) _STR(x) From patchwork Thu Jun 10 11:33:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19302C48BD1 for ; Thu, 10 Jun 2021 11:35:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 00582613E9 for ; Thu, 10 Jun 2021 11:35:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230436AbhFJLhj (ORCPT ); Thu, 10 Jun 2021 07:37:39 -0400 Received: from mail-wr1-f42.google.com ([209.85.221.42]:35543 "EHLO mail-wr1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230247AbhFJLhf (ORCPT ); Thu, 10 Jun 2021 07:37:35 -0400 Received: by mail-wr1-f42.google.com with SMTP id m18so1921274wrv.2 for ; Thu, 10 Jun 2021 04:35:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5/Ptno904wbIfdQ2YLSEMRtXDnRxxep6QWoMNal48Tw=; b=nrYZY3DwSRKg3SbW1rrD9SkYRH/Q0ptuvt+e09sLqxSAZPrJBKNhMRMo7+Q4UWfWNv qTwTHYF2Frtqvt6yNVyN4tw1c1eIu03/M2gOHUbVBLzWlrKD9AC/YV8us1rBJobNtd4N 9UcmnjxJpikqzOsSxcMgypgKeDS+/AqRlEw28PgvSeXr8otGogR/4oDMjaIwd1dbCRpV SwbB+/6OWDP50/Fk0l6r3eyi2HB3k2n+/qq1qRsvFmz5UpseD+vDdxfKLTC0DfbXkV4n E+K4zNZwKeZtmhDxgjQFAYrLt/kHR87wWwtKmVbw1/O0Gk5SaYNvtItOc9w3Lo2dCWpY KpDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5/Ptno904wbIfdQ2YLSEMRtXDnRxxep6QWoMNal48Tw=; b=NU6RCsIlcvT+q3oHX0lvjQ3KyawJspf0OS0LxO0/ZkXTNWTmQRxvjpIk0IGYzKlsjq dTJGzFX7GS8Xks2wh0vtP+wg0joEeBZgcv+GioejDeQJFeDspDBJczZHpXyj6hO2ermP jI/rMaQYg4ib8+CB7tEc4CPKoPOwAnk0SUWy8jORO3AuR2SfBlJOmrQUglQCMoSPRjIl dj1GsyfSseUXRwvzELzPTQEWJq+puYvF5sQF/223PHYlA5evtc9KS6k9t5CtKIdOFRZ7 rtrSY8O0G1g11ESMeM7kjFTg/UIkZC+hjs3UJRAF50rCxt8fGgv2Gbg+mLutQloXawZJ egBQ== X-Gm-Message-State: AOAM530d7UOPqPWQMZnt7teHIrmSgSm+zPdvgb3GEVc1/5usHegRAx4f EOrZ0sprnBDP4mMj5KKWwKw= X-Google-Smtp-Source: ABdhPJyCkMJGsLOlFplbsflpYjnIz1eV+b+Due6D66FVWMMOjJV3aEpCncTB6V6oFLYlvYkOBXb3fg== X-Received: by 2002:a5d:474d:: with SMTP id o13mr4812253wrs.176.1623324878387; Thu, 10 Jun 2021 04:34:38 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:37 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 10/40] trace-cmd library: Compress part of the trace file Date: Thu, 10 Jun 2021 14:33:56 +0300 Message-Id: <20210610113426.257931-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Compress part of the trace.dat file metadata. If there is compression support, chose file version 7 and compress these parts of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name A new compression header is added in the file, right after the page size information. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-output.c | 145 ++++++++++++++++++++---- 2 files changed, 127 insertions(+), 22 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 3332572b..bf90eae8 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -34,4 +34,8 @@ void tracecmd_info(const char *fmt, ...); int tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_compression_start(struct tracecmd_output *handle); +int out_compression_end(struct tracecmd_output *handle); +void out_compression_reset(struct tracecmd_output *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index bacda23b..0d89fe13 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -58,6 +58,10 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + + bool do_compress; + struct tracecmd_compression *compress; + struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -78,12 +82,31 @@ struct list_event_system { static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { + if (handle->do_compress && handle->compress) + return tracecmd_compress_write(handle->compress, data, size); + if (handle->msg_handle) return tracecmd_msg_data_send(handle->msg_handle, data, size); return __do_write_check(handle->fd, data, size); } +static inline int do_lseek(struct tracecmd_output *handle, off_t offset, int whence) +{ + if (handle->do_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek64(handle->fd, offset, whence); +} + +static inline int do_preed(struct tracecmd_output *handle, void *dst, int len, off_t offset) +{ + if (handle->do_compress) + return tracecmd_compress_pread(handle->compress, dst, len, offset); + else + return pread(handle->fd, dst, len, offset); +} + static short convert_endian_2(struct tracecmd_output *handle, short val) { if (!handle->pevent) @@ -109,6 +132,31 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +__hidden void out_compression_reset(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return; + tracecmd_compress_reset(handle->compress); + handle->do_compress = false; +} + +__hidden int out_compression_start(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + tracecmd_compress_reset(handle->compress); + handle->do_compress = true; + return 0; +} + +__hidden int out_compression_end(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + handle->do_compress = false; + return tracecmd_compress_block(handle->compress); +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen @@ -159,6 +207,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) free(option); } free(handle->trace_clock); + tracecmd_compress_destroy(handle->compress); free(handle); } @@ -659,13 +708,17 @@ static int read_ftrace_files(struct tracecmd_output *handle) } create_event_list_item(handle, &systems, &list); - + out_compression_start(handle); ret = copy_event_system(handle, systems); + if (!ret) + ret = out_compression_end(handle); + else + out_compression_reset(handle); free_list_events(systems); - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - + if (!ret) + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; return ret; } @@ -715,6 +768,7 @@ static int read_event_files(struct tracecmd_output *handle, for (slist = systems; slist; slist = slist->next) count++; + out_compression_start(handle); ret = -1; endian4 = convert_endian_4(handle, count); if (do_write_check(handle, &endian4, 4)) @@ -730,8 +784,14 @@ static int read_event_files(struct tracecmd_output *handle, ret = copy_event_system(handle, slist); } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + if (!ret) + ret = out_compression_end(handle); out_free: + if (!ret) + handle->file_state = TRACECMD_FILE_ALL_EVENTS; + else + out_compression_reset(handle); + free_list_events(systems); return ret; @@ -793,20 +853,20 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, if (kallsyms) path = kallsyms; - + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ size = 0; endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; - return 0; + ret = do_write_check(handle, &endian4, 4); + goto out; } size = get_size(path); endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; + ret = do_write_check(handle, &endian4, 4); + if (ret) + goto out; set_proc_kptr_restrict(0); check_size = copy_file(handle, path); @@ -814,13 +874,18 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, errno = EINVAL; tracecmd_warning("error in size of file '%s'", path); set_proc_kptr_restrict(1); - return -1; + ret = -1; + goto out; } set_proc_kptr_restrict(1); - handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = out_compression_end(handle); +out: + if (!ret) + handle->file_state = TRACECMD_FILE_KALLSYMS; + else + out_compression_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_output *handle) @@ -840,6 +905,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -861,11 +927,14 @@ static int read_ftrace_printk(struct tracecmd_output *handle) } out: - handle->file_state = TRACECMD_FILE_PRINTK; put_tracing_file(path); + if (out_compression_end(handle)) + return -1; + handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); + out_compression_reset(handle); return -1; } @@ -911,14 +980,39 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { - if (ihandle) + if (ihandle) { handle->file_version = tracecmd_get_in_file_version(ihandle); - else - handle->file_version = FILE_VERSION; + } else { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + } return 0; } +static int write_compression_header(struct tracecmd_output *handle) +{ + const char *name = NULL; + const char *ver = NULL; + char *buf; + int ret; + + ret = tracecmd_compress_proto_get_name(handle->compress, &name, &ver); + if (ret < 0 || !name || !ver) + return -1; + ret = asprintf(&buf, "%s %s", name, ver); + if (ret < 0) + return -1; + ret = do_write_check(handle, buf, strlen(buf) + 1); + + free(buf); + return ret; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -992,8 +1086,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, endian4 = convert_endian_4(handle, handle->page_size); if (do_write_check(handle, &endian4, 4)) goto out_free; + if (handle->file_version >= 7 && write_compression_header(handle)) + goto out_free; handle->file_state = TRACECMD_FILE_INIT; - if (ihandle) return handle; @@ -1285,11 +1380,17 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) handle->file_state); return ret; } + out_compression_start(handle); + ret = save_tracing_file_data(handle, "saved_cmdlines"); - if (ret < 0) + if (ret < 0) { + out_compression_reset(handle); return ret; - handle->file_state = TRACECMD_FILE_CMD_LINES; - return 0; + } + ret = out_compression_end(handle); + if (!ret) + handle->file_state = TRACECMD_FILE_CMD_LINES; + return ret; } struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) From patchwork Thu Jun 10 11:33:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38810C48BE0 for ; Thu, 10 Jun 2021 11:35:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2011361406 for ; Thu, 10 Jun 2021 11:35:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230247AbhFJLhk (ORCPT ); Thu, 10 Jun 2021 07:37:40 -0400 Received: from mail-wm1-f41.google.com ([209.85.128.41]:44962 "EHLO mail-wm1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230396AbhFJLhg (ORCPT ); Thu, 10 Jun 2021 07:37:36 -0400 Received: by mail-wm1-f41.google.com with SMTP id m41-20020a05600c3b29b02901b9e5d74f02so3100812wms.3 for ; Thu, 10 Jun 2021 04:35:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t3intCMoSYuOhFvknXB492ibiTZthsbHP5Iqqq3yFFg=; b=JuV7z+ocDz/tmpNWo93ie38GcArHfxMTeVKyJBVlQq8m4nLowNSVjFD8isMqsFsc8p HeFQbn6lB2fGiL8fsKw/HGCNmT7HmDdMVsCPY8a02KkqSbnWyMm0zWaU5YFjBU18Dk7d 9MlOM53GwIXp4NIBl6apEUQIw4LVNoTc2p/Lmd+i4znQtgn/7OxY5phBTuzqldbfvc4d TWEeUfMfaIVCGyFI/rOn1N3HcO5EVmZ415/i1Jk1JuWRXfmaCovag6T5pw9Yk3EQJOaR ADiiPih9BS/uipTsIavD5c2HF8JfOwMMHdioj6KepyS/dvbn4s6RRM4GR8oAtbYgK//m CQ7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t3intCMoSYuOhFvknXB492ibiTZthsbHP5Iqqq3yFFg=; b=pDzQQBNZzTPefwEh6UsKRKzf8bUuacaLKlqaRNpDibuHKb0SpnCiB0ZKXpkXIxSpNn pz7UlpUlPZ5Bw9dM0QdWHOQhdf09ApHBSy6R5CzvZp2sKvPXmKGoauDMCxDMQtxSCzQf aYUImmczGkFIib6gi2886XMcArDXgYU9sWouY5u0949tK6fUOijW4vPK7EA/cGLERdJ0 BAX8vIGc2ssegsyMUgFDtn38Q93c5WeSm4RALy0YcbtmYLe43AOsgoX/c+Pk6Mj8dZpG CCY3XxFyemmys2cQZyPIsGqLY+atnaaR5ayHQxbfakus0AiVRQKDA1h66uZx4NrfTpnl 93sA== X-Gm-Message-State: AOAM531N+UrSa27ll2Z4ebGHr1DwU1sMRDC/u3YqIpBZlDjIJYuJbNX6 d8Xa7HbQ9nXlSRGy3JOUeDr9SRO0BHCEFQ== X-Google-Smtp-Source: ABdhPJx5I3+xFbRSebm9+r9NkufyZytLcDrD3NkEWpcEanzl2+GlLgkxi/UzoYj/ts9Zh9kIiJHbqQ== X-Received: by 2002:a05:600c:ada:: with SMTP id c26mr13858611wmr.189.1623324879179; Thu, 10 Jun 2021 04:34:39 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:38 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 11/40] trace-cmd library: Read compressed trace file Date: Thu, 10 Jun 2021 14:33:57 +0300 Message-Id: <20210610113426.257931-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If a trace file version 7 is detected, read compressed sections of it. Read the compression header, check if the compression algorithm and version, used to compress the file, are supported and use it to uncompress these sections of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-input.c | 257 ++++++++++++++++-------- 2 files changed, 178 insertions(+), 82 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index bf90eae8..24228d8f 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -38,4 +38,7 @@ int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); +void in_uncompress_reset(struct tracecmd_input *handle); +int in_uncompress_block(struct tracecmd_input *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 46619ac7..43d731e0 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -136,6 +136,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + bool read_compress; + struct tracecmd_compression *compress; + struct host_trace_info host; double ts2secs; char * cpustats; @@ -255,12 +258,28 @@ static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) return tot; } +static inline int do_lseek(struct tracecmd_input *handle, int offset, int whence) +{ + if (handle->read_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek(handle->fd, offset, whence); +} + +static inline ssize_t do_read_compressed(struct tracecmd_input *handle, void *data, size_t size) +{ + if (handle->read_compress) + return tracecmd_compress_read(handle->compress, data, size); + else + return do_read(handle, data, size); +} + static ssize_t do_read_check(struct tracecmd_input *handle, void *data, size_t size) { ssize_t ret; - ret = do_read(handle, data, size); + ret = do_read_compressed(handle, data, size); if (ret < 0) return ret; if (ret != size) @@ -278,10 +297,8 @@ static char *read_string(struct tracecmd_input *handle) ssize_t r; for (;;) { - r = do_read(handle, buf, BUFSIZ); - if (r < 0) - goto fail; - if (!r) + r = do_read_compressed(handle, buf, BUFSIZ); + if (r <= 0) goto fail; for (i = 0; i < r; i++) { @@ -307,7 +324,7 @@ static char *read_string(struct tracecmd_input *handle) } /* move the file descriptor to the end of the string */ - r = lseek(handle->fd, -(r - (i+1)), SEEK_CUR); + r = do_lseek(handle, -(r - (i+1)), SEEK_CUR); if (r < 0) goto fail; @@ -359,6 +376,26 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +__hidden void in_uncompress_reset(struct tracecmd_input *handle) +{ + if (handle->compress && handle->file_version >= 7) { + handle->read_compress = false; + tracecmd_compress_reset(handle->compress); + } +} + +__hidden int in_uncompress_block(struct tracecmd_input *handle) +{ + int ret = 0; + + if (handle->compress && handle->file_version >= 7) { + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->read_compress = true; + } + return ret; +} + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -602,34 +639,40 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) + if (in_uncompress_block(handle)) return -1; + ret = read4(handle, &count); + if (ret < 0) + goto out; + for (i = 0; i < count; i++) { - if (read8(handle, &size) < 0) - return -1; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_ftrace_file(handle, size, print_all, ereg); if (ret < 0) - return -1; + goto out; } handle->event_files_start = lseek64(handle->fd, 0, SEEK_CUR); + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + ret = 0; +out: if (sreg) { regfree(sreg); regfree(ereg); } - - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - - return 0; + in_uncompress_reset(handle); + return ret; } static int read_event_files(struct tracecmd_input *handle, const char *regex) { unsigned long long size; - char *system; + char *system = NULL; regex_t spreg; regex_t epreg; regex_t *sreg = NULL; @@ -654,13 +697,19 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) return -1; } - if (read4(handle, &systems) < 0) + if (in_uncompress_block(handle)) return -1; + ret = read4(handle, &systems); + if (ret < 0) + goto out; + for (i = 0; i < systems; i++) { system = read_string(handle); - if (!system) - return -1; + if (!system) { + ret = -1; + goto out; + } sys_printed = 0; print_all = 0; @@ -687,103 +736,117 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) - goto failed; + ret = read4(handle, &count); + if (ret < 0) + goto out; for (x=0; x < count; x++) { - if (read8(handle, &size) < 0) - goto failed; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_event_file(handle, system, size, print_all, &sys_printed, reg); if (ret < 0) - goto failed; + goto out; } free(system); - } - - if (sreg) { - regfree(sreg); - regfree(ereg); + system = NULL; } handle->file_state = TRACECMD_FILE_ALL_EVENTS; - - return 0; - - failed: + ret = 0; + out: + in_uncompress_reset(handle); if (sreg) { regfree(sreg); regfree(ereg); } free(system); - return -1; + return ret; } static int read_proc_kallsyms(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; + struct tep_handle *tep = handle->pevent; unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_KALLSYMS) return 0; - if (read4(handle, &size) < 0) + if (in_uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ - buf = malloc(size+1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)){ - free(buf); - return -1; + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_KALLSYMS; + goto out; /* OK? */ } - buf[size] = 0; - - tep_parse_kallsyms(pevent, buf); - free(buf); + buf = malloc(size+1); + if (!buf) { + ret = -1; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; + buf[size] = 0; + tep_parse_kallsyms(tep, buf); handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = 0; +out: + free(buf); + in_uncompress_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_input *handle) { unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_PRINTK) return 0; - if (read4(handle, &size) < 0) + if (in_uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ + + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_PRINTK; + goto out; /* OK? */ + } buf = malloc(size + 1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)) { - free(buf); - return -1; + if (!buf) { + ret = -1; + goto out; } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; buf[size] = 0; tep_parse_printk_formats(handle->pevent, buf); - - free(buf); - handle->file_state = TRACECMD_FILE_PRINTK; + ret = 0; - return 0; +out: + free(buf); + in_uncompress_reset(handle); + return ret; } static int read_and_parse_cmdlines(struct tracecmd_input *handle); @@ -2974,20 +3037,30 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; unsigned long long size; - char *cmdlines; + char *cmdlines = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_CMD_LINES) return 0; - if (read_data_and_size(handle, &cmdlines, &size) < 0) + if (in_uncompress_block(handle)) return -1; + + ret = read_data_and_size(handle, &cmdlines, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_CMD_LINES; + goto out; + } cmdlines[size] = 0; tep_parse_saved_cmdlines(pevent, cmdlines); - free(cmdlines); - handle->file_state = TRACECMD_FILE_CMD_LINES; - - return 0; + ret = 0; +out: + free(cmdlines); + in_uncompress_reset(handle); + return ret; } static void extract_trace_clock(struct tracecmd_input *handle, char *line) @@ -3268,7 +3341,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) struct tracecmd_input *handle; char test[] = TRACECMD_MAGIC; unsigned int page_size; - char *version; + size_t offset; + char *str = NULL; + char *zver; char buf[BUFSIZ]; unsigned long ver; @@ -3291,11 +3366,11 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (memcmp(buf, "tracing", 7) != 0) goto failed_read; - version = read_string(handle); - if (!version) + str = read_string(handle); + if (!str) goto failed_read; - tracecmd_info("version = %s\n", version); - ver = strtol(version, NULL, 10); + tracecmd_info("version = %s\n", str); + ver = strtol(str, NULL, 10); if (!ver && errno) goto failed_read; if (!tracecmd_is_version_supported(ver)) { @@ -3303,7 +3378,8 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) goto failed_read; } handle->file_version = ver; - free(version); + free(str); + str = NULL; if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3328,20 +3404,36 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) read4(handle, &page_size); handle->page_size = page_size; - handle->header_files_start = - lseek64(handle->fd, 0, SEEK_CUR); + offset = lseek64(handle->fd, 0, SEEK_CUR); + handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); + lseek64(handle->fd, offset, SEEK_SET); - handle->total_file_size = - lseek64(handle->fd, 0, SEEK_END); + if (handle->file_version >= 7) { + str = read_string(handle); + if (!str) + goto failed_read; + zver = strchr(str, ' '); + if (!zver) + goto failed_read; + *zver = '\0'; + handle->compress = tracecmd_compress_alloc(str, zver + 1, + handle->fd, handle->pevent, NULL); + if (!handle->compress) { + tracecmd_warning("Unsupported file compression %s %s", str, zver + 1); + goto failed_read; + } + free(str); + str = NULL; + } handle->header_files_start = - lseek64(handle->fd, handle->header_files_start, SEEK_SET); - + lseek64(handle->fd, 0, SEEK_CUR); handle->file_state = TRACECMD_FILE_INIT; return handle; failed_read: + free(str); free(handle); return NULL; @@ -3519,7 +3611,8 @@ void tracecmd_close(struct tracecmd_input *handle) if (handle->flags & TRACECMD_FL_BUFFER_INSTANCE) tracecmd_close(handle->parent); else { - /* Only main handle frees plugins and pevent */ + /* Only main handle frees plugins, pevent and compression context */ + tracecmd_compress_destroy(handle->compress); tep_unload_plugins(handle->plugin_list, handle->pevent); tep_free(handle->pevent); } From patchwork Thu Jun 10 11:33:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A22AC47094 for ; Thu, 10 Jun 2021 11:35:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4578A61406 for ; Thu, 10 Jun 2021 11:35:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230416AbhFJLho (ORCPT ); Thu, 10 Jun 2021 07:37:44 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:40923 "EHLO mail-wr1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230410AbhFJLhh (ORCPT ); Thu, 10 Jun 2021 07:37:37 -0400 Received: by mail-wr1-f50.google.com with SMTP id y7so1900133wrh.7 for ; Thu, 10 Jun 2021 04:35:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jNmgwtkM7E5IR4z79tTsl2FEDjRbmtrhOhWVpEAWIrY=; b=WHAcUlEZkJNb6az1QFZONW6qg3fOu3WK9nb8PPGQCwyPj366Rk55yiilTH7kQIbOQA 3KdeATDV2m52tUD/9ShigZLd0YqgAtVTu71kyBXWCdyDADiU+aP2Csxy90pBGbzEPPu1 8NP/ZNxIgD7HPr4M54p4Gt7JCAlTcoTyfPdSifeh/VEYOemtcXn+MlGlgPArYLBftKjj hBLbX952cmIadu0R4wP5oEoGXwrrgMUHsbE3jfMvt8deEFaYQRvaUgXZlo3g9k5MAoBY IR1dOAtAzz0Wbk/N2VVcoO9OcFGLY4UwVg8Kg21h98vm+ZQIq6i9bdpERfY9IzsHkpRu 5tQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jNmgwtkM7E5IR4z79tTsl2FEDjRbmtrhOhWVpEAWIrY=; b=m7P7/eA3vh6QrvcRaALFu4RK0+b67KPXm10dvjXR6gbBhYCO/DgdBi3u5SCwZj/fzo rbY/0KoSNVjjbHlTwSb792vgAhj7oKzLEB38cNA7yBN/26vd4pjozr1XevbSptP2TbrX CRRo3eM3wqlul7iGEleBzbyJKIuPncTpW8WSwO+6Sp6GirsCBkUXNpuzOgkIcgjQ1CQc O2XGSw2RH/Dtuz60PIDxzJN8VfMV6Vw860xGLoRa2wN9gfyxlIic/Zly2yg2ATZ8cpsM FSpR4j/S2WI/2ab9ix0UBPm35ScqtkU5+4ftGwbBkdLRH0sPqcAwtUUYe8Tq/9bnA4EO HNLQ== X-Gm-Message-State: AOAM5330o/Nj3OE9FGF4ZMRbNHD9Muy61tontVTl4iokVPkcX58LYwIs +KqY55ukl+PTnT2ldXCoJB8= X-Google-Smtp-Source: ABdhPJy6uXnsIwsUAGIMQXYYGutdUK1+UzXUkfgfA11EumqB3PBRp9R/NNtg13MWXwm3jzJvZ4h5nQ== X-Received: by 2002:adf:a45a:: with SMTP id e26mr5001065wra.222.1623324880153; Thu, 10 Jun 2021 04:34:40 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:39 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 12/40] trace-cmd library: Add new API to get compression of input handler Date: Thu, 10 Jun 2021 14:33:58 +0300 Message-Id: <20210610113426.257931-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The new API returns name and version of the compression algorithm, used to compress the trace file associated with given input file handler: tracecmd_get_file_compress_proto() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 16 ++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ec534d51..624d0afb 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -236,6 +236,8 @@ unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 43d731e0..0b17b58d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4148,6 +4148,22 @@ unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle) return handle->file_version; } +/** + * tracecmd_get_file_compress_proto - get name and version of compression algorithm, + * used to compress the trace file + * @handle: input handle for the trace.dat file + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version) +{ + return tracecmd_compress_proto_get_name(handle->compress, name, version); +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Thu Jun 10 11:33:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37607C48BD1 for ; Thu, 10 Jun 2021 11:34:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2177E61406 for ; Thu, 10 Jun 2021 11:34:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229895AbhFJLgw (ORCPT ); Thu, 10 Jun 2021 07:36:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230264AbhFJLgw (ORCPT ); Thu, 10 Jun 2021 07:36:52 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DCEBC061760 for ; Thu, 10 Jun 2021 04:34:44 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id l2so1908611wrw.6 for ; Thu, 10 Jun 2021 04:34:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=il8iUhJksidiq3MngizPXyhw7RHXGLQZpoprRtC3+P0=; b=osvaI1f+k6ZNt0oIgjOnkXOX6b0UsQqwCArtOdq3nTgX+/cmkvWicBT9vH4Q8BEDZF wsDwOVQ8xpVTwBFq4nqnBhNn9T9crMk0sDJioH8RFgWQ7l6YgaD67G1yPcgiwBRQNJyD I6JP0sdcqnHod0J3NsDJ+IJo9wF3z9Rn8X4UQPA8XVOsyDqZnc/AB9WPEhE0K92f+1F4 tC8VyDsCNXflVNV0Z28G/p+8iO3q8Hz/JoFfGscooJzzOkrvihi4cjIsI0FXRHOOVxzQ huPsA0LBcpwVw7wYgHRpo/SaXXze9uF+wPVzXL9/SzeYg6c6WLU33sVSfflor7bnvBWm lUEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=il8iUhJksidiq3MngizPXyhw7RHXGLQZpoprRtC3+P0=; b=fNSiPzZqB+aFwky8rvAvO0DTk0FhoD/GWRNW06UMmPUsPkEuo/hDrDzCE/wzdaIYXK A0iNQ3T4TW3+1z9ZXcH92uA5sgrbg7HotcrNAcH27KVYlxU9UPSmSMuLWRYfFJ5Zmjzr YDZ9By19l+1M0S9WES21Pq6tbMNv1yOvb1CgLiRLo34hlXMrKx1miHWBL7Far8heibWB 32X8Bg0g6VHjGZ5VFpHNPd4w/FvMZ9QZc4xcYtr8GL4cQ8gpfDxIgZC+uqmmjN93xsq7 Gz+8ZpYQSa+yTPn4jUojtj61ZPRbl9xVpltGdMCtuvrWMvbEDDjNn+EpYbDv7Kx5xIdz A8Eg== X-Gm-Message-State: AOAM533IL2GzufJVi5eJvB2bqEYw26ikIgPbKHgGdJrRtC1vqWcceic+ UlHD9hMEsT3j1uAleZUDUDM= X-Google-Smtp-Source: ABdhPJzs8AWXxbHqCiQGl+aIpK+rnxjbz1FgVY2MpWZjENBCIebmWCgJgyj3WHgOqoo5CExb0ZXuYQ== X-Received: by 2002:a5d:4a43:: with SMTP id v3mr4786638wrs.397.1623324881014; Thu, 10 Jun 2021 04:34:41 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:40 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 13/40] trace-cmd library: Inherit compression algorithm from input file Date: Thu, 10 Jun 2021 14:33:59 +0300 Message-Id: <20210610113426.257931-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace file output handler is allocated, based on given trace file input handler - use the same compression algorithm. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 0d89fe13..172462e7 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -980,17 +980,27 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { + const char *cname = NULL; + const char *cver = NULL; + if (ihandle) { handle->file_version = tracecmd_get_in_file_version(ihandle); - } else { - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); - if (handle->compress) - handle->file_version = 7; - else - handle->file_version = 6; + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + return -1; + } + return 0; } + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + return 0; } @@ -1645,6 +1655,8 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) { struct tracecmd_output *handle = NULL; struct tracecmd_input *ihandle; + const char *cname = NULL; + const char *cver = NULL; int fd2; /* Move the file descriptor to the beginning */ @@ -1682,6 +1694,12 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->file_version = tracecmd_get_in_file_version(ihandle); list_head_init(&handle->options); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + goto out_free; + } tracecmd_close(ihandle); return handle; From patchwork Thu Jun 10 11:34:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28241C48BE6 for ; Thu, 10 Jun 2021 11:35:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 15FAE61406 for ; Thu, 10 Jun 2021 11:35:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230401AbhFJLhv (ORCPT ); Thu, 10 Jun 2021 07:37:51 -0400 Received: from mail-wr1-f43.google.com ([209.85.221.43]:36493 "EHLO mail-wr1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230265AbhFJLhu (ORCPT ); Thu, 10 Jun 2021 07:37:50 -0400 Received: by mail-wr1-f43.google.com with SMTP id e11so1926898wrg.3 for ; Thu, 10 Jun 2021 04:35:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NqdwP9PtDAQbN3d2HuWkt3iR2ue3U7FNAnglrV9syuU=; b=QwSyCj3aa43HO6qdXWLGHjfvmreS05gt8zwUIi/bvJmRoupazvjGXD9IqwUmvkXC6s Jn1sunsO3IxlXgSX8ZYUJfDS1Bc1jN21aEhc6ZFEH4UhiPjHKwVN4mHsEwFtBVDY7vsL PH+H1oYjogdLj3nSvQz+LYt8Q3aQpAD0Pim2SwfmQLH7Mp8ICdXaYB4fz4yQJlzlDxR6 aMPAlR+BE8vN8hZG7Em/KwHKGr7Qb4x2z8h9D3hDyqI9pxYeGfAjI5vDJD0liyzE1/6T GPnN+aM+va5vJgbHT3XiTqZviaRzbhdYPrC8M+Mh7AknXafj43NKoNUDuliFqK8NL2O1 Tygw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NqdwP9PtDAQbN3d2HuWkt3iR2ue3U7FNAnglrV9syuU=; b=YJHxiuebYYJlhV0O7YaX9Y+HFALxh0AbMvCtItd/Cmoxfj3/i8kMpA2RzSlTKx9iNC ftEaTU0K4nTjP+TSt99MEWnviuGhO3c4FMkJxyrnMZwHVq7BglKtnIJPovaR151PIdGs 8v4zlWfegVZdxlkKXt6g4c0coEEDeMG9xoVY0+hECRDyqWUO4tRuMTLbidgQvUgkjWzG FnA39IltuGm2DLh5COYsfN4FCqM6M6FznzU3piW0WHSgaApOTlEmud0puYQZsFICmTLb IB7KxF7yjitTmikx0NQHEiHJq+8vebj+M6YJekP2mO0bQiwaSkZBIGiGzNbulav68gsu ZEbg== X-Gm-Message-State: AOAM533OK3Df5qcaCRGQjq9Ii5Qq7UF68OGPu9GdMPA+8tVFEUIlD/lA Ju6d9jqiBetwpQ6zQ1WGhvdtlgw7o1ZMyw== X-Google-Smtp-Source: ABdhPJyOvGVKoYAM043XkJ6k/JwFEecBPU+uo4gB4+5b4M8b5CEvofhi0ShkK66L+cSHJ94lXlRpXw== X-Received: by 2002:a5d:4692:: with SMTP id u18mr4785552wrq.318.1623324881910; Thu, 10 Jun 2021 04:34:41 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:41 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 14/40] trace-cmd library: Extend the create file APIs to support different compression Date: Thu, 10 Jun 2021 14:34:00 +0300 Message-Id: <20210610113426.257931-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added additional parameter for file compression to all trace-cmd library APIs for creating a new trace file. The caller could specify what compression algorithm should be used when creating a new trace file: tracecmd_create_file_latency tracecmd_create_init_file_glob tracecmd_create_init_fd_glob tracecmd_create_init_fd_msg tracecmd_create_init_file tracecmd_create_init_file_override tracecmd_create_init_fd The new parameter is of type string and can be one of: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 19 +++-- lib/trace-cmd/trace-output.c | 79 ++++++++++++------- tracecmd/trace-record.c | 12 ++- tracecmd/trace-restore.c | 4 +- tracecmd/trace-stream.c | 2 +- 5 files changed, 70 insertions(+), 46 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 624d0afb..ee73325c 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -271,20 +271,23 @@ struct tracecmd_event_list { struct tracecmd_option; struct tracecmd_msg_handle; -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression); struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_fd(int fd); +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_fd(int fd, const char *compression); struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list); +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression); struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_file(const char *output_file); + struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression); struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms); + const char *kallsyms, + const char *compression); struct tracecmd_option *tracecmd_add_option(struct tracecmd_output *handle, unsigned short id, int size, const void *data); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 172462e7..f249ee91 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -978,24 +978,41 @@ out_free: } static int select_file_version(struct tracecmd_output *handle, - struct tracecmd_input *ihandle) + struct tracecmd_input *ihandle, + const char *compression) { const char *cname = NULL; const char *cver = NULL; - if (ihandle) { - handle->file_version = tracecmd_get_in_file_version(ihandle); + if (compression) { /* If compression is specified - use it */ + if (!strcmp(compression, "any")) { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("No compression algorithms are not supported"); + return -1; + } + } else if (strcmp(compression, "none")) { + handle->compress = tracecmd_compress_alloc(compression, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("Compression algorithm %s is not supported", + compression); + return -1; + } + } + } else if (ihandle) { /* If there is input handler - inherit compression */ if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, handle->pevent, handle->msg_handle); if (!handle->compress) return -1; } - return 0; + } else { /* default - auto select the best available compression algorithm */ + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); } - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); if (handle->compress) handle->file_version = 7; else @@ -1028,7 +1045,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, struct tracecmd_event_list *list, - struct tracecmd_msg_handle *msg_handle) + struct tracecmd_msg_handle *msg_handle, + const char *compression) { struct tracecmd_output *handle; struct tep_handle *pevent; @@ -1049,7 +1067,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; - if (select_file_version(handle, ihandle)) + if (select_file_version(handle, ihandle, compression)) goto out_free; list_head_init(&handle->options); @@ -1128,7 +1146,8 @@ static struct tracecmd_output *create_file(const char *output_file, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, + const char *compression) { struct tracecmd_output *handle; int fd; @@ -1137,7 +1156,7 @@ static struct tracecmd_output *create_file(const char *output_file, if (fd < 0) return NULL; - handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL); + handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL, compression); if (!handle) { close(fd); unlink(output_file); @@ -1403,13 +1422,14 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return ret; } -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression) { struct tracecmd_output *handle; char *path; int ret; - handle = create_file(output_file, NULL, NULL, NULL, &all_event_list); + handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) return NULL; @@ -1710,41 +1730,43 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output *tracecmd_create_init_fd(int fd) +struct tracecmd_output *tracecmd_create_init_fd(int fd, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL, compression); } struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(msg_handle->fd, NULL, NULL, NULL, list, msg_handle); + return create_file_fd(msg_handle->fd, NULL, NULL, NULL, + list, msg_handle, compression); } struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list) +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, list, NULL, compression); } struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list) +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression) { - return create_file(output_file, NULL, NULL, NULL, list); + return create_file(output_file, NULL, NULL, NULL, list, compression); } -struct tracecmd_output *tracecmd_create_init_file(const char *output_file) +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression) { - return create_file(output_file, NULL, NULL, NULL, &all_event_list); + return create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); } struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms) + const char *kallsyms, + const char *compression) { - return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list); + return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list, compression); } /** @@ -1756,12 +1778,13 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) { struct tracecmd_output *handle; + const char *compr_name = NULL; - handle = create_file(file, ihandle, NULL, NULL, &all_event_list); + tracecmd_get_file_compress_proto(ihandle, &compr_name, NULL); + handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compr_name); if (!handle) return NULL; diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 60ee5fb3..29f84b7c 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3700,7 +3700,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events); + network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3718,8 +3718,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, - listed_events); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4067,8 +4066,7 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, - listed_events); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4460,7 +4458,7 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count); + handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4491,7 +4489,7 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events); + handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 280a37f0..96b4fb5d 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -91,7 +91,7 @@ void trace_restore (int argc, char **argv) } handle = tracecmd_create_init_file_override(output, tracing_dir, - kallsyms); + kallsyms, NULL); if (!handle) die("Unabled to create output file %s", output); if (tracecmd_write_cmdlines(handle) < 0) @@ -128,7 +128,7 @@ void trace_restore (int argc, char **argv) handle = tracecmd_copy(ihandle, output); tracecmd_close(ihandle); } else - handle = tracecmd_create_init_file(output); + handle = tracecmd_create_init_file(output, NULL); if (!handle) die("error writing to %s", output); diff --git a/tracecmd/trace-stream.c b/tracecmd/trace-stream.c index b47b208c..317de1b2 100644 --- a/tracecmd/trace-stream.c +++ b/tracecmd/trace-stream.c @@ -43,7 +43,7 @@ trace_stream_init(struct buffer_instance *instance, int cpu, int fd, int cpus, tfd = fileno(fp); ofd = dup(tfd); - trace_output = tracecmd_create_init_fd(ofd); + trace_output = tracecmd_create_init_fd(ofd, NULL); if (!trace_output) { fclose(fp); return NULL; From patchwork Thu Jun 10 11:34:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 276C6C48BDF for ; Thu, 10 Jun 2021 11:35:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1312761407 for ; Thu, 10 Jun 2021 11:35:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230400AbhFJLhs (ORCPT ); Thu, 10 Jun 2021 07:37:48 -0400 Received: from mail-wr1-f46.google.com ([209.85.221.46]:41821 "EHLO mail-wr1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230389AbhFJLhj (ORCPT ); Thu, 10 Jun 2021 07:37:39 -0400 Received: by mail-wr1-f46.google.com with SMTP id o3so1903825wri.8 for ; Thu, 10 Jun 2021 04:35:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R1+fTqnmfQqIOk9aOZ4VXKr/VhZIoW38trAOjO96PI4=; b=bPAYDqBaq5ruwOyNVxUjfxA82hlLpgvtIv3blJhLXJWXeQ4dK81XikhW2vlj+GLSaI wg1M8/a+2pp0JnqIdAMNr4RZfl6n6IjgvT+LZ1CUCJ7K3Ur3ymb19vKYuw7G9CnrfZR3 YdZ3fpWbqoII69ubQqVvmr215FYRsgOpAmhomFA95D143X5khxHob/aHFJqrL6QsGoIr F0ZyW5bOLm7yS51jYSWBuYjuUbTuTSYqOisvHdQCgNjAPzy07c1sxfV9ggumn/C6UdSx XOnhFKgHlB1pwtLVVzVmlKtlo8qg5g0qcTzKIBjI5adSGdvxrohw1wQ0qaHPc0vlTj15 hpKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R1+fTqnmfQqIOk9aOZ4VXKr/VhZIoW38trAOjO96PI4=; b=m+BGAsLnzFnU4/ngo+KXEIbeGcabPeS25FC2ij8ttMru/97osyc2I7e14fb1dRhBzO 9ZBi8S84lfUMtCl5nqG2sQ0yebB+xyg/jsco3CD2lm9n2feQ3t5r+z8YP3/jfLYv6khl nb3DpWwOEJkr1jZlFnbo89ORfwS1pj0DAZm0V2b/goLkKfijSY4fhBFWS3haeVBbRJF2 q0eGq5/ucuS+qQmTwT1hyG04RrweeMJReZLTAPsKuWrnR9DgqwcExoJcY/CJNCVCmPLW lVyfqOA2VDXOBI0wywKLGfcBRDyUydTlGLqtbPlPjwlho16Ahu2BjDYxrdv5N6OfGy9X V7uQ== X-Gm-Message-State: AOAM533YrPhMlVMLo1SbTcm5fCZ56X0yx1KrU9Vb4qE8jEFA/kc7DcSO 7v6OxwDk+gxgbMlRTL4mnNmBwYzhkIbhPA== X-Google-Smtp-Source: ABdhPJxh4NlN7O9k26GfCJY31QchOYFCk4ecIdP07gQR375gE+ZqHZCL/C5J8YrhZREK9NIsMz1YEg== X-Received: by 2002:a5d:474f:: with SMTP id o15mr5013546wrs.298.1623324882704; Thu, 10 Jun 2021 04:34:42 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:42 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 15/40] trace-cmd record: Add new parameter --compression Date: Thu, 10 Jun 2021 14:34:01 +0300 Message-Id: <20210610113426.257931-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a new parameter "trace-cmd record --compression", can be used to select the desired compression algorithm for the trace output file. One of these strings can be passed: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms, available algorithms can be listed with "trace-cmd list -c" If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 24 +++++++++++++++++++----- tracecmd/trace-usage.c | 5 +++++ 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 29f84b7c..8f8ea08b 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -199,6 +199,7 @@ struct common_record_context { char *date2ts; char *user; const char *clock; + const char *compression; struct tsc_nsec tsc2nsec; int data_flags; int tsync_loop_interval; @@ -3700,7 +3701,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(msg_handle, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3718,7 +3720,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4066,7 +4069,8 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, + listed_events, ctx->compression); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4458,7 +4462,8 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); + handle = tracecmd_create_file_latency(ctx->output, + local_cpu_count, ctx->compression); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4489,7 +4494,8 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); + handle = tracecmd_create_init_file_glob(ctx->output, + listed_events, ctx->compression); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); @@ -5736,6 +5742,7 @@ void init_top_instance(void) } enum { + OPT_comporession = 239, OPT_tsc2nsec = 240, OPT_fork = 241, OPT_tsyncinterval = 242, @@ -6172,6 +6179,7 @@ static void parse_record_options(int argc, {"fork", no_argument, NULL, OPT_fork}, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, {"poll", no_argument, NULL, OPT_poll}, + {"compression", required_argument, NULL, OPT_comporession}, {NULL, 0, NULL, 0} }; @@ -6597,6 +6605,12 @@ static void parse_record_options(int argc, cmd_check_die(ctx, CMD_set, *(argv+1), "--poll"); recorder_flags |= TRACECMD_RECORD_POLL; break; + case OPT_comporession: + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + ctx->compression = strdup(optarg); + break; case OPT_quiet: case 'q': quiet = true; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index cd8ca391..c70f9919 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -69,6 +69,11 @@ static struct usage_help usage_help[] = { " If 0 is specified, no loop is performed - timestamps offset is calculated only twice," " at the beginnig and at the end of the trace\n" " --poll don't block while reading from the trace buffer\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { "set", From patchwork Thu Jun 10 11:34:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DD21C47094 for ; Thu, 10 Jun 2021 11:34:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7AE0D613FF for ; Thu, 10 Jun 2021 11:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230272AbhFJLgz (ORCPT ); Thu, 10 Jun 2021 07:36:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230264AbhFJLgy (ORCPT ); Thu, 10 Jun 2021 07:36:54 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 139BFC0617A6 for ; Thu, 10 Jun 2021 04:34:45 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id y7so1900330wrh.7 for ; Thu, 10 Jun 2021 04:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=Tx/RS3qab4KPAHw+QfUD0TMOHBRHJuhW2ah7GumeZf9fCCvGruJohh9ytzJ/KngHGl oow4QiyKzpz1HlJLfM0+A5n1Brr0jo5/dYAUuAlNV9Bg3iMuo3kw33L3rya+zh3iZ1A9 vwK5JKHVOe0t4Zn7mAMVpKZQFcp+aMxqSu6s7UeQW9/aQYix1wEjlxgoU9aUwlIU2FE5 lkzu/MUMPbf+oEqw7qPqFjCzt3pkOgEdpDMn2MpIP6Tw0v75RKfpBYHPgnrC/dZUnvXX Ds2dLvkjbVAflf3+264MJJUbKXGN1jYkcdcbuc9uxhhF2nvv37Qp8tzmGX0CaEMvwskJ gatA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=BVibbKa0Ol4m8KejPnG/DKvSQpkZTi0uuIM3DIsz9dCoLJ/Vkcl/u1tq/gvzsyFOT8 L1SzuAcNz3QgpFww2XCo0s3+Ofj9/q5nWzuM0G179wIg25DUKAfJyzg82gRhOjGLqqWa Qee7g+CV/eZq8YdA9dIDqa8xLqcAaY64iQ61I5KBbRSWGvzR7OQvIxEJBaWcHMxLbjvh /hhmd4215nNkb/bR/oKNiLOvkn9xks45RldjzH8K9etn2Qg/sLGV8iGkcsEQnaDH+w0m R5aruVViYRJgb7gI3B/HhxXv+otF/M6+4vpXm/sHnYCxkADPq1KcFSFUetT1eXGKMjKk k95Q== X-Gm-Message-State: AOAM530dbt9eIWxwnfHJAZBpdPNaHX/kmpkEzU1CCWTuYCRcBaVYYmwV skHaFmVQSMoXOMn/cZOPFpo= X-Google-Smtp-Source: ABdhPJxf4EN8owJB6BmGdtBuuEUBbl3+FhGpAxrR63iy1VZIS9hLDL0v2+/NkUscs+eq4qju+cMurg== X-Received: by 2002:adf:f346:: with SMTP id e6mr4787689wrp.179.1623324883620; Thu, 10 Jun 2021 04:34:43 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:43 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 16/40] trace-cmd dump: Add support for trace files version 7 Date: Thu, 10 Jun 2021 14:34:02 +0300 Message-Id: <20210610113426.257931-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added support to read and dump information of trace files version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 125 +++++++++++++++++++++++++++++++++++------- 1 file changed, 105 insertions(+), 20 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 03cc82b4..ed82104b 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -26,6 +26,9 @@ static struct tep_handle *tep; static unsigned int trace_cpus; static int has_clock; +static unsigned long file_version; +static bool read_compress; +static struct tracecmd_compression *compress; enum dump_items { SUMMARY = (1 << 0), @@ -52,46 +55,62 @@ enum dump_items verbosity; tracecmd_plog(fmt, ##__VA_ARGS__); \ } while (0) -static int read_file_string(int fd, char *dst, int len) +static int read_fd(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, 1); + r = read(fd, dst+size, len); if (r > 0) { - size++; - len--; + size += r; + len -= r; } else break; - if (!dst[size - 1]) - break; - } while (r > 0 && len); + } while (r > 0); - if (!size || dst[size - 1]) + if (len) return -1; - return 0; + return size; } -static int read_file_bytes(int fd, char *dst, int len) +static int read_compressed(int fd, char *dst, int len) +{ + + if (read_compress) + return tracecmd_compress_read(compress, dst, len); + return read_fd(fd, dst, len); +} + +static int read_file_string(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, len); + r = read_compressed(fd, dst+size, 1); if (r > 0) { - size += r; - len -= r; + size++; + len--; } else break; - } while (r > 0); + if (!dst[size - 1]) + break; + } while (r > 0 && len); - if (len) + if (!size || dst[size - 1]) return -1; return 0; } +static int read_file_bytes(int fd, char *dst, int len) +{ + int ret; + + ret = read_compressed(fd, dst, len); + return ret < 0 ? ret : 0; +} + static void read_dump_string(int fd, int size, enum dump_items id) { char buf[DUMP_SIZE]; @@ -146,7 +165,6 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; - unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -168,11 +186,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); - ver = strtol(buf, NULL, 10); - if (!ver && errno) + file_version = strtol(buf, NULL, 10); + if (!file_version && errno) die("Invalid file version string %s", buf); - if (!tracecmd_is_version_supported(ver)) - die("Unsupported file version %lu", ver); + if (!tracecmd_is_version_supported(file_version)) + die("Unsupported file version %lu", file_version); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) @@ -192,6 +210,29 @@ static void dump_initial_format(int fd) do_print(SUMMARY, "\t\t%d\t[Page size, bytes]\n", val4); } +static void dump_compress(int fd) +{ + char compr[DUMP_SIZE]; + char *ver = NULL; + + if (file_version < 7) + return; + + /* get compression header */ + if (read_file_string(fd, compr, DUMP_SIZE)) + die("no compression header"); + ver = strchr(compr, ' '); + if (!ver) + die("no compression version"); + *ver = '\0'; + do_print((SUMMARY), "\t\t%s\t[Compression algorithm]\n", compr); + do_print((SUMMARY), "\t\t%s\t[Compression version]\n", ver + 1); + + compress = tracecmd_compress_alloc(compr, ver + 1, fd, tep, NULL); + if (!compress) + die("cannot uncomress the file"); +} + static void dump_header_page(int fd) { unsigned long long size; @@ -234,11 +275,34 @@ static void dump_header_event(int fd) read_dump_string(fd, size, HEAD_EVENT); } +static void uncompress_reset(void) +{ + if (compress && file_version >= 7) { + read_compress = false; + tracecmd_compress_reset(compress); + } +} + +static int uncompress_block(void) +{ + int ret = 0; + + if (compress && file_version >= 7) { + ret = tracecmd_uncompress_block(compress); + if (!ret) + read_compress = true; + + } + return ret; +} + static void dump_ftrace_events_format(int fd) { unsigned long long size; unsigned int count; + if (uncompress_block()) + die("cannot uncompress file block"); do_print((SUMMARY | FTRACE_FORMAT), "\t[Ftrace format, "); if (read_file_number(fd, &count, 4)) die("cannot read the count of the ftrace events"); @@ -251,6 +315,7 @@ static void dump_ftrace_events_format(int fd) read_dump_string(fd, size, FTRACE_FORMAT); count--; } + uncompress_reset(); } static void dump_events_format(int fd) @@ -262,6 +327,9 @@ static void dump_events_format(int fd) do_print((SUMMARY | EVENT_FORMAT | EVENT_SYSTEMS), "\t[Events format, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &systems, 4)) die("cannot read the count of the event systems"); @@ -284,6 +352,7 @@ static void dump_events_format(int fd) } systems--; } + uncompress_reset(); } static void dump_kallsyms(int fd) @@ -292,12 +361,17 @@ static void dump_kallsyms(int fd) do_print((SUMMARY | KALLSYMS), "\t[Kallsyms, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the kallsyms"); do_print((SUMMARY | KALLSYMS), "%d bytes]\n", size); read_dump_string(fd, size, KALLSYMS); + + uncompress_reset(); } static void dump_printk(int fd) @@ -306,12 +380,17 @@ static void dump_printk(int fd) do_print((SUMMARY | TRACE_PRINTK), "\t[Trace printk, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the trace printk"); do_print((SUMMARY | TRACE_PRINTK), "%d bytes]\n", size); read_dump_string(fd, size, TRACE_PRINTK); + + uncompress_reset(); } static void dump_cmdlines(int fd) @@ -320,12 +399,17 @@ static void dump_cmdlines(int fd) do_print((SUMMARY | CMDLINES), "\t[Saved command lines, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 8)) die("cannot read the size of the saved command lines"); do_print((SUMMARY | CMDLINES), "%d bytes]\n", size); read_dump_string(fd, size, CMDLINES); + + uncompress_reset(); } static void dump_cpus_count(int fd) @@ -669,6 +753,7 @@ static void dump_file(const char *file) do_print(SUMMARY, "\n Tracing meta data in file %s:\n", file); dump_initial_format(fd); + dump_compress(fd); dump_header_page(fd); dump_header_event(fd); dump_ftrace_events_format(fd); From patchwork Thu Jun 10 11:34:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91BCCC47094 for ; Thu, 10 Jun 2021 11:35:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7BB5D61414 for ; Thu, 10 Jun 2021 11:35:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230269AbhFJLg5 (ORCPT ); Thu, 10 Jun 2021 07:36:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230291AbhFJLg4 (ORCPT ); Thu, 10 Jun 2021 07:36:56 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A6E1C0617A8 for ; Thu, 10 Jun 2021 04:34:46 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id z8so1892432wrp.12 for ; Thu, 10 Jun 2021 04:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KF+feKjMyKpX88QmGJlWIg3wMmDI0ahJDLjuPSqTtL0=; b=GBpUoHMqls7UkxcUVSlabWYCRfoOnSaZ7P+GrF20ukxz+i7vKzt0c8T89SaQpiSJEz CBmPIXEGQbRceu4yHmjqFmK6j+lcjABkY6NQvzKbhGKQypD5QGq8XMeNMddOCGlpZVdg GnrXZ5RnG8v6uZEFQpZkMzfx8i+Te0w7hpJwTZRYU0k0ogxvO7FZMcHdOiRFNEHIDWTS dLpjeT8ZUDWIkmeOa1+LlmL5gwoIbTVW6Ay0cyCMoiBOqLO+R8YiuMTTafCsFiQ5mGNs dARA3osoyJ3pu6gKdckHdw3yoVwnyntmOtNbj+LJ7QttlNgTHFKLVaGG/QPvJ39sqwiw qpnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KF+feKjMyKpX88QmGJlWIg3wMmDI0ahJDLjuPSqTtL0=; b=csXdKZX6IHWaZ2ZfsuG4Nh0lGXcYtH3Y1oZHVnhzgQoTSesRR9rCoTP3yu+qJmsxmX V9+xUoRpDv0a3807nwf4Ok5Vg4qF+zaIZlX4jZXbnfI0QAVtpb/HU23oVhSOXy36ejsI c8YdC7EX0uamP7BNWvF7ReUgva5pVg1Snt7xrHOFJHotKYcvucSrNNKqS7ix/671eK4w Mn0sYktQXxuADqNUUzUl8J24moZPo8d++L8I4DfdcEHpjill/L3btgwB5o80JeJFa52l WxnHnlHIY3pqjDcvFKhOY3JbTZOfq4u4PmloDqJI1an47MlGnlpSoLbz6GqLgkPNBhBl ZpBQ== X-Gm-Message-State: AOAM530WyiVzbzbiVD9R7ZbP+i6glp6rgnn/DeWomOyQ3oc/+RBSHBNm +YVzwy5Tw8TjrAF2RKBo2wGNcHZZNe0MZg== X-Google-Smtp-Source: ABdhPJy3zSJqz0V0I2WCuPtWLaLl/akJYNl1bOvOiUYjzywezy2sfIV4BW7XNj7V+Pk+lkKoAi4p0A== X-Received: by 2002:a5d:4a4e:: with SMTP id v14mr4919094wrs.74.1623324884482; Thu, 10 Jun 2021 04:34:44 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:44 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 17/40] trace-cmd library: Add support for zlib compression library Date: Thu, 10 Jun 2021 14:34:03 +0300 Message-Id: <20210610113426.257931-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If libz is available, use that library to provide trace file compression support. The library is detected runtime. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 10 ++ lib/trace-cmd/include/trace-cmd-local.h | 5 + lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++++++++++++++++ lib/trace-cmd/trace-compress.c | 8 ++ 4 files changed, 195 insertions(+) create mode 100644 lib/trace-cmd/trace-compress-zlib.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index bab4322d..83ba7016 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -7,6 +7,13 @@ ldir:=$(src)/lib/trace-cmd DEFAULT_TARGET = $(LIBTRACECMD_STATIC) +pound := \# +ZLIB_INSTALLED := $(shell if (printf "$(pound)include \n void main(){deflateInit(NULL, Z_BEST_COMPRESSION);}" | $(CC) -o /dev/null -x c - -lz >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) +ifeq ($(ZLIB_INSTALLED), 1) +CFLAGS += -DHAVE_ZLIB +$(info Have zlib compression support) +endif + OBJS = OBJS += trace-hash.o OBJS += trace-hooks.o @@ -26,6 +33,9 @@ OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif OBJS += trace-compress.o +ifeq ($(ZLIB_INSTALLED), 1) +OBJS += trace-compress-zlib.o +endif # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 24228d8f..80304141 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,6 +31,11 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +#ifdef HAVE_ZLIB +int tracecmd_zlib_init(void); +void tracecmd_zlib_free(void); +#endif + int tracecmd_compress_init(void); void tracecmd_compress_free(void); diff --git a/lib/trace-cmd/trace-compress-zlib.c b/lib/trace-cmd/trace-compress-zlib.c new file mode 100644 index 00000000..3208d57b --- /dev/null +++ b/lib/trace-cmd/trace-compress-zlib.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include + +#include "trace-cmd-private.h" + +#define __ZLIB_NAME "zlib" +#define __ZLIB_WEIGTH 10 +#define __ZLIB_FILE "libz.so" +#define ZLIB_FUNC_COMPRESS "compress2" +#define ZLIB_FUNC_DECOMOPRESS "uncompress" +#define ZLIB_FUNC_SIZE "compressBound" +#define ZLIB_FUNC_VERSION "zlibVersion" + +static void *zlib_handle; +static int (*_lib_compress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes, int level); +static int (*_libz_decompress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes); +static unsigned long (*_libz_compress_bound)(unsigned long in_bytes); +static const char *(*_libz_version)(void); + +static int zlib_compress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_lib_compress) + return -1; + + ret = _lib_compress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes, Z_BEST_COMPRESSION); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_STREAM_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static int zlib_decompress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_libz_decompress) + return -1; + + ret = _libz_decompress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_DATA_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static unsigned int zlib_compress_bound(unsigned int in_bytes) +{ + if (!_libz_compress_bound) + return 0; + return _libz_compress_bound(in_bytes); +} + +static bool zlib_is_supported(const char *name, const char *version) +{ + const char *zver; + + if (!name) + return false; + if (strlen(name) != strlen(__ZLIB_NAME) || strcmp(name, __ZLIB_NAME)) + return false; + + if (!version) + return true; + + if (!_libz_version) + return false; + zver = _libz_version(); + if (!zver) + return false; + + /* Compare the major version number */ + if (atoi(version) <= atoi(zver)) + return true; + + return false; +} + +int tracecmd_zlib_init(void) +{ + if (zlib_handle) + return 0; + + zlib_handle = dlopen(__ZLIB_FILE, RTLD_NOW | RTLD_GLOBAL); + if (!zlib_handle) + return -1; + _lib_compress = dlsym(zlib_handle, ZLIB_FUNC_COMPRESS); + if (!_lib_compress) + goto error; + _libz_decompress = dlsym(zlib_handle, ZLIB_FUNC_DECOMOPRESS); + if (!_libz_decompress) + goto error; + _libz_compress_bound = dlsym(zlib_handle, ZLIB_FUNC_SIZE); + if (!_libz_compress_bound) + goto error; + _libz_version = dlsym(zlib_handle, ZLIB_FUNC_VERSION); + if (!_libz_version) + goto error; + + return tracecmd_compress_proto_register(__ZLIB_NAME, _libz_version(), __ZLIB_WEIGTH, + zlib_compress, zlib_decompress, + zlib_compress_bound, zlib_is_supported); + +error: + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + dlclose(zlib_handle); + zlib_handle = NULL; + return -1; +} + +void tracecmd_zlib_free(void) +{ + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + + if (zlib_handle) { + dlclose(zlib_handle); + zlib_handle = NULL; + } + +} diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index ff742226..2b329d00 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -353,6 +353,10 @@ int tracecmd_compress_init(void) gettimeofday(&time, NULL); srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); +#ifdef HAVE_ZLIB + tracecmd_zlib_init(); +#endif + return 0; } @@ -530,6 +534,10 @@ void tracecmd_compress_free(void) struct compress_proto *proto = proto_list; struct compress_proto *del; +#ifdef HAVE_ZLIB + tracecmd_zlib_free(); +#endif + while (proto) { del = proto; proto = proto->next; From patchwork Thu Jun 10 11:34:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A42B1C48BDF for ; Thu, 10 Jun 2021 11:36:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8E1CA61406 for ; Thu, 10 Jun 2021 11:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230396AbhFJLh6 (ORCPT ); Thu, 10 Jun 2021 07:37:58 -0400 Received: from mail-wm1-f54.google.com ([209.85.128.54]:34748 "EHLO mail-wm1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230272AbhFJLh5 (ORCPT ); Thu, 10 Jun 2021 07:37:57 -0400 Received: by mail-wm1-f54.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso4597038wmc.1 for ; Thu, 10 Jun 2021 04:35:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9m9bTPjljOgtQSdfKwMA2giQ6G51c9OhQIMnQPsx4ik=; b=JvoBaaJrmSrDuXwN9qTeJUTOl+rP8O+Kxdru4goXE2kyjYM9659BzpvZ9FMsHidSvO HpCuajTxztqe+zT5DKxz4MJ2OY2awj4DP4a1hCTOj1sUJyJBBWxpDw9nEFrgEZXaNKfN SXFd3tYPOd19xCCA3ow3BzsrwQ+IHckOSszFMAy5ojz0WZdpZTLj1OMk8emcXdmSz+qQ 50UIEy2PGcDHhCBimRT7C1hRSPNCdF5bi38ENZiiux9RjdPVDAtrhOS38jQfG6dvEJb8 BOysovCi+VwBvy6pPaY+1tI7F1NklJC7V6EBWGaCD68vlDQGnBMfjJvvNEIVz4qaz3Wg JT3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9m9bTPjljOgtQSdfKwMA2giQ6G51c9OhQIMnQPsx4ik=; b=p9hCI1IZOguUpSHg1pIXNYj//1Zl9UTafniPK2EATyzH6fzdpO8bVtxkZQ7ChGReJC uOxBCUXbM38NT+SX7ORJ7YjaXVpuESqAUr360QaXSRu/toyAnCnlFX8h10G7ot36v4/p 4M0lB1jFn894OsNx0HithupxbN1KvvJBCSG4NnobGdJabcw/Pm0gAgj7CcfKYuprPp1v OdMvVkiA6icKmqVKFDJBy+qJ/qnbJ7EUXuhZQmPONs3zJYHEHSz4vVhOtPpNo5MDU4da G+aN0eNV409FDogNj0gXwDStWNzOlZJD/2grHfyizI/g2YI1k4cuDkVPENS/m1SSp8B8 owrA== X-Gm-Message-State: AOAM532+YKgYZuRnOujfSxGit0SIfvl/N1O5p3KY6IbvNcq2issNJn64 9gILLTyWMxO882BkdF2XsdI= X-Google-Smtp-Source: ABdhPJyqangeEdAs8RmyAfa/PZJ1z6bnyZbCGqzU3/kNMWVOjsckr1tCa+T4NNFdVFmS0XN6RF8qUg== X-Received: by 2002:a05:600c:1c22:: with SMTP id j34mr15111857wms.166.1623324885537; Thu, 10 Jun 2021 04:34:45 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:45 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 18/40] trace-cmd library: Hide the logic for updating buffer offset Date: Thu, 10 Jun 2021 14:34:04 +0300 Message-Id: <20210610113426.257931-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a trace buffer data are written in the trace file, the buffer option in the file metadata is updated with the file offset of the tracing data. Hide this logic into the trace-cmd library. Added new APIs: tracecmd_add_buffer_description() tracecmd_write_buffers_description() Changed APIs: tracecmd_append_buffer_cpu_data() Removed APIs: tracecmd_add_buffer_option() This internal refactoring is needed for changes, related to compression of the options sections of the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 8 +- lib/trace-cmd/trace-output.c | 84 +++++++++++++++++-- tracecmd/trace-record.c | 16 ++-- 3 files changed, 85 insertions(+), 23 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ee73325c..cbb578ec 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -295,8 +295,8 @@ struct tracecmd_option * tracecmd_add_option_v(struct tracecmd_output *handle, unsigned short id, const struct iovec *vector, int count); -struct tracecmd_option *tracecmd_add_buffer_option(struct tracecmd_output *handle, - const char *name, int cpus); +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus); +int tracecmd_write_buffers_description(struct tracecmd_output *handle); int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); @@ -312,9 +312,7 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files); - + const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index f249ee91..8abd6fb9 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -44,6 +44,14 @@ struct tracecmd_option { struct list_head list; }; +struct tracecmd_buffer { + int cpus; + void *name; + tsize_t offset; + struct tracecmd_option *option; + struct list_head list; +}; + enum { OUTPUT_FL_SEND_META = (1 << 0), }; @@ -63,6 +71,7 @@ struct tracecmd_output { struct tracecmd_compression *compress; struct list_head options; + struct list_head buffers; struct tracecmd_msg_handle *msg_handle; char *trace_clock; }; @@ -189,6 +198,7 @@ bool tracecmd_get_quiet(struct tracecmd_output *handle) void tracecmd_output_free(struct tracecmd_output *handle) { struct tracecmd_option *option; + struct tracecmd_buffer *buffer; if (!handle) return; @@ -199,6 +209,13 @@ void tracecmd_output_free(struct tracecmd_output *handle) if (handle->pevent) tep_unref(handle->pevent); + while (!list_empty(&handle->buffers)) { + buffer = container_of(handle->buffers.next, + struct tracecmd_buffer, list); + list_del(&buffer->list); + free(buffer->name); + free(buffer); + } while (!list_empty(&handle->options)) { option = container_of(handle->options.next, struct tracecmd_option, list); @@ -1071,6 +1088,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, goto out_free; list_head_init(&handle->options); + list_head_init(&handle->buffers); buf[0] = 23; buf[1] = 8; @@ -1369,9 +1387,8 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -struct tracecmd_option * -tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, - int cpus) +static struct tracecmd_option * +add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; @@ -1399,6 +1416,53 @@ tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, return option; } +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus) +{ + struct tracecmd_buffer *buf; + + buf = calloc(1, sizeof(struct tracecmd_buffer)); + if (!buf) + return -1; + buf->name = strdup(name); + buf->cpus = cpus; + if (!buf->name) { + free(buf); + return -1; + } + list_add_tail(&buf->list, &handle->buffers); + return 0; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + struct tracecmd_option *option; + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + option = add_buffer_option(handle, buf->name, buf->cpus); + if (!option) + return -1; + buf->option = option; + } + return 0; +} + +static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) +{ + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + if (strlen(name) == strlen(buf->name) && !strcmp(name, buf->name)) { + if (handle->file_version >= 7) + return buf->offset; + if (!buf->option) + break; + return buf->option->offset; + } + } + return 0; +} + int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; @@ -1643,18 +1707,23 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, } int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files) + const char *name, int cpus, char * const *cpu_data_files) { + tsize_t b_offset; tsize_t offset; stsize_t ret; + b_offset = get_buffer_file_offset(handle, name); + if (!b_offset) { + tracecmd_warning("Cannot find description for buffer %s\n", name); + return -1; + } offset = lseek64(handle->fd, 0, SEEK_CUR); /* Go to the option data, where will write the offest */ - ret = lseek64(handle->fd, option->offset, SEEK_SET); + ret = lseek64(handle->fd, b_offset, SEEK_SET); if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); + tracecmd_warning("could not seek to %lld\n", b_offset); return -1; } @@ -1713,6 +1782,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); list_head_init(&handle->options); + list_head_init(&handle->buffers); if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 8f8ea08b..3921071d 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4152,7 +4152,6 @@ static void touch_file(const char *file) } static void append_buffer(struct tracecmd_output *handle, - struct tracecmd_option *buffer_option, struct buffer_instance *instance, char **temp_files) { @@ -4180,7 +4179,7 @@ static void append_buffer(struct tracecmd_output *handle, touch_file(temp_files[i]); } - tracecmd_append_buffer_cpu_data(handle, buffer_option, + tracecmd_append_buffer_cpu_data(handle, tracefs_instance_get_name(instance->tracefs), cpu_count, temp_files); for (i = 0; i < instance->cpu_count; i++) { @@ -4441,7 +4440,6 @@ static void write_guest_file(struct buffer_instance *instance) static void record_data(struct common_record_context *ctx) { - struct tracecmd_option **buffer_options; struct tracecmd_output *handle; struct buffer_instance *instance; bool local = false; @@ -4512,9 +4510,6 @@ static void record_data(struct common_record_context *ctx) } if (buffers) { - buffer_options = malloc(sizeof(*buffer_options) * buffers); - if (!buffer_options) - die("Failed to allocate buffer options"); i = 0; for_each_instance(instance) { int cpus = instance->cpu_count != local_cpu_count ? @@ -4522,10 +4517,9 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; - - buffer_options[i++] = tracecmd_add_buffer_option(handle, - tracefs_instance_get_name(instance->tracefs), - cpus); + tracecmd_add_buffer_description(handle, + tracefs_instance_get_name(instance->tracefs), + cpus); add_buffer_stat(handle, instance); } } @@ -4560,7 +4554,7 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; print_stat(instance); - append_buffer(handle, buffer_options[i++], instance, temp_files); + append_buffer(handle, instance, temp_files); } } From patchwork Thu Jun 10 11:34:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA5FFC48BE0 for ; Thu, 10 Jun 2021 11:35:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B03B861407 for ; Thu, 10 Jun 2021 11:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230425AbhFJLht (ORCPT ); Thu, 10 Jun 2021 07:37:49 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:37721 "EHLO mail-wr1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230443AbhFJLhn (ORCPT ); Thu, 10 Jun 2021 07:37:43 -0400 Received: by mail-wr1-f50.google.com with SMTP id i94so1916459wri.4 for ; Thu, 10 Jun 2021 04:35:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=56+tOMEVuLQdZHVTYFPELp0wqhvqnbET+paZQ1bbMLQ=; b=Qa66b/xm6CoGztsfyQWjur9wHw03J8aiHaQFYCXHDhyB30DRXlYbrd579Y36kk0PV5 fw1JPgE/7mo4RJ1n0iLkQWuogVQhGyfErfV2FSu1gyLHPw8CT3RAqd4ee0A6oZHCCcK+ H/Mc/LWI9BYkT76HWYLNGWGQGAk6qP7dWdP/CS3U5Bpie4EMnP9ifJJ/E+5LPUytuxo9 Ko6vCHEuI6kLSqOPEKjX3XtZObZ0vMr18WSr8nY0iLd++KPl3hrgy6Ph0pzk8qq46S0t ACPIwpzxPvGNZDHGkHknl3rT0e1g9LxwJhya9Uh3Wpo9HDM74SSv4wXQxjzVlXw8bjXi GQCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=56+tOMEVuLQdZHVTYFPELp0wqhvqnbET+paZQ1bbMLQ=; b=bffZWtkkvLBTrdFKIZGAouPRnAXX5Wjii76FwhvhVwb94lW8dBsXXTJBUSOTt5XB/e a4s/mdlQGDHKSMFAekeBp/CwuTEq1xUqBbMYK+v+bbIvyHE2M21S16TjEjrQW7yoRYW4 eQRoY/ssB7GEBnkTUMb5CYCFAf+nyLIfrIVkCK7EhBQ1MuYbOUcb6xxJCGKbJX5f4uKT /Nw9E1RQWvYlyLgl12bJD2GlEcnANKLMz/LhO9ujvn1tkOxECVQHIrYvjNdiBe3sjfJc 3IlMt1V4Z9N6DuUnYt4SdHGcM8tuaM4704wZE6NDPc0gRvve0Gv4hWRhGfkdNL+KRAkp 1Ukg== X-Gm-Message-State: AOAM530HP6Uo56p4IU85Mnsn9psQagdQ+d0uIDn4JieYXB1KD8Uhhrxv +ZRbnsq+V3ANt0JXHFZ/icF9NCxftpccTA== X-Google-Smtp-Source: ABdhPJxKQAEOEK3oXX3ijzMOkLF7Wkrl8vYXnCBhZzoLtMy5eFDOLP5CRqCw8C1Hz8PLKgMDeTjAbA== X-Received: by 2002:a05:6000:cb:: with SMTP id q11mr4952204wrx.13.1623324886356; Thu, 10 Jun 2021 04:34:46 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:45 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 19/40] trace-cmd: Move buffers description outside of options Date: Thu, 10 Jun 2021 14:34:05 +0300 Message-Id: <20210610113426.257931-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new buffer data are written in the trace file, the file offset is stored in the buffer description. As the buffer description is part of the trace options, which are already written in the file, the offset field in this option is updated with the new value. This causes a serious problem when option section of the file is compressed. Updating a value in already compressed section may change the size of this section, which can make the calculated offset of the buffer trace data invalid. To solve that problem and to enable the compression of the option section of the file, the structure of the trace file is changed: - In trace file v7, the description of the buffers is moved to a separate section in the trace file medatata. It is stored after the CPU count section and before the option section. The format of this new buffers description is: 4 bytes - count of the buffers, stored in the file. The top buffer is not counted. For each buffer: 8 bytes, offset of the trace data for this buffer in the file. null-terminated string, name of the buffer. The new section is mandatory, in case of no buffers - the count 0 is stored in the first 4 bytes. - In trace file v6, the description of the buffers is not changed. These changes are needed for compression of the options sections from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-input.c | 40 +++++++++ lib/trace-cmd/trace-output.c | 85 ++++++++++++++++++- tracecmd/trace-dump.c | 24 ++++++ tracecmd/trace-listen.c | 3 + tracecmd/trace-record.c | 4 + 7 files changed, 156 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index cbb578ec..faa8c533 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -112,6 +112,7 @@ enum tracecmd_file_states { TRACECMD_FILE_PRINTK, TRACECMD_FILE_CMD_LINES, TRACECMD_FILE_CPU_COUNT, + TRACECMD_FILE_BUFERS, TRACECMD_FILE_OPTIONS, TRACECMD_FILE_CPU_LATENCY, TRACECMD_FILE_CPU_FLYRECORD, diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 80304141..59148930 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -46,4 +46,7 @@ void out_compression_reset(struct tracecmd_output *handle); void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); +int write_buffers_description_v7(struct tracecmd_output *handle); +int write_buffers_description_v6(struct tracecmd_output *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 0b17b58d..e953909f 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -881,6 +881,39 @@ static int read_cpus(struct tracecmd_input *handle) return 0; } +static int read_buffers(struct tracecmd_input *handle) +{ + struct input_buffer_instance *buffer; + unsigned long long offset; + unsigned int count; + int i; + + if (handle->file_version < 7 || handle->file_state >= TRACECMD_FILE_BUFERS) + return 0; + + if (read4(handle, &count) < 0) + return -1; + i = handle->nr_buffers; + handle->nr_buffers += count; + handle->buffers = realloc(handle->buffers, + sizeof(*handle->buffers) * handle->nr_buffers); + if (!handle->buffers) + return -1; + + for (; i < handle->nr_buffers; i++) { + buffer = &handle->buffers[i]; + if (read8(handle, &offset) < 0) + return -1; + buffer->offset = offset; + buffer->name = read_string(handle); + if (!buffer->name) + return -1; + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +} + /** * tracecmd_read_headers - read the header information from trace.dat * @handle: input handle for the trace.dat file @@ -955,6 +988,13 @@ int tracecmd_read_headers(struct tracecmd_input *handle, if (state <= handle->file_state) return 0; + if (read_buffers(handle) < 0) + return -1; + + if (state <= handle->file_state) + return 0; + + if (read_options_type(handle) < 0) return -1; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8abd6fb9..5bd33b70 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -376,6 +376,17 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) if (!handle) return -1; + if (handle->file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, handle->file_version); + return -1; + default: + break; + } + } + switch (new_state) { case TRACECMD_FILE_HEADERS: case TRACECMD_FILE_FTRACE_EVENTS: @@ -384,10 +395,16 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) case TRACECMD_FILE_PRINTK: case TRACECMD_FILE_CMD_LINES: case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_OPTIONS: + case TRACECMD_FILE_BUFERS: if (handle->file_state == (new_state - 1)) return 0; break; + case TRACECMD_FILE_OPTIONS: + if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) + return 0; + if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) + return 0; + break; case TRACECMD_FILE_CPU_LATENCY: case TRACECMD_FILE_CPU_FLYRECORD: if (handle->file_state == TRACECMD_FILE_OPTIONS) @@ -1433,7 +1450,7 @@ int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char * return 0; } -int tracecmd_write_buffers_description(struct tracecmd_output *handle) +__hidden int write_buffers_description_v6(struct tracecmd_output *handle) { struct tracecmd_option *option; struct tracecmd_buffer *buf; @@ -1447,6 +1464,63 @@ int tracecmd_write_buffers_description(struct tracecmd_output *handle) return 0; } +__hidden int write_buffers_description_v7(struct tracecmd_output *handle) +{ + struct tracecmd_buffer *buf; + char *data = NULL; + int count = 0; + int size; + int ret; + + if (!check_out_state(handle, TRACECMD_FILE_BUFERS)) { + tracecmd_warning("Cannot write buffers descriptions"); + return -1; + } + + list_for_each_entry(buf, &handle->buffers, list) { + count++; + } + + count = convert_endian_4(handle, count); + ret = do_write_check(handle, &count, 4); + if (ret < 0) + return ret; + + list_for_each_entry(buf, &handle->buffers, list) { + size = 8 + strlen(buf->name) + 1; + data = malloc(size); + if (!data) + goto error; + *(tsize_t *)data = 0; + strcpy(data + 8, buf->name); + /* Save the data location in case it needs to be updated */ + buf->offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, data, size)) + goto error; + + /* + * In case a buffer instance has different number of CPUs as the + * local machine. + */ + if (buf->cpus) + tracecmd_add_option(handle, TRACECMD_OPTION_CPUCOUNT, + sizeof(int), &buf->cpus); + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +error: + return -1; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + if (handle->file_version >= 7) + return write_buffers_description_v7(handle); + + return write_buffers_description_v6(handle); +} + static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) { struct tracecmd_buffer *buf; @@ -1505,7 +1579,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_cpus(handle, cpus) < 0) goto out_free; - + if (tracecmd_write_buffers_description(handle) < 0) + goto out_free; if (tracecmd_write_options(handle) < 0) goto out_free; @@ -1698,7 +1773,9 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, ret = tracecmd_write_cpus(handle, cpus); if (ret) return ret; - + ret = tracecmd_write_buffers_description(handle); + if (ret) + return ret; ret = tracecmd_write_options(handle); if (ret) return ret; diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index ed82104b..56acb01d 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -420,6 +420,29 @@ static void dump_cpus_count(int fd) do_print(SUMMARY, "\t%d [CPUs with tracing data]\n", trace_cpus); } +static void dump_buffers(int fd) +{ + unsigned long long offset; + char name[DUMP_SIZE]; + int count; + + if (file_version < 7) + return; + + if (read_file_number(fd, &count, 4)) + die("cannot read the buffers count"); + do_print(SUMMARY, "\t%d [buffers with tracing data]\n", count); + + while (count) { + if (read_file_number(fd, &offset, 8)) + die("cannot read the buffer offset"); + if (read_file_string(fd, name, DUMP_SIZE)) + die("cannot read the buffer name"); + do_print(SUMMARY, "\t\t 0x%llx %s[data offset, buffer name]\n", offset, name); + count--; + } +} + static void dump_option_string(int fd, int size, char *desc) { do_print(OPTIONS, "\t\t[Option %s, %d bytes]\n", desc, size); @@ -762,6 +785,7 @@ static void dump_file(const char *file) dump_printk(fd); dump_cmdlines(fd); dump_cpus_count(fd); + dump_buffers(fd); dump_therest(fd); tep_free(tep); diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index 0ae1c948..62debdb6 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -604,6 +604,9 @@ static int put_together_file(int cpus, int ofd, const char *node, if (write_options) { ret = tracecmd_write_cpus(handle, cpus); + if (ret) + goto out; + ret = tracecmd_write_buffers_description(handle); if (ret) goto out; ret = tracecmd_write_options(handle); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 3921071d..8bd771e0 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3711,6 +3711,9 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; ret = tracecmd_write_cpus(network_handle, instance->cpu_count); + if (ret) + goto error; + ret = tracecmd_write_buffers_description(network_handle); if (ret) goto error; ret = tracecmd_write_options(network_handle); @@ -4074,6 +4077,7 @@ static void setup_agent(struct buffer_instance *instance, add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_buffers_description(network_handle); tracecmd_write_options(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); instance->network_handle = network_handle; From patchwork Thu Jun 10 11:34:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03D37C48BD1 for ; Thu, 10 Jun 2021 11:35:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7DB761407 for ; Thu, 10 Jun 2021 11:35:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229895AbhFJLhu (ORCPT ); Thu, 10 Jun 2021 07:37:50 -0400 Received: from mail-wr1-f45.google.com ([209.85.221.45]:40935 "EHLO mail-wr1-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230444AbhFJLhn (ORCPT ); Thu, 10 Jun 2021 07:37:43 -0400 Received: by mail-wr1-f45.google.com with SMTP id y7so1900579wrh.7 for ; Thu, 10 Jun 2021 04:35:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mNhWGc9JKwMhf/44O4SEmqYecOnF74SysaVRk8k6W+M=; b=oU1yUKs5dY1elc/4RTz5DfLn5ned2ZThw9vMRBzXEkvlBGs3xGP4bZVAMKYFLobsqv 7DPgGjEiTUMJo7x4FyJy21mxHgyInc/XplKvjUGsH/4q4s1F9DPPNSc7WiaclhhzJFkq 2TVy+jIII/KB4TD/cvlb+DO9xUWwzp16Aiazf+tT645no3AP7LtmnLWiDJ29zHCLtdAk vAILD03LwA+doN4W2SP8Oa5L5Y6xKZAGr7chOUAlbdGuznoXc7lMksp/dk4vk/riDg3H qSSWP5212lrl0vDJ3QkemBcRXoD9l1VS3i1SVT3laQzK80vYRIAUNnc5O41oHnRyRQKX 3M2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mNhWGc9JKwMhf/44O4SEmqYecOnF74SysaVRk8k6W+M=; b=jeaNwiciRbE2Rt/8axZXjMSKJlqEZtvDO7r914AwV0U7L1ASqaQ3DPh28ghFBrHDkj MuX+VA78r3L9AxRXc2vzdkakdl01/F2iaI7H+QiojLSMzOaTEvN+Xa3jT3jZ/lod9yTC xg/SH6jFwXZ1jjS9uSDBvy/V6qj9Mboy+LEqoU3UkOIS7/rftpCtG89za2cbaZVAhrun qWtocVQ/jlIbKxeCzecI0W83tg+wItn0SGagOVECluOSVGanEtItE9iaJ9Ev2Xw8nYwc 4TQyswBHQk8UZQW6N7klspwVZgmEf1IPFLBAOhKaoqWel/xy2WZuw2rDVbWjHnrX4g26 W3xg== X-Gm-Message-State: AOAM533vhqetc/YJ4X9RpbaH4Zm2QxXfRfLt1DItnig8X4h4LsIZofFU BT0DsX3o5ZlxxHn/LhYukG8= X-Google-Smtp-Source: ABdhPJw+q50sLpJ4ZFVCpQZJm0m80wkuyHC1oDPWreMOSwi8Ng6tEMsORcdwbvsmnc8L2rmqU8nD5g== X-Received: by 2002:a05:6000:1b8d:: with SMTP id r13mr4904256wru.207.1623324887250; Thu, 10 Jun 2021 04:34:47 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:46 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 20/40] trace-cmd library: Track the offset in the option section in the trace file Date: Thu, 10 Jun 2021 14:34:06 +0300 Message-Id: <20210610113426.257931-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading / writing trace file, store the offset of the option section in the file. A new internal API is added, to get the offset of the option section from an input file handler: tracecmd_get_options_offset() These changes are needed for compression of the option section from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 11 +++++++++++ lib/trace-cmd/trace-output.c | 2 ++ 3 files changed, 14 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index faa8c533..10089389 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -239,6 +239,7 @@ tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, const char **name, const char **version); +size_t tracecmd_get_options_offset(struct tracecmd_input *handle); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e953909f..b76f2b47 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -157,6 +157,7 @@ struct tracecmd_input { size_t header_files_start; size_t ftrace_files_start; size_t event_files_start; + size_t options_start; size_t total_file_size; /* For custom profilers. */ @@ -2759,6 +2760,7 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); for (;;) { if (do_read_check(handle, &option, 2)) @@ -4213,6 +4215,15 @@ bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle) return handle->use_trace_clock; } +/** + * tracecmd_get_options_offset - get offset of the options sections in the file + * @handle: input handle for the trace.dat file + */ +size_t tracecmd_get_options_offset(struct tracecmd_input *handle) +{ + return handle->options_start; +} + /** * tracecmd_get_show_data_func - return the show data func * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 5bd33b70..1b9cf683 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -66,6 +66,7 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + size_t options_start; bool do_compress; struct tracecmd_compression *compress; @@ -1858,6 +1859,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); + handle->options_start = tracecmd_get_options_offset(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Thu Jun 10 11:34:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C15B0C48BDF for ; Thu, 10 Jun 2021 11:35:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ADBE9613FF for ; Thu, 10 Jun 2021 11:35:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230435AbhFJLhu (ORCPT ); Thu, 10 Jun 2021 07:37:50 -0400 Received: from mail-wr1-f47.google.com ([209.85.221.47]:43682 "EHLO mail-wr1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230401AbhFJLhp (ORCPT ); Thu, 10 Jun 2021 07:37:45 -0400 Received: by mail-wr1-f47.google.com with SMTP id r9so1894480wrz.10 for ; Thu, 10 Jun 2021 04:35:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vQ/Js3+yEnptXiJvdIVqM4WvS/t43l9fxOrwlaNbevg=; b=BycnLGjSeWLFwVe8SF4w+1+ExloE6WT9VAGITQjzWtaIfh0GS3KnGDlnTZthxym9hg hbT/MmZhaoeR8V28hkmj3lq44UEs9tVnrOFmPfesRcF8jvm/dl9vnMthUCvhPHrc2eMK 4imBiTnURSyQnF2fLGWgaAjo38JkPI1McnPcGbGkO3aB+l+7zOWL4u+9N/lZCiu07VCi Z+lef8DEiid2H4yPS0FOrdRmY7R2sOKSjQGqtsQk8kjqTufZLPSBtQAuxKa6SLZEmgrM A4lSZgSfhGVC+bS6zDimgo5/rrrjsbiZufZp23aNpBjx5xESyRS4dmEBEyrsjEcIitYT OlMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vQ/Js3+yEnptXiJvdIVqM4WvS/t43l9fxOrwlaNbevg=; b=nH/AyaOxq4ij0TaTfbC6FoBR/NuRRNLkblFIS4z4kkKl68JUVbevKHDM5xucEyd9zv wtxQK2hUgOsBzJNt2MFugXySXOv/mxpieXNXIMq9G2/uclPgyOzQk6MVI2UP7wRbVIq1 1r2e8LL4P+KuAM2nHj7riDZ3zy+VH/Jt1PdDN7PWguDFHOYEu/vtdDPPODkUNp0yBMNq Rxel1ZEZI8GQ/jefTsqjez+vyQk0gvk5fD9QtYu3J9Xy/6y2Kyz2TM3Wv+jKegTiIP/7 upZH2BY+ENGbLtPtRgrnt76pDZQTiiv/7lcOgZiGJup2evqxs+SSAInNjq/aigufb0wP EmuA== X-Gm-Message-State: AOAM5333gi8IGhmb9BWMPYegAkxn7k7caz1xJAeW5WUWhONULmxr74dx NTp0mwR6TxwaFGP+yhEKrUk= X-Google-Smtp-Source: ABdhPJxqXs7gXpTPElyVDcn0QYMfgEXPTWy8ezfKjQs52R5X+OazBBsKWiZdIWu3hGOylYqEQk92fw== X-Received: by 2002:adf:e84a:: with SMTP id d10mr4931884wrn.48.1623324888139; Thu, 10 Jun 2021 04:34:48 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:47 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 21/40] trace-cmd library: Add compression of the option section of the trace file Date: Thu, 10 Jun 2021 14:34:07 +0300 Message-Id: <20210610113426.257931-22-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Comperss the option section of the trace file. This section is not big currently and compressing it does not reduce significantly the size of the file. This could be useful in the future as new options can be added, storing a potentially huge amount of data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 46 +++++++++++------ lib/trace-cmd/trace-output.c | 69 +++++++++++++++++-------- tracecmd/trace-dump.c | 16 ++++-- 4 files changed, 92 insertions(+), 40 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 59148930..b77c49ff 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,7 @@ void tracecmd_zlib_free(void); int tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b76f2b47..8fff003e 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2761,23 +2761,29 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); - + if (in_uncompress_block(handle)) + return -1; for (;;) { - if (do_read_check(handle, &option, 2)) - return -1; + ret = do_read_check(handle, &option, 2); + if (ret) + goto out; if (option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (do_read_check(handle, &size, 4)) - return -1; + ret = do_read_check(handle, &size, 4); + if (ret) + goto out; size = tep_read_number(handle->pevent, &size, 4); buf = malloc(size); - if (!buf) - return -ENOMEM; - if (do_read_check(handle, buf, size)) - return -1; + if (!buf) { + ret = -ENOMEM; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret) + goto out; switch (option) { case TRACECMD_OPTION_DATE: @@ -2827,14 +2833,16 @@ static int handle_options(struct tracecmd_input *handle) buf + 8, 4); ret = tsync_cpu_offsets_load(handle, buf + 12, size - 12); if (ret < 0) - return ret; + goto out; tracecmd_enable_tsync(handle, true); break; case TRACECMD_OPTION_CPUSTAT: buf[size-1] = '\n'; cpustats = realloc(cpustats, cpustats_size + size + 1); - if (!cpustats) - return -ENOMEM; + if (!cpustats) { + ret = -ENOMEM; + goto out; + } memcpy(cpustats + cpustats_size, buf, size); cpustats_size += size; cpustats[cpustats_size] = 0; @@ -2844,14 +2852,17 @@ static int handle_options(struct tracecmd_input *handle) handle->nr_buffers++; handle->buffers = realloc(handle->buffers, sizeof(*handle->buffers) * handle->nr_buffers); - if (!handle->buffers) - return -ENOMEM; + if (!handle->buffers) { + ret = -ENOMEM; + goto out; + } buffer = &handle->buffers[handle->nr_buffers - 1]; buffer->name = strdup(buf + 8); if (!buffer->name) { free(handle->buffers); handle->buffers = NULL; - return -ENOMEM; + ret = -ENOMEM; + goto out; } offset = *(unsigned long long *)buf; buffer->offset = tep_read_number(handle->pevent, &offset, 8); @@ -2908,8 +2919,11 @@ static int handle_options(struct tracecmd_input *handle) } handle->cpustats = cpustats; + ret = 0; - return 0; +out: + in_uncompress_reset(handle); + return ret; } static int read_options_type(struct tracecmd_input *handle) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 1b9cf683..81d97d4a 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -150,6 +150,18 @@ __hidden void out_compression_reset(struct tracecmd_output *handle) handle->do_compress = false; } +__hidden int out_uncompress_block(struct tracecmd_output *handle) +{ + int ret = 0; + + if (handle->file_version < 7 || !handle->compress) + return 0; + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->do_compress = true; + return ret; +} + __hidden int out_compression_start(struct tracecmd_output *handle) { if (handle->file_version < 7 || !handle->compress) @@ -1326,32 +1338,38 @@ int tracecmd_write_options(struct tracecmd_output *handle) if (do_write_check(handle, "options ", 10)) return -1; - + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + out_compression_start(handle); list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; + goto error; + if (out_compression_end(handle)) + goto error; handle->file_state = TRACECMD_FILE_OPTIONS; return 0; +error: + out_compression_reset(handle); + return -1; } int tracecmd_append_options(struct tracecmd_output *handle) @@ -1367,42 +1385,51 @@ int tracecmd_append_options(struct tracecmd_output *handle) * We can append only if options are already written and tracing data * is not yet written */ - if (handle->file_state != TRACECMD_FILE_OPTIONS) + if (handle->file_state != TRACECMD_FILE_OPTIONS || !handle->options_start) return -1; - - if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) return -1; - offset = lseek64(handle->fd, -2, SEEK_CUR); - if (offset == (off_t)-1) + if (out_uncompress_block(handle)) return -1; - - r = pread(handle->fd, &option, 2, offset); + if (do_lseek(handle, 0, SEEK_END) == -1) + goto error; + offset = do_lseek(handle, -2, SEEK_CUR); + if (offset == (off_t)-1) + goto error; + r = do_preed(handle, &option, 2, offset); if (r != 2 || option != TRACECMD_OPTION_DONE) - return -1; - + goto error; list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } - option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; + goto error; + if (handle->file_version >= 7 && handle->compress) { + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) + goto error; + if (out_compression_end(handle)) + goto error; + } return 0; +error: + out_compression_reset(handle); + return -1; } static struct tracecmd_option * diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 56acb01d..8d0f2251 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -82,6 +82,13 @@ static int read_compressed(int fd, char *dst, int len) return read_fd(fd, dst, len); } +static int do_lseek(int fd, int offset, int whence) +{ + if (read_compress) + return tracecmd_compress_lseek(compress, offset, whence); + return lseek64(fd, offset, whence); +} + static int read_file_string(int fd, char *dst, int len) { size_t size = 0; @@ -625,6 +632,9 @@ static void dump_options(int fd) unsigned int size; int count = 0; + if (uncompress_block()) + die("cannot uncompress file block"); + for (;;) { if (read_file_number(fd, &option, 2)) die("cannot read the option id"); @@ -635,7 +645,7 @@ static void dump_options(int fd) count++; if (!DUMP_CHECK(OPTIONS) && !DUMP_CHECK(CLOCK) && !DUMP_CHECK(SUMMARY)) { - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); continue; } switch (option) { @@ -685,12 +695,12 @@ static void dump_options(int fd) default: do_print(OPTIONS, " %d %d\t[Unknown option, size - skipping]\n", option, size); - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); break; } } do_print(SUMMARY, "\t[%d options]\n", count); - + uncompress_reset(); } static void dump_latency(int fd) From patchwork Thu Jun 10 11:34:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FA11C48BE0 for ; Thu, 10 Jun 2021 11:36:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 634CE613E9 for ; Thu, 10 Jun 2021 11:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230265AbhFJLh5 (ORCPT ); Thu, 10 Jun 2021 07:37:57 -0400 Received: from mail-wm1-f48.google.com ([209.85.128.48]:54863 "EHLO mail-wm1-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230238AbhFJLh5 (ORCPT ); Thu, 10 Jun 2021 07:37:57 -0400 Received: by mail-wm1-f48.google.com with SMTP id m3so742634wms.4 for ; Thu, 10 Jun 2021 04:35:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GoJm7VsZJlhF8Q1T9Z+WwKmc6H4hjRgw29T+P08ERkU=; b=LzoZT11vZH2AMqbdOL0MH/SqbKnA25KkbIU4GhI/h5J+DBNEH82FgUaefukCqFH5uh arDp8DShoC8V3DOzeMnEdGS4sbi0OlIkOYf0q7u7SoJgO/UddeWKKyQkrt17SQhTREQw 4UnVG+VpRN5WKWptbf5CIP1wzcxlasO9a3kiw5ES8K3wasta6wDF8asgKilgOrpx2zvK +x7E8ix0+AGIFwVLndombdmkOFLlG2jSMjgsQ20V/jCH4D7n6yX54azfCkEQ9OWWpHEh 8KPhMOOPIE+pHoctI3L2OakhEYZ3nSf8tuh7mEck1fz5b8XlSLvUBV6xhxQk/gfh3go7 bfUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GoJm7VsZJlhF8Q1T9Z+WwKmc6H4hjRgw29T+P08ERkU=; b=PzWTtneyP1JbDFln3PepNxEAiR9/5ro0B7en0LwDp5LtZsF1B1L5RgWp1uXtGUHRGX 2SQiRES/zmpIrnz1QtbkIfedbQR1mX2Msckepov48l2ccjIjJHuSfeFOv1omdLgbRP83 wpuIrOy/CXVfELbbpOdZ3p/JK94gU4QA45sTTg8Mi+9C0+IK8pMtQx+xapbZwu4BOBTK Vj7xDMFJqWG54hy+DWY7nyRN6SP30tsjEvGeqs4AVMHhhzKAZpDUGb4yLi8xVe2+D9zj i4f+DioIVnMQrjKSiLARkZ53eX5eR71DlxpGxS50aASS5LShb6iB3xFFcKCWwoQmBnsu Q6Pg== X-Gm-Message-State: AOAM531FYlkVSb3RF6tqmvJngWelr7E00WY9Vj2UZueVfQViALcdXVLT QEcSW6xK5lme4UR/x27YwjE= X-Google-Smtp-Source: ABdhPJzC21RnvfC1wx2/tsI9nrJpfDr1JiEIIsEygUuxH+2mLT4jK/1snIMR8RBIEJH4zuWFJz9lNQ== X-Received: by 2002:a1c:9dcd:: with SMTP id g196mr14879230wme.135.1623324888959; Thu, 10 Jun 2021 04:34:48 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:48 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 22/40] trace-cmd library: Refactor the logic for writing trace data in the file Date: Thu, 10 Jun 2021 14:34:08 +0300 Message-Id: <20210610113426.257931-23-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Refactored the internal logic of tracecmd_write_cpu_data() API to be suitable for adding trace data compression. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 94 +++++++++++++++++------------------- 1 file changed, 44 insertions(+), 50 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 81d97d4a..860be759 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1679,11 +1679,17 @@ static char *get_clock(struct tracecmd_output *handle) return handle->trace_clock; } +struct data_file_write { + int file_size; + off64_t soffset; + off64_t data_offset; + off64_t doffset; +}; + int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { - off64_t *offsets = NULL; - unsigned long long *sizes = NULL; + struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; char *clock = NULL; @@ -1705,37 +1711,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (do_write_check(handle, "flyrecord", 10)) goto out_free; - offsets = malloc(sizeof(*offsets) * cpus); - if (!offsets) - goto out_free; - sizes = malloc(sizeof(*sizes) * cpus); - if (!sizes) - goto out_free; - - offset = lseek64(handle->fd, 0, SEEK_CUR); - - /* hold any extra data for data */ - offset += cpus * (16); - - /* - * Unfortunately, the trace_clock data was placed after the - * cpu data, and wasn't accounted for with the offsets. - * We need to save room for the trace_clock file. This means - * we need to find the size of it before we define the final - * offsets. - */ - clock = get_clock(handle); - if (!clock) + data_files = calloc(cpus, sizeof(struct data_file_write)); + if (!data_files) goto out_free; - /* Save room for storing the size */ - offset += 8; - offset += strlen(clock); - /* 2 bytes for [] around the clock */ - offset += 2; - - /* Page align offset */ - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + /* Write 0 for trace data offset and size and store offsets of these fields */ for (i = 0; i < cpus; i++) { file = cpu_data_files[i]; ret = stat(file, &st); @@ -1743,53 +1723,67 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, tracecmd_warning("can not stat '%s'", file); goto out_free; } - offsets[i] = offset; - sizes[i] = st.st_size; - offset += st.st_size; - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].file_size = st.st_size; - endian8 = convert_endian_8(handle, offsets[i]); + endian8 = 0; + data_files[i].doffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; - endian8 = convert_endian_8(handle, sizes[i]); + data_files[i].soffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; } - if (save_clock(handle, clock)) + clock = get_clock(handle); + if (clock && save_clock(handle, clock)) goto out_free; for (i = 0; i < cpus; i++) { + data_files[i].data_offset = lseek64(handle->fd, 0, SEEK_CUR); + /* Page align offset */ + data_files[i].data_offset = (data_files[i].data_offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].data_offset = lseek64(handle->fd, data_files[i].data_offset, SEEK_SET); + if (data_files[i].data_offset == (off64_t)-1) + goto out_free; if (!tracecmd_get_quiet(handle)) fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", - i, (unsigned long long) offsets[i]); - offset = lseek64(handle->fd, offsets[i], SEEK_SET); - if (offset == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offsets[i]); - goto out_free; - } + i, (unsigned long long) data_files[i].data_offset); + check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != sizes[i]) { + if (check_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, sizes[i]); + check_size, data_files[i].file_size); goto out_free; } + /* Write the real CPU data offset inthe file */ + offset = lseek64(handle->fd, data_files[i].doffset, SEEK_SET); + endian8 = convert_endian_8(handle, data_files[i].data_offset); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + /* Write the real CPU data size in the file */ + offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); + endian8 = convert_endian_8(handle, check_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + check_size; + offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", (unsigned long long)check_size); } - free(offsets); - free(sizes); + if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) + goto out_free; + free(data_files); handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; return 0; out_free: - free(offsets); - free(sizes); + lseek64(handle->fd, 0, SEEK_END); + free(data_files); return -1; } From patchwork Thu Jun 10 11:34:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D734FC48BE8 for ; Thu, 10 Jun 2021 11:35:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C437B61419 for ; Thu, 10 Jun 2021 11:35:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230389AbhFJLhu (ORCPT ); Thu, 10 Jun 2021 07:37:50 -0400 Received: from mail-wm1-f45.google.com ([209.85.128.45]:55886 "EHLO mail-wm1-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230446AbhFJLhq (ORCPT ); Thu, 10 Jun 2021 07:37:46 -0400 Received: by mail-wm1-f45.google.com with SMTP id g204so5954906wmf.5 for ; Thu, 10 Jun 2021 04:35:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XThZzPdOYJaqw5JOXWGo2FbsQo5b9Di4gify468C2WE=; b=oS4j3+cwJ7XOIHFjOrhYjqvALzBGey/oR3bGyQPcvyFL/tPCDOxhq/xJ1U9bU4Aj3T PVMGm4FM9nBv+8U6rzLfglZR5sxofb7KJrwOO4A/11miBgGemLS5sjJIWeL+Bueqg4nq Vre9+7lJMCzrg9ooZLO1Vwx/MJ6naCEMaRdmuzih4JZcicYCj9eWZIWN8ekVZwrLX0/Q vcfh8OsEWBPLDz4fh9BkFwz+0BXG+23BFRY6DkEtY6d0q+XZxRmY51pcT0mxXdsI693/ u5GeaR+j8pyOH7k/tJj/xGjbIg5NjIgAbhwkqTUoQH8LVonPB9ui3ZKzfV5kDXT8Ckay 6q4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XThZzPdOYJaqw5JOXWGo2FbsQo5b9Di4gify468C2WE=; b=hcXTBiDr9u4SyV+93Bm826Rt5XP0Exk+RKKdt1K4VVpajIo6bkin1g0cqpvwt8eZGJ 9ef4JmpOoFEQdFRlm5CZ3/mfk0o7xPUCOuy/qXnVOmERe11d7+dkq4NKl4bZ/GbeYsw/ Ne2TTST3eVqvSDXejQprKL54C5p0nkr6WlcquQRXAxZ0cJnOwXOIjylLp4uWsLgUSwGK u4mCpgAY34+rJyeRrIHmheth1w/HVjsbDrAABEj87ygcqxJK1bX0NzRJPiDGZfCqnQHB V9bxjJLlzRZPEhzTLdIS7WHdf0qwUhAX19r//4jopUICNIX3H14EI5Udd2H738fByM/a utUQ== X-Gm-Message-State: AOAM533UTiK9SqDzftWF19HVfaSIngAQU75g7eIGu9je9zjNZojvizTK cBcfEhvqdvHQ6M0vQomNky6o0hXGq7poxw== X-Google-Smtp-Source: ABdhPJwJ3xCbl+pyOjhtS2LRH593cWjGA9mZaMu9zbCWaF4PXmcbkiJGwa9c9TOP2Wkuo+BNDqLizg== X-Received: by 2002:a7b:c20a:: with SMTP id x10mr4502370wmi.141.1623324889952; Thu, 10 Jun 2021 04:34:49 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:49 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 23/40] trace-cmd library: Add APIs for read and write compressed data in chunks Date: Thu, 10 Jun 2021 14:34:09 +0300 Message-Id: <20210610113426.257931-24-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New compression APIs are added for writing and reading compressed data from / to files in chunks: tracecmd_compress_copy_from() tracecmd_uncompress_copy_to() Format of the compressed data, used by these APIs, is: - 4 bytes, chunks count - for each chunk: - 4 bytes, size of compressed data in this chunk - 4 bytes, uncompressed size of the data in this chunk - data, bytes of Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 +- lib/trace-cmd/trace-compress.c | 200 ++++++++++++++++++ 2 files changed, 204 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 10089389..067ba34d 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -494,7 +494,10 @@ int tracecmd_compress_proto_register(const char *name, const char *version, int char *out, unsigned int *out_bytes), unsigned int (*comress_size)(unsigned int bytes), bool (*is_supported)(const char *name, const char *version)); - +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size); /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index 2b329d00..4a4707c2 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -594,3 +594,203 @@ error: free(v); return -1; } + +/** + * tracecmd_compress_copy_from - Copy and compress data from a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data + * @chunk_size: size of one compression chunk + * @read_size: in - max bytes to read from @fd, 0 to read till the EOF + * out - size of the uncompressed data read from @fd + * @write_size: return, size of the compressed data written into @handle + * + * This function reads uncompressed data from given @fd until EOF is reached, compresses + * the data using the @handle compression context and writes the compressed data into the fd + * associated with the @handle. The data is compressed on chunks with given @chunk_size size. + * The compressed data is written in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int chunks = 0; + unsigned int rsize = 0; + unsigned int rmax = 0; + unsigned int wsize = 0; + unsigned int csize; + unsigned int rchunk = 0; + unsigned int size; + unsigned int r; + off64_t offset; + char *buf_from; + char *buf_to; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->comress_block || !handle->proto->comress_size) + return 0; + if (read_size) + rmax = *read_size; + csize = handle->proto->comress_size(chunk_size); + buf_from = malloc(chunk_size); + if (!buf_from) + return -1; + buf_to = malloc(csize); + if (!buf_to) + return -1; + /* save the initial offset and write 0 chunks */ + offset = lseek64(handle->fd, 0, SEEK_CUR); + write_fd(handle->fd, &chunks, 4); + + do { + if (rmax > 0 && (rmax - rsize) < chunk_size) + rchunk = (rmax - rsize); + else + rchunk = chunk_size; + + r = read(fd, buf_from, rchunk); + if (r < 0 || (rmax > 0 && rsize >= rmax)) + break; + rsize += r; + size = csize; + if (r > 0) { + ret = handle->proto->comress_block(buf_from, r, buf_to, &size); + if (ret < 0) { + if (errno == EINTR) + continue; + break; + } + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &r, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write the compressed data */ + ret = write_fd(handle->fd, buf_to, size); + if (ret != size) + break; + /* data + compress header */ + wsize += (size + 8); + chunks++; + } + } while (r > 0); + free(buf_from); + free(buf_to); + if (r) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + endian4 = tep_read_number(handle->tep, &chunks, 4); + /* write chunks count*/ + write_fd(handle->fd, &chunks, 4); + lseek64(handle->fd, offset, SEEK_SET); + if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} + +/** + * tracecmd_uncompress_copy_to - Uncompress data and copy to a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data + * @read_size: return, size of the compressed data read from @handle + * @write_size: return, size of the uncompressed data written into @fd + * + * This function reads compressed data from the fd, associated with @handle, uncompresses it + * using the @handle compression context and writes the uncompressed data into the fd. + * The compressed data must be in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + unsigned int rsize = 0; + unsigned int wsize = 0; + char *bytes_out = NULL; + char *bytes_in = NULL; + int size_out; + int size_in; + int chunks; + char buf[4]; + char *tmp; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + chunks = tep_read_number(handle->tep, buf, 4); + rsize += 4; + while (chunks) { + if (read(handle->fd, buf, 4) != 4) + break; + s_compressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (read(handle->fd, buf, 4) != 4) + break; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (!bytes_in || size_in < s_compressed) { + tmp = realloc(bytes_in, s_compressed); + if (!tmp) + break; + bytes_in = tmp; + size_in = s_compressed; + } + + if (!bytes_out || size_out < s_uncompressed) { + tmp = realloc(bytes_out, s_uncompressed); + if (!tmp) + break; + bytes_out = tmp; + size_out = s_uncompressed; + } + + if (read_fd(handle->fd, bytes_in, s_compressed) < 0) + break; + rsize += s_compressed; + ret = handle->proto->uncompress_block(bytes_in, s_compressed, + bytes_out, &s_uncompressed); + if (ret) + break; + write_fd(fd, bytes_out, s_uncompressed); + wsize += s_uncompressed; + chunks--; + } + free(bytes_in); + free(bytes_out); + if (chunks) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} From patchwork Thu Jun 10 11:34:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D26BC48BD1 for ; Thu, 10 Jun 2021 11:36:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 464DE613FF for ; Thu, 10 Jun 2021 11:36:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230288AbhFJLiC (ORCPT ); Thu, 10 Jun 2021 07:38:02 -0400 Received: from mail-wm1-f43.google.com ([209.85.128.43]:54861 "EHLO mail-wm1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230272AbhFJLiC (ORCPT ); Thu, 10 Jun 2021 07:38:02 -0400 Received: by mail-wm1-f43.google.com with SMTP id m3so742688wms.4 for ; Thu, 10 Jun 2021 04:35:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NEhPGyloHjAqC5x3xEBshV9EuN0/dWE1r+Si3CUPt1k=; b=G30HsMaMMu4EvvDPCSKlOjil0xwynpFrQY0QC5dD3Xc6YAfuAdll07bGvPFKY+XbOK DD5ccjZxAqhf8Akm6JrvFYPNRf8rmIYN+588ohHORL4TrGggjeeSK4DkRGgA65vOZVmH WCas7pfbg4sfB3uL1UV0q75hYdYVl/oUPY/v3c0zGhMx6SLbwqH9WYV8yLMYvTfXuwuO VKy5JePMbM+nFmQGMGY64rSgoGjEUK75xrI1u4k9nyADfbCPz3j3ApE0/w3sCGd6Kecv AbzB140R03d21R8LYbBy2/EXjAqTV+7z1eYKEuVLBQLN+8JHpHCXDsDr5wYPS+Z7TzvT LS3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NEhPGyloHjAqC5x3xEBshV9EuN0/dWE1r+Si3CUPt1k=; b=JW1ue03ZiCQzlBW/+70pVhauqihbrTAzMrDo/L3PQVhY11oEprT/1Aj/WnunakKaWM OD3glMJWGJSk+h+3I1V4/99gz3xDnhoQ8ukxDFw0wvuqsGv1A1JG3eMBkFBXUanl5IA0 by1EtXvcY+/vICoG2zS066Ndmm33cH0YIKgUfVD7m20Q4CfmlpWXljGQFLNQAjNPEAYA LhyycXs4neMB5L/ENeMgKya4sHtorSOxUXvma/vDXeEvz8FXRPbKhRc4WcccJmHJZfU8 1/ed+L9hA8gAT77MplDrynUtfmHKSGDpMngLj0WRZSsB+TEnMEBL2yjHqRI/z5MLlJJ0 xlLg== X-Gm-Message-State: AOAM533jwH2mT8Cxnd4cO91m1kCG7nv3v+hABj3ifY5s+oqVIhCWvkey vspSfhUGf1fsqY5w5lodfA8= X-Google-Smtp-Source: ABdhPJxZ+OuofO5t6Y9LFrzV7NKcqqyRoI+wkaCVybS84KhOpk8TMqa++aNpoNKhhoKbQ8JzOb4r6Q== X-Received: by 2002:a1c:e91a:: with SMTP id q26mr14748660wmc.170.1623324890838; Thu, 10 Jun 2021 04:34:50 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:50 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 24/40] trace-cmd: Compress trace data Date: Thu, 10 Jun 2021 14:34:10 +0300 Message-Id: <20210610113426.257931-25-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a trace.dat file of version 7, compress the trace data. The data is compressed in chunks. The chunk size is a compile time parameter, set by default to 1MB. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 50 +++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 860be759..bd14452a 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -321,6 +321,41 @@ static tsize_t copy_file(struct tracecmd_output *handle, return size; } +#define COMPRESS_CHUNK_SIZE (1*1024*1024) +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + unsigned long long rsize = 0; + unsigned long long wsize = 0; + tsize_t size; + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + if (handle->file_version >= 7) { + ret = tracecmd_compress_copy_from(handle->compress, fd, + COMPRESS_CHUNK_SIZE, &rsize, &wsize); + if (ret < 0) { + tracecmd_warning("Can't compress '%s'", file); + close(fd); + return 0; + } + size = rsize; + *write_size = wsize; + } else { + size = copy_file_fd(handle, fd); + *write_size = size; + } + + close(fd); + return size; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -1692,8 +1727,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; + unsigned long long read_size; + unsigned long long write_size; char *clock = NULL; - off64_t check_size; char *file; struct stat st; int ret; @@ -1749,11 +1785,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", i, (unsigned long long) data_files[i].data_offset); - check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != data_files[i].file_size) { + read_size = copy_file_compress(handle, cpu_data_files[i], &write_size); + if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, data_files[i].file_size); + read_size, data_files[i].file_size); goto out_free; } /* Write the real CPU data offset inthe file */ @@ -1763,14 +1799,14 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, goto out_free; /* Write the real CPU data size in the file */ offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); - endian8 = convert_endian_8(handle, check_size); + endian8 = convert_endian_8(handle, write_size); if (do_write_check(handle, &endian8, 8)) goto out_free; - offset = data_files[i].data_offset + check_size; + offset = data_files[i].data_offset + write_size; offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", - (unsigned long long)check_size); + (unsigned long long)write_size); } if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) From patchwork Thu Jun 10 11:34:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39516C47094 for ; Thu, 10 Jun 2021 11:36:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2479B61406 for ; Thu, 10 Jun 2021 11:36:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230238AbhFJLiB (ORCPT ); Thu, 10 Jun 2021 07:38:01 -0400 Received: from mail-wm1-f45.google.com ([209.85.128.45]:54864 "EHLO mail-wm1-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230272AbhFJLiB (ORCPT ); Thu, 10 Jun 2021 07:38:01 -0400 Received: by mail-wm1-f45.google.com with SMTP id m3so742716wms.4 for ; Thu, 10 Jun 2021 04:35:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=05lK6z1tRPoWI6LsDB6q2WoJqm7gnqhaRWTmXEeBIrg=; b=NDtVgA+s+EOTt+QwBvTH6dupeY2FOV4ddHnWPJXdh9o6PlKjP5uzdAVGNyxyt/hyxO BoaybKt2je6/2zF7HkqIzAamuY+1Ka/KDLKTHy6mQcLcGhuZ6DAubBVn/QAi+jJcFMAC v+Kt80UWLwynx9bWPgnmQRDDNKqsjc3ocsmhU0pAAvtdoL+lHxXzsq+6MOQZnUm99FzC hUV77ikAjEapfhjagSnt7FiGJKJ3xm0gLw9x/sy+QGVV6LXy3CJzM4mtiZaoktowjrdT N2auaBXGt247t0jJQElAHx4Q4Ukg2qgNjSgTl0w7+wuF4X68xC+MSKrobS2uiY2EDYdR JLsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=05lK6z1tRPoWI6LsDB6q2WoJqm7gnqhaRWTmXEeBIrg=; b=J738UoCX1cDnxdODyOHgeM+q5CrzkKUrr+mRgqfkVH0lAFsK6TlTplNJBpJ6Q8zrJK Ii+sPhxPtHrMg0qE0SWFvpOwj0h7dOFcG0CkIkZJSZ4ekhr51dbUwBJvve9Gq5dRQAbT VTHwAWp4rpJ8LniMviqLn8wJyqIs1hkzsnFItFhSIA6hRkDFZyw4jcEb8BysyCk96pzT 2sC36lvT+/owCLLl/LtzST0Pjfv8fFdwmPCxovXC/7nMw3tcwi1wYFnjgu7arKWyN1H+ qX+RmJoXM1Mqkg59D7YhqQh2XGrZOKUAr22Gho5xtQbOV3rh4xgPqpc552xAn/7Az4gr PgvQ== X-Gm-Message-State: AOAM5329voj43v4IokzT1Z2hJzvZXqlKuKoFB7IdSNx30CikEYJt4n/+ wz+hVo+G7VtkD9ubJaUUSrhd2DU0SLvkVw== X-Google-Smtp-Source: ABdhPJyz5YTymm6widXM0pZ8gD7O1PjV92qJDA6gT/nDHgIEUNWbmonu+dmECpWZajTMQ+6YTTu+Ug== X-Received: by 2002:a05:600c:4b88:: with SMTP id e8mr4485261wmp.107.1623324891754; Thu, 10 Jun 2021 04:34:51 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:51 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 25/40] trace-cmd: Read compressed trace data Date: Thu, 10 Jun 2021 14:34:11 +0300 Message-Id: <20210610113426.257931-26-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace.dat file of version 7, uncompress the trace data. The trace data for each CPU is uncompressed in a temporary file, located in /tmp directory with prefix "trace_cpu_data". Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 74 +++++++++++++++++++++++++++++-------- tracecmd/trace-read.c | 8 ++++ 2 files changed, 67 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 8fff003e..327082a2 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,7 @@ struct page { #endif }; +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +73,10 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + + /* temporary file for uncompressed cpu data */ + int cfd; + char cfile[26]; /* strlen(COMPR_TEMP_FILE) */ }; struct input_buffer_instance { @@ -1080,6 +1085,7 @@ static void *allocate_page_map(struct tracecmd_input *handle, off64_t map_offset; void *map; int ret; + int fd; if (handle->read_page) { map = malloc(handle->page_size); @@ -1119,12 +1125,15 @@ static void *allocate_page_map(struct tracecmd_input *handle, map_size -= map_offset + map_size - (cpu_data->file_offset + cpu_data->file_size); + if (cpu_data->cfd >= 0) + fd = cpu_data->cfd; + else + fd = handle->fd; again: page_map->size = map_size; page_map->offset = map_offset; - page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, - handle->fd, map_offset); + page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, fd, map_offset); if (page_map->map == MAP_FAILED) { /* Try a smaller map */ @@ -2316,13 +2325,41 @@ tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record) /* Not reached */ } +static int cpu_data_uncompress(struct tracecmd_input *handle, int cpu, unsigned long long *size) +{ + struct cpu_data *cpu_data; + + cpu_data = &handle->cpu_data[cpu]; + strcpy(cpu_data->cfile, COMPR_TEMP_FILE); + cpu_data->cfd = mkstemp(cpu_data->cfile); + if (cpu_data->cfd < 0) + return -1; + return tracecmd_uncompress_copy_to(handle->compress, cpu_data->cfd, NULL, size); +} + static int init_cpu(struct tracecmd_input *handle, int cpu) { struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + unsigned long long size; + off64_t offset; int i; - cpu_data->offset = cpu_data->file_offset; - cpu_data->size = cpu_data->file_size; + if (handle->file_version >= 7 && cpu_data->file_size > 0) { + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + if (cpu_data_uncompress(handle, cpu, &size) < 0) + return -1; + cpu_data->offset = 0; + cpu_data->file_offset = 0; + cpu_data->file_size = size; + cpu_data->size = size; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + } else { + cpu_data->offset = cpu_data->file_offset; + cpu_data->size = cpu_data->file_size; + } cpu_data->timestamp = 0; list_head_init(&cpu_data->page_maps); @@ -3015,6 +3052,7 @@ static int read_cpu_data(struct tracecmd_input *handle) handle->cpu_data[cpu].file_offset = offset; handle->cpu_data[cpu].file_size = size; + handle->cpu_data[cpu].cfd = -1; if (size > max_size) max_size = size; @@ -3635,17 +3673,23 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].cfd >= 0) { + close(handle->cpu_data[cpu].cfd); + unlink(handle->cpu_data[cpu].cfile); + } } } diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 0cf6e773..d605d05a 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1363,7 +1363,14 @@ struct tracecmd_input *read_trace_header(const char *file, int flags) static void sig_end(int sig) { + struct handle_list *handles; + fprintf(stderr, "trace-cmd: Received SIGINT\n"); + + list_for_each_entry(handles, &handle_list, list) { + tracecmd_close(handles->handle); + } + exit(0); } @@ -1924,6 +1931,7 @@ void trace_report (int argc, char **argv) /* and version overrides uname! */ if (show_version) otype = OUTPUT_VERSION_ONLY; + read_data_info(&handle_list, otype, global, align_ts); list_for_each_entry(handles, &handle_list, list) { From patchwork Thu Jun 10 11:34:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF8FBC48BD1 for ; Thu, 10 Jun 2021 11:35:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC47F61406 for ; Thu, 10 Jun 2021 11:35:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230298AbhFJLhF (ORCPT ); Thu, 10 Jun 2021 07:37:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230317AbhFJLhC (ORCPT ); Thu, 10 Jun 2021 07:37:02 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3021C061574 for ; Thu, 10 Jun 2021 04:34:53 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id l11-20020a05600c4f0bb029017a7cd488f5so6243324wmq.0 for ; Thu, 10 Jun 2021 04:34:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Oyvni9NwSMBXewBmzBWQtJRoikj0AVHUlPO3dy5pHdU=; b=luGhXp5SklsR+W1RMOZ54X6mS1Hrrs5Grc8TZEqjozxSTIh0ESqhUhu0kYfHjSNyUQ Uk5qowIR+mR85gw3TqZ0J/hjFBkVRc9NHMoq5/yQ9TOsW5mszlnsqYqo2dQupQyyd40S 7o4CbP8ZWnnIW45a7qv1tfS94Z2QyiuwtZX7rBddvZ0cx8oeZZKOALKUPjaaDUhQHbgy 41S3mltjOIWfekVXoIwbefKY7clNTrdFUbkkqIhrUZebQ5mFuLFNHFv4PhEiV+SipBqw kEkWRVjjC0Mzu5Yokbjlgb71DHs7srmUG0VwO9NJhv1bB2i3H2dQuVRLrhG5sFbqsAPW KfHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Oyvni9NwSMBXewBmzBWQtJRoikj0AVHUlPO3dy5pHdU=; b=XhD7XVDQf/3lr9AvSaCwx3xSUMaPR4TnDflA+wZPD2NgCyKDE7u15326hTqeotj1dy 79CGt+6dnn5j/hBTPPRDlU2gaJeqBwMmLLBrDttP1YTBvYWzefWkM8uHEdW7Uxqk5Lss 1JtIdz4mQFq7D1mDihNarSubSjlHBBSiZCnsm614THnOysB7ZbWgpfiarBfA1eL/yH4n KseBOQ9SWJ61qfUrfUGWSkjDxc8YqaIxsHzgno8bl/hEwpMpbN6kWxqj2Vsz20BcZQu4 cqU3MhV4tjGCa+WwE1JMMIIXNMnxK1QJcMqtlZEYidvLwTM7/9WXvBKn+GF6929BLI4V HSqw== X-Gm-Message-State: AOAM531ybEF3RRTKXCQvjkd2YEbF7g731LfcsY8rmjMCyU1cCH1NtkyW tdals5aRBlQhL8+IVm0vDJg= X-Google-Smtp-Source: ABdhPJzsEpJ+Bl41KUZZfqY6eHNwcPIQP1BV9wCq13voh/lTAJ96A9VPKzM0JRjPJlZQsLMy2KM+OA== X-Received: by 2002:a1c:dcc3:: with SMTP id t186mr4829201wmg.23.1623324892639; Thu, 10 Jun 2021 04:34:52 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:52 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 26/40] trace-cmd library: Compress latency trace data Date: Thu, 10 Jun 2021 14:34:12 +0300 Message-Id: <20210610113426.257931-27-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating latency trace file of version 7, compress the latency trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index bd14452a..6a272503 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1661,7 +1661,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (!path) goto out_free; - copy_file(handle, path); + copy_file_compress(handle, path, NULL); put_tracing_file(path); From patchwork Thu Jun 10 11:34:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51F8AC47094 for ; Thu, 10 Jun 2021 11:36:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C61E61406 for ; Thu, 10 Jun 2021 11:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230410AbhFJLh5 (ORCPT ); Thu, 10 Jun 2021 07:37:57 -0400 Received: from mail-wm1-f53.google.com ([209.85.128.53]:44991 "EHLO mail-wm1-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230265AbhFJLh5 (ORCPT ); Thu, 10 Jun 2021 07:37:57 -0400 Received: by mail-wm1-f53.google.com with SMTP id m41-20020a05600c3b29b02901b9e5d74f02so3101243wms.3 for ; Thu, 10 Jun 2021 04:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O4tw/vtCdoHAkqVILpd8GpKTrQc88oL92rc/FfJiJWk=; b=DEWgAuq7qbCRRGbek9yHsnc8D5UeBUhbqLNpiLYFh7Hd+YTRXpSH84z+zU1AsXjbTk htwtpn/vIdjia2L1J5Hp1jvLhFCx7csa8+Oj4RaOxilrSv6KEJlK4rFC0pAd349tW4bW rspjTS0yATlpB0QXArrXQ17ZVe7PZnn3zCa7P9XOedmPBCrmi+D+Dom//FpJ8Zrik1s7 hvHeVRJNaDvFzCYAfTmJ82/hI+avqUX0wDsS78QMKGlZ5PE2gnajw/K3RU3Jzlb6veZH rEy428oYtYFSBvGCX+MbklzNUMbFIOG3cjTBhZMBiL9zcQ1xxPVIRVjDIYKLjtoy/XpQ xMSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O4tw/vtCdoHAkqVILpd8GpKTrQc88oL92rc/FfJiJWk=; b=B+Pw0GvzmQSPe1zyNLnY8mpFnj3eGt3AMbbjES8ROZR2dubLSowkiV1/s8rFMldjUw iM4dbX3nVaf9wipXyy5JgfwQ+qFNLdGTY8RpxmgGlCWL96hgboB2diTQ7YP1ogsnL4mn gwJoi18WYR6lEvvbCFkM1RcGYsDIa/sF7rd7pfzzW1NQEHN7MQT+dvcPvFyx4NJM+yJR ayULpt8JxQMpKcqMAhHwmrPv1P1t3G9PDFtjYGv5m4CGyPIVcA1akpAtZmx1CPDpkyYh NWZ1xlZsKp4bH3b3sc5th0c/xllfWlqDJOWLs8PX1RVqlkpvMheMmxWC9uchk6PHCeBd lHeQ== X-Gm-Message-State: AOAM532iQ1PeiwiXxVfytQ52AQoW8obm2XC1jF800ANpH5XIQBpl+Uz4 Wq8SQw0DmYOUFXzfSysseqY/+HrDVCPSOQ== X-Google-Smtp-Source: ABdhPJwXAMr2Y2+5yKjdTOPVXlzJ9gj9LQJzK1QB2TAuG9xwZbeHCLdllH/f5aGBPbrsAhsrNGfC8g== X-Received: by 2002:a7b:c19a:: with SMTP id y26mr14529901wmi.132.1623324893501; Thu, 10 Jun 2021 04:34:53 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:53 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 27/40] trace-cmd: Read compressed latency trace data Date: Thu, 10 Jun 2021 14:34:13 +0300 Message-Id: <20210610113426.257931-28-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading trace file version 7 with latency trace data, uncompress the data. A new trace-cmd API is introduced to handle that case: tracecmd_latency_data_read() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 42 ++++++++++++++++++- tracecmd/trace-read.c | 6 +-- 3 files changed, 46 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 067ba34d..4a3eb1bc 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -208,6 +208,8 @@ tracecmd_peek_data_ref(struct tracecmd_input *handle, int cpu) return rec; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, void *buf, size_t size); + struct tep_record * tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 327082a2..12876838 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -137,6 +137,9 @@ struct tracecmd_input { bool read_page; bool use_pipe; int file_version; + /* temporary file for uncompressed latency data */ + int lat_cfd; + char lat_cfile[26]; /* strlen(COMPR_TEMP_FILE) */ struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -2995,6 +2998,35 @@ static int read_options_type(struct tracecmd_input *handle) return 0; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, void *buf, size_t size) +{ + int fd, r; + + if (handle->file_state != TRACECMD_FILE_CPU_LATENCY) + return -1; + if (handle->lat_cfd >= 0) + fd = handle->lat_cfd; + else + fd = handle->fd; + r = read(fd, buf, size); + + return r; +} + +static int latency_data_uncompress(struct tracecmd_input *handle) +{ + int ret; + + strcpy(handle->lat_cfile, COMPR_TEMP_FILE); + handle->lat_cfd = mkstemp(handle->lat_cfile); + if (handle->lat_cfd < 0) + return -1; + ret = tracecmd_uncompress_copy_to(handle->compress, handle->lat_cfd, NULL, NULL); + if (!ret) + lseek64(handle->lat_cfd, 0, SEEK_SET); + return ret; +} + static int read_cpu_data(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -3009,8 +3041,11 @@ static int read_cpu_data(struct tracecmd_input *handle) /* * Check if this is a latency report or not. */ - if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) + if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) { + if (handle->file_version >= 7 && latency_data_uncompress(handle)) + return -1; return 1; + } /* We expect this to be flyrecord */ if (handle->file_state != TRACECMD_FILE_CPU_FLYRECORD) @@ -3448,6 +3483,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->fd = fd; handle->ref = 1; + handle->lat_cfd = -1; if (do_read_check(handle, buf, 3)) goto failed_read; @@ -3698,6 +3734,10 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->uname); free(handle->trace_clock); close(handle->fd); + if (handle->lat_cfd >= 0) { + close(handle->lat_cfd); + unlink(handle->lat_cfile); + } tracecmd_free_hooks(handle->hooks); handle->hooks = NULL; diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index d605d05a..dfb814b7 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -948,13 +948,13 @@ void trace_show_data(struct tracecmd_input *handle, struct tep_record *record) printf("\n"); } -static void read_rest(void) +static void read_latency(struct tracecmd_input *handle) { char buf[BUFSIZ + 1]; int r; do { - r = read(input_fd, buf, BUFSIZ); + r = tracecmd_latency_data_read(handle, buf, BUFSIZ); if (r > 0) { buf[r] = 0; printf("%s", buf); @@ -1241,7 +1241,7 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype if (ret > 0) { if (multi_inputs) die("latency traces do not work with multiple inputs"); - read_rest(); + read_latency(handles->handle); return; } From patchwork Thu Jun 10 11:34:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08648C47094 for ; Thu, 10 Jun 2021 11:34:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E4B4C613F1 for ; Thu, 10 Jun 2021 11:34:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230265AbhFJLgw (ORCPT ); Thu, 10 Jun 2021 07:36:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbhFJLgw (ORCPT ); Thu, 10 Jun 2021 07:36:52 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09BE3C0617AD for ; Thu, 10 Jun 2021 04:34:56 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id b205so1270082wmb.3 for ; Thu, 10 Jun 2021 04:34:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=66FlxMvR+6uLn0JyQQaiP+J5SvIQRJEbuwhocU3t7Is=; b=mq9Y+wBs6MO39kZrTSs5K59GpS06XUD3lBAmYf/mQ1P7tkW1g4/08dbr0hx1Q1gd9f fP6bbA35ByguhrT8Cru0MVf4ZN9Ra+RNb/jfDCiaGhTTeUjEZohQ8V3hxET5KnYV8ypV A86zs7HYI40cDyGA1EONzIDWVVZ8tIle7ItO79bAXmTUxbznecExwChl+pgyb3fKeh6k H/K7aTIdOPAJzOr+RN4XQEzOr+uFDusn0UBNmybPIumA1W3EvFbfbetv/3cDehcuWutG kuF7q+neG8KDUFvTFNilXVP1vjVT5JIrEjXiUMJA2Fhx4G2wcOqUFhBLkMp/VklkvdhT VOqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=66FlxMvR+6uLn0JyQQaiP+J5SvIQRJEbuwhocU3t7Is=; b=dYqFZB47w03UeQU2JexOVgR8wJt78ylt7hobeU9QK9FBdodQCSQjOF7mVSXEE/LZG1 5kMH2grZI1hi4fUk2WzXxIvEDPkswLScqdI4GVnl/VpDvGSatXvQlnFjP3P9E0z4hfJz mtQQIYSM6TMkUJHGKuy+o9ejFV4SYozGJNd42wqfh38wG3t3I/YwUMct3aZWy9XQemoA jB8dppk8EP1L9uwKkVvIw0LTlepiXmnIArzGq7vt6p3u9H0scIOjQNtJ2BbbkJaCmHb2 DmgsJN5veR1fp/6TS2l6d5yA4OchKpOjP1vIhfLGNcwVTyQiJKh2VZ1Com9+XH4IbXa1 hKrg== X-Gm-Message-State: AOAM530NDAqjp0sVBaRVnnUO9IgOu31WHkRBfskICFpsTx0MwgHtnavJ fYv6yDuUA98+NOI5rVAShnTl6PFCu9mBPA== X-Google-Smtp-Source: ABdhPJwn3jiAA90JDy0mdG+StiaWdXpexFdftNZ/xcP/IiDlFGoiXRQiF5gLqZwxGHv3bzAu5xvrUg== X-Received: by 2002:a7b:c099:: with SMTP id r25mr6110054wmh.144.1623324894577; Thu, 10 Jun 2021 04:34:54 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:54 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 28/40] trace-cmd library: Reuse within the library the function that checks file state. Date: Thu, 10 Jun 2021 14:34:14 +0300 Message-Id: <20210610113426.257931-29-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Make the function, that checks if the next file state is valid, global for the tracmd-library, so it can be reused. It is important the same check logic to be used in the whole library. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 ++ lib/trace-cmd/trace-input.c | 5 ++ lib/trace-cmd/trace-output.c | 83 ++++++------------------- lib/trace-cmd/trace-util.c | 41 ++++++++++++ 4 files changed, 70 insertions(+), 63 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index b77c49ff..fa1b46b0 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,10 @@ void tracecmd_zlib_free(void); int tracecmd_compress_init(void); void tracecmd_compress_free(void); +bool check_file_state(unsigned long file_version, int current_state, int new_state); +bool check_in_state(struct tracecmd_input *handle, int new_state); +bool check_out_state(struct tracecmd_output *handle, int new_state); + int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 12876838..d3c5ad18 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4436,3 +4436,8 @@ int tracecmd_enable_tsync(struct tracecmd_input *handle, bool enable) return 0; } + +__hidden bool check_in_state(struct tracecmd_input *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 6a272503..039a49cd 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -419,50 +419,6 @@ int tracecmd_ftrace_enable(int set) return ret; } -static int check_out_state(struct tracecmd_output *handle, int new_state) -{ - if (!handle) - return -1; - - if (handle->file_version < 7) { - switch (new_state) { - case TRACECMD_FILE_BUFERS: - tracecmd_warning("State %d is not supported in trace file version %d", - new_state, handle->file_version); - return -1; - default: - break; - } - } - - switch (new_state) { - case TRACECMD_FILE_HEADERS: - case TRACECMD_FILE_FTRACE_EVENTS: - case TRACECMD_FILE_ALL_EVENTS: - case TRACECMD_FILE_KALLSYMS: - case TRACECMD_FILE_PRINTK: - case TRACECMD_FILE_CMD_LINES: - case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_BUFERS: - if (handle->file_state == (new_state - 1)) - return 0; - break; - case TRACECMD_FILE_OPTIONS: - if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) - return 0; - if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) - return 0; - break; - case TRACECMD_FILE_CPU_LATENCY: - case TRACECMD_FILE_CPU_FLYRECORD: - if (handle->file_state == TRACECMD_FILE_OPTIONS) - return 0; - break; - } - - return -1; -} - static int read_header_files(struct tracecmd_output *handle) { tsize_t size, check_size, endian8; @@ -471,7 +427,7 @@ static int read_header_files(struct tracecmd_output *handle) int fd; int ret; - if (check_out_state(handle, TRACECMD_FILE_HEADERS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_HEADERS)) { tracecmd_warning("Cannot read header files, unexpected state 0x%X", handle->file_state); return -1; @@ -783,7 +739,7 @@ static int read_ftrace_files(struct tracecmd_output *handle) struct tracecmd_event_list list = { .glob = "ftrace/*" }; int ret; - if (check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS)) { tracecmd_warning("Cannot read ftrace files, unexpected state 0x%X", handle->file_state); return -1; @@ -828,7 +784,7 @@ static int read_event_files(struct tracecmd_output *handle, int endian4; int ret; - if (check_out_state(handle, TRACECMD_FILE_ALL_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_ALL_EVENTS)) { tracecmd_warning("Cannot read event files, unexpected state 0x%X", handle->file_state); return -1; @@ -927,7 +883,7 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, struct stat st; int ret; - if (check_out_state(handle, TRACECMD_FILE_KALLSYMS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_KALLSYMS)) { tracecmd_warning("Cannot read kallsyms, unexpected state 0x%X", handle->file_state); return -1; @@ -977,7 +933,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) char *path; int ret; - if (check_out_state(handle, TRACECMD_FILE_PRINTK) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_PRINTK)) { tracecmd_warning("Cannot read printk, unexpected state 0x%X", handle->file_state); return -1; @@ -1339,11 +1295,10 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CPU_COUNT); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_COUNT)) { tracecmd_warning("Cannot write CPU count into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } cpus = convert_endian_4(handle, cpus); ret = do_write_check(handle, &cpus, 4); @@ -1359,16 +1314,14 @@ int tracecmd_write_options(struct tracecmd_output *handle) unsigned short option; unsigned short endian2; unsigned int endian4; - int ret; /* If already written, ignore */ if (handle->file_state == TRACECMD_FILE_OPTIONS) return 0; - ret = check_out_state(handle, TRACECMD_FILE_OPTIONS); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_OPTIONS)) { tracecmd_warning("Cannot write options into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } if (do_write_check(handle, "options ", 10)) @@ -1604,11 +1557,10 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CMD_LINES); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CMD_LINES)) { tracecmd_warning("Cannot write command lines into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } out_compression_start(handle); @@ -1628,7 +1580,6 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in { struct tracecmd_output *handle; char *path; - int ret; handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) @@ -1647,8 +1598,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_options(handle) < 0) goto out_free; - ret = check_out_state(handle, TRACECMD_FILE_CPU_LATENCY); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_LATENCY)) { tracecmd_warning("Cannot write latency data into the file, unexpected state 0x%X", handle->file_state); goto out_free; @@ -1737,7 +1687,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, /* This can be called multiple times (when recording instances) */ ret = handle->file_state == TRACECMD_FILE_CPU_FLYRECORD ? 0 : - check_out_state(handle, TRACECMD_FILE_CPU_FLYRECORD); + check_file_state(handle->file_version, + handle->file_state, + TRACECMD_FILE_CPU_FLYRECORD); if (ret < 0) { tracecmd_warning("Cannot write trace data into the file, unexpected state 0x%X", handle->file_state); @@ -2006,3 +1958,8 @@ out_free: tracecmd_output_close(handle); return NULL; } + +__hidden bool check_out_state(struct tracecmd_output *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 136260c6..087d37ab 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -636,3 +636,44 @@ void tracecmd_lib_free(void) { tracecmd_compress_free(); } + +__hidden bool check_file_state(unsigned long file_version, int current_state, int new_state) +{ + if (file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, file_version); + return false; + default: + break; + } + } + + switch (new_state) { + case TRACECMD_FILE_HEADERS: + case TRACECMD_FILE_FTRACE_EVENTS: + case TRACECMD_FILE_ALL_EVENTS: + case TRACECMD_FILE_KALLSYMS: + case TRACECMD_FILE_PRINTK: + case TRACECMD_FILE_CMD_LINES: + case TRACECMD_FILE_CPU_COUNT: + case TRACECMD_FILE_BUFERS: + if (current_state == (new_state - 1)) + return true; + break; + case TRACECMD_FILE_OPTIONS: + if (file_version < 7 && current_state == TRACECMD_FILE_CPU_COUNT) + return true; + if (file_version >= 7 && current_state == TRACECMD_FILE_BUFERS) + return true; + break; + case TRACECMD_FILE_CPU_LATENCY: + case TRACECMD_FILE_CPU_FLYRECORD: + if (current_state == TRACECMD_FILE_OPTIONS) + return true; + break; + } + + return false; +} From patchwork Thu Jun 10 11:34:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33410C47094 for ; Thu, 10 Jun 2021 11:35:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A095613E9 for ; Thu, 10 Jun 2021 11:35:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230267AbhFJLhx (ORCPT ); Thu, 10 Jun 2021 07:37:53 -0400 Received: from mail-wr1-f48.google.com ([209.85.221.48]:40957 "EHLO mail-wr1-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230396AbhFJLhw (ORCPT ); Thu, 10 Jun 2021 07:37:52 -0400 Received: by mail-wr1-f48.google.com with SMTP id y7so1901048wrh.7 for ; Thu, 10 Jun 2021 04:35:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=86tIhNKoxf3eeyVQNQb2etWHDkZYbLQAO7WhqeGm2H0=; b=qu2hLvsb+1IziOUVlqqEm5ceeyTZrNUaisBRbC/Jv9pmdHbsSylXrOvUvepU/Se24m msqdXljuE5upF4xuYV7/O68amanTIs5OZMjD7KnhlrMWqSTkVhZEmtjZIc0St1DjRNMh RWupPWyIOl4B4l7dcmrlzBnIh3ZQGnZhzohD3iD5WCYslnFpXbpiuqkfRv7Xc6fEhqG6 ZCdPaF1uhsl/aAs/irZAy0VkfSgipunDtaVPcfHJKxmIjT4jAp1KwMJttDOS3mEX1JnC 9En2YFYUDjqXlancI2hDTSVadrPKNHFx6JYRtdYTj+jvKUYVncrWowJHNyAsXExlbOZI HTnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=86tIhNKoxf3eeyVQNQb2etWHDkZYbLQAO7WhqeGm2H0=; b=aZIo3CGKIbSbF3i7DeIBeEubpZtFy00IjoBxADVjkus0Kp4StuHSNunoAQRsgS3vWh zp77cqz7B26BreyEkkJHJ3lv5yFwK9PZ3GGT1YvxaWGcJQ+FpmC8EB1yQnJWmPDaNtr8 7SZWknnRJS6f4GIsIjnWt9KtSo3N5D0iCLSSWAA8On8h4a+uUyrnzsQR2yl5aK7EtOXQ vyzsy+CYH0WqR7oPJH2ZKelPRZdwfxLO1ypau9hcbxQpJC2Ef9GynXNdOwQEFQ+8DvMD /yKakrg86VoWmkCAGkYnZ+zkwLzGJ1V1tHYrLo6JTYGcZ8CzL1aPA6zuEkPcpWYfMea1 iR2w== X-Gm-Message-State: AOAM532W3gLfLs+jrML1BSRGRWYtA7Ph+I5AFwEmW0sJwECN8Mf6YxBW EbjZfvFXAfIn7essfZHpibCArzjdOAUxYA== X-Google-Smtp-Source: ABdhPJxPA9MqWP6WygutzmTXY+fc6OGNnBE7SxFkpp7lsYbstiPe34t4Wg1QuRLgoBjOIVJnM17DHw== X-Received: by 2002:a5d:4692:: with SMTP id u18mr4786474wrq.318.1623324895517; Thu, 10 Jun 2021 04:34:55 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:55 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 29/40] trace-cmd library: Make tracecmd_copy_headers() to work with output handler Date: Thu, 10 Jun 2021 14:34:15 +0300 Message-Id: <20210610113426.257931-30-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When copying headers between two trace files, use output handler context instead of just a fd. Using output handler will allow to use file compression, if needed. This change fixes "trace-cmd split" command to work with trace files v7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 +- lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-input.c | 244 ++++++++++++------ lib/trace-cmd/trace-output.c | 13 +- 4 files changed, 174 insertions(+), 90 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4a3eb1bc..65e80cec 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -170,7 +170,8 @@ int tracecmd_get_parsing_failures(struct tracecmd_input *handle); int tracecmd_long_size(struct tracecmd_input *handle); int tracecmd_page_size(struct tracecmd_input *handle); int tracecmd_cpus(struct tracecmd_input *handle); -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index fa1b46b0..135f52f3 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -51,7 +51,11 @@ void out_compression_reset(struct tracecmd_output *handle); void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); +void out_set_file_state(struct tracecmd_output *handle, int new_state); + int write_buffers_description_v7(struct tracecmd_output *handle); int write_buffers_description_v6(struct tracecmd_output *handle); +long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index d3c5ad18..74d1ebf7 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3759,44 +3759,47 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle); } -static int read_copy_size8(struct tracecmd_input *handle, int fd, unsigned long long *size) +static int read_copy_size8(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, unsigned long long *size) { /* read size */ - if (do_read_check(handle, size, 8)) + if (do_read_check(in_handle, size, 8)) return -1; - if (__do_write_check(fd, size, 8)) + if (do_write_check(out_handle, size, 8)) return -1; - *size = tep_read_number(handle->pevent, size, 8); + *size = tep_read_number(in_handle->pevent, size, 8); return 0; } -static int read_copy_size4(struct tracecmd_input *handle, int fd, unsigned int *size) +static int read_copy_size4(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, + unsigned int *size) { /* read size */ - if (do_read_check(handle, size, 4)) + if (do_read_check(in_handle, size, 4)) return -1; - if (__do_write_check(fd, size, 4)) + if (do_write_check(out_handle, size, 4)) return -1; - *size = tep_read_number(handle->pevent, size, 4); + *size = tep_read_number(in_handle->pevent, size, 4); return 0; } -static int read_copy_data(struct tracecmd_input *handle, - unsigned long long size, int fd) +static int read_copy_data(struct tracecmd_input *in_handle, + unsigned long long size, + struct tracecmd_output *out_handle) { char *buf; buf = malloc(size); if (!buf) return -1; - if (do_read_check(handle, buf, size)) + if (do_read_check(in_handle, buf, size)) goto failed_read; - if (__do_write_check(fd, buf, size)) + if (do_write_check(out_handle, buf, size)) goto failed_read; free(buf); @@ -3808,65 +3811,82 @@ static int read_copy_data(struct tracecmd_input *handle, return -1; } -static int copy_header_files(struct tracecmd_input *handle, int fd) +static int copy_header_files(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_HEADERS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_HEADERS) || + !check_out_state(out_handle, TRACECMD_FILE_HEADERS)) return -1; /* "header_page" */ - if (read_copy_data(handle, 12, fd) < 0) + if (read_copy_data(in_handle, 12, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; /* "header_event" */ - if (read_copy_data(handle, 13, fd) < 0) + if (read_copy_data(in_handle, 13, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_HEADERS; + in_handle->file_state = TRACECMD_FILE_HEADERS; + out_set_file_state(out_handle, in_handle->file_state); return 0; } -static int copy_ftrace_files(struct tracecmd_input *handle, int fd) +static int copy_ftrace_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; unsigned int count; unsigned int i; - if (handle->file_state != TRACECMD_FILE_FTRACE_EVENTS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_FTRACE_EVENTS) || + !check_out_state(out_handle, TRACECMD_FILE_FTRACE_EVENTS)) return -1; - if (read_copy_size4(handle, fd, &count) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (i = 0; i < count; i++) { - if (read_copy_size8(handle, fd, &size) < 0) - return -1; + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; } - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_event_files(struct tracecmd_input *handle, int fd) +static int copy_event_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; char *system; @@ -3874,103 +3894,159 @@ static int copy_event_files(struct tracecmd_input *handle, int fd) unsigned int count; unsigned int i,x; - if (handle->file_state != TRACECMD_FILE_ALL_EVENTS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_ALL_EVENTS) || + !check_out_state(out_handle, TRACECMD_FILE_ALL_EVENTS)) return -1; - if (read_copy_size4(handle, fd, &systems) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &systems) < 0) + goto error; for (i = 0; i < systems; i++) { - system = read_string(handle); + system = read_string(in_handle); if (!system) - return -1; - if (__do_write_check(fd, system, strlen(system) + 1)) { + goto error; + if (do_write_check(out_handle, system, strlen(system) + 1)) { free(system); - return -1; + goto error; } free(system); - if (read_copy_size4(handle, fd, &count) < 0) - return -1; + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (x=0; x < count; x++) { - if (read_copy_size8(handle, fd, &size) < 0) - return -1; + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; } } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_ALL_EVENTS; + + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_proc_kallsyms(struct tracecmd_input *handle, int fd) +static int copy_proc_kallsyms(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_KALLSYMS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_KALLSYMS) || + !check_out_state(out_handle, TRACECMD_FILE_KALLSYMS)) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; - handle->file_state = TRACECMD_FILE_KALLSYMS; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_KALLSYMS; + + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_ftrace_printk(struct tracecmd_input *handle, int fd) +static int copy_ftrace_printk(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_PRINTK - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_PRINTK) || + !check_out_state(out_handle, TRACECMD_FILE_PRINTK)) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; - if (!size) - return 0; /* OK? */ + out_compression_start(out_handle); - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; + if (!size) + goto out; /* OK? */ - handle->file_state = TRACECMD_FILE_PRINTK; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_PRINTK; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_command_lines(struct tracecmd_input *handle, int fd) +static int copy_command_lines(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_CMD_LINES - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_CMD_LINES) || + !check_out_state(out_handle, TRACECMD_FILE_CMD_LINES)) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; - handle->file_state = TRACECMD_FILE_CMD_LINES; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_CMD_LINES; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor - * @handle: input handle for the trace.dat file to copy from. - * @fd: The file descriptor to copy to. + * @in_handle: input handle for the trace.dat file to copy from. + * @out_handle: output handle to the trace.dat file to copy to. * @start_state: The file state to start copying from (zero for the beginnig) * @end_state: The file state to stop at (zero for up to cmdlines) * @@ -3981,7 +4057,8 @@ static int copy_command_lines(struct tracecmd_input *handle, int fd) * NOTE: The input handle is also modified, and ends at the end * state as well. */ -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state) { @@ -3997,68 +4074,67 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, if (end_state < TRACECMD_FILE_HEADERS) return 0; - - if (handle->file_state >= start_state) { + if (in_handle->file_state >= start_state) { /* Set the handle to just before the start state */ - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + lseek64(in_handle->fd, in_handle->header_files_start, SEEK_SET); /* Now that the file handle has moved, change its state */ - handle->file_state = TRACECMD_FILE_INIT; + in_handle->file_state = TRACECMD_FILE_INIT; } /* Try to bring the input up to the start state - 1 */ - ret = tracecmd_read_headers(handle, start_state - 1); + ret = tracecmd_read_headers(in_handle, start_state - 1); if (ret < 0) goto out; switch (start_state) { case TRACECMD_FILE_HEADERS: - ret = copy_header_files(handle, fd); + ret = copy_header_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_FTRACE_EVENTS: /* handle's state is now updating with the copies */ - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_files(handle, fd); + ret = copy_ftrace_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_ALL_EVENTS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_event_files(handle, fd); + ret = copy_event_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_KALLSYMS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_proc_kallsyms(handle, fd); + ret = copy_proc_kallsyms(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_PRINTK: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_printk(handle, fd); + ret = copy_ftrace_printk(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_CMD_LINES: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_command_lines(handle, fd); + ret = copy_command_lines(in_handle, out_handle); default: break; } diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 039a49cd..1b4210b5 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -89,8 +89,8 @@ struct list_event_system { char *name; }; -static stsize_t -do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) +__hidden long long +do_write_check(struct tracecmd_output *handle, const void *data, long long size) { if (handle->do_compress && handle->compress) return tracecmd_compress_write(handle->compress, data, size); @@ -1946,11 +1946,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - if (tracecmd_copy_headers(ihandle, handle->fd, 0, 0) < 0) + if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) goto out_free; - handle->file_state = tracecmd_get_file_state(ihandle); - /* The file is all ready to have cpu data attached */ return handle; @@ -1959,6 +1957,11 @@ out_free: return NULL; } +__hidden void out_set_file_state(struct tracecmd_output *handle, int new_state) +{ + handle->file_state = new_state; +} + __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); From patchwork Thu Jun 10 11:34:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCF8BC48BE0 for ; Thu, 10 Jun 2021 11:34:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6760613F1 for ; Thu, 10 Jun 2021 11:34:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230267AbhFJLgy (ORCPT ); Thu, 10 Jun 2021 07:36:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230264AbhFJLgx (ORCPT ); Thu, 10 Jun 2021 07:36:53 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 987A2C061760 for ; Thu, 10 Jun 2021 04:34:57 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso4597357wmc.1 for ; Thu, 10 Jun 2021 04:34:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BTU1iFO2V4ZzMjExOiVLwCGbuFnGNHp5D2Mj4ZKntEk=; b=OhJtmfhJMnsEkpERKdKNkvWJ313vInsTtwALigUcxpZ0Q0YIdM0MmB1xeNTgX/Gyl8 NCmMViRuh8vKd2Y6pa5VHNlFNqJ8roFNxmcL9I/prBKgvWi10UPRbPT8zj6Mb591SOTD lC93iwEG1ChZySnUxzYwnvv7LKeQyGdJGmJHzBGmKFZAMiYBdzqyrpoCVXbhwyY29V3G M2t7dc0es4iyEAMroSvjfi2xlMtesa/mHYQ86qCGhzeU0fc4GH19kmpu/e4t5fhEQEH2 smy5i2TAglv9uLSZ0yKA39KNaGeMj6Ibbl0F16RJPkTMeRMZLAC2Y68K0vWwseNJAJ10 2CNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BTU1iFO2V4ZzMjExOiVLwCGbuFnGNHp5D2Mj4ZKntEk=; b=OPb3/a1G1EgO3fsLSNgexxkRBUOEeE+KymGqHvuc15HJZKKvzk+7Wxd3MsyfW0nGbH yT8bpHKE+rBQ6rWnTF1teoZn/Fx+z05894a4UtlJ9rkC4zaR3izg9ZpEKHDaSRsgVwHC JLTk1uA00fWmktuRaBZjCwBlyd8fFDj91OSskH2b5CNkp6nA3DDq1YdVXqm/Sv/xmmkb FUTccTwfPN2K76PywCF+88s3T+UGJfBGhsow/gQpfy4v7ONwCorqaU/okiXxNMoniBtp MQeTIOO8JV9YWCW4TBZSif4NuQkf7eurj0f2dgtXrMdriyeOndU3/tce17uocBd1N+Tn WAsw== X-Gm-Message-State: AOAM531pAB2+OS0EkiQPHBPH8rELmDYZ0o6WQzdoboDMrRNNxAcSWrOK 58szybnDakZHaJmS9oJzk9iHmBwX6IIC3Q== X-Google-Smtp-Source: ABdhPJwC+jZhf1moQg5TejxFqJUcstK8Dmy9Q8OdPtz/eMMewxOUxNEkGiHNTCUysE3FMu2fsHfwMg== X-Received: by 2002:a05:600c:3789:: with SMTP id o9mr14869763wmr.78.1623324896244; Thu, 10 Jun 2021 04:34:56 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:55 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 30/40] trace-cmd: Do not use trace file compression with streams Date: Thu, 10 Jun 2021 14:34:16 +0300 Message-Id: <20210610113426.257931-31-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a temporary output handler for use with a stream, force it not not use trace file compression. As no data are saved in this file, there is no need to have a compression overhead. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-stream.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tracecmd/trace-stream.c b/tracecmd/trace-stream.c index 317de1b2..1ca3a79d 100644 --- a/tracecmd/trace-stream.c +++ b/tracecmd/trace-stream.c @@ -43,7 +43,7 @@ trace_stream_init(struct buffer_instance *instance, int cpu, int fd, int cpus, tfd = fileno(fp); ofd = dup(tfd); - trace_output = tracecmd_create_init_fd(ofd, NULL); + trace_output = tracecmd_create_init_fd(ofd, "none"); if (!trace_output) { fclose(fp); return NULL; From patchwork Thu Jun 10 11:34:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16CA8C48BE0 for ; Thu, 10 Jun 2021 11:35:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED021613F1 for ; Thu, 10 Jun 2021 11:35:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230317AbhFJLhG (ORCPT ); Thu, 10 Jun 2021 07:37:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230322AbhFJLhD (ORCPT ); Thu, 10 Jun 2021 07:37:03 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8345EC061760 for ; Thu, 10 Jun 2021 04:34:58 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id b145-20020a1c80970000b029019c8c824054so6320709wmd.5 for ; Thu, 10 Jun 2021 04:34:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tY0jSkhEty4HDlJj4DNPTV9zO0ERvs1vyxa1BBJLEgQ=; b=CLBNh51paME0x98+P2jztuMnsIHs0pwVbEYswRxI+LVYMa76EmWLC+HHjhJxLtr59+ IgS7GHZlqRupT7nuRUyjOuCH9Axs5G0M8k/KVAnHLSOEUNDtlTnWzWDk3si1cE9Po6F0 U6W52IEwQcB32z3p3RkBD8UCeL1GUhYCM1cwj2U4RRkYNbR2xARulfc96h0yYFcYeV6v uccWYWOATtsy6CQYhKrREb0RjPP9kHbMO7p4mEG3SPjLi2XwjZ7ISm6teXY/PkAfdOVb w6oTmeunPazjUw889S/+jT/bY/3naxmUz7r2HnWdaP+Bib/vmpCa1LEg6pB4wlGokHcW QkWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tY0jSkhEty4HDlJj4DNPTV9zO0ERvs1vyxa1BBJLEgQ=; b=cQu0dGCkwt3vb5tBMfQgkqW0+J5xPDMsIl1IIL+nn/sbCLA1GBHW5hCeQeIz4FQplm uizX4TzzdTFnbfVvQn4pTMrW/Z4YZdt6mkW0sp5gQVj/pQ4cifc9NfsIRUVfbQkqGEM+ vM0cw42mINOPAb8MwTgAhTekRubGBw/R8nGiCZ6xa7XKOpmpCxY7+2I6sAGrn83nli1Q 47IU3kILBqs5IIhuuk+kFGSDiqFIoKopwFTT7bn9qYP+R1XaMAzhWPmc/KwKdVqZZDK2 Dwmj5KFilkibspa0cReNd7NWJoNeilf1qmEEiV/yWni6yDjK+qTohzclnmOELTpV9sHz hZ6g== X-Gm-Message-State: AOAM530dF00QmV4gd8o23MwMko7rlbbV0ARud9KF3i8+jtPgy3zt6LFF cM0p/5RAMpnxq+8CJ+vxovz7qaxkYaMK1Q== X-Google-Smtp-Source: ABdhPJxiTXt0UuYh/Wq+9FqXGJcajjDkLFdoeufiVaNWshQUBq2btMkH6KiVFun3XASIxTwZ7kA/ww== X-Received: by 2002:a05:600c:354f:: with SMTP id i15mr4534298wmq.38.1623324897174; Thu, 10 Jun 2021 04:34:57 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:56 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 31/40] trace-cmd library: Add new API to get file version of output handler Date: Thu, 10 Jun 2021 14:34:17 +0300 Message-Id: <20210610113426.257931-32-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added an API to get the version of the trace file, associated with given output file handler. tracecmd_get_out_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-output.c | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 65e80cec..9a08a960 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -319,6 +319,7 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 1b4210b5..ff9c9a34 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1966,3 +1966,12 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); } + +/** + * tracecmd_get_out_file_version - return the trace.dat file version + * @handle: output handle for the trace.dat file + */ +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle) +{ + return handle->file_version; +} From patchwork Thu Jun 10 11:34:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE12AC47094 for ; Thu, 10 Jun 2021 11:35:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DAA2961410 for ; Thu, 10 Jun 2021 11:35:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230356AbhFJLhK (ORCPT ); Thu, 10 Jun 2021 07:37:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230329AbhFJLhD (ORCPT ); Thu, 10 Jun 2021 07:37:03 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BA19C0617A6 for ; Thu, 10 Jun 2021 04:34:59 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id 3-20020a05600c0243b029019f2f9b2b8aso6214462wmj.2 for ; Thu, 10 Jun 2021 04:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6d+ps2XI6q8t8p5XkZiTzQSOgVeK23JVAY/zSugXJQ0=; b=FVdEySxM5hs5y8uTrZC3+t/+/XhbukG9noL33LKU5fsEE2qK4ihNd4TdcL80D8boPb l77mypErCW/Lh/BSOJccnwq0oJCEIWUz0DDp/rRQM8FGDoDH6g2bjgsWE+Z1u1sfqKTw zaKmbzsDGMJ6LNMjeeonfyvME3krKk4zaWWZBk+0Nd2rRxX5jseMp9zSdQP5M4YdqRDD NhqKJ5ZrreUYFC3rCWZLqCA26Gfl58ka6rYWv+zM0KokD8VtSjsJii90qgbp7DHWsKT1 6g3byp8hP787YWHay0rTmp+9ZA1tmVVvRNS46ITKDxhRlwUT+aMzI+oIZ+XwyBSBzgak egiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6d+ps2XI6q8t8p5XkZiTzQSOgVeK23JVAY/zSugXJQ0=; b=MRzmv2Y68oJogiG9ghBFIgftXNHJMgNjSIidUHRB253dOwkUbDgOnYj0h0Je2Hxdla 3BZevq0FETTvuVg7f/zs4x45se2IpyGSO65ShQv4JFtmSFFn1vdlZWGZhKHSxe2+JEzK dpp8cWbiYYJDnaUKtYUVkhnbtY2ALr15wPl+0E5KqvaIva69ErKlyWKPui3icoSkhl8/ aWv5c7Qig0AbW5mIAg2+CN6QVS0AfRAtG+AJ6mfLJXqKpphaVeVMOZfmTQlHbobL/Gml jejlhepTT+AxWbU7jqff4ADn+a4Vo3b9UQC84CbYrR8oMUqteDMDvEeHy4et1eNQULyI v+Xw== X-Gm-Message-State: AOAM531mmFMAEwNrb8xX1akqvMC3dQ2/swLJIWVo5tTpgOD0pmb/Mw/+ q+GuyhvvN1plkW8lPIKQGB9Kp8CHX1uG9g== X-Google-Smtp-Source: ABdhPJyEEsZ7G3RHFSF6gzDoS4WLW0jESO7+R0yZ4ERk+hzMh7HOnAaLqbRNJktrG3O6n7oa3NFxHw== X-Received: by 2002:a1c:4b0d:: with SMTP id y13mr4530876wma.179.1623324898041; Thu, 10 Jun 2021 04:34:58 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:57 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 32/40] trace-cmd: Add file state parameter to tracecmd_copy Date: Thu, 10 Jun 2021 14:34:18 +0300 Message-Id: <20210610113426.257931-33-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The tracecmd_copy() API is used to create output trace handler from given input trace handler and to copy data from it. Currently it is used by trace-cmd split and restore commands, that's why it is hardcoded what data to be copied from input to output handler. Addig desired output handler file state makes the API more generic and allows it to be used in more use cases. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 4 ++-- lib/trace-cmd/trace-output.c | 8 ++++++-- tracecmd/trace-restore.c | 2 +- tracecmd/trace-split.c | 2 +- 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 9a08a960..2c31be5f 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -309,8 +309,8 @@ int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file); +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, + enum tracecmd_file_states state); int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index ff9c9a34..e3b018aa 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1931,13 +1931,16 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * tracecmd_copy - copy the headers of one trace.dat file for another * @ihandle: input handle of the trace.dat file to copy * @file: the trace.dat file to create + * @state: what data will be copied from the source handle * * Reads the header information and creates a new trace data file * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, + enum tracecmd_file_states state) { + enum tracecmd_file_states fstate; struct tracecmd_output *handle; const char *compr_name = NULL; @@ -1946,7 +1949,8 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) + fstate = state > TRACECMD_FILE_CMD_LINES ? TRACECMD_FILE_CMD_LINES : state; + if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; /* The file is all ready to have cpu data attached */ diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 96b4fb5d..7d439fd2 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -125,7 +125,7 @@ void trace_restore (int argc, char **argv) if (tracecmd_read_headers(ihandle, TRACECMD_FILE_CMD_LINES) < 0) die("error reading file %s headers", input); - handle = tracecmd_copy(ihandle, output); + handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES); tracecmd_close(ihandle); } else handle = tracecmd_create_init_file(output, NULL); diff --git a/tracecmd/trace-split.c b/tracecmd/trace-split.c index 8366d128..401e1d29 100644 --- a/tracecmd/trace-split.c +++ b/tracecmd/trace-split.c @@ -345,7 +345,7 @@ static double parse_file(struct tracecmd_input *handle, dir = dirname(output); base = basename(output); - ohandle = tracecmd_copy(handle, output_file); + ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES); cpus = tracecmd_cpus(handle); cpu_data = malloc(sizeof(*cpu_data) * cpus); From patchwork Thu Jun 10 11:34:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46E66C47094 for ; Thu, 10 Jun 2021 11:36:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 27BC3613FE for ; Thu, 10 Jun 2021 11:36:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230272AbhFJLiI (ORCPT ); Thu, 10 Jun 2021 07:38:08 -0400 Received: from mail-wm1-f53.google.com ([209.85.128.53]:42934 "EHLO mail-wm1-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230438AbhFJLiF (ORCPT ); Thu, 10 Jun 2021 07:38:05 -0400 Received: by mail-wm1-f53.google.com with SMTP id l7-20020a05600c1d07b02901b0e2ebd6deso6209926wms.1 for ; Thu, 10 Jun 2021 04:35:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ovTO2+bhZPsYG830hwQ0yxKfzcFEyCIHS3+j+KQAJbM=; b=NCI3y1AZzKbGTHm2DfAbluTuuvArMIhuJa+mP4g57FKkfy9BZap+p947ByhvYI4jEF Isu4c27QmaA+MbngUVqq0Pk3spy5ctsvfqK8dE1XVkWy/XhxJY+0+Kr8Qd180xF48t7y U0LOjvrwQ3oUlxwY9NR0kGbOTcimjjawj4xi/GUlSCZMV/4kl6wkC1159R+KoVMaRvaE jSzWKonBoV2qAyLgI9mvNGt6Uvj/BaSsH2eno4IiY+mf4GFLtgqxc1FzyllLd0LhnyhJ 0zkzPVIjecEZbG2Ye3sS6thzunHgJS6ln6b2wBF+F0LiloHJzLj8xjYFy1J85s33oFEL sumA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ovTO2+bhZPsYG830hwQ0yxKfzcFEyCIHS3+j+KQAJbM=; b=ejQ8d+9rek6wkAzVSRJPtKvDvYHzUp539FS/ADEtpOObz0iI/QA+evi9ex+NdsC8IQ N4r+Ov+L7HMraH2UmvGKVG/P6vMoUhBxY4dmZto04OuJaPG0cYsgZRsb4Z3CVUP/a9QZ mEx0+4fVq/aIcGhVmZj+Y6ijRMFXCtQWRI3q1CQiWKoQniqyrm2KLdqJ6O8YwKtxa4qg VvHwmV0YCO3NaOxDIrJJD/AvdP9rlaJYE8rq1BB9SGrHwhVw1ORYNLml4GnG3+TDX/12 zKVCvJ3Av5snuH8kWLMF3B1dX3vvHmUIxunxrT5Mz+P1qkydymMPT+o04cS8qnrY57HZ ihRA== X-Gm-Message-State: AOAM531ZVfIN6htxOD3vJon69KywtLeKjGbkG1qwVe1ABd8BiKxBNNS2 fQG4VT1Y+0ehO0gVmerNwTpdZiHlwBXVAA== X-Google-Smtp-Source: ABdhPJzCu8nja4HolmIxUD3S4phalU0ZBI9F1TbqvJ8IV1xCbPg6NpE3/iFGSlFA1WXbnvESms+8Zg== X-Received: by 2002:a7b:cf18:: with SMTP id l24mr14183369wmg.160.1623324898929; Thu, 10 Jun 2021 04:34:58 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:58 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 33/40] trace-cmd: Copy CPU count in tracecmd_copy Date: Thu, 10 Jun 2021 14:34:19 +0300 Message-Id: <20210610113426.257931-34-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of CPU count from input to output trace hanlder. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 30 ++++++++++++++++++++++++++++++ lib/trace-cmd/trace-output.c | 2 +- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 74d1ebf7..289102bf 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4043,6 +4043,23 @@ error: return -1; } +static int copy_cpu_count(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) +{ + unsigned int size; + + if (!check_in_state(in_handle, TRACECMD_FILE_CPU_COUNT) || + !check_out_state(out_handle, TRACECMD_FILE_CPU_COUNT)) + return -1; + + if (read_copy_size4(in_handle, out_handle, &size) < 0) + return -1; + + in_handle->file_state = TRACECMD_FILE_CPU_COUNT; + out_set_file_state(out_handle, in_handle->file_state); + + return 0; +} + /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor * @in_handle: input handle for the trace.dat file to copy from. @@ -4135,6 +4152,19 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, return 0; ret = copy_command_lines(in_handle, out_handle); + if (ret < 0) + goto out; + + /* fallthrough */ + case TRACECMD_FILE_CPU_COUNT: + if (end_state <= in_handle->file_state) + return 0; + + ret = copy_cpu_count(in_handle, out_handle); + if (ret < 0) + goto out; + + /* fallthrough */ default: break; } diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e3b018aa..a3d0f037 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1949,7 +1949,7 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - fstate = state > TRACECMD_FILE_CMD_LINES ? TRACECMD_FILE_CMD_LINES : state; + fstate = state > TRACECMD_FILE_CPU_COUNT ? TRACECMD_FILE_CPU_COUNT : state; if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; From patchwork Thu Jun 10 11:34:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B37BDC48BE6 for ; Thu, 10 Jun 2021 11:35:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 903D6613F1 for ; Thu, 10 Jun 2021 11:35:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230322AbhFJLhG (ORCPT ); Thu, 10 Jun 2021 07:37:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230323AbhFJLhD (ORCPT ); Thu, 10 Jun 2021 07:37:03 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C13EC0617A8 for ; Thu, 10 Jun 2021 04:35:01 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id i94so1917419wri.4 for ; Thu, 10 Jun 2021 04:35:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GOtu+Wh1wifdgaEDNzuuGtxIZ37pnZ0Ef/YmVURzFrM=; b=CO368FZMkGAYMtt4ZKFGjxwxHHzjGuppVPcZlRJgXNXS1Ucq8SkRi5bG9R+cywZadO XdcFkg4xtuG926lSqJNBR+HJaH2QJerwH4eY0tNjSSrkYAYYnZ9rPuncQaaHluAfWZVb sYIgwmWbkdgBur/I8Vpskk/GwcpHCC0qSCKzDF2LyBUbblvMT2PxKKjEEdcyY044VmbU Dv6FU72hO+LOkgqtVGW411TdAIBk9qGkzwLUACxBMGArh5nzpHWqgTaN/rOetIxQy2qn t/15gERTHLssjM00O8MNzMIBd8Qu6tuW02ijfXdLQnZGheQEWhqvdMeVQaZ5G4suF/sl FK7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GOtu+Wh1wifdgaEDNzuuGtxIZ37pnZ0Ef/YmVURzFrM=; b=tYIiDv2rNauIpcMpbil79LFez61koDdLMsX8RpXIFMjv8vulXcAtTNo6PD8JiTC1xR CPLvY5mLaT8g0OUUZiA8XmDyoyUDYJvoHNG4Nem9YrSNT3EczuBJn8fJzulXqVI/BjP5 EQYOh5H7Gz7fOiz5Soy5UEsC0D01D2gVOLdnWFYPTtsW+oU672MyTrezQfb50f/feFED 9Y3Gs22UHMKjkqxp8DWX2VH00LNZxjwlOVg5Wm4XNi+fbCTK3KULuFOwDFfTcmSfgSkc MoPcKWD+eiO/WXn/64GbU3xBWUbZrUMvA8+Pvc+/BZakvx3ruWmgb6vo9dme6iEn93VO bPQA== X-Gm-Message-State: AOAM530qUj7l7eU/AQUGA9FMZgEmJgEJeaqtRXyxJP0VO3VsAdxuk6En 8RjhABK66J5yyogyyi508JCUfLSBXdH/pg== X-Google-Smtp-Source: ABdhPJxuPiIHMYAFnfD6Csax1zyNUGjB5CEs+fL47nSxkvSAFwDnCXB8Wk+lZwDsdlXD2DjFI506Qg== X-Received: by 2002:a5d:5151:: with SMTP id u17mr5050940wrt.302.1623324899914; Thu, 10 Jun 2021 04:34:59 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.34.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:34:59 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 34/40] trace-cmd: Copy buffers description in tracecmd_copy Date: Thu, 10 Jun 2021 14:34:20 +0300 Message-Id: <20210610113426.257931-35-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of trace buffers description from input to output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 38 +++++++++++++++++++ lib/trace-cmd/trace-output.c | 3 ++ 3 files changed, 43 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 2c31be5f..3735586d 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -174,6 +174,8 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); +int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 289102bf..7d0a41c2 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4173,6 +4173,44 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, return ret < 0 ? -1 : 0; } +static int skip_buffers(struct tracecmd_input *handle) +{ + unsigned long long offset; + unsigned int count; + char *bname; + int i; + + if (read4(handle, &count) < 0) + return -1; + + for (i = 0; i < count; i++) { + if (read8(handle, &offset) < 0) + return -1; + bname = read_string(handle); + free(bname); + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +} + +int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int i; + + if (in_handle->file_version >= 7) { + if (!check_in_state(in_handle, TRACECMD_FILE_BUFERS)) + return -1; + skip_buffers(in_handle); + } + for (i = 0; i < in_handle->nr_buffers; i++) + tracecmd_add_buffer_description(out_handle, + in_handle->buffers[i].name, 0); + + return tracecmd_write_buffers_description(out_handle); +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a3d0f037..960d9a44 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1952,6 +1952,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char fstate = state > TRACECMD_FILE_CPU_COUNT ? TRACECMD_FILE_CPU_COUNT : state; if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; + if (state >= TRACECMD_FILE_BUFERS && + tracecmd_copy_buffer_descr(ihandle, handle) < 0) + goto out_free; /* The file is all ready to have cpu data attached */ return handle; From patchwork Thu Jun 10 11:34:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF4E9C47094 for ; Thu, 10 Jun 2021 11:35:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8460613F1 for ; Thu, 10 Jun 2021 11:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230349AbhFJLhQ (ORCPT ); Thu, 10 Jun 2021 07:37:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230355AbhFJLhP (ORCPT ); Thu, 10 Jun 2021 07:37:15 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 509DCC0617AD for ; Thu, 10 Jun 2021 04:35:02 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id o3so1904782wri.8 for ; Thu, 10 Jun 2021 04:35:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RltyCT2fa2M7gYV3t/HTr8YblFBU8FIl+m0E7lH5+As=; b=ib8Fx5S5HPzLqnbLtZvDSKSfXfrxZZYKn6zDRhxuX4JpKJTS8TA+N//nDIlysPJTX9 Xi/RucpkCoPZvyH1e2O9Xj3nM7aEH9+dKKMfMTKyKuXjmWhyv9DkjZlnbce4lxxk4O6H qhapMCweyd1zX6NLbO4gfOZPRsxsRFygt4y8Es/Gxha8Zp+NZ/naZ2DeToY+OdDqAGqZ D13ogUIy3sdG1Lz9/XqqobSRYAn9TaaYbYvgk6fGxdOQYZCINPDV/epk44AVQbHenAyW enDE4Ky/Vay70hCFcE0oEttdzY4wR6EgTHZkeLnIlmdo4A7gyQiwarUrnOj1tZ7OeibQ 3Ssg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RltyCT2fa2M7gYV3t/HTr8YblFBU8FIl+m0E7lH5+As=; b=nEiaKewvpv6GTgm2GQIcGcjj9pi8kPjm5FStUYU37zALamYKpuZbQDzdgxvIrxWoUU x0zkF6vGPuypuKECR9HTcEzax3E4Dxw1VQ74Y8UpraCt3Z/7q+LHsantxKaPbNnj12+q vPWrtQcUyUU4eoAvB/YUe0y9dz/qctCHRmivmV/wz6yOpWsucL3i3nbfZOPUWxaRErlr YOMny+ORUllZt6RnpMbSbRDmZnoEkaTJ0/sxzlS+iQycti2ZX63CyxJpYyD6u1sJMmgJ jhssAZDd9MQPpkQ1WQ47n2ByHrjKYCkwQ7yhHb5ahe4dpPS0jY9a6k5l2u+aEFsTEDRm kfDQ== X-Gm-Message-State: AOAM532jgfbGdvrzTVGz3ssQdEaLjXExcRgEBlOlhEyU42BLmndJRuAj qsxLM1yXPE9qDIvw/LWQ19ocFYUqMusbUg== X-Google-Smtp-Source: ABdhPJxuFRlY4JCJFW5jnX6mY5Yqkz2MhajJx/s9U8d0uQxgyPz3iWq185YFjZeGdoAOxEnOcWnbMQ== X-Received: by 2002:a05:6000:cb:: with SMTP id q11mr4953211wrx.13.1623324900895; Thu, 10 Jun 2021 04:35:00 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:35:00 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 35/40] trace-cmd: Copy options in tracecmd_copy Date: Thu, 10 Jun 2021 14:34:21 +0300 Message-Id: <20210610113426.257931-36-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of trace options section from input to output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 80 +++++++++++++++++++ lib/trace-cmd/trace-output.c | 8 ++ 4 files changed, 91 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 3735586d..595ba688 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -176,6 +176,8 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, enum tracecmd_file_states end_state); int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); +int tracecmd_copy_options(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 135f52f3..628af16d 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -52,6 +52,7 @@ void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); void out_set_file_state(struct tracecmd_output *handle, int new_state); +void out_save_options_offset(struct tracecmd_output *handle); int write_buffers_description_v7(struct tracecmd_output *handle); int write_buffers_description_v6(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 7d0a41c2..794f8ea3 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4211,6 +4211,86 @@ int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, return tracecmd_write_buffers_description(out_handle); } +static int copy_options(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) +{ + unsigned short option; + unsigned short en2; + unsigned int size; + unsigned int en4; + + if (in_uncompress_block(in_handle)) + return -1; + out_save_options_offset(out_handle); + out_compression_start(out_handle); + for (;;) { + if (do_read_check(in_handle, &option, 2)) + goto error; + en2 = tep_read_number(in_handle->pevent, &option, 2); + if (en2 == TRACECMD_OPTION_DONE) + break; + /* next 4 bytes is the size of the option */ + if (do_read_check(in_handle, &size, 4)) + goto error; + en4 = tep_read_number(in_handle->pevent, &size, 4); + /* Do not copy buffers description, as there is a file specific offset */ + if (en2 == TRACECMD_OPTION_BUFFER) { + /* Skip the option */ + do_lseek(in_handle, en4, SEEK_CUR); + continue; + } + + if (do_write_check(out_handle, &option, 2)) + goto error; + if (do_write_check(out_handle, &size, 4)) + goto error; + if (read_copy_data(in_handle, en4, out_handle)) + goto error; + } + if (do_write_check(out_handle, &option, 2)) + goto error; + in_uncompress_reset(in_handle); + if (out_compression_end(out_handle)) + goto error; + in_handle->file_state = TRACECMD_FILE_OPTIONS; + out_set_file_state(out_handle, in_handle->file_state); + /* Append local options */ + return tracecmd_append_options(out_handle); +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return 0; +} + +int tracecmd_copy_options(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + off64_t offset; + char buf[10]; + + if (!check_in_state(in_handle, TRACECMD_FILE_OPTIONS) || + !check_out_state(out_handle, TRACECMD_FILE_OPTIONS)) + return -1; + + /* Save where we currently are */ + offset = lseek64(in_handle->fd, 0, SEEK_CUR); + + if (do_read_check(in_handle, buf, 10)) + return -1; + /* check if this handles options */ + if (strncmp(buf, "options", 7) == 0) { + if (do_write_check(out_handle, "options ", 10)) + return -1; + if (copy_options(in_handle, out_handle) < 0) + return -1; + } else { + if (lseek64(in_handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + in_handle->file_state = TRACECMD_FILE_OPTIONS; + out_set_file_state(out_handle, in_handle->file_state); + } + return 0; +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 960d9a44..cbe641fd 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1955,6 +1955,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (state >= TRACECMD_FILE_BUFERS && tracecmd_copy_buffer_descr(ihandle, handle) < 0) goto out_free; + if (state >= TRACECMD_FILE_OPTIONS && + tracecmd_copy_options(ihandle, handle) < 0) + goto out_free; /* The file is all ready to have cpu data attached */ return handle; @@ -1974,6 +1977,11 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) return check_file_state(handle->file_version, handle->file_state, new_state); } +__hidden void out_save_options_offset(struct tracecmd_output *handle) +{ + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); +} + /** * tracecmd_get_out_file_version - return the trace.dat file version * @handle: output handle for the trace.dat file From patchwork Thu Jun 10 11:34:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DF9EC48BE0 for ; Thu, 10 Jun 2021 11:35:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37C1961406 for ; Thu, 10 Jun 2021 11:35:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230334AbhFJLhL (ORCPT ); Thu, 10 Jun 2021 07:37:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230360AbhFJLhK (ORCPT ); Thu, 10 Jun 2021 07:37:10 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C910C0617AE for ; Thu, 10 Jun 2021 04:35:03 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id i94so1917519wri.4 for ; Thu, 10 Jun 2021 04:35:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m86bEKIOnZmBFmC+Cm97MmQx3GEr7vGNCooD/PTULnM=; b=lwAEe6kw9WG6ulB2hJHGdJzkDftEQgIQiLOtktPK7v0ES5nGwUaBC7YHTlPUofMXW9 L9698c8jetGh7mzAbEpxqO2dHxqMdGFDWEDrAh4xuNHJBbXEL0lxEoiJQmn1aZzrVF8a hlSQv7mRm4jOBrgzFYLyMq6jFeB5f1dB5w+BQRmWDDc8X2TymwXa0BN9ONMRGUbFaEcP UG/JlkF1QI8PbcoT7m2M6bOUOc4N2oD+HVbCl6BpfLJihYZtqXgemz7lCmu/tS5PoRq8 hI9W2DDaRVneWMqpByzviOLiZsJJSDqbvwLih+8RSqlqptOuiWBBCq/BCfznS1PscQdf rErA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m86bEKIOnZmBFmC+Cm97MmQx3GEr7vGNCooD/PTULnM=; b=ahTd8etK21I5P5T+3a46LLUbk1IVvXj387aNGKvxr7drzYTbvHRGoMMDGO+rZ+wxP9 di8Tsn75lztcRRueyfCL8A7xb3n93osFoo+5z4+JbWG5TG7V8wBN222F+KKYo5V6UFi/ ICjEYFF5iCBMOab+aCKgFh4hrDtpWCKgtAS/DLvfPzAKxKeKL6OZL1pdWKpMfKIWwtBH jBnkWYSw/uHJJB8zmPzEgBxjTdSZiJNzA7jAHmaQaPmGQcbejmGByUsj62bWp+1hCKS7 gVEZbguuKWE3xOJ4iqYO07Pp7C6Mjpz3JWldDdbKdn/dTfDP7fgsV52vczjIABusOKpv VvfQ== X-Gm-Message-State: AOAM533j5ZzGHct4ecaue45lqDrUpoi4TKoPF2zJ8Cyh1IFMenllfJRh e9LFYIlLdLEYGtt36WZN7N5ioxOeLbRgfQ== X-Google-Smtp-Source: ABdhPJxfVQHAuovjMMHqnv98lBucfYfZW4f86QOPOdAocPUmRKse9lgDbOcQ0mcORYA0ho/p1IiDpQ== X-Received: by 2002:adf:ee52:: with SMTP id w18mr4945330wro.37.1623324901879; Thu, 10 Jun 2021 04:35:01 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.35.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:35:01 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 36/40] trace-cmd library: Refactor the logic for writing CPU trace data Date: Thu, 10 Jun 2021 14:34:22 +0300 Message-Id: <20210610113426.257931-37-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Refactored the internal library logic for writing trace CPU data in the trace file. The existing logic copies trace data from a temporary file into the trace file. In order to reuse the code, modify it to support copying the trace data not only from temporary file, but from given fd at given offset. These changes will be used in extending the tracecmd_copy API. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 13 +++ lib/trace-cmd/trace-output.c | 142 +++++++++++++++++------- 2 files changed, 114 insertions(+), 41 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 628af16d..e8c7968f 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -53,10 +53,23 @@ int in_uncompress_block(struct tracecmd_input *handle); void out_set_file_state(struct tracecmd_output *handle, int new_state); void out_save_options_offset(struct tracecmd_output *handle); +unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size); + int write_buffers_description_v7(struct tracecmd_output *handle); int write_buffers_description_v6(struct tracecmd_output *handle); long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); +struct cpu_data_source { + int fd; + int size; + off64_t offset; +}; + +int out_write_cpu_data(struct tracecmd_output *handle, int cpus, + struct cpu_data_source *data); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index cbe641fd..a3ccfba5 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -286,18 +286,26 @@ static unsigned long get_size(const char *file) return size; } -static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd) +static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd, unsigned long long max) { + tsize_t rsize = 0; tsize_t size = 0; char buf[BUFSIZ]; stsize_t r; do { - r = read(fd, buf, BUFSIZ); + if (max > 0 && (max - size) < BUFSIZ) + rsize = (max - size); + else + rsize = BUFSIZ; + + r = read(fd, buf, rsize); if (r > 0) { size += r; if (do_write_check(handle, buf, r)) return 0; + if (max > 0 && size >= max) + break; } } while (r > 0); @@ -315,47 +323,61 @@ static tsize_t copy_file(struct tracecmd_output *handle, tracecmd_warning("Can't read '%s'", file); return 0; } - size = copy_file_fd(handle, fd); + size = copy_file_fd(handle, fd, 0); close(fd); return size; } #define COMPRESS_CHUNK_SIZE (1*1024*1024) -static tsize_t copy_file_compress(struct tracecmd_output *handle, - const char *file, unsigned long long *write_size) +__hidden unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size) { unsigned long long rsize = 0; unsigned long long wsize = 0; - tsize_t size; + unsigned long long size; int ret; - int fd; - - fd = open(file, O_RDONLY); - if (fd < 0) { - tracecmd_warning("Can't read '%s'", file); - return 0; - } if (handle->file_version >= 7) { + rsize = max; ret = tracecmd_compress_copy_from(handle->compress, fd, COMPRESS_CHUNK_SIZE, &rsize, &wsize); - if (ret < 0) { - tracecmd_warning("Can't compress '%s'", file); - close(fd); + if (ret < 0) return 0; - } + size = rsize; - *write_size = wsize; + if (write_size) + *write_size = wsize; } else { - size = copy_file_fd(handle, fd); - *write_size = size; + size = copy_file_fd(handle, fd, max); + if (write_size) + *write_size = size; } - close(fd); return size; } +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + ret = out_copy_fd_compress(handle, fd, 0, write_size); + if (!ret) + tracecmd_warning("Can't compress '%s'", file); + + close(fd); + return ret; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -467,7 +489,7 @@ static int read_header_files(struct tracecmd_output *handle) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s' size=%lld read=%lld", path, size, check_size); @@ -493,7 +515,7 @@ static int read_header_files(struct tracecmd_output *handle) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s'", path); @@ -1671,8 +1693,8 @@ struct data_file_write { off64_t doffset; }; -int tracecmd_write_cpu_data(struct tracecmd_output *handle, - int cpus, char * const *cpu_data_files) +__hidden int out_write_cpu_data(struct tracecmd_output *handle, + int cpus, struct cpu_data_source *data) { struct data_file_write *data_files = NULL; off64_t offset; @@ -1680,8 +1702,6 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, unsigned long long read_size; unsigned long long write_size; char *clock = NULL; - char *file; - struct stat st; int ret; int i; @@ -1705,14 +1725,7 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, /* Write 0 for trace data offset and size and store offsets of these fields */ for (i = 0; i < cpus; i++) { - file = cpu_data_files[i]; - ret = stat(file, &st); - if (ret < 0) { - tracecmd_warning("can not stat '%s'", file); - goto out_free; - } - data_files[i].file_size = st.st_size; - + data_files[i].file_size = data[i].size; endian8 = 0; data_files[i].doffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) @@ -1736,13 +1749,19 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (!tracecmd_get_quiet(handle)) fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", i, (unsigned long long) data_files[i].data_offset); - - read_size = copy_file_compress(handle, cpu_data_files[i], &write_size); - if (read_size != data_files[i].file_size) { - errno = EINVAL; - tracecmd_warning("did not match size of %lld to %lld", - read_size, data_files[i].file_size); + offset = lseek64(data[i].fd, data[i].offset, SEEK_SET); + if (offset == (off64_t)-1) goto out_free; + if (data[i].size) { + read_size = out_copy_fd_compress(handle, data[i].fd, data[i].size, &write_size); + if (read_size != data_files[i].file_size) { + errno = EINVAL; + tracecmd_warning("did not match size of %lld to %lld", + read_size, data_files[i].file_size); + goto out_free; + } + } else { + write_size = 0; } /* Write the real CPU data offset inthe file */ offset = lseek64(handle->fd, data_files[i].doffset, SEEK_SET); @@ -1775,6 +1794,47 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, return -1; } +int tracecmd_write_cpu_data(struct tracecmd_output *handle, + int cpus, char * const *cpu_data_files) +{ + struct cpu_data_source *data; + struct stat st; + int ret; + int i; + + data = calloc(cpus, sizeof(struct cpu_data_source)); + if (!data) + return -1; + for (i = 0; i < cpus; i++) + data[i].fd = -1; + for (i = 0; i < cpus; i++) { + ret = stat(cpu_data_files[i], &st); + if (ret < 0) { + tracecmd_warning("can not stat '%s'", cpu_data_files[i]); + break; + } + data[i].fd = open(cpu_data_files[i], O_RDONLY); + if (data[i].fd < 0) { + tracecmd_warning("Can't read '%s'", data[i].fd); + break; + } + + data[i].size = st.st_size; + data[i].offset = 0; + } + + if (i < cpus) + ret = -1; + else + ret = out_write_cpu_data(handle, cpus, data); + for (i = 0; i < cpus; i++) { + if (data[i].fd >= 0) + close(data[i].fd); + } + free(data); + return ret; +} + int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { From patchwork Thu Jun 10 11:34:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 183B0C47094 for ; Thu, 10 Jun 2021 11:36:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED7DA613FE for ; Thu, 10 Jun 2021 11:36:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbhFJLiO (ORCPT ); Thu, 10 Jun 2021 07:38:14 -0400 Received: from mail-wm1-f48.google.com ([209.85.128.48]:34761 "EHLO mail-wm1-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230245AbhFJLiO (ORCPT ); Thu, 10 Jun 2021 07:38:14 -0400 Received: by mail-wm1-f48.google.com with SMTP id u5-20020a7bc0450000b02901480e40338bso4597515wmc.1 for ; Thu, 10 Jun 2021 04:36:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=laA4gGGumfq0sJQFfiL10durfgyLHHcEFJonCoS0JyE=; b=hOLOw8o2Z/un6Pl9tTd65a50CD8cKU649RBQi9NFV6jAzPPKu8lpOjw2ZVFbkJk0F0 +875qUTlVI/Bw2qlQieb/hPWkI+42U1clnnGAkNSG4QWqcoIqz+A1Oqt4XFF3gD3N2Ot YjWZB2B/9eLcoth4B0uemBE95wN691QvLdkvqUmLNi/rsRpNxsoMZN14u7ir/P2QU7ZR bNNJezT7iyxCJw/CKqj1sIeQ3ru7iE6G5GM9EVdPIcODHN2yw8P9IKBCtoEZBd+x8EM0 6WhXqKGPIIR/IKOniKYSDVdlUsv5NG0OVhIQMak7j/sHD4LIj5bNGVebWt2Nb1c/N2Iu 8Vfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=laA4gGGumfq0sJQFfiL10durfgyLHHcEFJonCoS0JyE=; b=o0SpmtGjHP3ZYcFQDnIdnl7HnNRwIIF4Xj32k0lTSNpJS6ve4xxFPWcmtgfTi7nPw9 AXMrZAvlmMJj3zzdNcs0NLpgXpK3NGdpLdurFAOpcpxAFck/pHtZENbIZfgHZQb/LpH3 rt1h4gZfayndC7rwsHTBqQEhBXuYMJoI1lHAminGNQYShPDQ+akqDbsw2t3ROGE4IehH wOkn7shXW+66E9YAIzC1AWdaUuJt+oPNEmB9g/UesVlLVPDzfMB+zDHZnAKh7atbka5b gbe1mCnqpZKAVwBknOwPuexgLhlfa69cGADk2cUD9Qxfq7E0iquOXSbUDzjMIps+zx5E 5rhQ== X-Gm-Message-State: AOAM533ErCxfZcstcOACmjUvhfi8sfKWzec9we5vmysDeJWXCujWAdLE KR7brbos4wmuWSIL5bbmJlQ= X-Google-Smtp-Source: ABdhPJxOME9x8oRMsaP2KZCxLDRLueHLrTCSCbRIitXmLLceysn379p+4B/mJbfCFS1QsfpGDat6QA== X-Received: by 2002:a7b:c394:: with SMTP id s20mr4574229wmj.33.1623324902836; Thu, 10 Jun 2021 04:35:02 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.35.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:35:02 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 37/40] trace-cmd library: Refactor the logic for writing CPU instance trace data Date: Thu, 10 Jun 2021 14:34:23 +0300 Message-Id: <20210610113426.257931-38-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a helper function, internal to the library, for updating CPU instance offset in the trace file. This allows to reuse the logic inside the library, will be used in the tracecmd_copy API. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 ++ lib/trace-cmd/trace-output.c | 11 +++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index e8c7968f..14908f3c 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -72,4 +72,6 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data); +int out_update_buffer_cpu_offset(struct tracecmd_output *handle, const char *name); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a3ccfba5..9933224c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1853,8 +1853,7 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, return tracecmd_write_cpu_data(handle, cpus, cpu_data_files); } -int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - const char *name, int cpus, char * const *cpu_data_files) +__hidden int out_update_buffer_cpu_offset(struct tracecmd_output *handle, const char *name) { tsize_t b_offset; tsize_t offset; @@ -1883,6 +1882,14 @@ int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, tracecmd_warning("could not seek to %lld\n", offset); return -1; } + return 0; +} + +int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, + const char *name, int cpus, char * const *cpu_data_files) +{ + if (out_update_buffer_cpu_offset(handle, name)) + return -1; return tracecmd_write_cpu_data(handle, cpus, cpu_data_files); } From patchwork Thu Jun 10 11:34:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A59EC48BD1 for ; Thu, 10 Jun 2021 11:36:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 62FDF613E9 for ; Thu, 10 Jun 2021 11:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230245AbhFJLiP (ORCPT ); Thu, 10 Jun 2021 07:38:15 -0400 Received: from mail-wr1-f50.google.com ([209.85.221.50]:39578 "EHLO mail-wr1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230250AbhFJLiP (ORCPT ); Thu, 10 Jun 2021 07:38:15 -0400 Received: by mail-wr1-f50.google.com with SMTP id l2so1909828wrw.6 for ; Thu, 10 Jun 2021 04:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H/Nld6QJK8ob2cGGOO/ZYw75wyLuGDA6phwis20loh4=; b=hmv4uCFlR2hqiaAHZvX+ff14aJPNAgKd4LIITVF0BaHCN/gIPFBevOoy3NIVWcI9MA OSCCwYbMGa33/ymSwKT1hLKUFaUdq3orI0UQOFs3bkm+3ukxsVtJhR1O3TLT8UMBZUqp b74rS7ivdlOcLLfSNwWOvbZLXnVYUEP3/YQr3Ie6RKWWRQmi0E6jC5XBQrDHEC031gUu OCf8ZU9LjM6YD2IVruGVIrcf/XSBh2dPDGmvpW3ePv57I0jMiXGEaRNgVGEhCyrtalUd qXU68GxJwp4Nl4sfbX9/DzJIQAOeX9TIinYs93FYeYORO6SKVS19KJDC+Muu/bA3GO1E 12oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H/Nld6QJK8ob2cGGOO/ZYw75wyLuGDA6phwis20loh4=; b=hAByD4j8t/Pdg/NSGKjIRM9uUlmKH8ZnybCR7uKumYRvgPRT9SREtN3vXC+IbksvwK k9jz4Kpa6Y9jA1oM8sHzaesgclKIEzjk/Nl6pq/e3QYtbcxmnRDwWw/7u0kCj/6ycG79 jfDfnaxWQ7VCYcnu7A/fEcJiWQ0A1u1yX7U36s7jsz5asKjBpeIG4CLzNWAVjcsVd/Yj b2Ad3VFbvKhz4/IyoU/YkVfWvtyK+MDMXWlkqW6q/mYuUYE4ClMQaiBETqVSfvqPybT2 rghA4S3bnS4q2xwCCtJjyfiyo+y1j9CUB09K8cLo035igkBvLWX+xcj4JQIOJpy9ZCl1 6fmA== X-Gm-Message-State: AOAM532hQcp2hpX9c0xOzevpe0Srh7Alm4Mi+fBunA4H6WQsZ3PwsiPj 4H5E8J6rKSdXiUiVjpZtkdQ= X-Google-Smtp-Source: ABdhPJxFNKW4cbMVEGGQu1f7CM9v1VtIti0892cJTb3ZeoiXPEaRc2341uECrV4q8W1LaHYbyhZ56g== X-Received: by 2002:a5d:4a43:: with SMTP id v3mr4788223wrs.397.1623324903952; Thu, 10 Jun 2021 04:35:03 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.35.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:35:03 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 38/40] trace-cmd: Copy trace data in tracecmd_copy Date: Thu, 10 Jun 2021 14:34:24 +0300 Message-Id: <20210610113426.257931-39-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of trace data from input to output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 101 ++++++++++++++++++ lib/trace-cmd/trace-output.c | 3 + 3 files changed, 106 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 595ba688..79fef884 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -178,6 +178,8 @@ int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); int tracecmd_copy_options(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); +int tracecmd_copy_trace_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 794f8ea3..2544d825 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4291,6 +4291,107 @@ int tracecmd_copy_options(struct tracecmd_input *in_handle, return 0; } +static int copy_trace_latency(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int fd; + + if (do_write_check(out_handle, "latency ", 10)) + return -1; + + if (in_handle->lat_cfd >= 0) + fd = in_handle->lat_cfd; + else + fd = in_handle->fd; + + if (!out_copy_fd_compress(out_handle, fd, 0, NULL)) + return -1; + + out_set_file_state(out_handle, TRACECMD_FILE_CPU_LATENCY); + return 0; +} + +static int copy_trace_flyrecord_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + struct cpu_data_source *data; + int ret; + int i; + + data = calloc(in_handle->cpus, sizeof(struct cpu_data_source)); + if (!data) + return -1; + for (i = 0; i < in_handle->cpus; i++) { + data[i].size = in_handle->cpu_data[i].file_size; + if (in_handle->cpu_data[i].cfd >= 0) { + data[i].fd = in_handle->cpu_data[i].cfd; + data[i].offset = 0; + } else { + data[i].fd = in_handle->fd; + data[i].offset = in_handle->cpu_data[i].file_offset; + } + } + ret = out_write_cpu_data(out_handle, in_handle->cpus, data); + + return ret; +} + +static int copy_trace_flyrecord(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + struct tracecmd_input *instance; + const char *name; + int ret; + int i; + + /* top instance */ + ret = copy_trace_flyrecord_data(in_handle, out_handle); + if (ret) + return ret; + + for (i = 0; i < in_handle->nr_buffers; i++) { + name = tracecmd_buffer_instance_name(in_handle, i); + if (!name) + continue; + instance = tracecmd_buffer_instance_handle(in_handle, i); + if (!instance) + continue; + if (!tracecmd_get_quiet(out_handle)) + fprintf(stderr, "\nBuffer: %s\n\n", name); + + if (!out_update_buffer_cpu_offset(out_handle, name)) + copy_trace_flyrecord_data(instance, out_handle); + + tracecmd_close(instance); + } + + return 0; +} + +int tracecmd_copy_trace_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + char buf[10]; + + if (!check_in_state(in_handle, TRACECMD_FILE_CPU_FLYRECORD) || + !check_out_state(out_handle, TRACECMD_FILE_CPU_FLYRECORD)) + return -1; + + tracecmd_set_out_clock(out_handle, in_handle->trace_clock); + + if (do_read_check(in_handle, buf, 10)) + return -1; + if (strncmp(buf, "latency", 7) == 0) { + in_handle->file_state = TRACECMD_FILE_CPU_LATENCY; + return copy_trace_latency(in_handle, out_handle); + } else if (strncmp(buf, "flyrecord", 9) == 0) { + in_handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; + return copy_trace_flyrecord(in_handle, out_handle); + } + + return -1; +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9933224c..fd7ea06c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2025,6 +2025,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (state >= TRACECMD_FILE_OPTIONS && tracecmd_copy_options(ihandle, handle) < 0) goto out_free; + if (state >= TRACECMD_FILE_CPU_LATENCY && + tracecmd_copy_trace_data(ihandle, handle) < 0) + goto out_free; /* The file is all ready to have cpu data attached */ return handle; From patchwork Thu Jun 10 11:34:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 026A3C48BD1 for ; Thu, 10 Jun 2021 11:35:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D51D961425 for ; Thu, 10 Jun 2021 11:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230355AbhFJLhR (ORCPT ); Thu, 10 Jun 2021 07:37:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230366AbhFJLhP (ORCPT ); Thu, 10 Jun 2021 07:37:15 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3FDC6C0617AF for ; Thu, 10 Jun 2021 04:35:06 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id a11so1890351wrt.13 for ; Thu, 10 Jun 2021 04:35:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Vcdi+S5Ov6ZUCcxdLyp6oZPs81fi1WITyWp1GWifBDk=; b=Oc4dti27ulG2s977K1Z+KhZSZBweJfSRRxremgffyj30xpvDa3yY/NHYMC1+F1vVdE 7Es7O+gRFEReuzeH5PzHR/PKuey2pWUjBGmAVibFvFJ72P92OuQ8C/2mo/lE0fz/8MJw cg7woXu3NiEVFE/gp0gUUZCBIOnaag6HAvURK3b6ZDt29JZ8dOW06g8B9nKj61tRgxCa yY7apGrfqKwS5dvGB1m6ExsQAFsjOq3P2hVJ5kEiXP2rsOVRrOPr/ko3jmmAfjRexkhn RQIoyGCm8GC1HYxOArkBMNYkBqFMLiU5qqC6MtkZxDFrn6CR2p+ElcMS8thXSkXHJ8Mg V9gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Vcdi+S5Ov6ZUCcxdLyp6oZPs81fi1WITyWp1GWifBDk=; b=noasNsYgJW3+sEEy0cCpgd86HqVYjNWLLC9Ykla0iIozyiSvDX0JwJsieqotpaeXs8 4tY/1BRdT96sZqDbOLRaVQVB9FjRIOge4lhplLQdd3Lequfnras1GlDWApBe/MHMT9sA uD/92kRXmn+OHDQsTM367FMnk8Nv3hZXMSHPtYZ308VgeAVBsKKgbdRUBlTQJVmDRgyQ pjRGSYhZba3OK26ALxar6WBmzNFn45v8m4AKPwpIOcp7YueLrDi8YvDc7ZBPjkWxuACl ZAFyrYH6FdUItw505zBFf43CKLb6rFaorWXIbX1TBBbdAwRFwdy9w+JlRbf/K7yOuVh6 LS2A== X-Gm-Message-State: AOAM530DwVjdrIgpdX4V99Sq4BYoKRcpMu2k77D3PQurQtcU+5YPTFX0 ArH8HCgCAEaE86/JbLtg778= X-Google-Smtp-Source: ABdhPJyLTVMY2LLKwSu6ievZONKf6nuHrsunAtNVl9R3TGBjfnhZh0CVewOhlZPuza0MoMHPikLBRQ== X-Received: by 2002:adf:f690:: with SMTP id v16mr5066352wrp.247.1623324904847; Thu, 10 Jun 2021 04:35:04 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.35.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:35:04 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 39/40] trace-cmd: Add compression parameter to tracecmd_copy Date: Thu, 10 Jun 2021 14:34:25 +0300 Message-Id: <20210610113426.257931-40-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added compression parameter to the tracecmd_copy() API, which allows to select the compression of the created output file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 +- lib/trace-cmd/trace-output.c | 11 +++++++---- tracecmd/trace-restore.c | 2 +- tracecmd/trace-split.c | 2 +- 4 files changed, 10 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 79fef884..c8f0ee93 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -316,7 +316,7 @@ int tracecmd_append_options(struct tracecmd_output *handle); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, - enum tracecmd_file_states state); + enum tracecmd_file_states state, const char *compression); int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index fd7ea06c..01f08b61 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1999,20 +1999,23 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * @ihandle: input handle of the trace.dat file to copy * @file: the trace.dat file to create * @state: what data will be copied from the source handle + * @compression: compression of the output file, can be one of: + * NULL - inherit compression from the input file + * "any" - compress the output file with the best available algorithm + * "none" - do not compress the output file + * algorithm_name - compress the output file with specified algorithm * * Reads the header information and creates a new trace data file * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, - enum tracecmd_file_states state) + enum tracecmd_file_states state, const char *compression) { enum tracecmd_file_states fstate; struct tracecmd_output *handle; - const char *compr_name = NULL; - tracecmd_get_file_compress_proto(ihandle, &compr_name, NULL); - handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compr_name); + handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compression); if (!handle) return NULL; diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 7d439fd2..58913fa7 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -125,7 +125,7 @@ void trace_restore (int argc, char **argv) if (tracecmd_read_headers(ihandle, TRACECMD_FILE_CMD_LINES) < 0) die("error reading file %s headers", input); - handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES); + handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES, NULL); tracecmd_close(ihandle); } else handle = tracecmd_create_init_file(output, NULL); diff --git a/tracecmd/trace-split.c b/tracecmd/trace-split.c index 401e1d29..b98e6fc9 100644 --- a/tracecmd/trace-split.c +++ b/tracecmd/trace-split.c @@ -345,7 +345,7 @@ static double parse_file(struct tracecmd_input *handle, dir = dirname(output); base = basename(output); - ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES); + ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES, NULL); cpus = tracecmd_cpus(handle); cpu_data = malloc(sizeof(*cpu_data) * cpus); From patchwork Thu Jun 10 11:34:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12312813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2617DC48BDF for ; Thu, 10 Jun 2021 11:35:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0EB50613FE for ; Thu, 10 Jun 2021 11:35:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230252AbhFJLhK (ORCPT ); Thu, 10 Jun 2021 07:37:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230330AbhFJLhE (ORCPT ); Thu, 10 Jun 2021 07:37:04 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2860BC0613A2 for ; Thu, 10 Jun 2021 04:35:07 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id h22-20020a05600c3516b02901a826f84095so6196942wmq.5 for ; Thu, 10 Jun 2021 04:35:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RtMwJbKncg4ckpg5SIUP3kDSQFOXLgYoO4/UYSv1wIw=; b=JUHpeMtdBZqYcr5EfCk/uQkvK/LuLXJ89L36i43G611PEMg0aqueZ6zaSmGFBu9kjg hfiixhqqE6eZJXqfLCFfA7Rc+2acCDwwdEl80PkHqeWgXzGcAK+SgzM3Omffa1CZagrg Bt2OVtBf9q5XuczVLcksd32UWAZtfN2c/4UDI3/jEfQKcMJYJ+j2q6UMsxZF8F5B6hTp 4f0qDu4PyShuo2OpktC/baVsap9g3Gnc+jl/qPmYoyki55/IUuhw6djaIk3bHKJEgNef 5kwadc4X+/lO5chU/xzs334tnBBX4thxoA63mcwVAWciC1rWfWFEdgp8UXS5TckmH3by Gh+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RtMwJbKncg4ckpg5SIUP3kDSQFOXLgYoO4/UYSv1wIw=; b=k1LS1iMvt9fTm+axOnhki9vvJ9/7qSdRGFdfKh7T5/AUJiZilhDueu+8svyG83IwUB SbSq/a4uhme58VIFtsV7AWDfCpoHIWYUlvSiyEdftN92B1Xjhse4nm/Cg9DpTydCCU5p hzuXgahqwmW+pv8xr7q+TyjY+eMlledcEimqw+3OlVIgI0HuGbtm7SAiJy6LloR3xnPp CTS1ECvphvrx8CDtpO6cdQ7QvOUhxH9YUyIkDjGbQ/Ny0AkD35CoIPxD30BmuGDrzA3m f93dw/BzjC/eF/AmFybnp4NtLHbPtpsjQ57+7aheZ5jbo85S+q7+9pYFTTKljqZakzcT Dxow== X-Gm-Message-State: AOAM530BbE6EKbPEuHROyV6b/6WkbnAX3243yAay97gYGw2AtJ2+xJMM Nv01nPDK6ZY2cqf/hX7N5GaUPCl38SGQQA== X-Google-Smtp-Source: ABdhPJzCwqTWVn1WPj8sDA0eB+RIhP4az4vTzF5nYAF/6HWDnRupJ+L/+ex5rD1z4d7PB3ZjmmEIaA== X-Received: by 2002:a05:600c:35c1:: with SMTP id r1mr14469341wmq.181.1623324905737; Thu, 10 Jun 2021 04:35:05 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id n42sm8898391wms.29.2021.06.10.04.35.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 04:35:05 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v5 40/40] trace-cmd: Add new command "trace-cmd convert" Date: Thu, 10 Jun 2021 14:34:26 +0300 Message-Id: <20210610113426.257931-41-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210610113426.257931-1-tz.stoyanov@gmail.com> References: <20210610113426.257931-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A new trace-cmd subcommand is added, used to convert trace files: trace-cmd convert -i -o --compression The command reads the given input trace file and creates output trace file with given compression. The --compression parameter is optional: - no --compression parameter: the output file inherits the compression from the input file. --compression none - the output file is not compressed. --compression any - use the best available compression algorithm to compress the output file --compression - use the specified compression algorithm, if available, to compress the output file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/Makefile | 1 + tracecmd/include/trace-local.h | 2 + tracecmd/trace-cmd.c | 1 + tracecmd/trace-convert.c | 93 ++++++++++++++++++++++++++++++++++ tracecmd/trace-usage.c | 12 +++++ 5 files changed, 109 insertions(+) create mode 100644 tracecmd/trace-convert.c diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 80c69bbb..35086b71 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -36,6 +36,7 @@ TRACE_CMD_OBJS += trace-usage.o TRACE_CMD_OBJS += trace-dump.o TRACE_CMD_OBJS += trace-clear.o TRACE_CMD_OBJS += trace-vm.o +TRACE_CMD_OBJS += trace-convert.o ifeq ($(VSOCK_DEFINED), 1) TRACE_CMD_OBJS += trace-agent.o diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index e9a0aea8..13dab44c 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -110,6 +110,8 @@ void trace_usage(int argc, char **argv); void trace_dump(int argc, char **argv); +void trace_convert(int argc, char **argv); + int trace_record_agent(struct tracecmd_msg_handle *msg_handle, int cpus, int *fds, int argc, char **argv, bool use_fifos, diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 71c8f6d6..e6274c35 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -130,6 +130,7 @@ struct command commands[] = { {"list", trace_list}, {"help", trace_usage}, {"dump", trace_dump}, + {"convert", trace_convert}, {"-h", trace_usage}, }; diff --git a/tracecmd/trace-convert.c b/tracecmd/trace-convert.c new file mode 100644 index 00000000..d75859c1 --- /dev/null +++ b/tracecmd/trace-convert.c @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov + */ +#include +#include +#include +#include +#include + +#include "trace-local.h" +#include "trace-cmd.h" +#include "trace-cmd-private.h" + +static void convert_file(const char *in, const char *out, char *compr) +{ + struct tracecmd_input *ihandle; + struct tracecmd_output *ohandle; + + ihandle = tracecmd_open(in, 0); + if (!ihandle) + die("error reading %s", in); + + ohandle = tracecmd_copy(ihandle, out, TRACECMD_FILE_CPU_FLYRECORD, compr); + if (!ohandle) + die("error writing %s", out); + tracecmd_output_close(ohandle); +} + +enum { + OPT_comporession = 255, +}; + +void trace_convert(int argc, char **argv) +{ + char *input_file = NULL; + char *output_file = NULL; + char *compression = NULL; + int c; + + if (argc < 2) + usage(argv); + + if (strcmp(argv[1], "convert") != 0) + usage(argv); + for (;;) { + int option_index = 0; + static struct option long_options[] = { + {"compression", required_argument, NULL, OPT_comporession}, + {"help", no_argument, NULL, '?'}, + {NULL, 0, NULL, 0} + }; + + c = getopt_long (argc-1, argv+1, "+hi:o:", long_options, &option_index); + if (c == -1) + break; + switch (c) { + case 'i': + if (input_file) + die("Only one input file is supported, %s already set", input_file); + input_file = optarg; + break; + case 'o': + if (output_file) + die("Only one output file is supported, %s already set", output_file); + output_file = optarg; + break; + case OPT_comporession: + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + compression = optarg; + break; + case 'h': + case '?': + default: + usage(argv); + } + } + + if ((argc - optind) >= 2) { + if (output_file) + usage(argv); + output_file = argv[optind + 1]; + } + + if (!input_file) + input_file = DEFAULT_INPUT_FILE; + if (!output_file) + usage(argv); + + convert_file(input_file, output_file, compression); +} diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index c70f9919..d0d12e87 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -392,6 +392,18 @@ static struct usage_help usage_help[] = { " --clock trace clock, saved in the file\n" " -h, --help show usage information\n" }, + { + "convert", + "convert trace file to different version", + " %s convert [options]\n" + " -i input file, default is trace.dat\n" + " -o output file, mandatory parameter.\n" + " The output file can be specified also as last argument of the command\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { NULL, NULL, NULL }