From patchwork Mon Jun 14 07:49:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F1D0C48BE6 for ; Mon, 14 Jun 2021 07:50:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 497BF61289 for ; Mon, 14 Jun 2021 07:50:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232520AbhFNHwh (ORCPT ); Mon, 14 Jun 2021 03:52:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232499AbhFNHwg (ORCPT ); Mon, 14 Jun 2021 03:52:36 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB72DC061766 for ; Mon, 14 Jun 2021 00:50:33 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id ce15so15022752ejb.4 for ; Mon, 14 Jun 2021 00:50:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SL302fTyFfaFnXlErdYp1JrAPy3ZPN35AL5S03E1of4=; b=bdy/q1bnSCqvYEu2xN0kjUgIC0obk/BxHsM4DCb2SIOZoyOa63UjW/TgqmuF4COVYq XVop6wCl0K2xWMC/jmh3nL6pWheNeD9tQRFPMja8wO7dYgSPk4GtRqxWn2CWVoJ926gD jh+yUXVZB01EZE3tX7zB0Tvn/kSYneQ+zfRTcIzcZzVBIBH8kuZZMLHpHeUle14jKFyj 1jJgmzVz9K0PvsGPLhmi/ZVphUO50k3KhXu71S1gVuh7LPRMgLZXm0xFcwaOjApdswF4 U4uIFhM5S6i8kNfD6sBTuvlq9f0X6huISp+BVAMxwN7Rc3ryFjdKsqUEmktKOaoOTA8T GXww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SL302fTyFfaFnXlErdYp1JrAPy3ZPN35AL5S03E1of4=; b=fFx0BvJIaDlhcvr+49Sck3HHWYRLtVaWrfP3b1sMtAw1rVCYUAkqUBosBHrz6r5+6U 1mv+ZAQurw+bnpn5s0H/gHNfeG4ZDrNcZpSEYkTCBvGPEb2g1U55fC8CQpMx2hujf7P/ VRyOrjiLU4UHzRRu6MOFY4hs+RgjxsOjWVxok4OxbycipUsuPK+CyRi+e63HPm/OCZXZ P+FsusTivEBb8CE6yNTS9pMQjXWq+tDY3SNtUFQyXuBQF0D7GSlJDoRWzYRouEelBYg4 lr8YVEQyv7feqBgyJghxGwAZ0AeBCT+799q703uOCGTxT4P/DnhEZWeA5c3Uuw2XgNDD 9rBQ== X-Gm-Message-State: AOAM533ytXBS5kAHlrwE9VEBWQbx0D2ppG4YwK7SIXIfnbvToFx8Nimx HIWsWMmY4FYEq1gYvFSyH4Y= X-Google-Smtp-Source: ABdhPJzjJUnHXVu92nSCciw2QHdtVKJfON1rg+BttDq0ja+AJbl9uN8N3aPTD1ku1rCI4uTbjdc2FQ== X-Received: by 2002:a17:906:919:: with SMTP id i25mr13606355ejd.171.1623657032572; Mon, 14 Jun 2021 00:50:32 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:31 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 01/45] trace-cmd library: Remove unused private APIs for creating trace files Date: Mon, 14 Jun 2021 10:49:45 +0300 Message-Id: <20210614075029.598048-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Remove these unused private trace-cmd library APIs: tracecmd_create_file() tracecmd_create_file_glob() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 6 ---- lib/trace-cmd/trace-output.c | 29 ------------------- 2 files changed, 35 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 5c2ab4ca..4bd36baa 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -263,12 +263,6 @@ struct tracecmd_option; struct tracecmd_msg_handle; struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files); -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *event_globs); struct tracecmd_output * tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a4a1eecc..9231c319 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1614,35 +1614,6 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *list) -{ - struct tracecmd_output *handle; - - handle = create_file(output_file, NULL, NULL, NULL, list); - if (!handle) - return NULL; - - if (tracecmd_write_cmdlines(handle)) - return NULL; - - if (tracecmd_append_cpu_data(handle, cpus, cpu_data_files) < 0) { - tracecmd_output_close(handle); - return NULL; - } - - return handle; -} - -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files) -{ - return tracecmd_create_file_glob(output_file, cpus, - cpu_data_files, &all_event_list); -} - struct tracecmd_output *tracecmd_create_init_fd(int fd) { return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); From patchwork Mon Jun 14 07:49:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C0FAC48BE8 for ; Mon, 14 Jun 2021 07:50:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68E7B61289 for ; Mon, 14 Jun 2021 07:50:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232499AbhFNHwp (ORCPT ); Mon, 14 Jun 2021 03:52:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232576AbhFNHwo (ORCPT ); Mon, 14 Jun 2021 03:52:44 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC47EC061766 for ; Mon, 14 Jun 2021 00:50:34 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id ba2so43439024edb.2 for ; Mon, 14 Jun 2021 00:50:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ul0cYskXBCYyoanWYB1IZdsjLcqgyRalazjxs5buo98=; b=tdAbbPcrRdlMxz98cVpyKjBaM3XbOK2kAs+DmShjTvdeeS5TnaeV1sNDkiBX13fO5R LLMwihuqhvHrJaFwqVIDfkAZgJscM0Rgd1HppITy2gBxMKdwRE1tiXoHjN73CqeF5BPc yuZR7BFcMdHCEz2chDVU334Z6zpSXxUdfDOhRC7vbEmlWaYhIPw+QWHUm1Jv0F+l5cNK 5FmdqREuq19o/eW7mRWxsZBfBsocrHsm2GuOjzka/xSxm+thrHKpWPXIJEeFfDb3c3b8 TN1MHqi+QJOY7qkd4qKor31aGCZKzAZvA52dkUOFerwtstZwkdax/SIyOVj6KV3PFHqE hwuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ul0cYskXBCYyoanWYB1IZdsjLcqgyRalazjxs5buo98=; b=Y0PxcvLqwjgVbiL8y0SAgR+27sV9ea3cewsfWZMWQzEHZt396Q/BpufH8Urk3X1dzz 0vuiob7IWFfC5eBOlNU3nlMJ/M2wp72Tw9DPUWm8xvQdyoOVwEPcUGUVwrU+7eN9GiCC 0v3HleyjtCBHLqCxwNL3uWXb2u26xT5bLusa2rcZW91Rm9BIowLGuwEMxeH4FNsA61R1 lnjuybtkq1vMK2VgAohy+UEuwnXuIiJl6m34Gd5YY3jdyodiaC7E9iGIWd9em3Xsq52R cGOttCPK9PM1r4Qw2VG0kOmX2Xhl90VJdKXGR8h4kpN/juWx1uhgdTQpj0sfOA+NlIVp j9Cw== X-Gm-Message-State: AOAM530hiL4fk5/8EhyUG9+cS3p2VdVKM9/wIPxEcUL3ZA5J/KXKRfAF Ul/Y+vPB+kT41joTxbPlVRze0a5frRP/DA== X-Google-Smtp-Source: ABdhPJy7wFS/RvAVUEs3nkoIB9WPXMMW18+6/HZi7lnpBwrUQpzBZ0+0vktmu/ago5jDACElZSbF1Q== X-Received: by 2002:a50:bf0f:: with SMTP id f15mr15429669edk.205.1623657033544; Mon, 14 Jun 2021 00:50:33 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:33 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 02/45] trace-cmd library: Remove unused API tracecmd_update_option Date: Mon, 14 Jun 2021 10:49:46 +0300 Message-Id: <20210614075029.598048-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Removed not used trace-cmd library API for updating the options section of already written trace file: tracecmd_update_option() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 -- lib/trace-cmd/trace-output.c | 40 ------------------- 2 files changed, 43 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4bd36baa..49539432 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -290,9 +290,6 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9231c319..c0aa58cf 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1229,46 +1229,6 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data) -{ - tsize_t offset; - stsize_t ret; - - if (size > option->size) { - tracecmd_warning("Can't update option with more data than allocated"); - return -1; - } - - if (handle->file_state < TRACECMD_FILE_OPTIONS) { - /* Hasn't been written yet. Just update current pointer */ - option->size = size; - memcpy(option->data, data, size); - return 0; - } - - /* Save current offset */ - offset = lseek64(handle->fd, 0, SEEK_CUR); - - ret = lseek64(handle->fd, option->offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); - return -1; - } - - if (do_write_check(handle, data, size)) - return -1; - - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offset); - return -1; - } - - return 0; -} - struct tracecmd_option * tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) From patchwork Mon Jun 14 07:49:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2483C48BE6 for ; Mon, 14 Jun 2021 07:50:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A76D61289 for ; Mon, 14 Jun 2021 07:50:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232536AbhFNHwj (ORCPT ); Mon, 14 Jun 2021 03:52:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232499AbhFNHwi (ORCPT ); Mon, 14 Jun 2021 03:52:38 -0400 Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5BFAC061767 for ; Mon, 14 Jun 2021 00:50:35 -0700 (PDT) Received: by mail-ed1-x52a.google.com with SMTP id s6so45187695edu.10 for ; Mon, 14 Jun 2021 00:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RiwZC1yBs8SQ9z1BwUSlU9j5di5yu+Mv55pgJ5/bATY=; b=sGSJoTy0SCqAzTU/GK5DRHpuaMY7B6dtjYBJEZUExFpEGSL+awarkPikl8xYmrGQuh UJKT4LTnzNbjIEu4yggZXWl12FykwnfOyYKWY1wzhtPI3Z+X1fDD+yyi4gjRA5lcm1a9 Bav37EgzvxIDNI+0OPVfrXJ3E80ff/QuSLwHpCCrma1Wccpdx58pcCuEcnL/uyPG7FZj 9/EU0Pa6qmruTKrtYQ7IPClBnBTy6bb6sB2cPujh+t+TGXT43HhOTu4d06gWNYntBD/k oG0WWOH0UT+Z20CnExOUgsmPFKNAF1hDIYyb554q+RxZ1f0xog9Oeh6hClfINLpafL1J A9Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RiwZC1yBs8SQ9z1BwUSlU9j5di5yu+Mv55pgJ5/bATY=; b=Xr8iY0LWvjIXzSnK41QC8NrDi6w64DQ+Lk/ka8NdL6KloF7LWZp5R+AQzNQeTSZa+P y7jhDBKwkXNYt3c/+HNB8e0FW+mPk29HE+v84NZGSiUlQHbjj/uteDHRGmygiUm7VOhB 6LSs1HA+HkI4mFFWo+UIIoMj6+KIP1O9/kqVSi6t82dFjDb1So2Jma+xetO4e8Ua7FVj cW/PZvcmItuJNR7C7XU60DmFkAKnozhZPEej0pRjo/JadP0CZlwOJtfBwAYisZzI4X4m J8CVboXpzFF0085rI3LNaABXjgPMNS+AQAg9l+Wfu8xCJc0Cr7l7bNTwar/6xDNSvkh5 nF9A== X-Gm-Message-State: AOAM532UturMiHKJMxIJBQEh9C6jyGMJ6RJyR5bfxWep0gFOmhfZXOje EtzgplYgP70nmN0z/1jqLEeP8BtKqK3zMQ== X-Google-Smtp-Source: ABdhPJyorSvqLD7IggN+XFxWwGwgPBEL1PrGp4bzcHMTkBFfFEEA1cFrBsaoIN7oQsApEcgRzZB2/g== X-Received: by 2002:aa7:d785:: with SMTP id s5mr2115550edq.19.1623657034423; Mon, 14 Jun 2021 00:50:34 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:34 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 03/45] trace-cmd: Check if file version is supported Date: Mon, 14 Jun 2021 10:49:47 +0300 Message-Id: <20210614075029.598048-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace file, version of the file is ignored. This could case problems when bumping the version number because of changes in in the structure of the file. The old code should detect unsupported file version and should not try to read it. A new trace-cmd library API is added to check if version is supported: tracecmd_is_version_supported() Checks are added in the code to ensure not trying to read trace file from unsupported version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 10 ++++++++++ lib/trace-cmd/trace-util.c | 7 +++++++ tracecmd/trace-dump.c | 7 +++++++ 4 files changed, 26 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 49539432..6fc18938 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -42,6 +42,8 @@ void tracecmd_record_ref(struct tep_record *record); void tracecmd_set_debug(bool set_debug); bool tracecmd_get_debug(void); +bool tracecmd_is_version_supported(unsigned int version); + struct tracecmd_output; struct tracecmd_recorder; struct hook_list; diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 5ee69b14..97ad0a5d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -131,6 +131,7 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + int file_version; struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -3269,6 +3270,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) unsigned int page_size; char *version; char buf[BUFSIZ]; + unsigned long ver; handle = malloc(sizeof(*handle)); if (!handle) @@ -3293,6 +3295,14 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (!version) goto failed_read; tracecmd_info("version = %s\n", version); + ver = strtol(version, NULL, 10); + if (!ver && errno) + goto failed_read; + if (!tracecmd_is_version_supported(ver)) { + tracecmd_warning("Unsupported file version %lu", ver); + goto failed_read; + } + handle->file_version = ver; free(version); if (do_read_check(handle, buf, 1)) diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b65f9dec..b0c98c72 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -617,3 +617,10 @@ unsigned long long tracecmd_generate_traceid(void) free(str); return hash; } + +bool tracecmd_is_version_supported(unsigned int version) +{ + if (version <= FILE_VERSION) + return true; + return false; +} diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 98425b98..03cc82b4 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "trace-local.h" @@ -145,6 +146,7 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; + unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -166,6 +168,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); + ver = strtol(buf, NULL, 10); + if (!ver && errno) + die("Invalid file version string %s", buf); + if (!tracecmd_is_version_supported(ver)) + die("Unsupported file version %lu", ver); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) From patchwork Mon Jun 14 07:49:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 324F0C49360 for ; Mon, 14 Jun 2021 07:50:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 156F261352 for ; Mon, 14 Jun 2021 07:50:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232524AbhFNHwj (ORCPT ); Mon, 14 Jun 2021 03:52:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232499AbhFNHwj (ORCPT ); Mon, 14 Jun 2021 03:52:39 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF5BCC061767 for ; Mon, 14 Jun 2021 00:50:36 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id ce15so15022963ejb.4 for ; Mon, 14 Jun 2021 00:50:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K/wi6U7hzPtl4la0xaLhqgtGsmlwcZ2Gfg6Lcb4Uo4I=; b=MCXDc4hyPCmO7+2hpgCdHowZwmARcJNrHYomUI1iXwuEKJPVgBZW0JyIRaeZM9sTZr xuEAUdgTUHlcBSoy1S3p0/8mm5O+UM/iMlxqzdoJGVoQB5lp7NoDi2lwspJfemIfXFA+ yyGyPPeUXWse42+SOyg4olljS6LDs9mJ8pLz5o4V7pljEPE6v2S4VpWWRfmLKxodXqSe xV79LTHAP+dYZsWQap+BbR1dg/Jkl2ZGDzshKgDr/MeHv1UqUey16kPtoasYIY//neR7 H+uuVrJHxbYpaaLAS5nbi6XpbNMlvRHLGw30bY/vDCpiTpZJ9jQetvNZS1MKuYCiKq54 WlPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K/wi6U7hzPtl4la0xaLhqgtGsmlwcZ2Gfg6Lcb4Uo4I=; b=QqGSLcRc0l59WVvf92FtpQFi7dlvgzFB5HeIiUkz1Q2E+JGCFKaUzrzce4AE0YI1oC 775ZIj6JWBU9MIueTnuU4wOI1vHUYKVWUmwBac9A/F2dLM4lx0ntgOT6IG3+Ab2mq6W2 CHWiMW+J6nO7hYsitnVSkzikCg3D5T026O7pxOxy3diagMksSh8gQhRo91muUDxHHgb9 vRsRbxGwNAumPnIdaJ1mX4PHnEpB9le8nCMlsXoDNqwYPaFdQj9DE6L4vaapjGsQcNsd PTTMtUjb7YMocs1ISnucwVkZHyE0Vo7EfM42LaXaR5ON7Th1br15hwlW7KKVIAR5oODF JWBg== X-Gm-Message-State: AOAM533l1jaIZx9IUe4JribhVuqkYOsrj32oCUQ8+H+V7CLOceRrOPWP h+Dr1mZHHR5DvnOZTyEY5LJQXIjsHxLpuw== X-Google-Smtp-Source: ABdhPJwmxeFFKrPZLnh8xhfXca/5CfBAtrWrDUJFFnpf7BoUwHvkjxik24hX1rgOsOrIM0i6QR+xHQ== X-Received: by 2002:a17:906:5488:: with SMTP id r8mr14260199ejo.374.1623657035620; Mon, 14 Jun 2021 00:50:35 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:35 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 04/45] trace-cmd library: Add new API to get file version of input handler Date: Mon, 14 Jun 2021 10:49:48 +0300 Message-Id: <20210614075029.598048-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added an API to get the version of the trace file, associated with given input file handler. tracecmd_get_in_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 6fc18938..01b12c47 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -232,6 +232,8 @@ int tracecmd_set_cursor(struct tracecmd_input *handle, unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); +unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); + int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); tracecmd_show_data_func diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 97ad0a5d..46619ac7 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4046,6 +4046,15 @@ struct tep_handle *tracecmd_get_tep(struct tracecmd_input *handle) return handle->pevent; } +/** + * tracecmd_get_in_file_version - return the trace.dat file version + * @handle: input handle for the trace.dat file + */ +unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle) +{ + return handle->file_version; +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Mon Jun 14 07:49:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE86BC48BE6 for ; Mon, 14 Jun 2021 07:51:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F30861370 for ; Mon, 14 Jun 2021 07:51:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232604AbhFNHxk (ORCPT ); Mon, 14 Jun 2021 03:53:40 -0400 Received: from mail-ed1-f50.google.com ([209.85.208.50]:43888 "EHLO mail-ed1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232606AbhFNHxj (ORCPT ); Mon, 14 Jun 2021 03:53:39 -0400 Received: by mail-ed1-f50.google.com with SMTP id s6so45187810edu.10 for ; Mon, 14 Jun 2021 00:51:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sP2SzL+jTkso72skt+AgZYS5f7e9OZMAJYuOulQ00BI=; b=S4haf12kr3TO4abFGlBUhYOhk5asaVXq8a7Eaq6oJziRN4BIcqXz0CU7MotyTNJBZq TGJzbFqPDpju/ZKGXLm57W0764ro5edVXMsYK/oa+98bZs56hX/VdDEqLLSOTLHAWzjd s0STcfUbf5WrRysIJVSI4Fw8mWuTsjF5dEMckkiPZo7y4V+Gvn7SHDtZEszyncO8g5AW ht4G1plEsgN7SWzRE6MMp0T/N2V2xUETscMwvNoz6cb032ALNuNTBjidf6zCC4Fdj+zg ThpTRrbJ5LveeWu0UArjy13XOHn3vrhBf+Wva8whFx0I33Y1qG82pL5uJbJfCCnBh0dc qj7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sP2SzL+jTkso72skt+AgZYS5f7e9OZMAJYuOulQ00BI=; b=Z1phzeqrAhk5by7eWDRJPF/jF7t47VIT+2CTt9dHuAp8ZAxsYHJ3p2auOhYVOeBO42 Ky9lRbTCyPW8B9XYihC8/hWEYiOcSz+USXfNs1jdtyD7K+/yWOQd+M6JbcYt+AgsKL2b YOFsoui7mV1pINsngZz/4GsYFHhE6vWu9y4AHklSX+mBXtDUz+49oCcCvQwcRNBipK6p wUelCMZsSPoPTLUzRUVRMHqO5+vWn3cB97OzIZTpgtbOATul3t22o4vK6Yefd/l8Q+Pd 85ISyDkjTmsQ2Jz83Qx1CeoEWZct+6zj+gMBzVhisjXtXQCZTtvpQFSSivG8Xkz40lwg Pydg== X-Gm-Message-State: AOAM5308h3z3C7yjJJ089ftTYczQl3Rq/eXPSh7jB3UHfERgzz9gcMl9 hdAP7tzlHxMDVpChFHAG+EY= X-Google-Smtp-Source: ABdhPJxzeksHSJv9Uu7/s5Bx2mDE4Ki/6oRXy2UK1ATKlubPyA4ZeimlIoTp56EsuDiJBrrjZQ7rig== X-Received: by 2002:a05:6402:31f3:: with SMTP id dy19mr15541436edb.153.1623657036609; Mon, 14 Jun 2021 00:50:36 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:36 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 05/45] trace-cmd library: Select the file version when writing trace file Date: Mon, 14 Jun 2021 10:49:49 +0300 Message-Id: <20210614075029.598048-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new output handler to trace file is allocated, select the proper file version. If this output handler is based on an existing input trace file handler, inherit the trace file version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c0aa58cf..bacda23b 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -57,6 +57,7 @@ struct tracecmd_output { int nr_options; bool quiet; unsigned long file_state; + unsigned long file_version; struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -907,6 +908,17 @@ out_free: return ret; } +static int select_file_version(struct tracecmd_output *handle, + struct tracecmd_input *ihandle) +{ + if (ihandle) + handle->file_version = tracecmd_get_in_file_version(ihandle); + else + handle->file_version = FILE_VERSION; + + return 0; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -933,6 +945,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; + if (select_file_version(handle, ihandle)) + goto out_free; + list_head_init(&handle->options); buf[0] = 23; @@ -943,7 +958,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, if (do_write_check(handle, buf, 10)) goto out_free; - if (do_write_check(handle, FILE_VERSION_STRING, strlen(FILE_VERSION_STRING) + 1)) + sprintf(buf, "%lu", handle->file_version); + if (do_write_check(handle, buf, strlen(buf) + 1)) goto out_free; /* get endian and page size */ @@ -1562,6 +1578,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->pevent = tracecmd_get_tep(ihandle); tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); + handle->file_version = tracecmd_get_in_file_version(ihandle); list_head_init(&handle->options); tracecmd_close(ihandle); From patchwork Mon Jun 14 07:49:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5C5EC48BE6 for ; Mon, 14 Jun 2021 07:50:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 88EF161352 for ; Mon, 14 Jun 2021 07:50:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232552AbhFNHwm (ORCPT ); Mon, 14 Jun 2021 03:52:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232499AbhFNHwm (ORCPT ); Mon, 14 Jun 2021 03:52:42 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA696C061767 for ; Mon, 14 Jun 2021 00:50:39 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id c10so15025158eja.11 for ; Mon, 14 Jun 2021 00:50:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2G0CILf23KT9a1hgIdunlhYcwh2NDqtDLL7z1ixvlIw=; b=BgszJMdOXI9TWHx6C+RFfz5ANOBl59lke5w+pxtbG6aPZazrH34xbSsVUf7A7tTQCF BdUvgCYnXzzq2dlS/2PN7CfaSP2ZIJY9ryd5qdXwTHgRM5x6gXLaiXzAwMRe3itYApst 08VH21/YhNNIr/JUTmpS5RYL0ewDlIghR0AiZgV/42PhFARSCpbbonRoMs5xs+6pPsPz ED5Z37XjQkXWY+u8N3Tvhs03xXxEchdCe80CYYJyIrMogYd/dzKQqEVcUeSIoP1ftLy/ k6JWuwWf4RQR/bapzHj+qxrxnBlfqc+CYPo+nHsJDetmcGpJa0RY/XdIBkdIfrYKdKsj xZDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2G0CILf23KT9a1hgIdunlhYcwh2NDqtDLL7z1ixvlIw=; b=P/3oULEr1fPXMG+J9/OwRT2V/ABmws70GyAIotuvttlcXSBmfF/slfXMA1LkIUk3zP 3hA8oznPAKE9cf9Lt7X7nQ9NaUaGFbgNYxVktCyo3b0fDiZeqvEiNSlGQ29bX0LwWINd Dkw85Hh2avWAKJfz8v55PQxuSQpbIUS9w6w7zDkM7ExoHhbcSWwCxzHMT54Is0piDoIy aQHWpliNBuBM5IQMDrvfFHx3YbopAeCIXUy5T0tR5VSjVRF4PuZo8ry9BfFSjNt9Nh8L E05zLxGQ40LfSwdkeAIifriDFQos+eAgz4YtaNlDRugPXtFL3AirCOVORQHw3444p0Sr nvfQ== X-Gm-Message-State: AOAM53297tn0Xr9EG77dL++ZyJBtNcmNqb12bc2FUl5C6TLj6nFc+DfW nZxF/GfPz0to1i7P7SO3PdI= X-Google-Smtp-Source: ABdhPJw0t7h9Tsiw3Vx+wKJbKpVhA4XA3bmxtlxugKd66T8DxWVNnmwtTRKssXG/3ersxYAif+o2cA== X-Received: by 2002:a17:906:3888:: with SMTP id q8mr13969007ejd.15.1623657037751; Mon, 14 Jun 2021 00:50:37 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:37 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 06/45] trace-cmd: Add APIs for library initialization and free Date: Mon, 14 Jun 2021 10:49:50 +0300 Message-Id: <20210614075029.598048-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The trace-cmd library has no APIs for initialization and free of the whole library. Added these new APIs: tracecmd_lib_init() tracecmd_lib_free() and call them in trace-cmd main function. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 3 +++ lib/trace-cmd/trace-util.c | 9 +++++++++ tracecmd/trace-cmd.c | 11 ++++++++--- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 01b12c47..be8b3c48 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -29,6 +29,9 @@ struct tep_plugin_list *trace_load_plugins(struct tep_handle *tep, int flags); +int tracecmd_lib_init(void); +void tracecmd_lib_free(void); + int *tracecmd_add_id(int *list, int id, int len); enum { diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b0c98c72..61054ad2 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -624,3 +624,12 @@ bool tracecmd_is_version_supported(unsigned int version) return true; return false; } + +int tracecmd_lib_init(void) +{ + return 0; +} + +void tracecmd_lib_free(void) +{ +} diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 00cdaa37..71c8f6d6 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -142,15 +142,20 @@ int main (int argc, char **argv) if (argc < 2) trace_usage(argc, argv); + tracecmd_lib_init(); + for (i = 0; i < ARRAY_SIZE(commands); ++i) { if (strcmp(argv[1], commands[i].name) == 0 ){ commands[i].run(argc, argv); - goto out; + break; } } + tracecmd_lib_free(); + /* No valid command found, show help */ - trace_usage(argc, argv); -out: + if (i == ARRAY_SIZE(commands)) + trace_usage(argc, argv); + exit(0); } From patchwork Mon Jun 14 07:49:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 581A4C49360 for ; Mon, 14 Jun 2021 07:51:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C28E61352 for ; Mon, 14 Jun 2021 07:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232614AbhFNHx4 (ORCPT ); Mon, 14 Jun 2021 03:53:56 -0400 Received: from mail-ed1-f54.google.com ([209.85.208.54]:35694 "EHLO mail-ed1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232608AbhFNHxz (ORCPT ); Mon, 14 Jun 2021 03:53:55 -0400 Received: by mail-ed1-f54.google.com with SMTP id ba2so43439333edb.2 for ; Mon, 14 Jun 2021 00:51:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aDJnSem5m9IjjShjp7Lc+1Wlm1Vn0fuQSvGLvISDyCo=; b=LyZFIeoW7pDPe10EDwEgGAyw/jyYlP/knNh5pThmVI5BWt/MSs9APeciwGZzRyeLnB eNAUdCXNG5eQ5EQU36qTsEaKvp29lRKuD7IjaYDZblGauPbHf3ajh4OAHGP5oqb0Fm6H KApc8osgL/DVUVet9DLmQFUcO1Bkngvo9lg8oBrTHbqyPJQPGU4956Eh6UHkxhMSLync ZubtW6AvAhJeFhHo47r1yTflPvWgnXtWaRDqeVsCg0UNcPKntWP6cad3Pc7HLHmZWY1M FyiMGVDnbNr/u/Cm4r1cE0R2sjZS+sEjU35fglnPJ2o8d4glkeN8LPkeOY6T8ml9KPe1 dzTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aDJnSem5m9IjjShjp7Lc+1Wlm1Vn0fuQSvGLvISDyCo=; b=df03ccdBn9OBhNo5ivr+5Q4SQVU72va+lkumWzy+ZUfQ8XdiWw6bG9w1xMi6hy8VkN zWDDCRtkiuv6U0dX2/Oz6h1Du5s8RcX+6miDhomoPUoDlgCMfP81I7vUB75PGoaUUll1 eoKopNN4MrEYF+jFm67C00oXGtSHXnwUAAI1Lxl7aG9wgsnfnE2bFkJYIX/Jbt34SnJA 65Alh5cdJhXnVw0rcqcFY8XmHnkHx9wL6o1XUe5q5R7p+XeAjhGLvdSsJoFe5YwApnZi d4nz9AT4qsRJINhD67z617q0gXLP3/hE6GbE9fuHOM2m+v8+TQ4y95fbemPlXUxBD3jY 8keA== X-Gm-Message-State: AOAM5319AA34VsGjk2CRINiEu8xabMOPmusq2MKRW5oJOwB4A89rj3RQ zIE4AiSxnyxUvQVzCyWiVfPxES2fZ0NRPg== X-Google-Smtp-Source: ABdhPJz5w6J5lbactlVa8LFRVgUsnzpmZrNIQQpCB7ztK+Qj6QNiliqmeO3vo6T+feRdiRHzVHHlDQ== X-Received: by 2002:a05:6402:1a:: with SMTP id d26mr15820729edu.105.1623657038907; Mon, 14 Jun 2021 00:50:38 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:38 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 07/45] trace-cmd library: Add support for compression algorithms Date: Mon, 14 Jun 2021 10:49:51 +0300 Message-Id: <20210614075029.598048-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added infrastructure to trace-cmd library for compression. Introduced various new APIs to work with this new functionality: struct tracecmd_compression tracecmd_compress_init() tracecmd_compress_free() tracecmd_compress_alloc() tracecmd_compress_destroy() tracecmd_compress_block() tracecmd_uncompress_block() tracecmd_compress_reset() tracecmd_compress_read() tracecmd_compress_pread() tracecmd_compress_write() tracecmd_compress_lseek() tracecmd_compress_proto_get_name() tracecmd_compress_is_supported() tracecmd_compress_protos_get() tracecmd_compress_proto_register() The compression algorithms are not part of this patch. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 1 + .../include/private/trace-cmd-private.h | 26 + lib/trace-cmd/include/trace-cmd-local.h | 2 + lib/trace-cmd/trace-compress.c | 589 ++++++++++++++++++ lib/trace-cmd/trace-util.c | 3 + 5 files changed, 621 insertions(+) create mode 100644 lib/trace-cmd/trace-compress.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index 17600318..bab4322d 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -25,6 +25,7 @@ ifeq ($(VSOCK_DEFINED), 1) OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif +OBJS += trace-compress.o # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index be8b3c48..ec534d51 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -464,6 +464,32 @@ void tracecmd_tsync_free(struct tracecmd_time_sync *tsync); int tracecmd_write_guest_time_shift(struct tracecmd_output *handle, struct tracecmd_time_sync *tsync); +/* --- Compression --- */ +struct tracecmd_compression; +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle); +void tracecmd_compress_destroy(struct tracecmd_compression *handle); +int tracecmd_compress_block(struct tracecmd_compression *handle); +int tracecmd_uncompress_block(struct tracecmd_compression *handle); +void tracecmd_compress_reset(struct tracecmd_compression *handle); +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len); +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset); +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size); +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence); +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version); +bool tracecmd_compress_is_supported(const char *name, const char *version); +int tracecmd_compress_protos_get(char ***names, char ***versions); +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)); + /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 821b5cdb..c6591e56 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,5 +31,7 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +void tracecmd_compress_init(void); +void tracecmd_compress_free(void); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c new file mode 100644 index 00000000..378a3e57 --- /dev/null +++ b/lib/trace-cmd/trace-compress.c @@ -0,0 +1,589 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include +#include + +#include "trace-cmd-private.h" +#include "trace-cmd-local.h" + +struct compress_proto { + struct compress_proto *next; + char *proto_name; + char *proto_version; + int weight; + + int (*compress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + int (*uncompress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + unsigned int (*compress_size)(unsigned int bytes); + bool (*is_supported)(const char *name, const char *version); +}; + +static struct compress_proto *proto_list; + +struct tracecmd_compression { + int fd; + unsigned int capacity; + unsigned long pointer; + char *buffer; + struct compress_proto *proto; + struct tep_handle *tep; + struct tracecmd_msg_handle *msg_handle; +}; + +static int read_fd(int fd, char *dst, int len) +{ + size_t size = 0; + int r; + + do { + r = read(fd, dst+size, len); + if (r > 0) { + size += r; + len -= r; + } else + break; + } while (r > 0); + + if (len) + return -1; + return size; +} + +static long long write_fd(int fd, const void *data, size_t size) +{ + long long tot = 0; + long long w; + + do { + w = write(fd, data + tot, size - tot); + tot += w; + + if (!w) + break; + if (w < 0) + return w; + } while (tot != size); + + return tot; +} + +static long long do_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + int ret; + + if (handle->msg_handle) { + ret = tracecmd_msg_data_send(handle->msg_handle, data, size); + if (ret) + return -1; + return size; + } + return write_fd(handle->fd, data, size); +} + +/** + * tracecmd_compress_lseek - Move the read/write pointer into the compression buffer + * @handle: compression handler + * @offset: number of bytes to move the pointer, can be negative or positive + * @whence: the starting position of the pointer movement, + * + * Returns the new file pointer on success, or -1 in case of an error. + */ +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence) +{ + unsigned long p, extend; + char *buf; + + if (!handle || !handle->buffer) + return (off_t)-1; + + switch (whence) { + case SEEK_CUR: + p = handle->pointer + offset; + break; + case SEEK_END: + p = handle->capacity + offset; + break; + case SEEK_SET: + p = offset; + break; + default: + return (off_t)-1; + } + + if (p <= handle->capacity) { + handle->pointer = p; + } else { + extend = p - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return (off_t)-1; + handle->buffer = buf; + handle->capacity += extend; + handle->pointer = p; + } + + return p; +} + +static int compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int s; + + if (handle->pointer + len > handle->capacity) + s = handle->capacity - handle->pointer; + else + s = len; + memcpy(dst, handle->buffer + handle->pointer, s); + + return s; +} + +/** + * tracecmd_compress_pread - pread() on compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * @offset: offset in the buffer of data to be read + * + * Read a @len of data from the compression buffer at given @offset, + * without updating the buffer pointer. + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset) +{ + int ret; + + if (!handle || !handle->buffer || offset > handle->capacity) + return -1; + + ret = tracecmd_compress_lseek(handle, offset, SEEK_SET); + if (ret < 0) + return ret; + return compress_read(handle, dst, len); +} + +/** + * tracecmd_compress_read - read() from compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * + * Read a @len of data from the compression buffer + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int ret; + + if (!handle || !handle->buffer) + return -1; + + ret = compress_read(handle, dst, len); + if (ret > 0) + handle->pointer += ret; + + return ret; +} + +/** + * tracecmd_compress_reset - Reset the compression buffer + * @handle: compression handler + * + * Reset the compression buffer, any data currently in the buffer will be destroyed. + * + */ +void tracecmd_compress_reset(struct tracecmd_compression *handle) +{ + if (!handle) + return; + + free(handle->buffer); + handle->buffer = NULL; + handle->pointer = 0; + handle->capacity = 0; +} + +/** + * tracecmd_uncompress_block - uncompress a memory block + * @handle: compression handler + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_uncompress_block(struct tracecmd_compression *handle) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + char *bytes = NULL; + char buf[4]; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + tracecmd_compress_reset(handle); + + if (read(handle->fd, buf, 4) != 4) + return -1; + s_compressed = tep_read_number(handle->tep, buf, 4); + if (read(handle->fd, buf, 4) != 4) + return -1; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + + handle->buffer = malloc(s_uncompressed); + if (!handle->buffer) + return -1; + bytes = malloc(s_compressed); + if (!bytes) + goto error; + + if (read_fd(handle->fd, bytes, s_compressed) < 0) + goto error; + ret = handle->proto->uncompress_block(bytes, s_compressed, + handle->buffer, &s_uncompressed); + if (ret) + goto error; + free(bytes); + handle->pointer = 0; + handle->capacity = s_uncompressed; + return 0; +error: + tracecmd_compress_reset(handle); + free(bytes); + return -1; +} + +/** + * tracecmd_compress_block - compress a memory block + * @handle: compression handler + * + * Compress the content of the internal memory buffer and write the compressed data in the file + * The tracecmd_compress_write() can be used to write data into the internal memory buffer, before + * calling this API. + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_compress_block(struct tracecmd_compression *handle) +{ + unsigned int size; + char *buf; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->compress_size || !handle->proto->compress_block) + return -1; + + size = handle->proto->compress_size(handle->pointer); + buf = malloc(size); + if (!buf) + return -1; + ret = handle->proto->compress_block(handle->buffer, handle->pointer, buf, &size); + if (ret < 0) + goto out; + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &handle->pointer, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write compressed data */ + ret = do_write(handle, buf, size); + ret = ((ret == size) ? 0 : -1); +out: + tracecmd_compress_reset(handle); + free(buf); + return ret; +} + +/** + * tracecmd_compress_write - write() to compression buffer + * @handle: compression handler + * @data: data to be written + * @size: size of @data + * + * Write @data of @size in the compression buffer + * + * Returns 0 on success, or -1 on failure. + */ +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + char *buf; + int extend; + + if (!handle) + return -1; + + if (handle->capacity < handle->pointer + size) { + extend = (handle->pointer + size) - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return -1; + handle->buffer = buf; + handle->capacity += extend; + } + memcpy(&handle->buffer[handle->pointer], data, size); + handle->pointer += size; + return 0; +} + +/** + * tracecmd_compress_init - initialize the library with available compression algorithms + */ +void tracecmd_compress_init(void) +{ + struct timeval time; + + gettimeofday(&time, NULL); + srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); +} + +static struct compress_proto *compress_proto_select(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *selected = NULL; + + while (proto) { + if (!selected || selected->weight > proto->weight) + selected = proto; + proto = proto->next; + } + + return selected; +} + +/** + * tracecmd_compress_alloc - Allocate a new compression context + * @name: name of the compression algorithm, if NULL - auto select the best available algorithm + * @version: version of the compression algorithm, can be NULL + * @fd: file descriptor for reading / writing data + * @tep: tep handler, used to encode the data + * @msg_handle: message handler, use it for reading / writing data instead of @fd + * + * Returns NULL on failure or pointer to allocated compression context. + * The returned context must be freed by tracecmd_compress_destroy() + */ +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle) +{ + struct tracecmd_compression *new; + struct compress_proto *proto; + + if (name) { + proto = proto_list; + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + break; + proto = proto->next; + } + } else { + proto = compress_proto_select(); + } + if (!proto) + return NULL; + + new = calloc(1, sizeof(*new)); + if (!new) + return NULL; + new->fd = fd; + new->tep = tep; + new->msg_handle = msg_handle; + new->proto = proto; + return new; +} + +/** + * tracecmd_compress_destroy - Free a compression context + * @handle: handle to the compression context that will be freed + */ +void tracecmd_compress_destroy(struct tracecmd_compression *handle) +{ + tracecmd_compress_reset(handle); + free(handle); +} + +/** + * tracecmd_compress_is_supported - check if compression algorithm with given name and + * version is supported + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * + * Returns true if the algorithm with given name and version is supported or false if it is not. + */ +bool tracecmd_compress_is_supported(const char *name, const char *version) +{ + struct compress_proto *proto = proto_list; + + if (!name) + return NULL; + + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + return true; + proto = proto->next; + } + return false; +} + +/** + * tracecmd_compress_proto_get_name - get name and version of compression algorithm + * @compress: compression handler. + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version) +{ + if (!compress || !compress->proto) + return -1; + if (name) + *name = compress->proto->proto_name; + if (version) + *version = compress->proto->proto_version; + return 0; +} + +/** + * tracecmd_compress_proto_register - register a new compression algorithm + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * @weight: weight of the compression algorithm, lower is better. + * @compress: compression hook, called to compress a memory block. + * @uncompress: uncompression hook, called to uncompress a memory block. + * @compress_size: hook, called to get the required minimum size of the buffer for compression + * given number of bytes. + * @is_supported: check hook, called to check if compression with given name and version is + * supported by this plugin. + * + * Returns 0 on success, or -1 in case of an error. If algorithm with given name and version is + * already registered, -1 is returned. + */ +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*compress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)) +{ + struct compress_proto *new; + + if (!name || !compress || !uncompress) + return -1; + if (tracecmd_compress_is_supported(name, version)) + return -1; + + new = calloc(1, sizeof(*new)); + if (!new) + return -1; + + new->proto_name = strdup(name); + if (!new->proto_name) + goto error; + new->proto_version = strdup(version); + if (!new->proto_version) + goto error; + new->compress_block = compress; + new->uncompress_block = uncompress; + new->compress_size = compress_size; + new->is_supported = is_supported; + new->weight = weight; + new->next = proto_list; + proto_list = new; + return 0; + +error: + free(new->proto_name); + free(new->proto_version); + free(new); + return -1; +} + +/** + * tracecmd_compress_free - free the library resources, related to available compression algorithms + * + */ +void tracecmd_compress_free(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *del; + + while (proto) { + del = proto; + proto = proto->next; + free(del->proto_name); + free(del->proto_version); + free(del); + } + proto_list = NULL; +} + +/** + * tracecmd_compress_protos_get - get a list of all supported compression algorithms and versions + * @names: return, array with names of all supported compression algorithms + * @versions: return, array with versions of all supported compression algorithms + * + * On success, the size of @names and @versions arrays is returned. Those arrays are allocated by + * the API and must be freed with free() by the caller. Both arrays are with same size, each name + * from @names corresponds to a version from @versions. + * On error -1 is returned and @names and @versions arrays are not allocated. + */ +int tracecmd_compress_protos_get(char ***names, char ***versions) +{ + struct compress_proto *proto = proto_list; + char **n = NULL; + char **v = NULL; + int c, i; + + for (c = 0; proto; proto = proto->next) + c++; + + if (c < 1) + return c; + + n = calloc(c, sizeof(char *)); + if (!n) + goto error; + v = calloc(c, sizeof(char *)); + if (!v) + goto error; + + proto = proto_list; + for (i = 0; i < c && proto; i++) { + n[i] = proto->proto_name; + v[i] = proto->proto_version; + proto = proto->next; + } + + *names = n; + *versions = v; + return c; + +error: + free(n); + free(v); + return -1; +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 61054ad2..136260c6 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -627,9 +627,12 @@ bool tracecmd_is_version_supported(unsigned int version) int tracecmd_lib_init(void) { + + tracecmd_compress_init(); return 0; } void tracecmd_lib_free(void) { + tracecmd_compress_free(); } From patchwork Mon Jun 14 07:49:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A03AC49360 for ; Mon, 14 Jun 2021 07:50:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E2FC61352 for ; Mon, 14 Jun 2021 07:50:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232562AbhFNHwo (ORCPT ); Mon, 14 Jun 2021 03:52:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232499AbhFNHwn (ORCPT ); Mon, 14 Jun 2021 03:52:43 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D4B9C061767 for ; Mon, 14 Jun 2021 00:50:41 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id ce15so15023277ejb.4 for ; Mon, 14 Jun 2021 00:50:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3KFLzzFKc9m2UFTOpsVh7cbeeh2hvHNTp1bJlNMgn90=; b=E2dFiMh6DNYbyVIK563rxIvBJU7uJtlOIKAkFi5brRsyrpeZadcVvOb24HtBqgpEW6 fkd9sBgdYlzFk8m0uIb77XhelAwYJ+DuXf8qbqLYSQFOzs9eBlPyXPf6aUXZG2SyVPYI tMImgyrbM+Ir8BDk7CAbDa+gnXqqOaGOM2YwyGkhrwz5AMD1Cm6zzsjzbxcokOvzau3T Koourv1Cx0vvWyYxAt886LjA0REhLqSraKk9S1RDTO6NOeYo1L6ZHVJIhRs2e2NaAsyf QfRv+sVrPNId3A4iWX3EDlG4IqnPvYkWzxaLD9tqyHdU9indKOBanQ8FHhQApU4Vn8IW 7nHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3KFLzzFKc9m2UFTOpsVh7cbeeh2hvHNTp1bJlNMgn90=; b=lep6P4uoBI3tQXAxczoJEqQ1385LSf0xmGv7OS6HOUe69l6IOBF8Xyi2GUMmMgEEyH 7djXirtCFGpN/kSukBSPdD1T8JGbaVkUYny4+2fXLiIG66teA75UfqLq4g/sHeOUS25D pVLeCAbx1yQ6id1qsdEFC22dQ5k7n+H/PS10YmEWVNqa8AZrOgMBkWlfv9pCeql8T6n7 ZCgc6t5facOcEqIWBxGwV/y4IS3IB/IrfQsNEJN5E41aeNvfJz4+WlkjVNxOevEeoceY BfPg/a7vg2Tp9zu3RoH1Jj0VqNCfdJjmItrIdcP3cSk4tqktfCLCCZ5DDaTA6/S39djw XsEA== X-Gm-Message-State: AOAM533Nm5x+DBJ7iwG4ouFhRPDIUfzgaouWhMqXqgkw6ii2/l9dq2SA XDVzzuX6oZj3qQ51gYfGd5aUnF2Eo74kxQ== X-Google-Smtp-Source: ABdhPJzThOnSkCcGBbKmjEVOHRBhOGQkjZtJLtJ7GJw6kZWpaJ4L2t2/XT5tDGq163jxBip40J5FDA== X-Received: by 2002:a17:906:6dc3:: with SMTP id j3mr14265692ejt.448.1623657039817; Mon, 14 Jun 2021 00:50:39 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:39 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 08/45] trace-cmd list: Show supported compression algorithms Date: Mon, 14 Jun 2021 10:49:52 +0300 Message-Id: <20210614075029.598048-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add new parameter "trace-cmd list -c" to show supported compression algorithms. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-list.c | 26 ++++++++++++++++++++++++++ tracecmd/trace-usage.c | 1 + 2 files changed, 27 insertions(+) diff --git a/tracecmd/trace-list.c b/tracecmd/trace-list.c index d060c810..900da73b 100644 --- a/tracecmd/trace-list.c +++ b/tracecmd/trace-list.c @@ -549,6 +549,24 @@ static void show_plugins(void) tep_free(pevent); } +static void show_compression(void) +{ + char **versions, **names; + int c, i; + + c = tracecmd_compress_protos_get(&names, &versions); + if (c <= 0) { + printf("No compression algorithms are supported\n"); + return; + } + printf("Supported compression algorithms:\n"); + for (i = 0; i < c; i++) + printf("\t%s, %s\n", names[i], versions[i]); + + free(names); + free(versions); +} + void trace_list(int argc, char **argv) { int events = 0; @@ -562,6 +580,7 @@ void trace_list(int argc, char **argv) int flags = 0; int systems = 0; int show_all = 1; + int compression = 0; int i; const char *arg; const char *funcre = NULL; @@ -626,6 +645,10 @@ void trace_list(int argc, char **argv) systems = 1; show_all = 0; break; + case 'c': + compression = 1; + show_all = 0; + break; case '-': if (strcmp(argv[i], "--debug") == 0) { tracecmd_set_debug(true); @@ -670,6 +693,8 @@ void trace_list(int argc, char **argv) show_clocks(); if (systems) show_systems(); + if (compression) + show_compression(); if (show_all) { printf("event systems:\n"); show_systems(); @@ -679,6 +704,7 @@ void trace_list(int argc, char **argv) show_tracers(); printf("\noptions:\n"); show_options(); + show_compression(); } return; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index c0033ce8..cd8ca391 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -331,6 +331,7 @@ static struct usage_help usage_help[] = { " -O list plugin options\n" " -B list defined buffer instances\n" " -C list the defined clocks (and active one)\n" + " -c list the supported trace file compression algorithms\n" }, { "restore", From patchwork Mon Jun 14 07:49:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33D37C49361 for ; Mon, 14 Jun 2021 07:50:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 180AA61370 for ; Mon, 14 Jun 2021 07:50:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232544AbhFNHxA (ORCPT ); Mon, 14 Jun 2021 03:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232579AbhFNHxA (ORCPT ); Mon, 14 Jun 2021 03:53:00 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE085C061574 for ; Mon, 14 Jun 2021 00:50:42 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id k7so15031934ejv.12 for ; Mon, 14 Jun 2021 00:50:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k3qTP15/KW1GvymOMvtOciY6oQlool890KwlGyG99eE=; b=U7H7LeN/p12pXhVhOKgXIX8erBu6M5o2ezNkPK1VwqNZu0Jf2QqgFJVHh+UEFzgEvD w9l+R5l7/7TOo+hRct+nJDDNcBeax8rDqvmo3iEoNArg2uiS+mVQWhgmypFZXssZRF1v iqLJCN5NDXQOcoz0Ir1aewy+zLczJxpqLryZXlzuqs0PokegmeyrivuaaWAUyHf3ZMA9 H5bEqbugfjz7QoPvMJN6/EW9avU0fkmfNRIp685tRHIfF/oZ5d90PiNN+m1OSDRJEVsK whVD8LyJ6J9HygsqDWMDJrf7D3uGERgD7V8L1oduwEWBlNyWBYThciUA3WfAEUFylNuv wepg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k3qTP15/KW1GvymOMvtOciY6oQlool890KwlGyG99eE=; b=iKEnN48rZ++SPndBrg6c7e/LMSUXZ7UX/jhtBtXG6HWu1JQpqTPwGyK8UlAYo8HM5F oTCugW2wac56n2m5s8W/mENcTuU114qY0/ahGZ/LAdMNtUyxwrYQ69DZBy4w8ops0PPq IBuO12B2Lmi1fDCjo/Qnr5Xv3yVUO7CZboNE5jlKGjBexX0XSncjlpx4vXg4tFZ2zbJ8 6gcP2RnAcEgBtu9Rcs0oZD1YdaYNfHCUZtXIQWVlAe9KRr4p5SYcSTra6LllGHpqHGbQ VZ9/RvX388RhWS2Tsl7Plr40s4e6iGyGJR5+ogkDiJJkJRvvGNTc74VqDVBNjlRoRSop 5E7w== X-Gm-Message-State: AOAM533yNJUq15TQH7kVfG+DV/9n4O+rNIxX5Fk4jWZo8xgzfPvZBknN bWgXpeiXzzpc15z5z1bynko= X-Google-Smtp-Source: ABdhPJyxTNoYAXyIlpwV+OGz9gcKhrsuRx41Qsw37QnFq364UUm1JZdvb/GTyh3u7mH5VjpPbUEkgw== X-Received: by 2002:a17:906:b2d1:: with SMTP id cf17mr14408920ejb.225.1623657040693; Mon, 14 Jun 2021 00:50:40 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:40 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 09/45] trace-cmd library: Bump the trace file version to 7 Date: Mon, 14 Jun 2021 10:49:53 +0300 Message-Id: <20210614075029.598048-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Adding a compression of the trace.dat file will change its structure. These changes are not backward compatible, the old trace-cmd binaries will not be able to read compressed trace files. Bumping the version to 7 will prevent old trace-cmd to read such files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index c6591e56..4d5d1455 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -15,7 +15,7 @@ void tracecmd_critical(const char *fmt, ...); void tracecmd_info(const char *fmt, ...); /* trace.dat file format version */ -#define FILE_VERSION 6 +#define FILE_VERSION 7 #define _STR(x) #x #define STR(x) _STR(x) From patchwork Mon Jun 14 07:49:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28045C48BE8 for ; Mon, 14 Jun 2021 07:50:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FF9261289 for ; Mon, 14 Jun 2021 07:50:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232557AbhFNHw7 (ORCPT ); Mon, 14 Jun 2021 03:52:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232546AbhFNHw7 (ORCPT ); Mon, 14 Jun 2021 03:52:59 -0400 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1134C061766 for ; Mon, 14 Jun 2021 00:50:45 -0700 (PDT) Received: by mail-ej1-x62f.google.com with SMTP id ho18so15041509ejc.8 for ; Mon, 14 Jun 2021 00:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KZHQCnRvJyBn5lxybtd0Sz7gA7tBvALRfNjLaMT/uL0=; b=Ge/5t8VIdZz4hMcdfs+vA7JdwzDH3A9YzBAJqrIEYLgB9Zme1araLmMbqRBFhC5g0m LCC1KwDk95oFdBGe554t8l3NTVj2PeYi598UdHjnvFe59WrIZDwzSuppaV30w28qDEFG 5J/ZFn4sTEjpbSWA2BLLhORfQFinzii5qSE9KdI4nmbxyYsWf+O1rE4RNu6dju1Vt6B/ qzE5P8w3i9zllsVM/cF8eFjqrrgqMy0XRIk+F3paMlNF/YTzFvQiy3e0s5ropZDETUen 4Eobemj9rkSTXd8wMZf2xGzrMr+6JP+Qiwso1/jy+QkxNhBlv+WyWSMGH/N2XLEha2Sp lM5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KZHQCnRvJyBn5lxybtd0Sz7gA7tBvALRfNjLaMT/uL0=; b=GHyDn3u1bM12+79UA1xuQ4ySuorKU4Wuzr7PeRDVHz+IOu2gZiLtedE6gkZ++yEeyO wW39Pm3j9fj+GXS37OR4SNtKS3piVEFuWrq9dcNFyNm0itD42U5tikMhpIor/vWUbkD6 /o5mgX89TSydbLBeWwXRnkgkZ5PEnWRN3YQ5wUDLAQkNLhsBE7TPmLv2sudmURORTDUp vC8SBvhhOPNh1elzQnXPu9JriXhxBnvWDWTy7v+xUDo7LWmnOFiqAILcly2ba75Vz36m nVfTtPO0KZ81NVZe15xQVg+aqVPorx+tv7khax8JEk9hg0KlMMeT4Xn5DusS24s4NV3O aaow== X-Gm-Message-State: AOAM532CLJ4/jywkTKCV9GIhjG6nSRYtvyGx63x6LYYkYrN/x+FDYKDy dPqBjl1vxlPcBYkLI87EIrfM4Y1nEOWWng== X-Google-Smtp-Source: ABdhPJxpD41vNKX/AlnNLXKgbUz2NUVUkes1nx2DiO6DqfqcmIXaASqZy3WduvJQeWF5k76cZ4omxg== X-Received: by 2002:a17:906:b317:: with SMTP id n23mr14341455ejz.324.1623657041798; Mon, 14 Jun 2021 00:50:41 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:41 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 10/45] trace-cmd library: Compress part of the trace file Date: Mon, 14 Jun 2021 10:49:54 +0300 Message-Id: <20210614075029.598048-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Compress part of the trace.dat file metadata. If there is compression support, chose file version 7 and compress these parts of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name A new compression header is added in the file, right after the page size information. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-output.c | 145 ++++++++++++++++++++---- 2 files changed, 127 insertions(+), 22 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 4d5d1455..93613cd7 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -34,4 +34,8 @@ void tracecmd_info(const char *fmt, ...); void tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_compression_start(struct tracecmd_output *handle); +int out_compression_end(struct tracecmd_output *handle); +void out_compression_reset(struct tracecmd_output *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index bacda23b..87816b37 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -58,6 +58,10 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + + bool do_compress; + struct tracecmd_compression *compress; + struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -78,12 +82,31 @@ struct list_event_system { static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { + if (handle->do_compress) + return tracecmd_compress_write(handle->compress, data, size); + if (handle->msg_handle) return tracecmd_msg_data_send(handle->msg_handle, data, size); return __do_write_check(handle->fd, data, size); } +static inline int do_lseek(struct tracecmd_output *handle, off_t offset, int whence) +{ + if (handle->do_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek64(handle->fd, offset, whence); +} + +static inline int do_preed(struct tracecmd_output *handle, void *dst, int len, off_t offset) +{ + if (handle->do_compress) + return tracecmd_compress_pread(handle->compress, dst, len, offset); + else + return pread(handle->fd, dst, len, offset); +} + static short convert_endian_2(struct tracecmd_output *handle, short val) { if (!handle->pevent) @@ -109,6 +132,31 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +__hidden void out_compression_reset(struct tracecmd_output *handle) +{ + if (handle->file_version < 7) + return; + tracecmd_compress_reset(handle->compress); + handle->do_compress = false; +} + +__hidden int out_compression_start(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + tracecmd_compress_reset(handle->compress); + handle->do_compress = true; + return 0; +} + +__hidden int out_compression_end(struct tracecmd_output *handle) +{ + if (handle->file_version < 7) + return 0; + handle->do_compress = false; + return tracecmd_compress_block(handle->compress); +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen @@ -159,6 +207,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) free(option); } free(handle->trace_clock); + tracecmd_compress_destroy(handle->compress); free(handle); } @@ -659,13 +708,17 @@ static int read_ftrace_files(struct tracecmd_output *handle) } create_event_list_item(handle, &systems, &list); - + out_compression_start(handle); ret = copy_event_system(handle, systems); + if (!ret) + ret = out_compression_end(handle); + else + out_compression_reset(handle); free_list_events(systems); - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - + if (!ret) + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; return ret; } @@ -715,6 +768,7 @@ static int read_event_files(struct tracecmd_output *handle, for (slist = systems; slist; slist = slist->next) count++; + out_compression_start(handle); ret = -1; endian4 = convert_endian_4(handle, count); if (do_write_check(handle, &endian4, 4)) @@ -730,8 +784,14 @@ static int read_event_files(struct tracecmd_output *handle, ret = copy_event_system(handle, slist); } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + if (!ret) + ret = out_compression_end(handle); out_free: + if (!ret) + handle->file_state = TRACECMD_FILE_ALL_EVENTS; + else + out_compression_reset(handle); + free_list_events(systems); return ret; @@ -793,20 +853,20 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, if (kallsyms) path = kallsyms; - + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ size = 0; endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; - return 0; + ret = do_write_check(handle, &endian4, 4); + goto out; } size = get_size(path); endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; + ret = do_write_check(handle, &endian4, 4); + if (ret) + goto out; set_proc_kptr_restrict(0); check_size = copy_file(handle, path); @@ -814,13 +874,18 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, errno = EINVAL; tracecmd_warning("error in size of file '%s'", path); set_proc_kptr_restrict(1); - return -1; + ret = -1; + goto out; } set_proc_kptr_restrict(1); - handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = out_compression_end(handle); +out: + if (!ret) + handle->file_state = TRACECMD_FILE_KALLSYMS; + else + out_compression_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_output *handle) @@ -840,6 +905,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -861,11 +927,14 @@ static int read_ftrace_printk(struct tracecmd_output *handle) } out: - handle->file_state = TRACECMD_FILE_PRINTK; put_tracing_file(path); + if (out_compression_end(handle)) + return -1; + handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); + out_compression_reset(handle); return -1; } @@ -911,14 +980,39 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { - if (ihandle) + if (ihandle) { handle->file_version = tracecmd_get_in_file_version(ihandle); - else - handle->file_version = FILE_VERSION; + } else { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + } return 0; } +static int write_compression_header(struct tracecmd_output *handle) +{ + const char *name = NULL; + const char *ver = NULL; + char *buf; + int ret; + + ret = tracecmd_compress_proto_get_name(handle->compress, &name, &ver); + if (ret < 0 || !name || !ver) + return -1; + ret = asprintf(&buf, "%s %s", name, ver); + if (ret < 0) + return -1; + ret = do_write_check(handle, buf, strlen(buf) + 1); + + free(buf); + return ret; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -992,8 +1086,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, endian4 = convert_endian_4(handle, handle->page_size); if (do_write_check(handle, &endian4, 4)) goto out_free; + if (handle->file_version >= 7 && write_compression_header(handle)) + goto out_free; handle->file_state = TRACECMD_FILE_INIT; - if (ihandle) return handle; @@ -1285,11 +1380,17 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) handle->file_state); return ret; } + out_compression_start(handle); + ret = save_tracing_file_data(handle, "saved_cmdlines"); - if (ret < 0) + if (ret < 0) { + out_compression_reset(handle); return ret; - handle->file_state = TRACECMD_FILE_CMD_LINES; - return 0; + } + ret = out_compression_end(handle); + if (!ret) + handle->file_state = TRACECMD_FILE_CMD_LINES; + return ret; } struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) From patchwork Mon Jun 14 07:49:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 153FAC49360 for ; Mon, 14 Jun 2021 07:51:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F260E61352 for ; Mon, 14 Jun 2021 07:51:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232618AbhFNHyA (ORCPT ); Mon, 14 Jun 2021 03:54:00 -0400 Received: from mail-ej1-f47.google.com ([209.85.218.47]:44741 "EHLO mail-ej1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232616AbhFNHx7 (ORCPT ); Mon, 14 Jun 2021 03:53:59 -0400 Received: by mail-ej1-f47.google.com with SMTP id c10so15025547eja.11 for ; Mon, 14 Jun 2021 00:51:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zf5AOhAEAPDVyQzk2iMX2j/CaJ3B/G0Hew1xFRyohqg=; b=pOlYaQluCfDn7PtthAFuyu4xPI3NVUF72qGjxLyF9dxEtc83YTi4PrikSMu8IvekTO bHF4/OXNg29/DAP3CJHmOEBJxQ0mfKRhHr9KpYqTpGlKiE4jLT7tA0eY7wA/MgmiFxyz HpSs6pM6CbVGEfzNjFmjHIVW3DA4zzKKDBAPIlcTHqxe682COZ3JnLOrq92XDBbx8YYn sd3AQDcvImL9p63kkIFei8FPT4QJLoVh4WwGxk/dt8LLW0QkbCI0pkqeb5fd7NrYOhai QJyEA0uIxrdgkvjpTgjwjMuHayUkAsiRLVnxRGJUtcSFlWfwY7kqmdzS4prcCEu8YGhU 2dvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zf5AOhAEAPDVyQzk2iMX2j/CaJ3B/G0Hew1xFRyohqg=; b=eDp/keZUhYNVaLIBINmNWJPzlVsSrFvNj5dVhA9ncZTB1NWI/WiNKQPWucmytk3sXB wTfDyHuItTihPXsWtqagk1q7IyI8dIoNyo9V64JbWlkmvTqKs6BaLS+75HUgxy9MH0tq S1RElvmk8z9stQKB3FJjbSibB36j7Srhip94meliOp1l81zKbatUne0L/o3KR3rk67Ol jzRHYAnJBaxOVyXpeEtdffqOCSdSuifVo9eCVHNNtZzDa4n1qwUjXqQBP22vdvDqoESK GdUpbtjNAkZ0a5vLi8jA5G1gtRp1YPdhs4H+LgCwTIUmSAmJWUvSbLTAcTRVhUDmP+Xh MzHw== X-Gm-Message-State: AOAM530/1hwjgXqCbrQlvkmVtSgoI8fqovUnmOZgqUHS7sq8XjZblDjH sW8tDt3LHdWRM86qU92wOYjEyNEhnjsAhA== X-Google-Smtp-Source: ABdhPJxSHvoyWpP768+NxMH3iir6mpsEDKG67vZ7TKCe0lV0wUFuxgBrBuWe1FMnXUSSdT6Kejnbqw== X-Received: by 2002:a17:906:fa93:: with SMTP id lt19mr14520107ejb.54.1623657042876; Mon, 14 Jun 2021 00:50:42 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:42 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 11/45] trace-cmd library: Read compressed trace file Date: Mon, 14 Jun 2021 10:49:55 +0300 Message-Id: <20210614075029.598048-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If a trace file version 7 is detected, read compressed sections of it. Read the compression header, check if the compression algorithm and version, used to compress the file, are supported and use it to uncompress these sections of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-input.c | 257 ++++++++++++++++-------- 2 files changed, 178 insertions(+), 82 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 93613cd7..419bfee0 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -38,4 +38,7 @@ int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); +void in_uncompress_reset(struct tracecmd_input *handle); +int in_uncompress_block(struct tracecmd_input *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 46619ac7..43d731e0 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -136,6 +136,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + bool read_compress; + struct tracecmd_compression *compress; + struct host_trace_info host; double ts2secs; char * cpustats; @@ -255,12 +258,28 @@ static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) return tot; } +static inline int do_lseek(struct tracecmd_input *handle, int offset, int whence) +{ + if (handle->read_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek(handle->fd, offset, whence); +} + +static inline ssize_t do_read_compressed(struct tracecmd_input *handle, void *data, size_t size) +{ + if (handle->read_compress) + return tracecmd_compress_read(handle->compress, data, size); + else + return do_read(handle, data, size); +} + static ssize_t do_read_check(struct tracecmd_input *handle, void *data, size_t size) { ssize_t ret; - ret = do_read(handle, data, size); + ret = do_read_compressed(handle, data, size); if (ret < 0) return ret; if (ret != size) @@ -278,10 +297,8 @@ static char *read_string(struct tracecmd_input *handle) ssize_t r; for (;;) { - r = do_read(handle, buf, BUFSIZ); - if (r < 0) - goto fail; - if (!r) + r = do_read_compressed(handle, buf, BUFSIZ); + if (r <= 0) goto fail; for (i = 0; i < r; i++) { @@ -307,7 +324,7 @@ static char *read_string(struct tracecmd_input *handle) } /* move the file descriptor to the end of the string */ - r = lseek(handle->fd, -(r - (i+1)), SEEK_CUR); + r = do_lseek(handle, -(r - (i+1)), SEEK_CUR); if (r < 0) goto fail; @@ -359,6 +376,26 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +__hidden void in_uncompress_reset(struct tracecmd_input *handle) +{ + if (handle->compress && handle->file_version >= 7) { + handle->read_compress = false; + tracecmd_compress_reset(handle->compress); + } +} + +__hidden int in_uncompress_block(struct tracecmd_input *handle) +{ + int ret = 0; + + if (handle->compress && handle->file_version >= 7) { + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->read_compress = true; + } + return ret; +} + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -602,34 +639,40 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) + if (in_uncompress_block(handle)) return -1; + ret = read4(handle, &count); + if (ret < 0) + goto out; + for (i = 0; i < count; i++) { - if (read8(handle, &size) < 0) - return -1; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_ftrace_file(handle, size, print_all, ereg); if (ret < 0) - return -1; + goto out; } handle->event_files_start = lseek64(handle->fd, 0, SEEK_CUR); + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + ret = 0; +out: if (sreg) { regfree(sreg); regfree(ereg); } - - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - - return 0; + in_uncompress_reset(handle); + return ret; } static int read_event_files(struct tracecmd_input *handle, const char *regex) { unsigned long long size; - char *system; + char *system = NULL; regex_t spreg; regex_t epreg; regex_t *sreg = NULL; @@ -654,13 +697,19 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) return -1; } - if (read4(handle, &systems) < 0) + if (in_uncompress_block(handle)) return -1; + ret = read4(handle, &systems); + if (ret < 0) + goto out; + for (i = 0; i < systems; i++) { system = read_string(handle); - if (!system) - return -1; + if (!system) { + ret = -1; + goto out; + } sys_printed = 0; print_all = 0; @@ -687,103 +736,117 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) - goto failed; + ret = read4(handle, &count); + if (ret < 0) + goto out; for (x=0; x < count; x++) { - if (read8(handle, &size) < 0) - goto failed; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_event_file(handle, system, size, print_all, &sys_printed, reg); if (ret < 0) - goto failed; + goto out; } free(system); - } - - if (sreg) { - regfree(sreg); - regfree(ereg); + system = NULL; } handle->file_state = TRACECMD_FILE_ALL_EVENTS; - - return 0; - - failed: + ret = 0; + out: + in_uncompress_reset(handle); if (sreg) { regfree(sreg); regfree(ereg); } free(system); - return -1; + return ret; } static int read_proc_kallsyms(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; + struct tep_handle *tep = handle->pevent; unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_KALLSYMS) return 0; - if (read4(handle, &size) < 0) + if (in_uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ - buf = malloc(size+1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)){ - free(buf); - return -1; + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_KALLSYMS; + goto out; /* OK? */ } - buf[size] = 0; - - tep_parse_kallsyms(pevent, buf); - free(buf); + buf = malloc(size+1); + if (!buf) { + ret = -1; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; + buf[size] = 0; + tep_parse_kallsyms(tep, buf); handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = 0; +out: + free(buf); + in_uncompress_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_input *handle) { unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_PRINTK) return 0; - if (read4(handle, &size) < 0) + if (in_uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ + + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_PRINTK; + goto out; /* OK? */ + } buf = malloc(size + 1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)) { - free(buf); - return -1; + if (!buf) { + ret = -1; + goto out; } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; buf[size] = 0; tep_parse_printk_formats(handle->pevent, buf); - - free(buf); - handle->file_state = TRACECMD_FILE_PRINTK; + ret = 0; - return 0; +out: + free(buf); + in_uncompress_reset(handle); + return ret; } static int read_and_parse_cmdlines(struct tracecmd_input *handle); @@ -2974,20 +3037,30 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; unsigned long long size; - char *cmdlines; + char *cmdlines = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_CMD_LINES) return 0; - if (read_data_and_size(handle, &cmdlines, &size) < 0) + if (in_uncompress_block(handle)) return -1; + + ret = read_data_and_size(handle, &cmdlines, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_CMD_LINES; + goto out; + } cmdlines[size] = 0; tep_parse_saved_cmdlines(pevent, cmdlines); - free(cmdlines); - handle->file_state = TRACECMD_FILE_CMD_LINES; - - return 0; + ret = 0; +out: + free(cmdlines); + in_uncompress_reset(handle); + return ret; } static void extract_trace_clock(struct tracecmd_input *handle, char *line) @@ -3268,7 +3341,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) struct tracecmd_input *handle; char test[] = TRACECMD_MAGIC; unsigned int page_size; - char *version; + size_t offset; + char *str = NULL; + char *zver; char buf[BUFSIZ]; unsigned long ver; @@ -3291,11 +3366,11 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (memcmp(buf, "tracing", 7) != 0) goto failed_read; - version = read_string(handle); - if (!version) + str = read_string(handle); + if (!str) goto failed_read; - tracecmd_info("version = %s\n", version); - ver = strtol(version, NULL, 10); + tracecmd_info("version = %s\n", str); + ver = strtol(str, NULL, 10); if (!ver && errno) goto failed_read; if (!tracecmd_is_version_supported(ver)) { @@ -3303,7 +3378,8 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) goto failed_read; } handle->file_version = ver; - free(version); + free(str); + str = NULL; if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3328,20 +3404,36 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) read4(handle, &page_size); handle->page_size = page_size; - handle->header_files_start = - lseek64(handle->fd, 0, SEEK_CUR); + offset = lseek64(handle->fd, 0, SEEK_CUR); + handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); + lseek64(handle->fd, offset, SEEK_SET); - handle->total_file_size = - lseek64(handle->fd, 0, SEEK_END); + if (handle->file_version >= 7) { + str = read_string(handle); + if (!str) + goto failed_read; + zver = strchr(str, ' '); + if (!zver) + goto failed_read; + *zver = '\0'; + handle->compress = tracecmd_compress_alloc(str, zver + 1, + handle->fd, handle->pevent, NULL); + if (!handle->compress) { + tracecmd_warning("Unsupported file compression %s %s", str, zver + 1); + goto failed_read; + } + free(str); + str = NULL; + } handle->header_files_start = - lseek64(handle->fd, handle->header_files_start, SEEK_SET); - + lseek64(handle->fd, 0, SEEK_CUR); handle->file_state = TRACECMD_FILE_INIT; return handle; failed_read: + free(str); free(handle); return NULL; @@ -3519,7 +3611,8 @@ void tracecmd_close(struct tracecmd_input *handle) if (handle->flags & TRACECMD_FL_BUFFER_INSTANCE) tracecmd_close(handle->parent); else { - /* Only main handle frees plugins and pevent */ + /* Only main handle frees plugins, pevent and compression context */ + tracecmd_compress_destroy(handle->compress); tep_unload_plugins(handle->plugin_list, handle->pevent); tep_free(handle->pevent); } From patchwork Mon Jun 14 07:49:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3644FC48BE6 for ; Mon, 14 Jun 2021 07:51:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 107C561380 for ; Mon, 14 Jun 2021 07:51:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232605AbhFNHxr (ORCPT ); Mon, 14 Jun 2021 03:53:47 -0400 Received: from mail-ej1-f50.google.com ([209.85.218.50]:43782 "EHLO mail-ej1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232594AbhFNHxq (ORCPT ); Mon, 14 Jun 2021 03:53:46 -0400 Received: by mail-ej1-f50.google.com with SMTP id ci15so15063005ejc.10 for ; Mon, 14 Jun 2021 00:51:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jNmgwtkM7E5IR4z79tTsl2FEDjRbmtrhOhWVpEAWIrY=; b=rw9F4E3BGgxGQWiG9Mi0L3R0Fs8NyXGs5JqpJEYgsVxT21wkNAOLsLMloj15/5nRSe TRr9cuYR9nB9df0AQ6E4vuZJ5C/OEw+z07YlqieDpMmgHSfmOyi9iv6kGcGaBEuvFPbW RhM2e7OEOkZFaloUiB78tA+EwhOO54wKz2P4jFvITpwNb4xOa3PcncsAuX3r9DPjhvrS H3D/0atbwg6+BJRJ5E3aPpikCxIqf7qsu9IVOfUljjcoC1BYN5Kt2oJQoV8ESd2qoup0 7++3YB2bZWevaH+ls4rr8yB/5+HYUCGF9rnokSmF2/JPUOjkB1K2H16Q6vBHwfUKhC1R ImEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jNmgwtkM7E5IR4z79tTsl2FEDjRbmtrhOhWVpEAWIrY=; b=NDlf31h5ACIzPh2BtRP4nnnnPEsR7Chzl6vDOs8VyPodto8SrMGqqpsiUVrmIZvW7T GNWLrUOijeJZPUWQmQ2TaMn5crIOMu02ATnr4HdJRBKEMFa4hA0/exoOg4tVft13PcwP M4g+MANrdAf6yK0myWAtv+XSB47jVbwuzFq+3OtHJFvzKVAiLT8289NRHJiPjNTx8kik TOKFrvsXTytKE6hwu2OS5W1yUlhzQpWGbaCv9FrogLWxRmDtddoGiym7xKfWUvyqzEI2 1l0tO0NxIBrDsB32ERLYXyEjvw6IFKbjP7bxSY97mLa3T5WKshcldPx1PMjOenxqB1So nooA== X-Gm-Message-State: AOAM533Er2fHnHsmXH9IwBSxx22qGf2icvp3RHuykMAk/z82TgLv7UsP Jt3pTQNteYfOSLU5VG0GFhA= X-Google-Smtp-Source: ABdhPJxYRS1oWX2IFxliO5zME40BqfMrbVWdIf1EouFsl12XWdXijU7uxkQQhX11KiAil5XzTZi43w== X-Received: by 2002:a17:906:d8cf:: with SMTP id re15mr14157190ejb.278.1623657043782; Mon, 14 Jun 2021 00:50:43 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:43 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 12/45] trace-cmd library: Add new API to get compression of input handler Date: Mon, 14 Jun 2021 10:49:56 +0300 Message-Id: <20210614075029.598048-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The new API returns name and version of the compression algorithm, used to compress the trace file associated with given input file handler: tracecmd_get_file_compress_proto() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 16 ++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ec534d51..624d0afb 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -236,6 +236,8 @@ unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 43d731e0..0b17b58d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4148,6 +4148,22 @@ unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle) return handle->file_version; } +/** + * tracecmd_get_file_compress_proto - get name and version of compression algorithm, + * used to compress the trace file + * @handle: input handle for the trace.dat file + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version) +{ + return tracecmd_compress_proto_get_name(handle->compress, name, version); +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Mon Jun 14 07:49:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 024EAC48BE8 for ; Mon, 14 Jun 2021 07:51:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D793061370 for ; Mon, 14 Jun 2021 07:51:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232594AbhFNHxs (ORCPT ); Mon, 14 Jun 2021 03:53:48 -0400 Received: from mail-ed1-f47.google.com ([209.85.208.47]:35711 "EHLO mail-ed1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232603AbhFNHxr (ORCPT ); Mon, 14 Jun 2021 03:53:47 -0400 Received: by mail-ed1-f47.google.com with SMTP id ba2so43439673edb.2 for ; Mon, 14 Jun 2021 00:51:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R7tpJUkfWQT6boWeVeVRny1IdcYdoZq+HnAPbHcv2Wg=; b=suKvipB4gJp8f/+nPQE6/hT+rh9kBhESV6xeGb2l5o327WleG+A7q2PT8URFfvPQ0Y 8AfUoeBY3Y3BcNJAoMZevZ4hE9gFiyp6PzhTNWUgl5sI6dnaoSn9laA0yrZeL+E7/z6v NSpnpeeGN/AvjYIGaF2eCJxIrtWI3x5CWw2KHM8eHDlddXYJv+EqFbqkXS3hhaNp5IjX F8MiBoYNjffXMeIPsxquuu9OWo1CqPZzIsR7gw8Jawz8CZh4ztB7AAfPCaS36sWAWEBk xtTi/5yQacuRIo7NtYTpWOtWYfTmlB1SH5PyWziFd4AN4fLtAv778fz9KddDX94NBRNe gTww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R7tpJUkfWQT6boWeVeVRny1IdcYdoZq+HnAPbHcv2Wg=; b=lIn7waHr7zvX9OaVBM/XLHy8z1uEkuju/uvxzCBwojsxVHIThHy5bhdNPyQAqLJWkd aI2wzFLtXweNUeDvuFw5rYyGJo3QTN42zs55HBTRz2K4Eg6EBHXbhyznJuPdmlcnFexs AAW5wXWi3hsIh+GasvYxGYh1T6SVp0gvpLLGLcXLEd1TBiogSryJ9TYauwC3/5o7nj7u MmFcNrHhA3XaOlmCXps8C94/WZVjVj57ZdjclhlBdh3ZyUKxCtu/XAIcdWCDEE56BaX6 OlKTuP47OWpsozMY3meNQlzYq3b8AFhYo+p2L/PeUieW3p7IlYAnoC7IUnXqAtMHzqd2 0jnw== X-Gm-Message-State: AOAM530l7p+/avZvZBZd0ZCTO86qyZIsNuWGl1FKuCr9BaasKNwdSC83 k/a5fb9AqK2aR53CbdTBk4Q3PajfR18UcQ== X-Google-Smtp-Source: ABdhPJytCH9DB0lGIaLgtdXQyx1IJWkzelMqL44fDhRF7wfRdsF/NgNLNjRbaNRQrVnnr8Z0ZDw3Vw== X-Received: by 2002:a50:eb92:: with SMTP id y18mr15606023edr.249.1623657044778; Mon, 14 Jun 2021 00:50:44 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:44 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 13/45] trace-cmd library: Inherit compression algorithm from input file Date: Mon, 14 Jun 2021 10:49:57 +0300 Message-Id: <20210614075029.598048-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace file output handler is allocated, based on given trace file input handler - use the same compression algorithm. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 87816b37..82b3e5df 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -980,17 +980,27 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { + const char *cname = NULL; + const char *cver = NULL; + if (ihandle) { handle->file_version = tracecmd_get_in_file_version(ihandle); - } else { - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); - if (handle->compress) - handle->file_version = 7; - else - handle->file_version = 6; + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + return -1; + } + return 0; } + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + return 0; } @@ -1645,6 +1655,8 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) { struct tracecmd_output *handle = NULL; struct tracecmd_input *ihandle; + const char *cname = NULL; + const char *cver = NULL; int fd2; /* Move the file descriptor to the beginning */ @@ -1682,6 +1694,12 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->file_version = tracecmd_get_in_file_version(ihandle); list_head_init(&handle->options); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + goto out_free; + } tracecmd_close(ihandle); return handle; From patchwork Mon Jun 14 07:49:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DEE1C48BE6 for ; Mon, 14 Jun 2021 07:51:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 33AE161289 for ; Mon, 14 Jun 2021 07:51:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232616AbhFNHyA (ORCPT ); Mon, 14 Jun 2021 03:54:00 -0400 Received: from mail-ed1-f52.google.com ([209.85.208.52]:45591 "EHLO mail-ed1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232608AbhFNHx7 (ORCPT ); Mon, 14 Jun 2021 03:53:59 -0400 Received: by mail-ed1-f52.google.com with SMTP id r7so30788989edv.12 for ; Mon, 14 Jun 2021 00:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gtQ5F8jrQUF2N6tOvDi45fySOa/hLLTaZHUpfsjugYM=; b=PzwVEsyYiZssY+5rqGm4dmN1/OG8dtNotk+6tdrBKlNjmXmHQw7duFDkWzOcD+5frK cRqihW/tvlFLsPGWzfnoUYr+7EDAJkROfsr2OJ678l8hEGskDNeJOZ0vk217yFF/TkdZ luTG/uL5DA5CX+zBqhNQXzb4kHgLC8BYgU8UTkpkXaAnYjQPe1FhWcYumTNb+4/e7t+1 zgDHURRS5ubiBMCcavWDCI7yp+GirfxCtKb93L+s8vY/T1jsVkUx6q/P58MLxYyrgu7f LXM6z2oPZmN27jLWE6LTolv19nrTnI8IZKIeNEwXqEjXJpFqVA/INcQs/NQCSENJ5rOV i97Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gtQ5F8jrQUF2N6tOvDi45fySOa/hLLTaZHUpfsjugYM=; b=Y9eq8BIOzzOKzZXhLFn4S0xtBzPLAYXlD6Rxv5J8aYitawxKGNNLSYfimF0Ki0a3wG q1m5+SJtwBmuJ4kDaSMzPzDicANGj+oQQMutB0bYACr5p2oX+5VJsKZIjkY65aGFK/qr IVlhl2JIR3SvZEc5jhEdCcAccNWGkNnlqaoe/uB3IaeJkyJiblYVt+Fo0cgabDPq2Wnz +wIM2Lw5oz4wt6sytzhD7P993mk7adT8E2SHd/xXMVIEwnDH7MHr5ko+bZCcTzm8l8jv oqJUnD7Q7tv3ECHZio+zLH3XuoitmdhyaFNTj9DqBaXDRX/rZCL6FtQobkuqFJXPxNoi PJbg== X-Gm-Message-State: AOAM530kfzOyHiXN0KALSNAGjPsD/Pz26UeVzYbsenZ1X+rkk/cJ9pAf O06AiSBQkWgDSlTAOQePRZg= X-Google-Smtp-Source: ABdhPJxXOnoYmMZSvTWVMQa/Pc8gm7SIzzg2/ZpYhi/Qi+1v+J+l4rnECRyHSHaMFpwLN9XZtXA3HQ== X-Received: by 2002:a05:6402:35cc:: with SMTP id z12mr3089352edc.45.1623657045666; Mon, 14 Jun 2021 00:50:45 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:45 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 14/45] trace-cmd library: Extend the create file APIs to support different compression Date: Mon, 14 Jun 2021 10:49:58 +0300 Message-Id: <20210614075029.598048-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added additional parameter for file compression to all trace-cmd library APIs for creating a new trace file. The caller could specify what compression algorithm should be used when creating a new trace file: tracecmd_create_file_latency tracecmd_create_init_file_glob tracecmd_create_init_fd_glob tracecmd_create_init_fd_msg tracecmd_create_init_file tracecmd_create_init_file_override tracecmd_create_init_fd The new parameter is of type string and can be one of: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 19 +++-- lib/trace-cmd/trace-output.c | 79 ++++++++++++------- tracecmd/trace-record.c | 12 ++- tracecmd/trace-restore.c | 4 +- tracecmd/trace-stream.c | 2 +- 5 files changed, 70 insertions(+), 46 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 624d0afb..ee73325c 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -271,20 +271,23 @@ struct tracecmd_event_list { struct tracecmd_option; struct tracecmd_msg_handle; -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression); struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_fd(int fd); +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_fd(int fd, const char *compression); struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list); +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression); struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_file(const char *output_file); + struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression); struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms); + const char *kallsyms, + const char *compression); struct tracecmd_option *tracecmd_add_option(struct tracecmd_output *handle, unsigned short id, int size, const void *data); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 82b3e5df..7c7d3d76 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -978,24 +978,41 @@ out_free: } static int select_file_version(struct tracecmd_output *handle, - struct tracecmd_input *ihandle) + struct tracecmd_input *ihandle, + const char *compression) { const char *cname = NULL; const char *cver = NULL; - if (ihandle) { - handle->file_version = tracecmd_get_in_file_version(ihandle); + if (compression) { /* If compression is specified - use it */ + if (!strcmp(compression, "any")) { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("No compression algorithms are not supported"); + return -1; + } + } else if (strcmp(compression, "none")) { + handle->compress = tracecmd_compress_alloc(compression, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("Compression algorithm %s is not supported", + compression); + return -1; + } + } + } else if (ihandle) { /* If there is input handler - inherit compression */ if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, handle->pevent, handle->msg_handle); if (!handle->compress) return -1; } - return 0; + } else { /* default - auto select the best available compression algorithm */ + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); } - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); if (handle->compress) handle->file_version = 7; else @@ -1028,7 +1045,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, struct tracecmd_event_list *list, - struct tracecmd_msg_handle *msg_handle) + struct tracecmd_msg_handle *msg_handle, + const char *compression) { struct tracecmd_output *handle; struct tep_handle *pevent; @@ -1049,7 +1067,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; - if (select_file_version(handle, ihandle)) + if (select_file_version(handle, ihandle, compression)) goto out_free; list_head_init(&handle->options); @@ -1128,7 +1146,8 @@ static struct tracecmd_output *create_file(const char *output_file, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, + const char *compression) { struct tracecmd_output *handle; int fd; @@ -1137,7 +1156,7 @@ static struct tracecmd_output *create_file(const char *output_file, if (fd < 0) return NULL; - handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL); + handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL, compression); if (!handle) { close(fd); unlink(output_file); @@ -1403,13 +1422,14 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return ret; } -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression) { struct tracecmd_output *handle; char *path; int ret; - handle = create_file(output_file, NULL, NULL, NULL, &all_event_list); + handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) return NULL; @@ -1710,41 +1730,43 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output *tracecmd_create_init_fd(int fd) +struct tracecmd_output *tracecmd_create_init_fd(int fd, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL, compression); } struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(msg_handle->fd, NULL, NULL, NULL, list, msg_handle); + return create_file_fd(msg_handle->fd, NULL, NULL, NULL, + list, msg_handle, compression); } struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list) +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, list, NULL, compression); } struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list) +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression) { - return create_file(output_file, NULL, NULL, NULL, list); + return create_file(output_file, NULL, NULL, NULL, list, compression); } -struct tracecmd_output *tracecmd_create_init_file(const char *output_file) +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression) { - return create_file(output_file, NULL, NULL, NULL, &all_event_list); + return create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); } struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms) + const char *kallsyms, + const char *compression) { - return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list); + return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list, compression); } /** @@ -1756,12 +1778,13 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) { struct tracecmd_output *handle; + const char *compr_name = NULL; - handle = create_file(file, ihandle, NULL, NULL, &all_event_list); + tracecmd_get_file_compress_proto(ihandle, &compr_name, NULL); + handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compr_name); if (!handle) return NULL; diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 60ee5fb3..29f84b7c 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3700,7 +3700,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events); + network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3718,8 +3718,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, - listed_events); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4067,8 +4066,7 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, - listed_events); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4460,7 +4458,7 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count); + handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4491,7 +4489,7 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events); + handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 280a37f0..96b4fb5d 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -91,7 +91,7 @@ void trace_restore (int argc, char **argv) } handle = tracecmd_create_init_file_override(output, tracing_dir, - kallsyms); + kallsyms, NULL); if (!handle) die("Unabled to create output file %s", output); if (tracecmd_write_cmdlines(handle) < 0) @@ -128,7 +128,7 @@ void trace_restore (int argc, char **argv) handle = tracecmd_copy(ihandle, output); tracecmd_close(ihandle); } else - handle = tracecmd_create_init_file(output); + handle = tracecmd_create_init_file(output, NULL); if (!handle) die("error writing to %s", output); diff --git a/tracecmd/trace-stream.c b/tracecmd/trace-stream.c index b47b208c..317de1b2 100644 --- a/tracecmd/trace-stream.c +++ b/tracecmd/trace-stream.c @@ -43,7 +43,7 @@ trace_stream_init(struct buffer_instance *instance, int cpu, int fd, int cpus, tfd = fileno(fp); ofd = dup(tfd); - trace_output = tracecmd_create_init_fd(ofd); + trace_output = tracecmd_create_init_fd(ofd, NULL); if (!trace_output) { fclose(fp); return NULL; From patchwork Mon Jun 14 07:49:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 849FBC48BE6 for ; Mon, 14 Jun 2021 07:52:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 662C561289 for ; Mon, 14 Jun 2021 07:52:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232631AbhFNHyH (ORCPT ); Mon, 14 Jun 2021 03:54:07 -0400 Received: from mail-ed1-f50.google.com ([209.85.208.50]:38875 "EHLO mail-ed1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232617AbhFNHyG (ORCPT ); Mon, 14 Jun 2021 03:54:06 -0400 Received: by mail-ed1-f50.google.com with SMTP id d13so31702277edt.5 for ; Mon, 14 Jun 2021 00:51:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cVvHmzASpCdfwkT4dhPnYQr3vlzUuxmkupRu990yC9Y=; b=Azbc5hq/ICqEJVfC+IN9XXOHjTQbjNy0Jmbr0cKAaCvDnQ6ZIU4MECr/NWbyN4GF+7 CSHyNWs39GlGx/9mv5ef5q8LCLpXCEnawWKpGgul+CE9Q/dhn5pEZVQQYdLu/fz1L1mv EhwChEnQ3LWc2CBFUIWBpBY1gJNMYr40wqpD0iTdib/qdFETzaqJWc3nk0jAV50XON2W FbTREidqs2SMF69eAilJMBDLeljOBL0c6pS7uVcdCu19gdPzBOECmLyISTYruNeqGL9o eccHCt7bkp+hqNiCH/aG9L296q9InadGrzykXKBIVoiNURonzo/xfpLRWwX69Se1ZKp4 tT+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cVvHmzASpCdfwkT4dhPnYQr3vlzUuxmkupRu990yC9Y=; b=Gj1Qpw4DgZrO0fHZTeMa65W40mjOQ72d4+kQ1UqDTSM0MafSqskn41+dDKhPB5wG11 ZrV8PZPL5hHVWHTV/SBaFz78tEAOy3dxmZaR2jSZVGAraqKzz6rU5LOeG4BNvP9Js9JG RV75a+QLd7LeRM6eRIA7yZ2zU4pg9dXq9aDUr01eU7CN3iU3oDGQHssi0shcqH6JgxGU UES127hjcmsErZNE3Uhpnf7KXc499MpmB4sHICaeJsOmx9x0VuXquAPnF3KI8FLsX/TG QT70K2q+gEVop1NPEnuqjv2rr0C/Wsjo7o8S5WaCrwfosWFj/VwDIEUqIgTRL6wlkYB5 RcVQ== X-Gm-Message-State: AOAM531rTqVm53tMeKdZ1tdwjPhih3Zkew5Zz2tNyu6MpkXa/hMPOcb3 SH+6kB4twZ1j3Ioz5L2XbXDEJUruIWL1+g== X-Google-Smtp-Source: ABdhPJwCErv90SBmiBjROQMG4o1GIg9+kqLq/UBeRINhm7wTZTusq+n+jwDG/I1bEjqTlrcv5buHZw== X-Received: by 2002:a05:6402:27d1:: with SMTP id c17mr15486331ede.28.1623657046494; Mon, 14 Jun 2021 00:50:46 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:46 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 15/45] trace-cmd record: Add new parameter --compression Date: Mon, 14 Jun 2021 10:49:59 +0300 Message-Id: <20210614075029.598048-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a new parameter "trace-cmd record --compression", can be used to select the desired compression algorithm for the trace output file. One of these strings can be passed: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms, available algorithms can be listed with "trace-cmd list -c" If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 30 +++++++++++++++++++++++++----- tracecmd/trace-usage.c | 5 +++++ 2 files changed, 30 insertions(+), 5 deletions(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 29f84b7c..d3362e5b 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -199,6 +199,7 @@ struct common_record_context { char *date2ts; char *user; const char *clock; + const char *compression; struct tsc_nsec tsc2nsec; int data_flags; int tsync_loop_interval; @@ -3700,7 +3701,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(msg_handle, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3718,7 +3720,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4066,7 +4069,8 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, + listed_events, ctx->compression); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4458,7 +4462,8 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); + handle = tracecmd_create_file_latency(ctx->output, + local_cpu_count, ctx->compression); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4489,7 +4494,8 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); + handle = tracecmd_create_init_file_glob(ctx->output, + listed_events, ctx->compression); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); @@ -5736,6 +5742,7 @@ void init_top_instance(void) } enum { + OPT_comporession = 239, OPT_tsc2nsec = 240, OPT_fork = 241, OPT_tsyncinterval = 242, @@ -6172,6 +6179,7 @@ static void parse_record_options(int argc, {"fork", no_argument, NULL, OPT_fork}, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, {"poll", no_argument, NULL, OPT_poll}, + {"compression", required_argument, NULL, OPT_comporession}, {NULL, 0, NULL, 0} }; @@ -6597,6 +6605,18 @@ static void parse_record_options(int argc, cmd_check_die(ctx, CMD_set, *(argv+1), "--poll"); recorder_flags |= TRACECMD_RECORD_POLL; break; + case OPT_comporession: + cmd_check_die(ctx, CMD_start, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_set, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_extract, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_stream, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_profile, *(argv+1), "--compression"); + cmd_check_die(ctx, CMD_record_agent, *(argv+1), "--compression"); + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + ctx->compression = strdup(optarg); + break; case OPT_quiet: case 'q': quiet = true; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index cd8ca391..c70f9919 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -69,6 +69,11 @@ static struct usage_help usage_help[] = { " If 0 is specified, no loop is performed - timestamps offset is calculated only twice," " at the beginnig and at the end of the trace\n" " --poll don't block while reading from the trace buffer\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { "set", From patchwork Mon Jun 14 07:50:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98D1AC48BE6 for ; Mon, 14 Jun 2021 07:51:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 837CC61289 for ; Mon, 14 Jun 2021 07:51:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232579AbhFNHxI (ORCPT ); Mon, 14 Jun 2021 03:53:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232591AbhFNHxH (ORCPT ); Mon, 14 Jun 2021 03:53:07 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB470C061767 for ; Mon, 14 Jun 2021 00:50:48 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id k7so15032397ejv.12 for ; Mon, 14 Jun 2021 00:50:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=VCFPikeJ2J96IkyZuglyia4/J2ZC5coitp5sOOyE4ve6cspm5bjzltCpM62u6NKvff RwL7BBRA/ANSkZkq+qpJamiuvRkPZRSxeaWT18r1OOC45eD9Zua26yABG3RChr1JtRub sPsy2IouHgIuGKQZ8xRgcXw82VfM9X5iVCfE+fGSErLmfeOcFFzz4JIQkIE7oYav04fX zTJxQ/MPV/nLwWqMfWPHtBXuDsDtl5CBM7TjbNdhgKrrpfGBZySAsD/fFdejeP3ZnHR4 RQ+KDOCeLSoSxU2fqzyaeIPZ4dRT3SeFpNxy44teTGpHthx01x4scEpTEmlbooUV/7v9 kVpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=lgKKAhibnPrkKCcgL4l6joDFxx0B2vEslEjXp1sCJxiBTtiTCRvtxrG4US52s0420x 41KC4GHFTsZa3cSS77NMLqsakue5c8AzgoEnbmmDqr/LuFKwNJwV3Jo4Jk3NAlRC5luZ jxPHMRh0cJtjmBq1ajUX/Ie2k7wyl5iiOoGHziLChzWrZ8Ey1zL0wl0lP7xOUryiZlI7 kaB/w1E7N5Hi94GdR6aaa/vVVywRO49SDrIUwaMvlN53Fy9v+1ZTDwQoFpjLlSLDmLi4 ammfmsUSKPfyfRCzPplCYeVkG4HakypEFGfeczV/2duBLLKHvRgoNrlNQz6RrHtWCTVO N1IA== X-Gm-Message-State: AOAM530MEA0J7N2+dCT+YTEtrHv9mWRehU2X2gCwIXlIgiZNwv5ZKIjI Qy3Rt/jfoY4vEINRi7eKM1fDC8h1DCUBZA== X-Google-Smtp-Source: ABdhPJx8M34Jqudn+O7qt9s0T7LEpaOHybo4QPS7jVDej6XLKYqSuDT98UKWY293fhJMoNuTSyaqcw== X-Received: by 2002:a17:906:7012:: with SMTP id n18mr3955316ejj.236.1623657047480; Mon, 14 Jun 2021 00:50:47 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:47 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 16/45] trace-cmd dump: Add support for trace files version 7 Date: Mon, 14 Jun 2021 10:50:00 +0300 Message-Id: <20210614075029.598048-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added support to read and dump information of trace files version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 125 +++++++++++++++++++++++++++++++++++------- 1 file changed, 105 insertions(+), 20 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 03cc82b4..ed82104b 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -26,6 +26,9 @@ static struct tep_handle *tep; static unsigned int trace_cpus; static int has_clock; +static unsigned long file_version; +static bool read_compress; +static struct tracecmd_compression *compress; enum dump_items { SUMMARY = (1 << 0), @@ -52,46 +55,62 @@ enum dump_items verbosity; tracecmd_plog(fmt, ##__VA_ARGS__); \ } while (0) -static int read_file_string(int fd, char *dst, int len) +static int read_fd(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, 1); + r = read(fd, dst+size, len); if (r > 0) { - size++; - len--; + size += r; + len -= r; } else break; - if (!dst[size - 1]) - break; - } while (r > 0 && len); + } while (r > 0); - if (!size || dst[size - 1]) + if (len) return -1; - return 0; + return size; } -static int read_file_bytes(int fd, char *dst, int len) +static int read_compressed(int fd, char *dst, int len) +{ + + if (read_compress) + return tracecmd_compress_read(compress, dst, len); + return read_fd(fd, dst, len); +} + +static int read_file_string(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, len); + r = read_compressed(fd, dst+size, 1); if (r > 0) { - size += r; - len -= r; + size++; + len--; } else break; - } while (r > 0); + if (!dst[size - 1]) + break; + } while (r > 0 && len); - if (len) + if (!size || dst[size - 1]) return -1; return 0; } +static int read_file_bytes(int fd, char *dst, int len) +{ + int ret; + + ret = read_compressed(fd, dst, len); + return ret < 0 ? ret : 0; +} + static void read_dump_string(int fd, int size, enum dump_items id) { char buf[DUMP_SIZE]; @@ -146,7 +165,6 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; - unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -168,11 +186,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); - ver = strtol(buf, NULL, 10); - if (!ver && errno) + file_version = strtol(buf, NULL, 10); + if (!file_version && errno) die("Invalid file version string %s", buf); - if (!tracecmd_is_version_supported(ver)) - die("Unsupported file version %lu", ver); + if (!tracecmd_is_version_supported(file_version)) + die("Unsupported file version %lu", file_version); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) @@ -192,6 +210,29 @@ static void dump_initial_format(int fd) do_print(SUMMARY, "\t\t%d\t[Page size, bytes]\n", val4); } +static void dump_compress(int fd) +{ + char compr[DUMP_SIZE]; + char *ver = NULL; + + if (file_version < 7) + return; + + /* get compression header */ + if (read_file_string(fd, compr, DUMP_SIZE)) + die("no compression header"); + ver = strchr(compr, ' '); + if (!ver) + die("no compression version"); + *ver = '\0'; + do_print((SUMMARY), "\t\t%s\t[Compression algorithm]\n", compr); + do_print((SUMMARY), "\t\t%s\t[Compression version]\n", ver + 1); + + compress = tracecmd_compress_alloc(compr, ver + 1, fd, tep, NULL); + if (!compress) + die("cannot uncomress the file"); +} + static void dump_header_page(int fd) { unsigned long long size; @@ -234,11 +275,34 @@ static void dump_header_event(int fd) read_dump_string(fd, size, HEAD_EVENT); } +static void uncompress_reset(void) +{ + if (compress && file_version >= 7) { + read_compress = false; + tracecmd_compress_reset(compress); + } +} + +static int uncompress_block(void) +{ + int ret = 0; + + if (compress && file_version >= 7) { + ret = tracecmd_uncompress_block(compress); + if (!ret) + read_compress = true; + + } + return ret; +} + static void dump_ftrace_events_format(int fd) { unsigned long long size; unsigned int count; + if (uncompress_block()) + die("cannot uncompress file block"); do_print((SUMMARY | FTRACE_FORMAT), "\t[Ftrace format, "); if (read_file_number(fd, &count, 4)) die("cannot read the count of the ftrace events"); @@ -251,6 +315,7 @@ static void dump_ftrace_events_format(int fd) read_dump_string(fd, size, FTRACE_FORMAT); count--; } + uncompress_reset(); } static void dump_events_format(int fd) @@ -262,6 +327,9 @@ static void dump_events_format(int fd) do_print((SUMMARY | EVENT_FORMAT | EVENT_SYSTEMS), "\t[Events format, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &systems, 4)) die("cannot read the count of the event systems"); @@ -284,6 +352,7 @@ static void dump_events_format(int fd) } systems--; } + uncompress_reset(); } static void dump_kallsyms(int fd) @@ -292,12 +361,17 @@ static void dump_kallsyms(int fd) do_print((SUMMARY | KALLSYMS), "\t[Kallsyms, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the kallsyms"); do_print((SUMMARY | KALLSYMS), "%d bytes]\n", size); read_dump_string(fd, size, KALLSYMS); + + uncompress_reset(); } static void dump_printk(int fd) @@ -306,12 +380,17 @@ static void dump_printk(int fd) do_print((SUMMARY | TRACE_PRINTK), "\t[Trace printk, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the trace printk"); do_print((SUMMARY | TRACE_PRINTK), "%d bytes]\n", size); read_dump_string(fd, size, TRACE_PRINTK); + + uncompress_reset(); } static void dump_cmdlines(int fd) @@ -320,12 +399,17 @@ static void dump_cmdlines(int fd) do_print((SUMMARY | CMDLINES), "\t[Saved command lines, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 8)) die("cannot read the size of the saved command lines"); do_print((SUMMARY | CMDLINES), "%d bytes]\n", size); read_dump_string(fd, size, CMDLINES); + + uncompress_reset(); } static void dump_cpus_count(int fd) @@ -669,6 +753,7 @@ static void dump_file(const char *file) do_print(SUMMARY, "\n Tracing meta data in file %s:\n", file); dump_initial_format(fd); + dump_compress(fd); dump_header_page(fd); dump_header_event(fd); dump_ftrace_events_format(fd); From patchwork Mon Jun 14 07:50:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AC65C48BE8 for ; Mon, 14 Jun 2021 07:51:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAA5761370 for ; Mon, 14 Jun 2021 07:51:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232606AbhFNHxw (ORCPT ); Mon, 14 Jun 2021 03:53:52 -0400 Received: from mail-ej1-f43.google.com ([209.85.218.43]:35671 "EHLO mail-ej1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232603AbhFNHxv (ORCPT ); Mon, 14 Jun 2021 03:53:51 -0400 Received: by mail-ej1-f43.google.com with SMTP id h24so15073348ejy.2 for ; Mon, 14 Jun 2021 00:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j8CaV7j/Xvc/zZdVkqv6nWG2mQY5ije+yLHSgTZXxSs=; b=d1hLkVMQJUWdXJqgoN8xdA+KDG5fynwIWa7gQ9lTRrw8fv2zuR7mF7KUUwtns//JFT aCLIzQvIbSlf0Hwz4HEDdocTuCIfbaN4aplw7CI75/qmjemQOAqTnxof7+DxR4ntEx7C 1wnd2mKCA5A78e3rr7KTE4iuIIkEOAMaOrccKPiu7gzkeF+6URqRs0B1cpbdxMoxsIpI /X5c5t1jqQMSOVrBbyVJidKBqlsyCtm7r6wBzFrHVQBUebqxdkqZCJi6/Q8OdlBy9XhQ MsYyTWScjkJV93lljm0VWxxzqW3qMZgvNHTPoHzrU3esiJVVrOSYx+YdJASassSxhatw aX1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j8CaV7j/Xvc/zZdVkqv6nWG2mQY5ije+yLHSgTZXxSs=; b=if7UJhsUetb1TKAtctod+Qea7Xhm4xKNywDEZdYMXv8d6q61magFdJ1TYaNZX23DS7 +ojgxt7b+f+NUVCJS7zQ2MXwuRmgnyofE60a4fmqhfjyWazPxgkW44JVOjrE/Qz5whZ6 TULZUrBhaJ9FIQk3cCbDJOqzaNqqYqRH3AVhabb1nF+ueaDRxmbSwZdDmTyA+O4zQ6aI Q4wywalYK9YmSlpKSX123KzHV9P7c0mUJ6chGclYEQUrgHehxoEuhx4z54afbTMayl+O 4J84ONh4OkSPej1gq0EMqn9S6no0wzFfUhgX/VNa1XR//z+ije2AnLQPNov+7wa2VNuM 6CLg== X-Gm-Message-State: AOAM531c70/OPY5R2nWqZ5ehyP0D8kMWKNICiFS+9T5dEoKe2dDVA1w2 3tC3GTyNS1pqgmWQnxi/LxI= X-Google-Smtp-Source: ABdhPJwbflxDN31qgneZxIvXwx3ChIIhz78eTjXow0sGzXM3HlJis47CHbRwj1cpv2nbYI8nLwPDAw== X-Received: by 2002:a17:906:3d9:: with SMTP id c25mr14391514eja.63.1623657048409; Mon, 14 Jun 2021 00:50:48 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:48 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 17/45] trace-cmd library: Add support for zlib compression library Date: Mon, 14 Jun 2021 10:50:01 +0300 Message-Id: <20210614075029.598048-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If libz is available, use that library to provide trace file compression support. The library is detected runtime. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 10 ++ lib/trace-cmd/include/trace-cmd-local.h | 5 + lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++++++++++++++++ lib/trace-cmd/trace-compress.c | 8 ++ 4 files changed, 195 insertions(+) create mode 100644 lib/trace-cmd/trace-compress-zlib.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index bab4322d..83ba7016 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -7,6 +7,13 @@ ldir:=$(src)/lib/trace-cmd DEFAULT_TARGET = $(LIBTRACECMD_STATIC) +pound := \# +ZLIB_INSTALLED := $(shell if (printf "$(pound)include \n void main(){deflateInit(NULL, Z_BEST_COMPRESSION);}" | $(CC) -o /dev/null -x c - -lz >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) +ifeq ($(ZLIB_INSTALLED), 1) +CFLAGS += -DHAVE_ZLIB +$(info Have zlib compression support) +endif + OBJS = OBJS += trace-hash.o OBJS += trace-hooks.o @@ -26,6 +33,9 @@ OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif OBJS += trace-compress.o +ifeq ($(ZLIB_INSTALLED), 1) +OBJS += trace-compress-zlib.o +endif # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 419bfee0..40a6a40d 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,6 +31,11 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +#ifdef HAVE_ZLIB +int tracecmd_zlib_init(void); +void tracecmd_zlib_free(void); +#endif + void tracecmd_compress_init(void); void tracecmd_compress_free(void); diff --git a/lib/trace-cmd/trace-compress-zlib.c b/lib/trace-cmd/trace-compress-zlib.c new file mode 100644 index 00000000..3208d57b --- /dev/null +++ b/lib/trace-cmd/trace-compress-zlib.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include + +#include "trace-cmd-private.h" + +#define __ZLIB_NAME "zlib" +#define __ZLIB_WEIGTH 10 +#define __ZLIB_FILE "libz.so" +#define ZLIB_FUNC_COMPRESS "compress2" +#define ZLIB_FUNC_DECOMOPRESS "uncompress" +#define ZLIB_FUNC_SIZE "compressBound" +#define ZLIB_FUNC_VERSION "zlibVersion" + +static void *zlib_handle; +static int (*_lib_compress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes, int level); +static int (*_libz_decompress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes); +static unsigned long (*_libz_compress_bound)(unsigned long in_bytes); +static const char *(*_libz_version)(void); + +static int zlib_compress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_lib_compress) + return -1; + + ret = _lib_compress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes, Z_BEST_COMPRESSION); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_STREAM_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static int zlib_decompress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_libz_decompress) + return -1; + + ret = _libz_decompress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_DATA_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static unsigned int zlib_compress_bound(unsigned int in_bytes) +{ + if (!_libz_compress_bound) + return 0; + return _libz_compress_bound(in_bytes); +} + +static bool zlib_is_supported(const char *name, const char *version) +{ + const char *zver; + + if (!name) + return false; + if (strlen(name) != strlen(__ZLIB_NAME) || strcmp(name, __ZLIB_NAME)) + return false; + + if (!version) + return true; + + if (!_libz_version) + return false; + zver = _libz_version(); + if (!zver) + return false; + + /* Compare the major version number */ + if (atoi(version) <= atoi(zver)) + return true; + + return false; +} + +int tracecmd_zlib_init(void) +{ + if (zlib_handle) + return 0; + + zlib_handle = dlopen(__ZLIB_FILE, RTLD_NOW | RTLD_GLOBAL); + if (!zlib_handle) + return -1; + _lib_compress = dlsym(zlib_handle, ZLIB_FUNC_COMPRESS); + if (!_lib_compress) + goto error; + _libz_decompress = dlsym(zlib_handle, ZLIB_FUNC_DECOMOPRESS); + if (!_libz_decompress) + goto error; + _libz_compress_bound = dlsym(zlib_handle, ZLIB_FUNC_SIZE); + if (!_libz_compress_bound) + goto error; + _libz_version = dlsym(zlib_handle, ZLIB_FUNC_VERSION); + if (!_libz_version) + goto error; + + return tracecmd_compress_proto_register(__ZLIB_NAME, _libz_version(), __ZLIB_WEIGTH, + zlib_compress, zlib_decompress, + zlib_compress_bound, zlib_is_supported); + +error: + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + dlclose(zlib_handle); + zlib_handle = NULL; + return -1; +} + +void tracecmd_zlib_free(void) +{ + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + + if (zlib_handle) { + dlclose(zlib_handle); + zlib_handle = NULL; + } + +} diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index 378a3e57..039f60c5 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -355,6 +355,10 @@ void tracecmd_compress_init(void) gettimeofday(&time, NULL); srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); + +#ifdef HAVE_ZLIB + tracecmd_zlib_init(); +#endif } static struct compress_proto *compress_proto_select(void) @@ -531,6 +535,10 @@ void tracecmd_compress_free(void) struct compress_proto *proto = proto_list; struct compress_proto *del; +#ifdef HAVE_ZLIB + tracecmd_zlib_free(); +#endif + while (proto) { del = proto; proto = proto->next; From patchwork Mon Jun 14 07:50:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06778C48BE6 for ; Mon, 14 Jun 2021 07:50:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D05826124B for ; Mon, 14 Jun 2021 07:50:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232576AbhFNHwy (ORCPT ); Mon, 14 Jun 2021 03:52:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232557AbhFNHwx (ORCPT ); Mon, 14 Jun 2021 03:52:53 -0400 Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB941C061787 for ; Mon, 14 Jun 2021 00:50:50 -0700 (PDT) Received: by mail-ej1-x635.google.com with SMTP id l1so14990028ejb.6 for ; Mon, 14 Jun 2021 00:50:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IJNFqqX/hbiSS7OIa+JNEw+bJG0elOcOWFPIQmVJ/Kw=; b=U5WcTA+E3FIoVqH8Ub9onfIRxndh5DSTjp+/X2WAOZltunBEoWvgQBMD3Xhy1vTmr4 ss8cIzw9h/q3pxVXkmbgiESdNobWwB5NHB5lUUeWJnBHlemCQ9q5+Nv+eHDgnu77tY9r EXEZ6WdXEVQE9dLP6rkX7L4hTZ+fsIFeFCIpYTF9PdoCHN8jMMaC16g9JQqGHslbccvD vV6t2/quSqGklntmsgmk8BYJ17wLhj7okZXYwelY+owulqMUTS6lPetVYOiwSjxSrsD0 0cpdfm+e8IOV6+kN+iJN5Biw8KbgULpz4yj1YZ4lw+YTvhdyv8d2IEwyK+acnTR2PLPL zfrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IJNFqqX/hbiSS7OIa+JNEw+bJG0elOcOWFPIQmVJ/Kw=; b=WhEMyGVUHRxXsZqMLc4b3qZhXlgh9vCiENGCZx+kdL9W90yVz+QsgFNO5Qr55q+tSw WqFMrwxxxaBdggOJrAFufEwmKjK7e3S69R9cbx8jncLQQurFcCEhnrziKAJz3f6x+wTp ych17zueM7Z7yHhSHXiaI5XTa2t/+XnXqeC7x2ib4rpT/ZcVYtFXbB1m3bC9J73fXLnZ WFiZtXxUJEVVpNZWrPo75jb7EiT6sY4fw1ku4Mj6TF0IJ59iL5IGchBM4mSWbhISc+zp Hii659WRnk4Xt7ka4dzcCBUKWjrIa4o/05J3bHB5sniPsl4dihdKNGmgOow3nYp65dy3 Tn5w== X-Gm-Message-State: AOAM532Td6SR6ppbsbmuv4S5WOUW6EoqvpQFdjTLlJZJtEmsUffTox/f 7H8nLpBE/GWjF0WUWsiz5jfLjCm4K+zvqA== X-Google-Smtp-Source: ABdhPJwqK2ZjuqAuwkcx7B/sl9RSxyGLqYXaYAB9p5w7t3OLoWje7AbUP4JSInECxIxKXsUMHUEs+g== X-Received: by 2002:a17:906:fa13:: with SMTP id lo19mr14489986ejb.468.1623657049395; Mon, 14 Jun 2021 00:50:49 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:49 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 18/45] trace-cmd library: Hide the logic for updating buffer offset Date: Mon, 14 Jun 2021 10:50:02 +0300 Message-Id: <20210614075029.598048-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a trace buffer data are written in the trace file, the buffer option in the file metadata is updated with the file offset of the tracing data. Hide this logic into the trace-cmd library. Added new APIs: tracecmd_add_buffer_description() tracecmd_write_buffers_description() Changed APIs: tracecmd_append_buffer_cpu_data() Removed APIs: tracecmd_add_buffer_option() This internal refactoring is needed for changes, related to compression of the options sections of the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 8 +- lib/trace-cmd/trace-output.c | 84 +++++++++++++++++-- tracecmd/trace-record.c | 16 ++-- 3 files changed, 85 insertions(+), 23 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ee73325c..cbb578ec 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -295,8 +295,8 @@ struct tracecmd_option * tracecmd_add_option_v(struct tracecmd_output *handle, unsigned short id, const struct iovec *vector, int count); -struct tracecmd_option *tracecmd_add_buffer_option(struct tracecmd_output *handle, - const char *name, int cpus); +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus); +int tracecmd_write_buffers_description(struct tracecmd_output *handle); int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); @@ -312,9 +312,7 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files); - + const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 7c7d3d76..8f8ca164 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -44,6 +44,14 @@ struct tracecmd_option { struct list_head list; }; +struct tracecmd_buffer { + int cpus; + void *name; + tsize_t offset; + struct tracecmd_option *option; + struct list_head list; +}; + enum { OUTPUT_FL_SEND_META = (1 << 0), }; @@ -63,6 +71,7 @@ struct tracecmd_output { struct tracecmd_compression *compress; struct list_head options; + struct list_head buffers; struct tracecmd_msg_handle *msg_handle; char *trace_clock; }; @@ -189,6 +198,7 @@ bool tracecmd_get_quiet(struct tracecmd_output *handle) void tracecmd_output_free(struct tracecmd_output *handle) { struct tracecmd_option *option; + struct tracecmd_buffer *buffer; if (!handle) return; @@ -199,6 +209,13 @@ void tracecmd_output_free(struct tracecmd_output *handle) if (handle->pevent) tep_unref(handle->pevent); + while (!list_empty(&handle->buffers)) { + buffer = container_of(handle->buffers.next, + struct tracecmd_buffer, list); + list_del(&buffer->list); + free(buffer->name); + free(buffer); + } while (!list_empty(&handle->options)) { option = container_of(handle->options.next, struct tracecmd_option, list); @@ -1071,6 +1088,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, goto out_free; list_head_init(&handle->options); + list_head_init(&handle->buffers); buf[0] = 23; buf[1] = 8; @@ -1369,9 +1387,8 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -struct tracecmd_option * -tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, - int cpus) +static struct tracecmd_option * +add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; @@ -1399,6 +1416,53 @@ tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, return option; } +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus) +{ + struct tracecmd_buffer *buf; + + buf = calloc(1, sizeof(struct tracecmd_buffer)); + if (!buf) + return -1; + buf->name = strdup(name); + buf->cpus = cpus; + if (!buf->name) { + free(buf); + return -1; + } + list_add_tail(&buf->list, &handle->buffers); + return 0; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + struct tracecmd_option *option; + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + option = add_buffer_option(handle, buf->name, buf->cpus); + if (!option) + return -1; + buf->option = option; + } + return 0; +} + +static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) +{ + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + if (strlen(name) == strlen(buf->name) && !strcmp(name, buf->name)) { + if (handle->file_version >= 7) + return buf->offset; + if (!buf->option) + break; + return buf->option->offset; + } + } + return 0; +} + int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; @@ -1643,18 +1707,23 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, } int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files) + const char *name, int cpus, char * const *cpu_data_files) { + tsize_t b_offset; tsize_t offset; stsize_t ret; + b_offset = get_buffer_file_offset(handle, name); + if (!b_offset) { + tracecmd_warning("Cannot find description for buffer %s\n", name); + return -1; + } offset = lseek64(handle->fd, 0, SEEK_CUR); /* Go to the option data, where will write the offest */ - ret = lseek64(handle->fd, option->offset, SEEK_SET); + ret = lseek64(handle->fd, b_offset, SEEK_SET); if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); + tracecmd_warning("could not seek to %lld\n", b_offset); return -1; } @@ -1713,6 +1782,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); list_head_init(&handle->options); + list_head_init(&handle->buffers); if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index d3362e5b..eff6f2f0 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4152,7 +4152,6 @@ static void touch_file(const char *file) } static void append_buffer(struct tracecmd_output *handle, - struct tracecmd_option *buffer_option, struct buffer_instance *instance, char **temp_files) { @@ -4180,7 +4179,7 @@ static void append_buffer(struct tracecmd_output *handle, touch_file(temp_files[i]); } - tracecmd_append_buffer_cpu_data(handle, buffer_option, + tracecmd_append_buffer_cpu_data(handle, tracefs_instance_get_name(instance->tracefs), cpu_count, temp_files); for (i = 0; i < instance->cpu_count; i++) { @@ -4441,7 +4440,6 @@ static void write_guest_file(struct buffer_instance *instance) static void record_data(struct common_record_context *ctx) { - struct tracecmd_option **buffer_options; struct tracecmd_output *handle; struct buffer_instance *instance; bool local = false; @@ -4512,9 +4510,6 @@ static void record_data(struct common_record_context *ctx) } if (buffers) { - buffer_options = malloc(sizeof(*buffer_options) * buffers); - if (!buffer_options) - die("Failed to allocate buffer options"); i = 0; for_each_instance(instance) { int cpus = instance->cpu_count != local_cpu_count ? @@ -4522,10 +4517,9 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; - - buffer_options[i++] = tracecmd_add_buffer_option(handle, - tracefs_instance_get_name(instance->tracefs), - cpus); + tracecmd_add_buffer_description(handle, + tracefs_instance_get_name(instance->tracefs), + cpus); add_buffer_stat(handle, instance); } } @@ -4560,7 +4554,7 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; print_stat(instance); - append_buffer(handle, buffer_options[i++], instance, temp_files); + append_buffer(handle, instance, temp_files); } } From patchwork Mon Jun 14 07:50:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 649CAC48BE6 for ; Mon, 14 Jun 2021 07:52:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 49A2D61357 for ; Mon, 14 Jun 2021 07:52:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232624AbhFNHyK (ORCPT ); Mon, 14 Jun 2021 03:54:10 -0400 Received: from mail-ej1-f54.google.com ([209.85.218.54]:33581 "EHLO mail-ej1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232620AbhFNHyJ (ORCPT ); Mon, 14 Jun 2021 03:54:09 -0400 Received: by mail-ej1-f54.google.com with SMTP id g20so15141901ejt.0 for ; Mon, 14 Jun 2021 00:51:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Iy0KlMckZ6j95jaPz7nvNgAmigL8aRzxAtBdyEFv58I=; b=ueoaHVAMZyGZ2puDo23TvFuzERaLPjj7uJu9KMKidECm00vQatUXa+hbtpooCQi3s1 8VqruoWd9wZX5zkpKvk0jdS+w7eMVqpDbR15pePiQG6nr04vNi29ZIbNZx6LlbbshT8E yCvLmBFl58XWwNO8YY1EIUqadqZvTPi0UcPjGqxNBtKT5KS8Qn9uAHlpTx0YzooTLF3A F9ZXg5wMNq2XuqWkhnXoCIOfI72vhl9YR6PD1w6QKtuEoP55QdZus5M3eim89DJHsTuQ qeZuR4m8c5DoDVv04h8zUyMOjQmAL8ZoSWbayLFnCOB6tghS7jGrtbzz/tM3JgCCcRLC dehg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Iy0KlMckZ6j95jaPz7nvNgAmigL8aRzxAtBdyEFv58I=; b=Zq2eJS/LsMHhyL7kngid+mPVLHXSMCr58KAAVCWvETmMDth/uvuAGSjpAwvoY3BJm6 Z3VMaPbw4qy0Od9IF7Rsf2nk9jNrIrW6qSGy5/mppLHQx5blzPRBKPpXTDh0dmiipJM+ lAd1GEu7RRb5vXFdZeD8X0BNBj7o9N2GsVFesM8krJAWGTLwMr96PzRaWeBf4tXp4cy7 2KQFvI+S95SgwLsLq3uYHWbQ8yJDUy0r2rezQXnA8V8f6DjqERECwUsYjIDK0o3ZzcA/ BA6w1//cD/7485ajPD7LzjNpJAa7RAbeYCbh67y66m91PjPtLBJ74j6wej2QFJnZlD2h fLHw== X-Gm-Message-State: AOAM532fI9kav5UV2S6Muk0unLtVYQXGJ7uRdiDpjhiPmd2d9uefvGtt EFcr2tOIneWB+M9aPc0cnXU= X-Google-Smtp-Source: ABdhPJxcwXFjvG4gYAzKqVr+ilkDgI155cS29+n+oqWEtFXNFCamo4wBdTyH92WULwKAj2bE8UIo6g== X-Received: by 2002:a17:907:2057:: with SMTP id pg23mr13911088ejb.113.1623657050386; Mon, 14 Jun 2021 00:50:50 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:49 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 19/45] trace-cmd: Move buffers description outside of options Date: Mon, 14 Jun 2021 10:50:03 +0300 Message-Id: <20210614075029.598048-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new buffer data are written in the trace file, the file offset is stored in the buffer description. As the buffer description is part of the trace options, which are already written in the file, the offset field in this option is updated with the new value. This causes a serious problem when option section of the file is compressed. Updating a value in already compressed section may change the size of this section, which can make the calculated offset of the buffer trace data invalid. To solve that problem and to enable the compression of the option section of the file, the structure of the trace file is changed: - In trace file v7, the description of the buffers is moved to a separate section in the trace file medatata. It is stored after the CPU count section and before the option section. The format of this new buffers description is: 4 bytes - count of the buffers, stored in the file. The top buffer is not counted. For each buffer: 8 bytes, offset of the trace data for this buffer in the file. null-terminated string, name of the buffer. The new section is mandatory, in case of no buffers - the count 0 is stored in the first 4 bytes. - In trace file v6, the description of the buffers is not changed. These changes are needed for compression of the options sections from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-input.c | 40 +++++++++ lib/trace-cmd/trace-output.c | 85 ++++++++++++++++++- tracecmd/trace-dump.c | 24 ++++++ tracecmd/trace-listen.c | 3 + tracecmd/trace-record.c | 4 + 7 files changed, 156 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index cbb578ec..faa8c533 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -112,6 +112,7 @@ enum tracecmd_file_states { TRACECMD_FILE_PRINTK, TRACECMD_FILE_CMD_LINES, TRACECMD_FILE_CPU_COUNT, + TRACECMD_FILE_BUFERS, TRACECMD_FILE_OPTIONS, TRACECMD_FILE_CPU_LATENCY, TRACECMD_FILE_CPU_FLYRECORD, diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 40a6a40d..2f421103 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -46,4 +46,7 @@ void out_compression_reset(struct tracecmd_output *handle); void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); +int write_buffers_description_v7(struct tracecmd_output *handle); +int write_buffers_description_v6(struct tracecmd_output *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 0b17b58d..e953909f 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -881,6 +881,39 @@ static int read_cpus(struct tracecmd_input *handle) return 0; } +static int read_buffers(struct tracecmd_input *handle) +{ + struct input_buffer_instance *buffer; + unsigned long long offset; + unsigned int count; + int i; + + if (handle->file_version < 7 || handle->file_state >= TRACECMD_FILE_BUFERS) + return 0; + + if (read4(handle, &count) < 0) + return -1; + i = handle->nr_buffers; + handle->nr_buffers += count; + handle->buffers = realloc(handle->buffers, + sizeof(*handle->buffers) * handle->nr_buffers); + if (!handle->buffers) + return -1; + + for (; i < handle->nr_buffers; i++) { + buffer = &handle->buffers[i]; + if (read8(handle, &offset) < 0) + return -1; + buffer->offset = offset; + buffer->name = read_string(handle); + if (!buffer->name) + return -1; + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +} + /** * tracecmd_read_headers - read the header information from trace.dat * @handle: input handle for the trace.dat file @@ -955,6 +988,13 @@ int tracecmd_read_headers(struct tracecmd_input *handle, if (state <= handle->file_state) return 0; + if (read_buffers(handle) < 0) + return -1; + + if (state <= handle->file_state) + return 0; + + if (read_options_type(handle) < 0) return -1; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 8f8ca164..7830f643 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -376,6 +376,17 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) if (!handle) return -1; + if (handle->file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, handle->file_version); + return -1; + default: + break; + } + } + switch (new_state) { case TRACECMD_FILE_HEADERS: case TRACECMD_FILE_FTRACE_EVENTS: @@ -384,10 +395,16 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) case TRACECMD_FILE_PRINTK: case TRACECMD_FILE_CMD_LINES: case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_OPTIONS: + case TRACECMD_FILE_BUFERS: if (handle->file_state == (new_state - 1)) return 0; break; + case TRACECMD_FILE_OPTIONS: + if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) + return 0; + if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) + return 0; + break; case TRACECMD_FILE_CPU_LATENCY: case TRACECMD_FILE_CPU_FLYRECORD: if (handle->file_state == TRACECMD_FILE_OPTIONS) @@ -1433,7 +1450,7 @@ int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char * return 0; } -int tracecmd_write_buffers_description(struct tracecmd_output *handle) +__hidden int write_buffers_description_v6(struct tracecmd_output *handle) { struct tracecmd_option *option; struct tracecmd_buffer *buf; @@ -1447,6 +1464,63 @@ int tracecmd_write_buffers_description(struct tracecmd_output *handle) return 0; } +__hidden int write_buffers_description_v7(struct tracecmd_output *handle) +{ + struct tracecmd_buffer *buf; + char *data = NULL; + int count = 0; + int size; + int ret; + + if (!check_out_state(handle, TRACECMD_FILE_BUFERS)) { + tracecmd_warning("Cannot write buffers descriptions"); + return -1; + } + + list_for_each_entry(buf, &handle->buffers, list) { + count++; + } + + count = convert_endian_4(handle, count); + ret = do_write_check(handle, &count, 4); + if (ret < 0) + return ret; + + list_for_each_entry(buf, &handle->buffers, list) { + size = 8 + strlen(buf->name) + 1; + data = malloc(size); + if (!data) + goto error; + *(tsize_t *)data = 0; + strcpy(data + 8, buf->name); + /* Save the data location in case it needs to be updated */ + buf->offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, data, size)) + goto error; + + /* + * In case a buffer instance has different number of CPUs as the + * local machine. + */ + if (buf->cpus) + tracecmd_add_option(handle, TRACECMD_OPTION_CPUCOUNT, + sizeof(int), &buf->cpus); + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +error: + return -1; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + if (handle->file_version >= 7) + return write_buffers_description_v7(handle); + + return write_buffers_description_v6(handle); +} + static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) { struct tracecmd_buffer *buf; @@ -1505,7 +1579,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_cpus(handle, cpus) < 0) goto out_free; - + if (tracecmd_write_buffers_description(handle) < 0) + goto out_free; if (tracecmd_write_options(handle) < 0) goto out_free; @@ -1698,7 +1773,9 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, ret = tracecmd_write_cpus(handle, cpus); if (ret) return ret; - + ret = tracecmd_write_buffers_description(handle); + if (ret) + return ret; ret = tracecmd_write_options(handle); if (ret) return ret; diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index ed82104b..56acb01d 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -420,6 +420,29 @@ static void dump_cpus_count(int fd) do_print(SUMMARY, "\t%d [CPUs with tracing data]\n", trace_cpus); } +static void dump_buffers(int fd) +{ + unsigned long long offset; + char name[DUMP_SIZE]; + int count; + + if (file_version < 7) + return; + + if (read_file_number(fd, &count, 4)) + die("cannot read the buffers count"); + do_print(SUMMARY, "\t%d [buffers with tracing data]\n", count); + + while (count) { + if (read_file_number(fd, &offset, 8)) + die("cannot read the buffer offset"); + if (read_file_string(fd, name, DUMP_SIZE)) + die("cannot read the buffer name"); + do_print(SUMMARY, "\t\t 0x%llx %s[data offset, buffer name]\n", offset, name); + count--; + } +} + static void dump_option_string(int fd, int size, char *desc) { do_print(OPTIONS, "\t\t[Option %s, %d bytes]\n", desc, size); @@ -762,6 +785,7 @@ static void dump_file(const char *file) dump_printk(fd); dump_cmdlines(fd); dump_cpus_count(fd); + dump_buffers(fd); dump_therest(fd); tep_free(tep); diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index 0ae1c948..62debdb6 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -604,6 +604,9 @@ static int put_together_file(int cpus, int ofd, const char *node, if (write_options) { ret = tracecmd_write_cpus(handle, cpus); + if (ret) + goto out; + ret = tracecmd_write_buffers_description(handle); if (ret) goto out; ret = tracecmd_write_options(handle); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index eff6f2f0..2a379d09 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3711,6 +3711,9 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; ret = tracecmd_write_cpus(network_handle, instance->cpu_count); + if (ret) + goto error; + ret = tracecmd_write_buffers_description(network_handle); if (ret) goto error; ret = tracecmd_write_options(network_handle); @@ -4074,6 +4077,7 @@ static void setup_agent(struct buffer_instance *instance, add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_buffers_description(network_handle); tracecmd_write_options(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); instance->network_handle = network_handle; From patchwork Mon Jun 14 07:50:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1035CC48BE6 for ; Mon, 14 Jun 2021 07:51:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE82C61357 for ; Mon, 14 Jun 2021 07:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232607AbhFNHxy (ORCPT ); Mon, 14 Jun 2021 03:53:54 -0400 Received: from mail-ej1-f41.google.com ([209.85.218.41]:38579 "EHLO mail-ej1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232603AbhFNHxy (ORCPT ); Mon, 14 Jun 2021 03:53:54 -0400 Received: by mail-ej1-f41.google.com with SMTP id og14so15066183ejc.5 for ; Mon, 14 Jun 2021 00:51:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LfohAIDa1mQgoxdChOqInkj7fzUNXVsse9vwxp5gcE4=; b=g780ofVyD8yXCcCay5uLyoKPrbhYVxC11g8luvnDkCNXqoB7UlmY9m9MLvnqd/xTg1 fjMjVa/lvSMjzLAUYFoCW4ouGPy9/02pLUedx1clfpiGZlbnfOybS+5qYL2BTDNjyRUJ mfRl0khWNL4HvNl3JbzaEz0rHcSbQjUIJm3O6R2Hz8Vpv3brGVLXE2Pba0vq2LUhabzw VAXG5Z0cVzJB/47GUJeZfUpdnWB1yJkDSnyMpHdpc4Q/BhlJVgMCAZ7WwnLc53LJrlvk k8dD3k1bD6ol0gNVM8UWuZNGHwhz/YHnm1R6x33f4GdsZBS+RTioODnalehtw17HjIrL hhzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LfohAIDa1mQgoxdChOqInkj7fzUNXVsse9vwxp5gcE4=; b=aGWxe1KYzZ2YiTdrh0FS151Qu3+9x7R/x2QzOzPMWj6ierJoO0SoRC7cISw9wTmh6K Z//95EHDwEtlHy7VI+l+0vnOl3HtNxvN/KF7imn5j9Q8JrqdE2+KSV5dl8v2vZXMzcn9 FA0kZk0bgFTZiuKe93p6LgzBC9Dk7VMETUDhoJTcEpKa3cCfbYUaDmQrxWxWh3cvofzK JrkLmHznRqTAu5bbPtN8YFJR3J+8eCsU9y3yb002t9hsodGWiyr96vESAcgmaTKWpw9z 5prgGcVAmOLrCLf6ffIaJwpyKBx4KrcrAmX8gHq3R4m2Gd3/F5vWUZkd+E97SPy+r7VZ Gf8A== X-Gm-Message-State: AOAM531c0YqyAS2vIeKrvCVh3CNAc3tYJj17fk3OnqIvq5k1Q1hccYuV ZaiVjASdP33F3zIR3te7XkA= X-Google-Smtp-Source: ABdhPJx6mqEa+uiAQDpx5D+rj1V72VzT+QALuoeNE52dNShxIir+C4KHuQu97XYgRtiVb90eFAJ9xQ== X-Received: by 2002:a17:906:5488:: with SMTP id r8mr14260934ejo.374.1623657051216; Mon, 14 Jun 2021 00:50:51 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:50 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 20/45] trace-cmd library: Track the offset in the option section in the trace file Date: Mon, 14 Jun 2021 10:50:04 +0300 Message-Id: <20210614075029.598048-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading / writing trace file, store the offset of the option section in the file. A new internal API is added, to get the offset of the option section from an input file handler: tracecmd_get_options_offset() These changes are needed for compression of the option section from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 11 +++++++++++ lib/trace-cmd/trace-output.c | 2 ++ 3 files changed, 14 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index faa8c533..10089389 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -239,6 +239,7 @@ tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_in_file_version(struct tracecmd_input *handle); int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, const char **name, const char **version); +size_t tracecmd_get_options_offset(struct tracecmd_input *handle); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e953909f..b76f2b47 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -157,6 +157,7 @@ struct tracecmd_input { size_t header_files_start; size_t ftrace_files_start; size_t event_files_start; + size_t options_start; size_t total_file_size; /* For custom profilers. */ @@ -2759,6 +2760,7 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); for (;;) { if (do_read_check(handle, &option, 2)) @@ -4213,6 +4215,15 @@ bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle) return handle->use_trace_clock; } +/** + * tracecmd_get_options_offset - get offset of the options sections in the file + * @handle: input handle for the trace.dat file + */ +size_t tracecmd_get_options_offset(struct tracecmd_input *handle) +{ + return handle->options_start; +} + /** * tracecmd_get_show_data_func - return the show data func * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 7830f643..bad234c4 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -66,6 +66,7 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + size_t options_start; bool do_compress; struct tracecmd_compression *compress; @@ -1858,6 +1859,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_in_file_version(ihandle); + handle->options_start = tracecmd_get_options_offset(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Mon Jun 14 07:50:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28880C48BE8 for ; Mon, 14 Jun 2021 07:52:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 134B361380 for ; Mon, 14 Jun 2021 07:52:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232617AbhFNHyI (ORCPT ); Mon, 14 Jun 2021 03:54:08 -0400 Received: from mail-ed1-f43.google.com ([209.85.208.43]:35738 "EHLO mail-ed1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232629AbhFNHyH (ORCPT ); Mon, 14 Jun 2021 03:54:07 -0400 Received: by mail-ed1-f43.google.com with SMTP id ba2so43440161edb.2 for ; Mon, 14 Jun 2021 00:51:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OQ7uF0lhZZ996H2CtqnUokIHplGG0A0iEPr25+eVg7Q=; b=QPT5+ErOgUtIoznlYChmyV0+H8y5qY60jlyJmyfmHmkknMVANjEn/EqhWS+jYwrnUz pTiQEaWgMeuNBrwioG7KTQ9UzsVhM+wJaAn8ZPr2kg9D8oBtm4wkMWLRhYPthmXz+mFE fEqQDKUoHj8oxX6FA1FGgzfcIVOCHALxe7QGGiyEQGDE1kEP2asfn+hMucYwQdu1mEYA Tc36RWW1tgs9svPT/jtmnMemRIup+hgJi2e1hcJ8efiUtPEqax5OAf34pdmnaL8WnE/I rx7p+XYLtM/fNhOQzoipBBfe3u6qSkhU9UhIsbd+Vc0IphpLNtfwe5R1L2VPQaVfH7F0 zP7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OQ7uF0lhZZ996H2CtqnUokIHplGG0A0iEPr25+eVg7Q=; b=bGraWO4Hd24IK0HG52YuLB2qmXr4azkfM5mL07tZMTIwR5Z8C7N9kcpTRBtgB+z07F Ahm49vx5sR2cyoyKPz7OBg+wPyDaQ3hpDONfa6t4N71azxgZIW7Z8i2AH34JuaEjbyQz AvKeGKXwWdblYzl2D9zezmU7aXn5shuFbJR16v6zTTHJtE6g7GgNyjAUvdfPZEf73lKI RdgWvBZpMJ7E/mpbWMgi7+DEfp5X1SyyrDumWdnsqiAilpCdUO61sBM8ZsaOjN3kePOl dR+uFXnf2uClviDfFtM04HYDeLvJeThyv6uMTnRCQGppCe1n/xhtwn+fj6acr2UPR4IZ 7xtw== X-Gm-Message-State: AOAM5316e/lsCu/2xPZdkPI6F8Ezgq1zc5XYr/KDIV5dZBF97Mz9yqio lr4C5QWkWlLIpVtFYJBYLX0= X-Google-Smtp-Source: ABdhPJxQ2yDGhkqpnau1Ipu5mTAeNZPPXmhtcR1jJMgA0fU+cK9BguvhTStcat1bmLLmtuPj1ztpjQ== X-Received: by 2002:aa7:d857:: with SMTP id f23mr16229357eds.41.1623657052169; Mon, 14 Jun 2021 00:50:52 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:51 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 21/45] trace-cmd library: Add compression of the option section of the trace file Date: Mon, 14 Jun 2021 10:50:05 +0300 Message-Id: <20210614075029.598048-22-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Comperss the option section of the trace file. This section is not big currently and compressing it does not reduce significantly the size of the file. This could be useful in the future as new options can be added, storing a potentially huge amount of data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 46 +++++++++++------ lib/trace-cmd/trace-output.c | 69 +++++++++++++++++-------- tracecmd/trace-dump.c | 16 ++++-- 4 files changed, 92 insertions(+), 40 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 2f421103..03d2a9a3 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,7 @@ void tracecmd_zlib_free(void); void tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b76f2b47..8fff003e 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2761,23 +2761,29 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); - + if (in_uncompress_block(handle)) + return -1; for (;;) { - if (do_read_check(handle, &option, 2)) - return -1; + ret = do_read_check(handle, &option, 2); + if (ret) + goto out; if (option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (do_read_check(handle, &size, 4)) - return -1; + ret = do_read_check(handle, &size, 4); + if (ret) + goto out; size = tep_read_number(handle->pevent, &size, 4); buf = malloc(size); - if (!buf) - return -ENOMEM; - if (do_read_check(handle, buf, size)) - return -1; + if (!buf) { + ret = -ENOMEM; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret) + goto out; switch (option) { case TRACECMD_OPTION_DATE: @@ -2827,14 +2833,16 @@ static int handle_options(struct tracecmd_input *handle) buf + 8, 4); ret = tsync_cpu_offsets_load(handle, buf + 12, size - 12); if (ret < 0) - return ret; + goto out; tracecmd_enable_tsync(handle, true); break; case TRACECMD_OPTION_CPUSTAT: buf[size-1] = '\n'; cpustats = realloc(cpustats, cpustats_size + size + 1); - if (!cpustats) - return -ENOMEM; + if (!cpustats) { + ret = -ENOMEM; + goto out; + } memcpy(cpustats + cpustats_size, buf, size); cpustats_size += size; cpustats[cpustats_size] = 0; @@ -2844,14 +2852,17 @@ static int handle_options(struct tracecmd_input *handle) handle->nr_buffers++; handle->buffers = realloc(handle->buffers, sizeof(*handle->buffers) * handle->nr_buffers); - if (!handle->buffers) - return -ENOMEM; + if (!handle->buffers) { + ret = -ENOMEM; + goto out; + } buffer = &handle->buffers[handle->nr_buffers - 1]; buffer->name = strdup(buf + 8); if (!buffer->name) { free(handle->buffers); handle->buffers = NULL; - return -ENOMEM; + ret = -ENOMEM; + goto out; } offset = *(unsigned long long *)buf; buffer->offset = tep_read_number(handle->pevent, &offset, 8); @@ -2908,8 +2919,11 @@ static int handle_options(struct tracecmd_input *handle) } handle->cpustats = cpustats; + ret = 0; - return 0; +out: + in_uncompress_reset(handle); + return ret; } static int read_options_type(struct tracecmd_input *handle) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index bad234c4..f7e6c3dd 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -150,6 +150,18 @@ __hidden void out_compression_reset(struct tracecmd_output *handle) handle->do_compress = false; } +__hidden int out_uncompress_block(struct tracecmd_output *handle) +{ + int ret = 0; + + if (handle->file_version < 7) + return 0; + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->do_compress = true; + return ret; +} + __hidden int out_compression_start(struct tracecmd_output *handle) { if (handle->file_version < 7 || !handle->compress) @@ -1326,32 +1338,38 @@ int tracecmd_write_options(struct tracecmd_output *handle) if (do_write_check(handle, "options ", 10)) return -1; - + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + out_compression_start(handle); list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; + goto error; + if (out_compression_end(handle)) + goto error; handle->file_state = TRACECMD_FILE_OPTIONS; return 0; +error: + out_compression_reset(handle); + return -1; } int tracecmd_append_options(struct tracecmd_output *handle) @@ -1367,42 +1385,51 @@ int tracecmd_append_options(struct tracecmd_output *handle) * We can append only if options are already written and tracing data * is not yet written */ - if (handle->file_state != TRACECMD_FILE_OPTIONS) + if (handle->file_state != TRACECMD_FILE_OPTIONS || !handle->options_start) return -1; - - if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) return -1; - offset = lseek64(handle->fd, -2, SEEK_CUR); - if (offset == (off_t)-1) + if (out_uncompress_block(handle)) return -1; - - r = pread(handle->fd, &option, 2, offset); + if (do_lseek(handle, 0, SEEK_END) == -1) + goto error; + offset = do_lseek(handle, -2, SEEK_CUR); + if (offset == (off_t)-1) + goto error; + r = do_preed(handle, &option, 2, offset); if (r != 2 || option != TRACECMD_OPTION_DONE) - return -1; - + goto error; list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } - option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; + goto error; + if (handle->file_version >= 7 && handle->compress) { + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) + goto error; + if (out_compression_end(handle)) + goto error; + } return 0; +error: + out_compression_reset(handle); + return -1; } static struct tracecmd_option * diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 56acb01d..8d0f2251 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -82,6 +82,13 @@ static int read_compressed(int fd, char *dst, int len) return read_fd(fd, dst, len); } +static int do_lseek(int fd, int offset, int whence) +{ + if (read_compress) + return tracecmd_compress_lseek(compress, offset, whence); + return lseek64(fd, offset, whence); +} + static int read_file_string(int fd, char *dst, int len) { size_t size = 0; @@ -625,6 +632,9 @@ static void dump_options(int fd) unsigned int size; int count = 0; + if (uncompress_block()) + die("cannot uncompress file block"); + for (;;) { if (read_file_number(fd, &option, 2)) die("cannot read the option id"); @@ -635,7 +645,7 @@ static void dump_options(int fd) count++; if (!DUMP_CHECK(OPTIONS) && !DUMP_CHECK(CLOCK) && !DUMP_CHECK(SUMMARY)) { - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); continue; } switch (option) { @@ -685,12 +695,12 @@ static void dump_options(int fd) default: do_print(OPTIONS, " %d %d\t[Unknown option, size - skipping]\n", option, size); - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); break; } } do_print(SUMMARY, "\t[%d options]\n", count); - + uncompress_reset(); } static void dump_latency(int fd) From patchwork Mon Jun 14 07:50:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 183AFC48BE6 for ; Mon, 14 Jun 2021 07:51:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F31AC61370 for ; Mon, 14 Jun 2021 07:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232603AbhFNHx4 (ORCPT ); Mon, 14 Jun 2021 03:53:56 -0400 Received: from mail-ed1-f46.google.com ([209.85.208.46]:41955 "EHLO mail-ed1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232609AbhFNHx4 (ORCPT ); Mon, 14 Jun 2021 03:53:56 -0400 Received: by mail-ed1-f46.google.com with SMTP id g18so43225229edq.8 for ; Mon, 14 Jun 2021 00:51:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ad5WRDGSmHIxRG6OKUBwzd4Bx+HY5iH9CAWdyYF9xFg=; b=jhCXrCORuFFKHOp46YVSRSV8/zVfwSj2S3CIXijLkwpPETTbwhDIPu4+hAXXbFDjp0 inC+v3fzZJ9IAtlon3TlQqe5k6VgfIyVEM/ykmGqo4AQoEMN73nEx+tq6W9JEcjk64Cs P80YCKYHu59vsGi6Aogxolkb3EXDALOhnjtG7HAdprFXH9iN6oCh3WmC0A5d5mN1AFAx iWludlMmOnbkmItW1h4RiIc+/HaflTs56eKJ4zLLKb7Go2UOO8iNsuSrjMjrVH5s6Aya ozq1G6Tbj27Uxxy3exoXffknu5zkWv5LywmCO6PC0l/N/AMWAXK10Uf3YAm/d4SffqXm SgHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ad5WRDGSmHIxRG6OKUBwzd4Bx+HY5iH9CAWdyYF9xFg=; b=gPfxteJ9v0T4gTLOYvWlLUJSKOi0OC549/8ogW1M4lv5s2fUslc4z8UF+AJL/CjtQb LYXCM03ufu5+GpHOlYvrUf21+5aWx2iaijF7tlDFw+z8OE4ybqmlVtY2E0Z7baWzq6Nb JcR+xhtqop9q/clEb+oREu6xMikNtJ0WLf428DR3zgmBeIJZjs37O9vH0uvnj1+Vs+qK Kk0f2dz1FHmO8FyG9Wgk8mw0t62iHHR3wfzoH1qKiSuo7oFbfDVIR8JePz8yUurCjEzi yyoz3WmvyofiHUiixzOIHD3zkSgAkMoYKhkfA/k9Gmdtlj3MrGcxYTc4Q88oiUA2NrnB oQFg== X-Gm-Message-State: AOAM530ksuzo/HUFlCblwD6MdWYVHWGG67NCdb6eYHExudo5IGKMawO8 XOUlCPU+76nfLPU8ljdJJzSF/fi1wzvPtg== X-Google-Smtp-Source: ABdhPJyXGz4+X2/Yqi3Akxas5/lhnS/aT+vhcOXYq/CRQpLGrI2OSVhDlCO8BTgdmFfmOiekoKPA7g== X-Received: by 2002:a05:6402:34c6:: with SMTP id w6mr15721294edc.174.1623657053170; Mon, 14 Jun 2021 00:50:53 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:52 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 22/45] trace-cmd library: Refactor the logic for writing trace data in the file Date: Mon, 14 Jun 2021 10:50:06 +0300 Message-Id: <20210614075029.598048-23-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Refactored the internal logic of tracecmd_write_cpu_data() API to be suitable for adding trace data compression. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 94 +++++++++++++++++------------------- 1 file changed, 44 insertions(+), 50 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index f7e6c3dd..ed4ef638 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1679,11 +1679,17 @@ static char *get_clock(struct tracecmd_output *handle) return handle->trace_clock; } +struct data_file_write { + int file_size; + off64_t soffset; + off64_t data_offset; + off64_t doffset; +}; + int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { - off64_t *offsets = NULL; - unsigned long long *sizes = NULL; + struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; char *clock = NULL; @@ -1705,37 +1711,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (do_write_check(handle, "flyrecord", 10)) goto out_free; - offsets = malloc(sizeof(*offsets) * cpus); - if (!offsets) - goto out_free; - sizes = malloc(sizeof(*sizes) * cpus); - if (!sizes) - goto out_free; - - offset = lseek64(handle->fd, 0, SEEK_CUR); - - /* hold any extra data for data */ - offset += cpus * (16); - - /* - * Unfortunately, the trace_clock data was placed after the - * cpu data, and wasn't accounted for with the offsets. - * We need to save room for the trace_clock file. This means - * we need to find the size of it before we define the final - * offsets. - */ - clock = get_clock(handle); - if (!clock) + data_files = calloc(cpus, sizeof(struct data_file_write)); + if (!data_files) goto out_free; - /* Save room for storing the size */ - offset += 8; - offset += strlen(clock); - /* 2 bytes for [] around the clock */ - offset += 2; - - /* Page align offset */ - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + /* Write 0 for trace data offset and size and store offsets of these fields */ for (i = 0; i < cpus; i++) { file = cpu_data_files[i]; ret = stat(file, &st); @@ -1743,53 +1723,67 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, tracecmd_warning("can not stat '%s'", file); goto out_free; } - offsets[i] = offset; - sizes[i] = st.st_size; - offset += st.st_size; - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].file_size = st.st_size; - endian8 = convert_endian_8(handle, offsets[i]); + endian8 = 0; + data_files[i].doffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; - endian8 = convert_endian_8(handle, sizes[i]); + data_files[i].soffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; } - if (save_clock(handle, clock)) + clock = get_clock(handle); + if (clock && save_clock(handle, clock)) goto out_free; for (i = 0; i < cpus; i++) { + data_files[i].data_offset = lseek64(handle->fd, 0, SEEK_CUR); + /* Page align offset */ + data_files[i].data_offset = (data_files[i].data_offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].data_offset = lseek64(handle->fd, data_files[i].data_offset, SEEK_SET); + if (data_files[i].data_offset == (off64_t)-1) + goto out_free; if (!tracecmd_get_quiet(handle)) fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", - i, (unsigned long long) offsets[i]); - offset = lseek64(handle->fd, offsets[i], SEEK_SET); - if (offset == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offsets[i]); - goto out_free; - } + i, (unsigned long long) data_files[i].data_offset); + check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != sizes[i]) { + if (check_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, sizes[i]); + check_size, data_files[i].file_size); goto out_free; } + /* Write the real CPU data offset inthe file */ + offset = lseek64(handle->fd, data_files[i].doffset, SEEK_SET); + endian8 = convert_endian_8(handle, data_files[i].data_offset); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + /* Write the real CPU data size in the file */ + offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); + endian8 = convert_endian_8(handle, check_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + check_size; + offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", (unsigned long long)check_size); } - free(offsets); - free(sizes); + if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) + goto out_free; + free(data_files); handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; return 0; out_free: - free(offsets); - free(sizes); + lseek64(handle->fd, 0, SEEK_END); + free(data_files); return -1; } From patchwork Mon Jun 14 07:50:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26755C48BE8 for ; Mon, 14 Jun 2021 07:52:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1230261357 for ; Mon, 14 Jun 2021 07:52:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232622AbhFNHyL (ORCPT ); Mon, 14 Jun 2021 03:54:11 -0400 Received: from mail-ej1-f42.google.com ([209.85.218.42]:40676 "EHLO mail-ej1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232623AbhFNHyK (ORCPT ); Mon, 14 Jun 2021 03:54:10 -0400 Received: by mail-ej1-f42.google.com with SMTP id my49so15063856ejc.7 for ; Mon, 14 Jun 2021 00:51:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fK0+f39KReYwmIZ4HszCv/9BYALVUEKPLQVsLsNwq94=; b=OOv7N4OfuJhyAxhW0lNISF/K2fJdgp5VZ6aoNGj2JwlSeTgrklZ37sf1qic5VUvXyT i3eb2tlGlu999P/whSSiuRCw+ABPJjRKfxJyTlgyufTqau/Z/DggkIshUs5Mf4V55a0F AFOgmO77o2Y5MZmOtZ3tiIbh2pXol/YdKF5wGYr1NKPHAQa2LLS0M7LfONY0JCbF3KQC Tdtwrhy2FXQ5c/nFbXyur2CsaH94fjbUikqaUFtkG+Oopsz221ELQxsJ2ZChcSTTfy4f U/0ogWgz1C04vdF+N9d6jPln5k6JA5KLthHwpWDgnQBRIEMAI4Ygh+tsXoCTgmKvHAZN Z3Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fK0+f39KReYwmIZ4HszCv/9BYALVUEKPLQVsLsNwq94=; b=a3eqjm/HaOlO4Xima81X1ZlCQCgfjyBJNxSV3Gjk5G6J0lAhDSD14q5TXLenWp3w+r oMrqC+GPNwlwj9N2iczbvrAgulB/fQj+phQ1QOlIkAzr0mUwYmZTRZt/Ir2c/h1CGY9N Ya9/FRa/2+x0xhzdPfP3zTAgra/lJqiLJRy1Si/Bd1tat8pFosp4zyBQvfeKcu25iS2k QJDiUTmxLnop+oeeoPdpMentnwQypO4ZMwlJ9dx/cfkIbgsK+C9PtMMaZxBHij/cP52z Pjkt585Xd3uqrGtKogSRAFd8Umk7YTnxv6nGsNr9NFGQF7ksaXBFAGVYzSQ7LNm/DC9p w2ZA== X-Gm-Message-State: AOAM532KjLsr+AeZXzG+NZmxiydX21adYKXZapHxsmPikxHddnmp5+jE dEiWsCLq1jvD2UT0MFGmAA3vmltCTO0kRg== X-Google-Smtp-Source: ABdhPJyU1wEzfgG1GIS53C5EGqItGWlo05979nYIo9kyfDUFhaiV7OZ2bGhehtNPUeg0YHtMhQMtUg== X-Received: by 2002:a17:906:1912:: with SMTP id a18mr14152458eje.117.1623657054072; Mon, 14 Jun 2021 00:50:54 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:53 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 23/45] trace-cmd library: Add APIs for read and write compressed data in chunks Date: Mon, 14 Jun 2021 10:50:07 +0300 Message-Id: <20210614075029.598048-24-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New compression APIs are added for writing and reading compressed data from / to files in chunks: tracecmd_compress_copy_from() tracecmd_uncompress_copy_to() Format of the compressed data, used by these APIs, is: - 4 bytes, chunks count - for each chunk: - 4 bytes, size of compressed data in this chunk - 4 bytes, uncompressed size of the data in this chunk - data, bytes of Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 +- lib/trace-cmd/trace-compress.c | 200 ++++++++++++++++++ 2 files changed, 204 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 10089389..067ba34d 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -494,7 +494,10 @@ int tracecmd_compress_proto_register(const char *name, const char *version, int char *out, unsigned int *out_bytes), unsigned int (*comress_size)(unsigned int bytes), bool (*is_supported)(const char *name, const char *version)); - +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size); /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index 039f60c5..dfed7b0e 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -595,3 +595,203 @@ error: free(v); return -1; } + +/** + * tracecmd_compress_copy_from - Copy and compress data from a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data to copy from + * @chunk_size: size of one compression chunk + * @read_size: in - max bytes to read from @fd, 0 to read till the EOF + * out - size of the uncompressed data read from @fd + * @write_size: return, size of the compressed data written into @handle + * + * This function reads uncompressed data from given @fd, compresses the data using the @handle + * compression context and writes the compressed data into the fd associated with the @handle. + * The data is compressed on chunks with given @chunk_size size. + * The compressed data is written in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int rchunk = 0; + unsigned int chunks = 0; + unsigned int wsize = 0; + unsigned int rsize = 0; + unsigned int rmax = 0; + unsigned int csize; + unsigned int size; + unsigned int r; + off64_t offset; + char *buf_from; + char *buf_to; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->compress_block || !handle->proto->compress_size) + return 0; + if (read_size) + rmax = *read_size; + csize = handle->proto->compress_size(chunk_size); + buf_from = malloc(chunk_size); + if (!buf_from) + return -1; + buf_to = malloc(csize); + if (!buf_to) + return -1; + /* save the initial offset and write 0 chunks */ + offset = lseek64(handle->fd, 0, SEEK_CUR); + write_fd(handle->fd, &chunks, 4); + + do { + if (rmax > 0 && (rmax - rsize) < chunk_size) + rchunk = (rmax - rsize); + else + rchunk = chunk_size; + + r = read(fd, buf_from, rchunk); + if (r < 0 || (rmax > 0 && rsize >= rmax)) + break; + rsize += r; + size = csize; + if (r > 0) { + ret = handle->proto->compress_block(buf_from, r, buf_to, &size); + if (ret < 0) { + if (errno == EINTR) + continue; + break; + } + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &r, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write the compressed data */ + ret = write_fd(handle->fd, buf_to, size); + if (ret != size) + break; + /* data + compress header */ + wsize += (size + 8); + chunks++; + } + } while (r > 0); + free(buf_from); + free(buf_to); + if (r) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + endian4 = tep_read_number(handle->tep, &chunks, 4); + /* write chunks count*/ + write_fd(handle->fd, &chunks, 4); + lseek64(handle->fd, offset, SEEK_SET); + if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} + +/** + * tracecmd_uncompress_copy_to - Uncompress data and copy to a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data to copy into + * @read_size: return, size of the compressed data read from @handle + * @write_size: return, size of the uncompressed data written into @fd + * + * This function reads compressed data from the fd, associated with @handle, uncompresses it + * using the @handle compression context and writes the uncompressed data into the fd. + * The compressed data must be in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + unsigned int rsize = 0; + unsigned int wsize = 0; + char *bytes_out = NULL; + char *bytes_in = NULL; + int size_out; + int size_in; + int chunks; + char buf[4]; + char *tmp; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + chunks = tep_read_number(handle->tep, buf, 4); + rsize += 4; + while (chunks) { + if (read(handle->fd, buf, 4) != 4) + break; + s_compressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (read(handle->fd, buf, 4) != 4) + break; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (!bytes_in || size_in < s_compressed) { + tmp = realloc(bytes_in, s_compressed); + if (!tmp) + break; + bytes_in = tmp; + size_in = s_compressed; + } + + if (!bytes_out || size_out < s_uncompressed) { + tmp = realloc(bytes_out, s_uncompressed); + if (!tmp) + break; + bytes_out = tmp; + size_out = s_uncompressed; + } + + if (read_fd(handle->fd, bytes_in, s_compressed) < 0) + break; + rsize += s_compressed; + ret = handle->proto->uncompress_block(bytes_in, s_compressed, + bytes_out, &s_uncompressed); + if (ret) + break; + write_fd(fd, bytes_out, s_uncompressed); + wsize += s_uncompressed; + chunks--; + } + free(bytes_in); + free(bytes_out); + if (chunks) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} From patchwork Mon Jun 14 07:50:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40DEFC48BE8 for ; Mon, 14 Jun 2021 07:51:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 28C6861352 for ; Mon, 14 Jun 2021 07:51:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232609AbhFNHx7 (ORCPT ); Mon, 14 Jun 2021 03:53:59 -0400 Received: from mail-ed1-f49.google.com ([209.85.208.49]:37793 "EHLO mail-ed1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232608AbhFNHx6 (ORCPT ); Mon, 14 Jun 2021 03:53:58 -0400 Received: by mail-ed1-f49.google.com with SMTP id b11so45215626edy.4 for ; Mon, 14 Jun 2021 00:51:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cJCBtvbxRLcQPMRtRDEoPrHPiqKGSweLWU23nOhOF0E=; b=W0a8jbbQnLU0RVCaot6H+6NzXogfKOUV4upnHkl0KipRyuJzxTEDC3/CIVN0rAdxSW Dv2MAsqXyr6WVtwRyKWW+yBqX2YOjdWC9khJB9kmUDi9nAioBqI9Gr5BzxkYIO1T7iy6 gpbZh5/bbGOf4q+l1oCkKWeB2QKdHASBSVMgiX30PodpPHGBZBXghI6hJ/xt5u+w/2HM IWgaPiXRPQyhTgw/I77o8gb477nYu2qIwGzz13ifBC5YlnEYfbeGp6POC5dYgXIDfVQg 5Zxz/qFkqOdsmHXGcPw8nimiPF/hNrmDy5LHUVlDHCE+o2yd8s4CmVmSMxoTpfzQEs1t 5Mzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cJCBtvbxRLcQPMRtRDEoPrHPiqKGSweLWU23nOhOF0E=; b=iMtiTDTG0lj0Wyo3MK0+nJ/GSzOzt6fK6FHk10lJ1+kRcMneKUxW72btFioVi5luT9 tjTa2hWeMDJyQcIxX2yHHMATUePB4J0/oLEPB1iqqBIAjdbqTfQ6QvTibP7M50ytba7q tXQpj7RhDabWPxdnLD/CRV168jMvevljmclYJMIlmI/3kSWNHVVmrHAJGA7LxIBod68s f6jBt9XdCr/2Oxktf5Dh+8gwBWfMqRibF2ZgXdk4vZ3FQh/DdcElr5FHBQYEE+ZLz7hs ARPqKLEzo7QN2AcTx05aSV+xrAJH+hrumdaob6ihU1KMU3NdukkJhjKBKC+S+fZyx+6M W29A== X-Gm-Message-State: AOAM531SVwyJCQP9Cs5dTc6bSIrXmBTmhmrX/8PeOc61Dl7qd9aDRulh oPJ4APTQt5Yb0p736DK8yraqpIBa9AZpiQ== X-Google-Smtp-Source: ABdhPJzCtXsJGQsxWF7qc7wPTrdhzU3UkWtrBVQ7w/X5t9mbHwvMfEB7YGCEbJVr+bSPe717LKWsQw== X-Received: by 2002:a05:6402:1c11:: with SMTP id ck17mr16273132edb.102.1623657055044; Mon, 14 Jun 2021 00:50:55 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:54 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 24/45] trace-cmd: Compress trace data Date: Mon, 14 Jun 2021 10:50:08 +0300 Message-Id: <20210614075029.598048-25-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a trace.dat file of version 7, compress the trace data. The data is compressed in chunks. The chunk size is a compile time parameter, set by default to 1MB. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 50 +++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index ed4ef638..20a646af 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -321,6 +321,41 @@ static tsize_t copy_file(struct tracecmd_output *handle, return size; } +#define COMPRESS_CHUNK_SIZE (1*1024*1024) +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + unsigned long long rsize = 0; + unsigned long long wsize = 0; + tsize_t size; + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + if (handle->file_version >= 7) { + ret = tracecmd_compress_copy_from(handle->compress, fd, + COMPRESS_CHUNK_SIZE, &rsize, &wsize); + if (ret < 0) { + tracecmd_warning("Can't compress '%s'", file); + close(fd); + return 0; + } + size = rsize; + *write_size = wsize; + } else { + size = copy_file_fd(handle, fd); + *write_size = size; + } + + close(fd); + return size; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -1692,8 +1727,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; + unsigned long long read_size; + unsigned long long write_size; char *clock = NULL; - off64_t check_size; char *file; struct stat st; int ret; @@ -1749,11 +1785,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", i, (unsigned long long) data_files[i].data_offset); - check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != data_files[i].file_size) { + read_size = copy_file_compress(handle, cpu_data_files[i], &write_size); + if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, data_files[i].file_size); + read_size, data_files[i].file_size); goto out_free; } /* Write the real CPU data offset inthe file */ @@ -1763,14 +1799,14 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, goto out_free; /* Write the real CPU data size in the file */ offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); - endian8 = convert_endian_8(handle, check_size); + endian8 = convert_endian_8(handle, write_size); if (do_write_check(handle, &endian8, 8)) goto out_free; - offset = data_files[i].data_offset + check_size; + offset = data_files[i].data_offset + write_size; offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", - (unsigned long long)check_size); + (unsigned long long)write_size); } if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) From patchwork Mon Jun 14 07:50:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E000BC48BE6 for ; Mon, 14 Jun 2021 07:50:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C3E4E6124B for ; Mon, 14 Jun 2021 07:50:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232546AbhFNHxA (ORCPT ); Mon, 14 Jun 2021 03:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232544AbhFNHxA (ORCPT ); Mon, 14 Jun 2021 03:53:00 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5ECB6C061766 for ; Mon, 14 Jun 2021 00:50:57 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id s6so45188974edu.10 for ; Mon, 14 Jun 2021 00:50:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=05lK6z1tRPoWI6LsDB6q2WoJqm7gnqhaRWTmXEeBIrg=; b=kgO+J+DAPGGchbQ+XBgwJoIqslhGmExGBWKImI3d+5z5scwVjscZJ2ECbcZTsS9Oi1 oovfKzPTUna8AuNJi8NhiTRdNet0CnvZWrWjRVCT/hIG6PLy3BY+og6CTxcP6T1qDw4s AkL59X1S8xHPLP1sdI7NGD5PVbtMVH/cw8F9XIhIBluBTT74r5hX3jjAkogwzzSkgxyY iRFYyTUTPwCC0EyVdJ8H+mfjY4hNOA1AY/By4EXD653hlgDXoWRoFzfNhJ+uRnNgLBm+ nQc89yyC3hAYNwCO1BxPdp7ZoPo6nuA9ANFWQ5a1Jz3BNVsh7ZWLWjIgAXifhfZfE+gj 8hEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=05lK6z1tRPoWI6LsDB6q2WoJqm7gnqhaRWTmXEeBIrg=; b=MR9sCmogfnmmErsmnUooTWY2t+wjWj0AwX2seSy2WMf9I3gv1C+5odWUuJaa8npBZW 609Hz9EkuVQW2j1jGXdBkpgCuQDhVtHZyL9xM6g/lfxTbSpcTQxfHDiRJ2ReneMDPghZ yXaei6PoW4lwP5bR7rUk5AVvDFtFWESBaNOvJlH+Mn3OiY3n+4n2NL18OBfpMK19lYCr 5G8cWKg8dVRCcRAQYONeKH2xpTGT7SSnfG5L44BoS4KOx0Eujt7XfdjbsV5he63/x0L7 yPwT7Ftpv0pKyyQ5JwixD78uEVd92IxqpzsXl1kTeV0faSKEIC5GP9i59qsmrkIt+kb/ xw+Q== X-Gm-Message-State: AOAM530Xpdsp8YSyo89A/HIC0g9t3B4OBi3xjZbMfpMGh+fd1GWU7BDi OJC8noGJe1tZj0oVolrQtc82TfhFeLf31w== X-Google-Smtp-Source: ABdhPJzoIRNqdHi0CtQ5dewgBcB/DMJz/YGdDrHXdnon2VtxMKBarDkXCIQpwftyOTjOQuF/hQz3gw== X-Received: by 2002:aa7:d785:: with SMTP id s5mr2116713edq.19.1623657055922; Mon, 14 Jun 2021 00:50:55 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:55 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 25/45] trace-cmd: Read compressed trace data Date: Mon, 14 Jun 2021 10:50:09 +0300 Message-Id: <20210614075029.598048-26-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace.dat file of version 7, uncompress the trace data. The trace data for each CPU is uncompressed in a temporary file, located in /tmp directory with prefix "trace_cpu_data". Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 74 +++++++++++++++++++++++++++++-------- tracecmd/trace-read.c | 8 ++++ 2 files changed, 67 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 8fff003e..327082a2 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,7 @@ struct page { #endif }; +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +73,10 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + + /* temporary file for uncompressed cpu data */ + int cfd; + char cfile[26]; /* strlen(COMPR_TEMP_FILE) */ }; struct input_buffer_instance { @@ -1080,6 +1085,7 @@ static void *allocate_page_map(struct tracecmd_input *handle, off64_t map_offset; void *map; int ret; + int fd; if (handle->read_page) { map = malloc(handle->page_size); @@ -1119,12 +1125,15 @@ static void *allocate_page_map(struct tracecmd_input *handle, map_size -= map_offset + map_size - (cpu_data->file_offset + cpu_data->file_size); + if (cpu_data->cfd >= 0) + fd = cpu_data->cfd; + else + fd = handle->fd; again: page_map->size = map_size; page_map->offset = map_offset; - page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, - handle->fd, map_offset); + page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, fd, map_offset); if (page_map->map == MAP_FAILED) { /* Try a smaller map */ @@ -2316,13 +2325,41 @@ tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record) /* Not reached */ } +static int cpu_data_uncompress(struct tracecmd_input *handle, int cpu, unsigned long long *size) +{ + struct cpu_data *cpu_data; + + cpu_data = &handle->cpu_data[cpu]; + strcpy(cpu_data->cfile, COMPR_TEMP_FILE); + cpu_data->cfd = mkstemp(cpu_data->cfile); + if (cpu_data->cfd < 0) + return -1; + return tracecmd_uncompress_copy_to(handle->compress, cpu_data->cfd, NULL, size); +} + static int init_cpu(struct tracecmd_input *handle, int cpu) { struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + unsigned long long size; + off64_t offset; int i; - cpu_data->offset = cpu_data->file_offset; - cpu_data->size = cpu_data->file_size; + if (handle->file_version >= 7 && cpu_data->file_size > 0) { + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + if (cpu_data_uncompress(handle, cpu, &size) < 0) + return -1; + cpu_data->offset = 0; + cpu_data->file_offset = 0; + cpu_data->file_size = size; + cpu_data->size = size; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + } else { + cpu_data->offset = cpu_data->file_offset; + cpu_data->size = cpu_data->file_size; + } cpu_data->timestamp = 0; list_head_init(&cpu_data->page_maps); @@ -3015,6 +3052,7 @@ static int read_cpu_data(struct tracecmd_input *handle) handle->cpu_data[cpu].file_offset = offset; handle->cpu_data[cpu].file_size = size; + handle->cpu_data[cpu].cfd = -1; if (size > max_size) max_size = size; @@ -3635,17 +3673,23 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].cfd >= 0) { + close(handle->cpu_data[cpu].cfd); + unlink(handle->cpu_data[cpu].cfile); + } } } diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 0cf6e773..d605d05a 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1363,7 +1363,14 @@ struct tracecmd_input *read_trace_header(const char *file, int flags) static void sig_end(int sig) { + struct handle_list *handles; + fprintf(stderr, "trace-cmd: Received SIGINT\n"); + + list_for_each_entry(handles, &handle_list, list) { + tracecmd_close(handles->handle); + } + exit(0); } @@ -1924,6 +1931,7 @@ void trace_report (int argc, char **argv) /* and version overrides uname! */ if (show_version) otype = OUTPUT_VERSION_ONLY; + read_data_info(&handle_list, otype, global, align_ts); list_for_each_entry(handles, &handle_list, list) { From patchwork Mon Jun 14 07:50:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A2CFC49361 for ; Mon, 14 Jun 2021 07:51:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DA2F61289 for ; Mon, 14 Jun 2021 07:51:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232608AbhFNHyB (ORCPT ); Mon, 14 Jun 2021 03:54:01 -0400 Received: from mail-ej1-f49.google.com ([209.85.218.49]:36652 "EHLO mail-ej1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232617AbhFNHx7 (ORCPT ); Mon, 14 Jun 2021 03:53:59 -0400 Received: by mail-ej1-f49.google.com with SMTP id nd37so7326494ejc.3 for ; Mon, 14 Jun 2021 00:51:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qXFp8nHwN12Mahu0aozsJZDGSYor14IBt4n92eO2+hI=; b=p/hkdiRmaeWg0GHAkIrkkbmrVRGMVFl/y8doHKHIN6ftcuV5LHwIFHuKrqNtevKtRl KvMQfybTLFSS0hGaSMjPM1lNDCBGVhy18d0pp6lI7ocTT52IdXOXffQT4vbIjfNa+qJj GbIP3fEI0ti7ry36ypFURsTzS8qGpa23IQgb0yw+QN7xutdT8veOnAlbnFE3VHJlYTuT g0ZMuhue7ppD8k5f1miRnK6dUFJz3CGPj/Mku2EeTg7R2KifGuuemoiCeIjw7JgypBTV uDzoOYVdQyWMjEe2Y6keNvZvrgNimN29RhMpLiHqGv1+YOx4Xyc+ozTE0/gxQPo8c6Dp I00g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qXFp8nHwN12Mahu0aozsJZDGSYor14IBt4n92eO2+hI=; b=XyoEWQQ7ZQjmja0P7WBQW0zXevpPi05g58QCo995n23oKHm4BeYO3y0k/Ed/todfQ9 Jf73Bp5h9DRLnz+0HDx3/AN+fepH0Nqbe+3RS6Ln9ic3FrjuFgQZ1AujEiFc/gQvok+X 4KcuqiuO0VB9bxjidN5VdLLAhNTB+eCkPwIV+vPnTNni1306A+bpGEgMWaLv/KUvC3Lv 2VNCseaLOn0LfZV6BGbTsqpaKygHlnlrOvPfJxeK5FwIRE9ihIWHpk9rdIAo/JdLiU+H XFIRyuzne+zHm+xtRvqUBKAaGWIzmVTkgNMNBK4DYjFaSK89ZeQYAX8UBNej7JLn1D+/ JlMg== X-Gm-Message-State: AOAM532HHe6+yQYiz8K+zM8BSH4I/q+9Nh69YrBiA0OPqftURMhUsEUO 1FHCVlk2KsPl5PsMsphf3pw= X-Google-Smtp-Source: ABdhPJzxPxaGXfSxtrPGVFBHdHjvBS+U35HSVYzQeLh3TQiSLCGyPbNzVt46FrjM+FrvGCB2NWRJhg== X-Received: by 2002:a17:906:b2d1:: with SMTP id cf17mr14409672ejb.225.1623657056896; Mon, 14 Jun 2021 00:50:56 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:56 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 26/45] trace-cmd library: Compress latency trace data Date: Mon, 14 Jun 2021 10:50:10 +0300 Message-Id: <20210614075029.598048-27-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating latency trace file of version 7, compress the latency trace data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 20a646af..017b3d7d 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1661,7 +1661,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (!path) goto out_free; - copy_file(handle, path); + copy_file_compress(handle, path, NULL); put_tracing_file(path); From patchwork Mon Jun 14 07:50:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A49DC48BE8 for ; Mon, 14 Jun 2021 07:52:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 076A461352 for ; Mon, 14 Jun 2021 07:52:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232621AbhFNHyB (ORCPT ); Mon, 14 Jun 2021 03:54:01 -0400 Received: from mail-ed1-f41.google.com ([209.85.208.41]:33639 "EHLO mail-ed1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232619AbhFNHyB (ORCPT ); Mon, 14 Jun 2021 03:54:01 -0400 Received: by mail-ed1-f41.google.com with SMTP id f5so40211409eds.0 for ; Mon, 14 Jun 2021 00:51:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=O4tw/vtCdoHAkqVILpd8GpKTrQc88oL92rc/FfJiJWk=; b=qQ+rVnXXDNFKKW4A8cvHzrCS+zUunzDF+A39iR5wqm7ijlRc2xv5oXrr3Sza2PKNkj qXB6ATgWAMMQQoYrz44GkD+M2qgH4JyeOvdiZ9l56WDvhnyeUvsWjhXgzk6jgNCHpDFi pD7GE986fdnWMT7NQfeXRipw9Vxk28tvmfpHR/9QKF1WPUZcpJxiO8kstpwxyEQkY2HZ 9FNxmCspa6t/6kObJgrwBmoTi6r3PE5yFC1WKJDQeyzKij9DX8+VBrJQW6FGmSSR5IOn /JPn7B4HIshhZY8ddue4nI2dfk4Bao2M+FzTVqx5Li3EuHMvo4S/eJ+XyyEiRGFTXmEC Yanw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O4tw/vtCdoHAkqVILpd8GpKTrQc88oL92rc/FfJiJWk=; b=egKPIav+7v6e2xvSY9hBf+mZ+ua1lr/XBGgenlAGJ9QWmbt6yO/DYVS7Sr0wEmtB24 j1rJfGZ6lzylxwXyTt/88Hzlrhayrw1DyanrACGKdmPtH1gAuaBRWS6eKvblLVg8Hjr6 7uo9fFcw1cf+PBqfyiqNUdt9iVOiC4NtR+7SzNQrywJsbLeTgFYILlIQFIUllGRW8twS uQZgj56KlbdopkBat7L7tBJK9MJNb9o4k0oSgQ68eXTidoZhRS8KnB6jAEE1+0/vt4nr vxMIUJRpyfrUNUnJmdf8C+kLo/FZMk4Uc4Ccb1FLdhogaaLWBRTIy219vch53GTl8pdS w+2A== X-Gm-Message-State: AOAM5313citKfJTLdTJwoTLhtj8MQfJA9NYzlVrrwmCzxXIua3S/SPr7 zEKF8fVHSWlvG5DSawnFP+4= X-Google-Smtp-Source: ABdhPJyWB2ca1Jgbm1/0266sIfnKfO7OXINSnsG5d3Wa5GxB2uRRXIBIMV40akOz2tknWUhlDSH9UA== X-Received: by 2002:a05:6402:1a:: with SMTP id d26mr15821709edu.105.1623657057856; Mon, 14 Jun 2021 00:50:57 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:57 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 27/45] trace-cmd: Read compressed latency trace data Date: Mon, 14 Jun 2021 10:50:11 +0300 Message-Id: <20210614075029.598048-28-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading trace file version 7 with latency trace data, uncompress the data. A new trace-cmd API is introduced to handle that case: tracecmd_latency_data_read() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 42 ++++++++++++++++++- tracecmd/trace-read.c | 6 +-- 3 files changed, 46 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 067ba34d..4a3eb1bc 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -208,6 +208,8 @@ tracecmd_peek_data_ref(struct tracecmd_input *handle, int cpu) return rec; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, void *buf, size_t size); + struct tep_record * tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 327082a2..12876838 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -137,6 +137,9 @@ struct tracecmd_input { bool read_page; bool use_pipe; int file_version; + /* temporary file for uncompressed latency data */ + int lat_cfd; + char lat_cfile[26]; /* strlen(COMPR_TEMP_FILE) */ struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -2995,6 +2998,35 @@ static int read_options_type(struct tracecmd_input *handle) return 0; } +int tracecmd_latency_data_read(struct tracecmd_input *handle, void *buf, size_t size) +{ + int fd, r; + + if (handle->file_state != TRACECMD_FILE_CPU_LATENCY) + return -1; + if (handle->lat_cfd >= 0) + fd = handle->lat_cfd; + else + fd = handle->fd; + r = read(fd, buf, size); + + return r; +} + +static int latency_data_uncompress(struct tracecmd_input *handle) +{ + int ret; + + strcpy(handle->lat_cfile, COMPR_TEMP_FILE); + handle->lat_cfd = mkstemp(handle->lat_cfile); + if (handle->lat_cfd < 0) + return -1; + ret = tracecmd_uncompress_copy_to(handle->compress, handle->lat_cfd, NULL, NULL); + if (!ret) + lseek64(handle->lat_cfd, 0, SEEK_SET); + return ret; +} + static int read_cpu_data(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -3009,8 +3041,11 @@ static int read_cpu_data(struct tracecmd_input *handle) /* * Check if this is a latency report or not. */ - if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) + if (handle->file_state == TRACECMD_FILE_CPU_LATENCY) { + if (handle->file_version >= 7 && latency_data_uncompress(handle)) + return -1; return 1; + } /* We expect this to be flyrecord */ if (handle->file_state != TRACECMD_FILE_CPU_FLYRECORD) @@ -3448,6 +3483,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->fd = fd; handle->ref = 1; + handle->lat_cfd = -1; if (do_read_check(handle, buf, 3)) goto failed_read; @@ -3698,6 +3734,10 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle->uname); free(handle->trace_clock); close(handle->fd); + if (handle->lat_cfd >= 0) { + close(handle->lat_cfd); + unlink(handle->lat_cfile); + } tracecmd_free_hooks(handle->hooks); handle->hooks = NULL; diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index d605d05a..dfb814b7 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -948,13 +948,13 @@ void trace_show_data(struct tracecmd_input *handle, struct tep_record *record) printf("\n"); } -static void read_rest(void) +static void read_latency(struct tracecmd_input *handle) { char buf[BUFSIZ + 1]; int r; do { - r = read(input_fd, buf, BUFSIZ); + r = tracecmd_latency_data_read(handle, buf, BUFSIZ); if (r > 0) { buf[r] = 0; printf("%s", buf); @@ -1241,7 +1241,7 @@ static void read_data_info(struct list_head *handle_list, enum output_type otype if (ret > 0) { if (multi_inputs) die("latency traces do not work with multiple inputs"); - read_rest(); + read_latency(handles->handle); return; } From patchwork Mon Jun 14 07:50:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 793B2C48BE6 for ; Mon, 14 Jun 2021 07:51:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F40361352 for ; Mon, 14 Jun 2021 07:51:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232582AbhFNHxD (ORCPT ); Mon, 14 Jun 2021 03:53:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232579AbhFNHxC (ORCPT ); Mon, 14 Jun 2021 03:53:02 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38AB6C061574 for ; Mon, 14 Jun 2021 00:51:00 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id t3so45241527edc.7 for ; Mon, 14 Jun 2021 00:51:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fm6GHm8rBYX0HOSR/Du9nrToQE1DoWVg7pjPtUjxL/M=; b=O92iOcvZHiACgfPFYL0jhuO7sukGEjI3cSgzixWuh8vO0+CNoUMRePIR39p97ApddS ud5df09wZmHAmJH10CMgSUMPFBhAKAFVUuLBhXMUEEcTaWuxg8wMgdu7oylm70bgonZ9 H1CSIGPZg+40xmxcfQf6TD4VlzdSjTn2ZTKCFNptG830STg6yoqXai74VKEgQPsdgV4k EEl1ZMPSJWFGBDr7VFvQQant43cMUnMAdEZvVcfkcq4GfrE0p7llA43jgz+kdmHZIYq1 RY3LJ4klUJth8u6FqXopVkcGT/mXXNCeDKhzySLSin9K8s7SNDPND2UnRxCGwUIWVTt+ z24Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fm6GHm8rBYX0HOSR/Du9nrToQE1DoWVg7pjPtUjxL/M=; b=kcraV5CBdUaecDK8i7oVuLqqy7y7QI4OFWpkmyL8HOl9AxSP9D5723R2somUOYMRQs Lh5lQNGgBV/q1wN9oL0xe4g/sSyhFPcP21vfgQz+WAs7P4tJGrKwrlC0CMVbswdA53tQ SqFt6/U74Xb1RLj1VTFA7JXpA4yImK5EbrjbKa3UGkspovrVOY/0WOFaNOFR6Yn9sEpp BgWqgIeKHCZvugURQyaHCunhkvPc1XbZq8lLRpEyT3S19tN4UAp038rplFl2pwouTCw5 Coh0VmaBnpmHV5JClfX8FMd0pG+DU9PpK1gP7XmAd1bMYixGy8IyM1pK3RKF+hroHUDo QDQA== X-Gm-Message-State: AOAM532psvGgPTE93l5s9jLC/DMUOdY5+k1xWq+csqvWKmMKwjtBRXGO PSsMjh/ztmP6nrMLqbnRxWBUWkPMBop2iA== X-Google-Smtp-Source: ABdhPJxFlkHESHjDqiKNF5HcMZwywkO95BwVdcWzEk6AV873fc3eXR0ZJMzUBVAMOxUQpmF/JhfmUQ== X-Received: by 2002:a05:6402:1906:: with SMTP id e6mr15585793edz.14.1623657058800; Mon, 14 Jun 2021 00:50:58 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:58 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 28/45] trace-cmd library: Reuse within the library the function that checks file state. Date: Mon, 14 Jun 2021 10:50:12 +0300 Message-Id: <20210614075029.598048-29-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Make the function, that checks if the next file state is valid, global for the tracmd-library, so it can be reused. It is important the same check logic to be used in the whole library. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 ++ lib/trace-cmd/trace-input.c | 5 ++ lib/trace-cmd/trace-output.c | 83 ++++++------------------- lib/trace-cmd/trace-util.c | 41 ++++++++++++ 4 files changed, 70 insertions(+), 63 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 03d2a9a3..d44b58a9 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,10 @@ void tracecmd_zlib_free(void); void tracecmd_compress_init(void); void tracecmd_compress_free(void); +bool check_file_state(unsigned long file_version, int current_state, int new_state); +bool check_in_state(struct tracecmd_input *handle, int new_state); +bool check_out_state(struct tracecmd_output *handle, int new_state); + int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 12876838..d3c5ad18 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4436,3 +4436,8 @@ int tracecmd_enable_tsync(struct tracecmd_input *handle, bool enable) return 0; } + +__hidden bool check_in_state(struct tracecmd_input *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 017b3d7d..c41dfb8f 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -419,50 +419,6 @@ int tracecmd_ftrace_enable(int set) return ret; } -static int check_out_state(struct tracecmd_output *handle, int new_state) -{ - if (!handle) - return -1; - - if (handle->file_version < 7) { - switch (new_state) { - case TRACECMD_FILE_BUFERS: - tracecmd_warning("State %d is not supported in trace file version %d", - new_state, handle->file_version); - return -1; - default: - break; - } - } - - switch (new_state) { - case TRACECMD_FILE_HEADERS: - case TRACECMD_FILE_FTRACE_EVENTS: - case TRACECMD_FILE_ALL_EVENTS: - case TRACECMD_FILE_KALLSYMS: - case TRACECMD_FILE_PRINTK: - case TRACECMD_FILE_CMD_LINES: - case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_BUFERS: - if (handle->file_state == (new_state - 1)) - return 0; - break; - case TRACECMD_FILE_OPTIONS: - if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) - return 0; - if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) - return 0; - break; - case TRACECMD_FILE_CPU_LATENCY: - case TRACECMD_FILE_CPU_FLYRECORD: - if (handle->file_state == TRACECMD_FILE_OPTIONS) - return 0; - break; - } - - return -1; -} - static int read_header_files(struct tracecmd_output *handle) { tsize_t size, check_size, endian8; @@ -471,7 +427,7 @@ static int read_header_files(struct tracecmd_output *handle) int fd; int ret; - if (check_out_state(handle, TRACECMD_FILE_HEADERS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_HEADERS)) { tracecmd_warning("Cannot read header files, unexpected state 0x%X", handle->file_state); return -1; @@ -783,7 +739,7 @@ static int read_ftrace_files(struct tracecmd_output *handle) struct tracecmd_event_list list = { .glob = "ftrace/*" }; int ret; - if (check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS)) { tracecmd_warning("Cannot read ftrace files, unexpected state 0x%X", handle->file_state); return -1; @@ -828,7 +784,7 @@ static int read_event_files(struct tracecmd_output *handle, int endian4; int ret; - if (check_out_state(handle, TRACECMD_FILE_ALL_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_ALL_EVENTS)) { tracecmd_warning("Cannot read event files, unexpected state 0x%X", handle->file_state); return -1; @@ -927,7 +883,7 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, struct stat st; int ret; - if (check_out_state(handle, TRACECMD_FILE_KALLSYMS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_KALLSYMS)) { tracecmd_warning("Cannot read kallsyms, unexpected state 0x%X", handle->file_state); return -1; @@ -977,7 +933,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) char *path; int ret; - if (check_out_state(handle, TRACECMD_FILE_PRINTK) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_PRINTK)) { tracecmd_warning("Cannot read printk, unexpected state 0x%X", handle->file_state); return -1; @@ -1339,11 +1295,10 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CPU_COUNT); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_COUNT)) { tracecmd_warning("Cannot write CPU count into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } cpus = convert_endian_4(handle, cpus); ret = do_write_check(handle, &cpus, 4); @@ -1359,16 +1314,14 @@ int tracecmd_write_options(struct tracecmd_output *handle) unsigned short option; unsigned short endian2; unsigned int endian4; - int ret; /* If already written, ignore */ if (handle->file_state == TRACECMD_FILE_OPTIONS) return 0; - ret = check_out_state(handle, TRACECMD_FILE_OPTIONS); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_OPTIONS)) { tracecmd_warning("Cannot write options into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } if (do_write_check(handle, "options ", 10)) @@ -1604,11 +1557,10 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CMD_LINES); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CMD_LINES)) { tracecmd_warning("Cannot write command lines into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } out_compression_start(handle); @@ -1628,7 +1580,6 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in { struct tracecmd_output *handle; char *path; - int ret; handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) @@ -1647,8 +1598,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_options(handle) < 0) goto out_free; - ret = check_out_state(handle, TRACECMD_FILE_CPU_LATENCY); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_LATENCY)) { tracecmd_warning("Cannot write latency data into the file, unexpected state 0x%X", handle->file_state); goto out_free; @@ -1737,7 +1687,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, /* This can be called multiple times (when recording instances) */ ret = handle->file_state == TRACECMD_FILE_CPU_FLYRECORD ? 0 : - check_out_state(handle, TRACECMD_FILE_CPU_FLYRECORD); + check_file_state(handle->file_version, + handle->file_state, + TRACECMD_FILE_CPU_FLYRECORD); if (ret < 0) { tracecmd_warning("Cannot write trace data into the file, unexpected state 0x%X", handle->file_state); @@ -2006,3 +1958,8 @@ out_free: tracecmd_output_close(handle); return NULL; } + +__hidden bool check_out_state(struct tracecmd_output *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 136260c6..087d37ab 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -636,3 +636,44 @@ void tracecmd_lib_free(void) { tracecmd_compress_free(); } + +__hidden bool check_file_state(unsigned long file_version, int current_state, int new_state) +{ + if (file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, file_version); + return false; + default: + break; + } + } + + switch (new_state) { + case TRACECMD_FILE_HEADERS: + case TRACECMD_FILE_FTRACE_EVENTS: + case TRACECMD_FILE_ALL_EVENTS: + case TRACECMD_FILE_KALLSYMS: + case TRACECMD_FILE_PRINTK: + case TRACECMD_FILE_CMD_LINES: + case TRACECMD_FILE_CPU_COUNT: + case TRACECMD_FILE_BUFERS: + if (current_state == (new_state - 1)) + return true; + break; + case TRACECMD_FILE_OPTIONS: + if (file_version < 7 && current_state == TRACECMD_FILE_CPU_COUNT) + return true; + if (file_version >= 7 && current_state == TRACECMD_FILE_BUFERS) + return true; + break; + case TRACECMD_FILE_CPU_LATENCY: + case TRACECMD_FILE_CPU_FLYRECORD: + if (current_state == TRACECMD_FILE_OPTIONS) + return true; + break; + } + + return false; +} From patchwork Mon Jun 14 07:50:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD6D1C49360 for ; Mon, 14 Jun 2021 07:51:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A766361289 for ; Mon, 14 Jun 2021 07:51:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232584AbhFNHxE (ORCPT ); Mon, 14 Jun 2021 03:53:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232579AbhFNHxD (ORCPT ); Mon, 14 Jun 2021 03:53:03 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30487C061574 for ; Mon, 14 Jun 2021 00:51:01 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id he7so14874170ejc.13 for ; Mon, 14 Jun 2021 00:51:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kWTWcXZIvFdwVZ5sOQ6RY01OSTOmcKvJZe3e+kAuak8=; b=U82HDavWikp6E4VKnMH1n471OwlzKeqeKWWYzEgwcvI7ZuL4u8+NiNT/hnSWvXB897 BiHF5FVxNM0FfuX24bA7Wzv1DNxLdgv2F1AwCdmraNBr5X2L375Vi28HrEQvkc8IvvXu n9hStiAUgKt/DTgV929DefMbe9OjQtK2bCnkD1g4bOGvj6pdme60SjGVwGryK1cwka6m Kkkpe1lG5Nc2sphidgNzIzM8HbYTzOJWXhG/UTvU8/GdHl6ZqYuxM0OK5O+y3y27N/p1 YosWuvsFvPZ5wjxgai7PuVKv1/gauV2vXUzXTHCck+16UPUz3E1BW+I4loOxs3JxUtjw JXdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kWTWcXZIvFdwVZ5sOQ6RY01OSTOmcKvJZe3e+kAuak8=; b=LXUXXcWPEPBVcZkAIz1bVjm3Pn3xZ0Dje2AB2M3ZBZfhlQhsPDYjKrnoSKXsBn1+nR o1S67GoewWdM7OTM+qtT57Wo2SpMvVG+MWsfuxVuHnDfSb4XUJyOEkpUP4jb27eDB2oK XIL/QAcr/gFh9kk7iRTAD8L6++wEkn6wrPLouL7CTVBwGDO9GrD/WQvkbItlASje/+vq HwHcJr0EzF1nxMY7m7agn0r9g6HaPI1nYly0F2gopAPq83g92mgod92aPAuHQkmB8Gy/ pYM5NBAVvC/omRUzIksGw7XLvV49nuTxyA4UNAVv67kzGk6k+Q4iPpYjl9P3VRu5Hbpu vLRw== X-Gm-Message-State: AOAM532mZOJexHgv1rbPDnZRm2Gw5oNnK4NgNgqGDUhH+BxSFFIZmmBO 0Sgv5UAhXJbs6djm4FgsKBaJkDOzn1Dflw== X-Google-Smtp-Source: ABdhPJzkzm1dWbKZ+uVj5+Vv5z5FJr+ofLc7h44jb2zUYoMoeqsBjiH+h9Cth24h8C+QcJkvFlVSDQ== X-Received: by 2002:a17:906:fa93:: with SMTP id lt19mr14520891ejb.54.1623657059724; Mon, 14 Jun 2021 00:50:59 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:50:59 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 29/45] trace-cmd library: Make tracecmd_copy_headers() to work with output handler Date: Mon, 14 Jun 2021 10:50:13 +0300 Message-Id: <20210614075029.598048-30-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When copying headers between two trace files, use output handler context instead of just a fd. Using output handler will allow to use file compression, if needed. This change fixes "trace-cmd split" command to work with trace files v7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 +- lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-input.c | 244 ++++++++++++------ lib/trace-cmd/trace-output.c | 13 +- 4 files changed, 174 insertions(+), 90 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4a3eb1bc..65e80cec 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -170,7 +170,8 @@ int tracecmd_get_parsing_failures(struct tracecmd_input *handle); int tracecmd_long_size(struct tracecmd_input *handle); int tracecmd_page_size(struct tracecmd_input *handle); int tracecmd_cpus(struct tracecmd_input *handle); -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index d44b58a9..8fb391fc 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -51,7 +51,11 @@ void out_compression_reset(struct tracecmd_output *handle); void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); +void out_set_file_state(struct tracecmd_output *handle, int new_state); + int write_buffers_description_v7(struct tracecmd_output *handle); int write_buffers_description_v6(struct tracecmd_output *handle); +long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index d3c5ad18..74d1ebf7 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3759,44 +3759,47 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle); } -static int read_copy_size8(struct tracecmd_input *handle, int fd, unsigned long long *size) +static int read_copy_size8(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, unsigned long long *size) { /* read size */ - if (do_read_check(handle, size, 8)) + if (do_read_check(in_handle, size, 8)) return -1; - if (__do_write_check(fd, size, 8)) + if (do_write_check(out_handle, size, 8)) return -1; - *size = tep_read_number(handle->pevent, size, 8); + *size = tep_read_number(in_handle->pevent, size, 8); return 0; } -static int read_copy_size4(struct tracecmd_input *handle, int fd, unsigned int *size) +static int read_copy_size4(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, + unsigned int *size) { /* read size */ - if (do_read_check(handle, size, 4)) + if (do_read_check(in_handle, size, 4)) return -1; - if (__do_write_check(fd, size, 4)) + if (do_write_check(out_handle, size, 4)) return -1; - *size = tep_read_number(handle->pevent, size, 4); + *size = tep_read_number(in_handle->pevent, size, 4); return 0; } -static int read_copy_data(struct tracecmd_input *handle, - unsigned long long size, int fd) +static int read_copy_data(struct tracecmd_input *in_handle, + unsigned long long size, + struct tracecmd_output *out_handle) { char *buf; buf = malloc(size); if (!buf) return -1; - if (do_read_check(handle, buf, size)) + if (do_read_check(in_handle, buf, size)) goto failed_read; - if (__do_write_check(fd, buf, size)) + if (do_write_check(out_handle, buf, size)) goto failed_read; free(buf); @@ -3808,65 +3811,82 @@ static int read_copy_data(struct tracecmd_input *handle, return -1; } -static int copy_header_files(struct tracecmd_input *handle, int fd) +static int copy_header_files(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_HEADERS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_HEADERS) || + !check_out_state(out_handle, TRACECMD_FILE_HEADERS)) return -1; /* "header_page" */ - if (read_copy_data(handle, 12, fd) < 0) + if (read_copy_data(in_handle, 12, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; /* "header_event" */ - if (read_copy_data(handle, 13, fd) < 0) + if (read_copy_data(in_handle, 13, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_HEADERS; + in_handle->file_state = TRACECMD_FILE_HEADERS; + out_set_file_state(out_handle, in_handle->file_state); return 0; } -static int copy_ftrace_files(struct tracecmd_input *handle, int fd) +static int copy_ftrace_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; unsigned int count; unsigned int i; - if (handle->file_state != TRACECMD_FILE_FTRACE_EVENTS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_FTRACE_EVENTS) || + !check_out_state(out_handle, TRACECMD_FILE_FTRACE_EVENTS)) return -1; - if (read_copy_size4(handle, fd, &count) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (i = 0; i < count; i++) { - if (read_copy_size8(handle, fd, &size) < 0) - return -1; + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; } - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_event_files(struct tracecmd_input *handle, int fd) +static int copy_event_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; char *system; @@ -3874,103 +3894,159 @@ static int copy_event_files(struct tracecmd_input *handle, int fd) unsigned int count; unsigned int i,x; - if (handle->file_state != TRACECMD_FILE_ALL_EVENTS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_ALL_EVENTS) || + !check_out_state(out_handle, TRACECMD_FILE_ALL_EVENTS)) return -1; - if (read_copy_size4(handle, fd, &systems) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &systems) < 0) + goto error; for (i = 0; i < systems; i++) { - system = read_string(handle); + system = read_string(in_handle); if (!system) - return -1; - if (__do_write_check(fd, system, strlen(system) + 1)) { + goto error; + if (do_write_check(out_handle, system, strlen(system) + 1)) { free(system); - return -1; + goto error; } free(system); - if (read_copy_size4(handle, fd, &count) < 0) - return -1; + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (x=0; x < count; x++) { - if (read_copy_size8(handle, fd, &size) < 0) - return -1; + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; } } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_ALL_EVENTS; + + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_proc_kallsyms(struct tracecmd_input *handle, int fd) +static int copy_proc_kallsyms(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_KALLSYMS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_KALLSYMS) || + !check_out_state(out_handle, TRACECMD_FILE_KALLSYMS)) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; - handle->file_state = TRACECMD_FILE_KALLSYMS; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_KALLSYMS; + + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_ftrace_printk(struct tracecmd_input *handle, int fd) +static int copy_ftrace_printk(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_PRINTK - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_PRINTK) || + !check_out_state(out_handle, TRACECMD_FILE_PRINTK)) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; - if (!size) - return 0; /* OK? */ + out_compression_start(out_handle); - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; + if (!size) + goto out; /* OK? */ - handle->file_state = TRACECMD_FILE_PRINTK; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_PRINTK; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_command_lines(struct tracecmd_input *handle, int fd) +static int copy_command_lines(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_CMD_LINES - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_CMD_LINES) || + !check_out_state(out_handle, TRACECMD_FILE_CMD_LINES)) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; - handle->file_state = TRACECMD_FILE_CMD_LINES; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_CMD_LINES; + if (out_compression_end(out_handle)) + goto error; + out_set_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor - * @handle: input handle for the trace.dat file to copy from. - * @fd: The file descriptor to copy to. + * @in_handle: input handle for the trace.dat file to copy from. + * @out_handle: output handle to the trace.dat file to copy to. * @start_state: The file state to start copying from (zero for the beginnig) * @end_state: The file state to stop at (zero for up to cmdlines) * @@ -3981,7 +4057,8 @@ static int copy_command_lines(struct tracecmd_input *handle, int fd) * NOTE: The input handle is also modified, and ends at the end * state as well. */ -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state) { @@ -3997,68 +4074,67 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, if (end_state < TRACECMD_FILE_HEADERS) return 0; - - if (handle->file_state >= start_state) { + if (in_handle->file_state >= start_state) { /* Set the handle to just before the start state */ - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + lseek64(in_handle->fd, in_handle->header_files_start, SEEK_SET); /* Now that the file handle has moved, change its state */ - handle->file_state = TRACECMD_FILE_INIT; + in_handle->file_state = TRACECMD_FILE_INIT; } /* Try to bring the input up to the start state - 1 */ - ret = tracecmd_read_headers(handle, start_state - 1); + ret = tracecmd_read_headers(in_handle, start_state - 1); if (ret < 0) goto out; switch (start_state) { case TRACECMD_FILE_HEADERS: - ret = copy_header_files(handle, fd); + ret = copy_header_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_FTRACE_EVENTS: /* handle's state is now updating with the copies */ - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_files(handle, fd); + ret = copy_ftrace_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_ALL_EVENTS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_event_files(handle, fd); + ret = copy_event_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_KALLSYMS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_proc_kallsyms(handle, fd); + ret = copy_proc_kallsyms(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_PRINTK: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_printk(handle, fd); + ret = copy_ftrace_printk(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_CMD_LINES: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_command_lines(handle, fd); + ret = copy_command_lines(in_handle, out_handle); default: break; } diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c41dfb8f..06d0738e 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -89,8 +89,8 @@ struct list_event_system { char *name; }; -static stsize_t -do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) +__hidden long long +do_write_check(struct tracecmd_output *handle, const void *data, long long size) { if (handle->do_compress) return tracecmd_compress_write(handle->compress, data, size); @@ -1946,11 +1946,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - if (tracecmd_copy_headers(ihandle, handle->fd, 0, 0) < 0) + if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) goto out_free; - handle->file_state = tracecmd_get_file_state(ihandle); - /* The file is all ready to have cpu data attached */ return handle; @@ -1959,6 +1957,11 @@ out_free: return NULL; } +__hidden void out_set_file_state(struct tracecmd_output *handle, int new_state) +{ + handle->file_state = new_state; +} + __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); From patchwork Mon Jun 14 07:50:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9619DC48BE6 for ; Mon, 14 Jun 2021 07:52:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7DBBA61370 for ; Mon, 14 Jun 2021 07:52:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232619AbhFNHyE (ORCPT ); Mon, 14 Jun 2021 03:54:04 -0400 Received: from mail-ej1-f54.google.com ([209.85.218.54]:40725 "EHLO mail-ej1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232617AbhFNHyD (ORCPT ); Mon, 14 Jun 2021 03:54:03 -0400 Received: by mail-ej1-f54.google.com with SMTP id my49so15064348ejc.7 for ; Mon, 14 Jun 2021 00:52:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BTU1iFO2V4ZzMjExOiVLwCGbuFnGNHp5D2Mj4ZKntEk=; b=NreabNewJTGKscnzf0f5kMwvJafThH3qICR8jVHDlBW4rFW5bZpCrU77BVvxhwwdkW qWUMs5QYct0zXuSjuyVr19maFZHjbto/0BuwGWnkPXOexPHzjJhNbsaw2cm5icLJP3BM 2BDB/zaqAZcRuCoXRlWagByrw9AheLZC45uIp8qczel3ErmkF1OVpiR4FBAUF3E8AtqL A51oraHX3l1AYMFl6Ey1Sk2oN2+Lhk05oYIcmfDvfOla9Cd5HRng9HTpExw+USoM1i8t PNxg2O1MoS12i2wKknovA1soAXeNoWqnjEKfccDBfeFvhJZMTdrvbZ5KqQXgG8MJYmy6 WTmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BTU1iFO2V4ZzMjExOiVLwCGbuFnGNHp5D2Mj4ZKntEk=; b=SaoDnzX2a7r5jROdfdrScytSWzr20hhlDazVGvzEqA0fqUhtQpukPagtwTT2NifNnQ 3wvqMuPuMS8GT9xzLIG+dtXAgQlNSAF4hQoc7OCtwfe16246MEEmj33JoJsTLeociPAC RMvc73W+auQMXbRJxEcJG/mrqJc854HRYKkf49clYpKpasiwWI2RwvMNanQkRrHb5eXy 7/jIskGyjRAyrywCDFUIOrwUJ6x9BPtT/Q8hvwurm4XfyuoK8y6/S+gDtJ4kOkCj8GJm GGOnL104XupKi00KhaVZNY960JVdyXsRqi3XIwu/lCmhIebVctUT3z+mhs3q9QWwzQA0 bbIA== X-Gm-Message-State: AOAM530AgzXfhUriqzZeOjBquHMAFZth5OrK3ACFQzMqF4uYAUPclJOk aTp+rnR6sMah7fctWGwvYJo= X-Google-Smtp-Source: ABdhPJwunKu6zIBxTD6LF32GIVZRH3FDLBkzhIMcB+kztf6mhsReR4KU2zhuuAh9Hp0Qvn4tbr+lFA== X-Received: by 2002:a17:906:d8cf:: with SMTP id re15mr14157964ejb.278.1623657060470; Mon, 14 Jun 2021 00:51:00 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.50.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:00 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 30/45] trace-cmd: Do not use trace file compression with streams Date: Mon, 14 Jun 2021 10:50:14 +0300 Message-Id: <20210614075029.598048-31-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a temporary output handler for use with a stream, force it not not use trace file compression. As no data are saved in this file, there is no need to have a compression overhead. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-stream.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tracecmd/trace-stream.c b/tracecmd/trace-stream.c index 317de1b2..1ca3a79d 100644 --- a/tracecmd/trace-stream.c +++ b/tracecmd/trace-stream.c @@ -43,7 +43,7 @@ trace_stream_init(struct buffer_instance *instance, int cpu, int fd, int cpus, tfd = fileno(fp); ofd = dup(tfd); - trace_output = tracecmd_create_init_fd(ofd, NULL); + trace_output = tracecmd_create_init_fd(ofd, "none"); if (!trace_output) { fclose(fp); return NULL; From patchwork Mon Jun 14 07:50:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BD46C48BE8 for ; Mon, 14 Jun 2021 07:51:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 74FAE61246 for ; Mon, 14 Jun 2021 07:51:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232590AbhFNHxV (ORCPT ); Mon, 14 Jun 2021 03:53:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232583AbhFNHxT (ORCPT ); Mon, 14 Jun 2021 03:53:19 -0400 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4031C061574 for ; Mon, 14 Jun 2021 00:51:02 -0700 (PDT) Received: by mail-ed1-x52f.google.com with SMTP id r7so30789974edv.12 for ; Mon, 14 Jun 2021 00:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p6IobRJ6Evi4eZu0SI8LMyVIQ413IC7AqsSEq/FbWVo=; b=JOAwPraUdoUGD3pNe4hqIByv+3Mf9wSK7aj4GH9VCxipcQrhJhFGwEKP/JLVlLe5Wz 8xibUHaGqnqZVSkzrLgXPcutYJRvXifKPwg2/lnAQlxde1oJMm3SyqR4kGI+INktdZVl XQOOWANgtJk39OvR4XojKM1b+SOdxixyh7YE4w7lyxIVpYD4rWm1xQt1OnvIxqHkTBap xX5En9Z4PEgYlW1UXyIifpQYJiWhj9R6hR7m605zHlY/Vwq+Etg0qbHqBt+YzeOD/UR8 heTGAToI9YSrdr8EYtdEvt5kucdA8lCmba6Nty4eORXz3NWJ+rmamjT9ZmpOya5gTe1i p/dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p6IobRJ6Evi4eZu0SI8LMyVIQ413IC7AqsSEq/FbWVo=; b=hNe8D6Bl6txaj9/+YHdohJ15pzK6VEBVSfBr4Vzqr0nO5TC9+U9WWSNWQNCLRR2i5J ooqGmcnSrSzVaCF4M7INQXEUpJ5oBfL63KTe4is0pMuqfJ0Y4CVZDyVe5I6/SvxUovOH 8yJdAcA9HRDdmde8mryCgVvS4oGGJR4CeizUBw4b5kwsbYH/yJlHCfIv/qbdHX5fPez3 0haQqsrZhEqrc6eevSDAuC1BQrFPNn/cecHYOftXfWLWYB/gFmoR17z56ZXmBY7TTtxH FbO5QYOXx0qwRlvKS/npsvxEGBKmEl/A+0bFTZwIetCUfyyjT8Ei4kdT3PgvZeBlCrUO lkdA== X-Gm-Message-State: AOAM530s0Tgespb/o5ZzcRwZgA50Qo3p9PBe93knjFrg2XXI42EoSEef aavsgVk98Wa61tTmaRQENwvZK5Y3PYlaEQ== X-Google-Smtp-Source: ABdhPJzgLIlho7OCOeJ0m3BGH7o5LkaiUx1pvXcRYaC2Ul1u2A1eTzew3A6vudr/hCjxhKt+WCvbHA== X-Received: by 2002:a05:6402:b5a:: with SMTP id bx26mr15162561edb.81.1623657061407; Mon, 14 Jun 2021 00:51:01 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:01 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 31/45] trace-cmd library: Add new API to get file version of output handler Date: Mon, 14 Jun 2021 10:50:15 +0300 Message-Id: <20210614075029.598048-32-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added an API to get the version of the trace file, associated with given output file handler. tracecmd_get_out_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-output.c | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 65e80cec..9a08a960 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -319,6 +319,7 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 06d0738e..9b83eb76 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1966,3 +1966,12 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); } + +/** + * tracecmd_get_out_file_version - return the trace.dat file version + * @handle: output handle for the trace.dat file + */ +unsigned long tracecmd_get_out_file_version(struct tracecmd_output *handle) +{ + return handle->file_version; +} From patchwork Mon Jun 14 07:50:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 868DCC49360 for ; Mon, 14 Jun 2021 07:52:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D7446138C for ; Mon, 14 Jun 2021 07:52:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232632AbhFNHyW (ORCPT ); Mon, 14 Jun 2021 03:54:22 -0400 Received: from mail-ed1-f53.google.com ([209.85.208.53]:39444 "EHLO mail-ed1-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232636AbhFNHyV (ORCPT ); Mon, 14 Jun 2021 03:54:21 -0400 Received: by mail-ed1-f53.google.com with SMTP id dj8so45193135edb.6 for ; Mon, 14 Jun 2021 00:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OL6z+eplL6Um4JCUwgrmpOe7SOkUvFTEKsytnrQIMvs=; b=tBL3xhbBePer2qFA8n+wh1wMGgG5QKX90Beaa1kkTB/PVeviOio91LjqUUAKr0t7Or S4C4Rq9L6XmXwAZAxpYPE3SY80A1MAgjeGhbQGRufIonIv78A8ABPLASjUXI896Zi4uq 6iW9zb7d+VwCHognbVkw8XIkaCa4ZYpjJx5NIUB1dufRYHx/kuQba9uR13/7FunHkV6C RlVHr3pLcUj5m+YNTb0yd+xmjMOnRyGadgHTFXTgCWfwX6qT80RHQe3vskacmCWYNUq+ tLFkJH8jwbg8iVlp01ElLLPMIPU8gJ0j7UOd/PzrHb1uBlh7hSbFo0ClqKcuFjZZoVEQ G9ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OL6z+eplL6Um4JCUwgrmpOe7SOkUvFTEKsytnrQIMvs=; b=PJe5L4AVxEpBu50M4phmceyf4QD+VZUQ6Gj59iwXwLs9Sd4hzXPJZvobJwdgC6MkT/ sVtGK7ZdjTTV6w/6kVzx7hdKwwTLRD4yGcCQaRmONAKZW10irifZXTdRbftvCjXaI7H1 nMdejvy8Vr07+DCKzAOn8LSfuNHF7p7JdWomAaDvrfTm25GeTGsyuhdK6CjhdvV9vy0p o+VVHIlXP5BgzCtmqDwYJsa28FzlEyAq3ZywJ1r5/ERwa6L4jK++r65nhvt9sp6fAWPD g4YOkBg4qQXvKGamqconzlyb+kCIrAbZoQT9LVsAldx2FPeaJAFY1fXAvdRCWdn9RVg2 ZcPQ== X-Gm-Message-State: AOAM5337XmPLtwpBHd+eypGLb7WiWkx6qMcvJulWM6qxI0sM7lxTk0Ku KV5PGmSHu4wJOPaEIH3BgsU= X-Google-Smtp-Source: ABdhPJxlpZdOavn/g0aLy2mM6QjLU5Rtf+nCfH+dG9EljhBGQq3yhh9tqSlQqgV8y2f2hj8RURW+Eg== X-Received: by 2002:aa7:c584:: with SMTP id g4mr15291731edq.335.1623657062489; Mon, 14 Jun 2021 00:51:02 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:01 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 32/45] trace-cmd: Add file state parameter to tracecmd_copy Date: Mon, 14 Jun 2021 10:50:16 +0300 Message-Id: <20210614075029.598048-33-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The tracecmd_copy() API is used to create output trace handler from given input trace handler and to copy data from it. Currently it is used by trace-cmd split and restore commands, that's why it is hardcoded what data to be copied from input to output handler. Addig desired output handler file state makes the API more generic and allows it to be used in more use cases. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 4 ++-- lib/trace-cmd/trace-output.c | 8 ++++++-- tracecmd/trace-restore.c | 2 +- tracecmd/trace-split.c | 2 +- 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 9a08a960..2c31be5f 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -309,8 +309,8 @@ int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file); +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, + enum tracecmd_file_states state); int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9b83eb76..806c833c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1931,13 +1931,16 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * tracecmd_copy - copy the headers of one trace.dat file for another * @ihandle: input handle of the trace.dat file to copy * @file: the trace.dat file to create + * @state: what data will be copied from the source handle * * Reads the header information and creates a new trace data file * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, + enum tracecmd_file_states state) { + enum tracecmd_file_states fstate; struct tracecmd_output *handle; const char *compr_name = NULL; @@ -1946,7 +1949,8 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) + fstate = state > TRACECMD_FILE_CMD_LINES ? TRACECMD_FILE_CMD_LINES : state; + if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; /* The file is all ready to have cpu data attached */ diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 96b4fb5d..7d439fd2 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -125,7 +125,7 @@ void trace_restore (int argc, char **argv) if (tracecmd_read_headers(ihandle, TRACECMD_FILE_CMD_LINES) < 0) die("error reading file %s headers", input); - handle = tracecmd_copy(ihandle, output); + handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES); tracecmd_close(ihandle); } else handle = tracecmd_create_init_file(output, NULL); diff --git a/tracecmd/trace-split.c b/tracecmd/trace-split.c index 8366d128..401e1d29 100644 --- a/tracecmd/trace-split.c +++ b/tracecmd/trace-split.c @@ -345,7 +345,7 @@ static double parse_file(struct tracecmd_input *handle, dir = dirname(output); base = basename(output); - ohandle = tracecmd_copy(handle, output_file); + ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES); cpus = tracecmd_cpus(handle); cpu_data = malloc(sizeof(*cpu_data) * cpus); From patchwork Mon Jun 14 07:50:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF6E8C49360 for ; Mon, 14 Jun 2021 07:51:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E6396124B for ; Mon, 14 Jun 2021 07:51:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232591AbhFNHxJ (ORCPT ); Mon, 14 Jun 2021 03:53:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232589AbhFNHxH (ORCPT ); Mon, 14 Jun 2021 03:53:07 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2E8FC061766 for ; Mon, 14 Jun 2021 00:51:04 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id ba2so43440943edb.2 for ; Mon, 14 Jun 2021 00:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=grOKVMmRfFecxyDe/8UxFkVTzxTChc98bauEYxiRBO0=; b=Z3Y7Rf6Jbi3XVKfEVqvnPkrNlKIpJiDQlpzj1GMqNTyNMxcZjXWSAuqFGQ6U4vfn0c 2TTLYtZYv+bUm1YWdQUlhViXHNp4ddhBrgQo+XUIi3DdA9kY/MdOb3IciusnCiJkYdKK dsrNaewMZ5IUluwd4XfTj1QrvJdjF52avGnisUnqIPk8M5buspOYKlHYoGLDZpFF6k++ fygboK8ZK+5NdCMwrMz/62u8XO3fPYTHR1O4MfOS/CsiZ/iueFZIgMON6gUD3mcc9xQw cMlfxLuCJDfm1lB7X2eOtkX6XJq/BaEdqFFag/t4qxJNJNMJelvu9JHi4TZFS+/a/mRS qF/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=grOKVMmRfFecxyDe/8UxFkVTzxTChc98bauEYxiRBO0=; b=smgzkjB7Yf0rhAwD6rc+kSYml8J1EnnUHjPRiY6glZtN/hsu5vrVssipIir5OZX39W mIvek9Po53YdU5KGnqBCQLO4mVArkGcAuF7CZGYOkSkvbDs6xQ6EOxVB79mVRKbaAdvv tBtWOHh5+L9dYJ4IfYLj2PtZZs6Drg3aD2AizKYa5vfN7wToD0bFv6SBQruk+hVqfXSH RC8TWf8NKCMR79BLeaIsH9gztERd3DaHPssWJUd4jRfU0mkLRbQcbeP7GB6rnYm3xQY9 VdFLwnRYQI9tTCkaqW47x+07dkKx1s6sYDug76LmwjHIysxlHQjwUojCb0PDNTgpTKav 5/gA== X-Gm-Message-State: AOAM530un/bVtQKbWEzC5QAQ+dcEv4goDoFzq8nZHgwn9hkT+D1XjR4s YckFtM+1WWrT+frzF43NPiY4GkVVZ6+S8w== X-Google-Smtp-Source: ABdhPJyG1iShsWAvJ0FA8aHuH44g67Qf3h0loJZWY1sAkiAkkKGeFRfF3sTPYuHT20X43o7oMp3oyg== X-Received: by 2002:a05:6402:27d1:: with SMTP id c17mr15487183ede.28.1623657063449; Mon, 14 Jun 2021 00:51:03 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:03 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 33/45] trace-cmd: Copy CPU count in tracecmd_copy Date: Mon, 14 Jun 2021 10:50:17 +0300 Message-Id: <20210614075029.598048-34-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of CPU count from input to output trace hanlder. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 30 ++++++++++++++++++++++++++++++ lib/trace-cmd/trace-output.c | 2 +- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 74d1ebf7..289102bf 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4043,6 +4043,23 @@ error: return -1; } +static int copy_cpu_count(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) +{ + unsigned int size; + + if (!check_in_state(in_handle, TRACECMD_FILE_CPU_COUNT) || + !check_out_state(out_handle, TRACECMD_FILE_CPU_COUNT)) + return -1; + + if (read_copy_size4(in_handle, out_handle, &size) < 0) + return -1; + + in_handle->file_state = TRACECMD_FILE_CPU_COUNT; + out_set_file_state(out_handle, in_handle->file_state); + + return 0; +} + /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor * @in_handle: input handle for the trace.dat file to copy from. @@ -4135,6 +4152,19 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, return 0; ret = copy_command_lines(in_handle, out_handle); + if (ret < 0) + goto out; + + /* fallthrough */ + case TRACECMD_FILE_CPU_COUNT: + if (end_state <= in_handle->file_state) + return 0; + + ret = copy_cpu_count(in_handle, out_handle); + if (ret < 0) + goto out; + + /* fallthrough */ default: break; } diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 806c833c..b4bdaeeb 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1949,7 +1949,7 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - fstate = state > TRACECMD_FILE_CMD_LINES ? TRACECMD_FILE_CMD_LINES : state; + fstate = state > TRACECMD_FILE_CPU_COUNT ? TRACECMD_FILE_CPU_COUNT : state; if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; From patchwork Mon Jun 14 07:50:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA421C48BE8 for ; Mon, 14 Jun 2021 07:51:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4D1F6124B for ; Mon, 14 Jun 2021 07:51:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232592AbhFNHxY (ORCPT ); Mon, 14 Jun 2021 03:53:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232593AbhFNHxX (ORCPT ); Mon, 14 Jun 2021 03:53:23 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3679C061787 for ; Mon, 14 Jun 2021 00:51:05 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id s6so45189514edu.10 for ; Mon, 14 Jun 2021 00:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L7aeLr4lc1KENvDwyPAuCgASeUBhtlmmst7Nub9ec/o=; b=meNCIekbRgMRnF4O1zkN5TFgSSKNU+unXvb7GMgFPKODuIRjqUIwSLvQH+SDm51HDK B99Eu3zGGKam8RJdUP9gN89xQs59GH4RIla8d9ITopQmp47tP9ik8S3uP//UKXT93PgT BhxtYS/XUnhrVZ5ZnZ9KUJW+1FifbOx7xxyP0jycBaW22diJa/Br3YJFrMxny6ofI2NY R6mbHi9NpYF3xI0IhEtAjm9UjO3Fe4lB3XrjdeNbQ8Eg7j5+eS6GFVERh/m659rWZe3g qHGQ1TmrIL25tz54kAJ/Zf9OfRjC0fpjkaLtDpATWQKLhm3m3yDUZotVZLoJQlvMpfFn 2GvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L7aeLr4lc1KENvDwyPAuCgASeUBhtlmmst7Nub9ec/o=; b=H9ogU1QDlfInIkXX+EaIOK0ZL/gMHGrJZ1wX3X6yM/TYuLG1eOxBNO84RzpijZU7GG 0xIRhZIaY6dH9EmlZKkQSyHZlCCmlnNHzZ/3y3GR8lKutEHrmmj84bIq7/s4BvcoHo4E 8Jple19EqpMINy5k6oRsXoIcgCdinKSoE1Aq4XoSBoH3syLkIrhOw8nBhhHbVbrleX2w Yb/v3bIMydinz07PWfB3cA/pd0sOI9PHJz8tDNCHy4K/kJJEM24V/fsopx/+/RGq0L3y EUALDZV814qQB5IaVFvcCm9Wgk4tIKou5p0bj4CgWYMFmvQrpK7QAlTSTsluqQylaGTT bryA== X-Gm-Message-State: AOAM533ok6pSGkmfz2y//SHxa0ciAVRsn4KiCX3UA/tPnophk2f4xS4h D/LNjkb9B7hf5IxCBowzwhlnNt54ZNOIgg== X-Google-Smtp-Source: ABdhPJzSnTUlsS49XZsT3J6NHdoSicY2IOnwlTHwQ+3TBb/N7RCxafvNUTXjIk2MHEctOyhl9U/pfg== X-Received: by 2002:a05:6402:2552:: with SMTP id l18mr15749775edb.166.1623657064310; Mon, 14 Jun 2021 00:51:04 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:03 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 34/45] trace-cmd: Copy buffers description in tracecmd_copy Date: Mon, 14 Jun 2021 10:50:18 +0300 Message-Id: <20210614075029.598048-35-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of trace buffers description from input to output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 38 +++++++++++++++++++ lib/trace-cmd/trace-output.c | 3 ++ 3 files changed, 43 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 2c31be5f..3735586d 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -174,6 +174,8 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); +int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 289102bf..7d0a41c2 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4173,6 +4173,44 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, return ret < 0 ? -1 : 0; } +static int skip_buffers(struct tracecmd_input *handle) +{ + unsigned long long offset; + unsigned int count; + char *bname; + int i; + + if (read4(handle, &count) < 0) + return -1; + + for (i = 0; i < count; i++) { + if (read8(handle, &offset) < 0) + return -1; + bname = read_string(handle); + free(bname); + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +} + +int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int i; + + if (in_handle->file_version >= 7) { + if (!check_in_state(in_handle, TRACECMD_FILE_BUFERS)) + return -1; + skip_buffers(in_handle); + } + for (i = 0; i < in_handle->nr_buffers; i++) + tracecmd_add_buffer_description(out_handle, + in_handle->buffers[i].name, 0); + + return tracecmd_write_buffers_description(out_handle); +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index b4bdaeeb..ca9024bf 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1952,6 +1952,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char fstate = state > TRACECMD_FILE_CPU_COUNT ? TRACECMD_FILE_CPU_COUNT : state; if (tracecmd_copy_headers(ihandle, handle, 0, fstate) < 0) goto out_free; + if (state >= TRACECMD_FILE_BUFERS && + tracecmd_copy_buffer_descr(ihandle, handle) < 0) + goto out_free; /* The file is all ready to have cpu data attached */ return handle; From patchwork Mon Jun 14 07:50:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62BF9C49361 for ; Mon, 14 Jun 2021 07:52:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5019761380 for ; Mon, 14 Jun 2021 07:52:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232637AbhFNHyW (ORCPT ); Mon, 14 Jun 2021 03:54:22 -0400 Received: from mail-ed1-f49.google.com ([209.85.208.49]:33675 "EHLO mail-ed1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232632AbhFNHyV (ORCPT ); Mon, 14 Jun 2021 03:54:21 -0400 Received: by mail-ed1-f49.google.com with SMTP id f5so40211937eds.0 for ; Mon, 14 Jun 2021 00:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rl0kPyNRg0Ei1SdJ3rLL1247YtZ7+uNI5K11AvHuLvs=; b=l1lJIpg19UrVJ7NmqrdKNTYM0n0CiQNHeFevNE57OYSRXQ0Pa0Xs5/wC4DispzdeTN YuEYK+zBSWCT50/6DsKFaU33VpWluBd+sLVDZiqB5yT7+O8UwsqhV99mW73Ql/EzQTFf h0rEyzWUAhIUBNr/mRdZN7gYYCVJLVr2fuSzELqaF43CTs+qF6p0YOkwGY9KzbmNUjuO byqhhdRzuwoYVDV1WFKBsbdy5SOxQ5765UMhbq6+rkQpTKln1+qtKmR9DJDpGD1I/+wg faE2MXoLJnGhaJ5LuVAXrI27fkYQu0xcf4H3kYaJent9XDa/B4ytpBEJGlRnlf+S2998 buDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rl0kPyNRg0Ei1SdJ3rLL1247YtZ7+uNI5K11AvHuLvs=; b=UNQGpLmQkXJF4VPbcPXqrMtSmQnxxeJk1QtbXWPaxBuiJzZEtbMFZmPbzpk9lrMvME duLPMp5eGaxyAz+Y7o6a43kKlk8exM2shyvetqwSkFBOLBMCmhP3BNQ6Rb8DG1ibtzyt UwMhNdroOplSojZrinA4gpnCSOBdpGzXIofLj7DGp8ZmGTVjxngdYma7tu51E2yNb1+P YNL36sej4MAm4TZxotQqbYBQUbv1XE9cXIBP4zNX5eQot8H9bxUSnnldQCW5t1Y1fqjA 0M4VSNK/Rq1dnhIl+ANPnzQ4ae3WZ87c1CqiN+VOaqOBH/ZBg1YJbi8rGC0ytWt5zxd2 GTBQ== X-Gm-Message-State: AOAM532cOjP/4Dm3SEA8nhwUTqabvIYk2Afz4LfN0JpJ8zN6org08gYA IKJjsR59yYPy5I+G2A3kp0I= X-Google-Smtp-Source: ABdhPJyu7RPBhYCclmSfMurRz1dKuVZHsuKY5EnBQv9GB2XhNddfGkLz4EAI4yeUSArHMRk+fJ9eag== X-Received: by 2002:aa7:c7c7:: with SMTP id o7mr15919084eds.231.1623657065263; Mon, 14 Jun 2021 00:51:05 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:04 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 35/45] trace-cmd: Copy options in tracecmd_copy Date: Mon, 14 Jun 2021 10:50:19 +0300 Message-Id: <20210614075029.598048-36-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of trace options section from input to output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 80 +++++++++++++++++++ lib/trace-cmd/trace-output.c | 8 ++ 4 files changed, 91 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 3735586d..595ba688 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -176,6 +176,8 @@ int tracecmd_copy_headers(struct tracecmd_input *in_handle, enum tracecmd_file_states end_state); int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); +int tracecmd_copy_options(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 8fb391fc..2e5682c0 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -52,6 +52,7 @@ void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); void out_set_file_state(struct tracecmd_output *handle, int new_state); +void out_save_options_offset(struct tracecmd_output *handle); int write_buffers_description_v7(struct tracecmd_output *handle); int write_buffers_description_v6(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 7d0a41c2..794f8ea3 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4211,6 +4211,86 @@ int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, return tracecmd_write_buffers_description(out_handle); } +static int copy_options(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) +{ + unsigned short option; + unsigned short en2; + unsigned int size; + unsigned int en4; + + if (in_uncompress_block(in_handle)) + return -1; + out_save_options_offset(out_handle); + out_compression_start(out_handle); + for (;;) { + if (do_read_check(in_handle, &option, 2)) + goto error; + en2 = tep_read_number(in_handle->pevent, &option, 2); + if (en2 == TRACECMD_OPTION_DONE) + break; + /* next 4 bytes is the size of the option */ + if (do_read_check(in_handle, &size, 4)) + goto error; + en4 = tep_read_number(in_handle->pevent, &size, 4); + /* Do not copy buffers description, as there is a file specific offset */ + if (en2 == TRACECMD_OPTION_BUFFER) { + /* Skip the option */ + do_lseek(in_handle, en4, SEEK_CUR); + continue; + } + + if (do_write_check(out_handle, &option, 2)) + goto error; + if (do_write_check(out_handle, &size, 4)) + goto error; + if (read_copy_data(in_handle, en4, out_handle)) + goto error; + } + if (do_write_check(out_handle, &option, 2)) + goto error; + in_uncompress_reset(in_handle); + if (out_compression_end(out_handle)) + goto error; + in_handle->file_state = TRACECMD_FILE_OPTIONS; + out_set_file_state(out_handle, in_handle->file_state); + /* Append local options */ + return tracecmd_append_options(out_handle); +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return 0; +} + +int tracecmd_copy_options(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + off64_t offset; + char buf[10]; + + if (!check_in_state(in_handle, TRACECMD_FILE_OPTIONS) || + !check_out_state(out_handle, TRACECMD_FILE_OPTIONS)) + return -1; + + /* Save where we currently are */ + offset = lseek64(in_handle->fd, 0, SEEK_CUR); + + if (do_read_check(in_handle, buf, 10)) + return -1; + /* check if this handles options */ + if (strncmp(buf, "options", 7) == 0) { + if (do_write_check(out_handle, "options ", 10)) + return -1; + if (copy_options(in_handle, out_handle) < 0) + return -1; + } else { + if (lseek64(in_handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + in_handle->file_state = TRACECMD_FILE_OPTIONS; + out_set_file_state(out_handle, in_handle->file_state); + } + return 0; +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index ca9024bf..95ca483c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1955,6 +1955,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (state >= TRACECMD_FILE_BUFERS && tracecmd_copy_buffer_descr(ihandle, handle) < 0) goto out_free; + if (state >= TRACECMD_FILE_OPTIONS && + tracecmd_copy_options(ihandle, handle) < 0) + goto out_free; /* The file is all ready to have cpu data attached */ return handle; @@ -1974,6 +1977,11 @@ __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) return check_file_state(handle->file_version, handle->file_state, new_state); } +__hidden void out_save_options_offset(struct tracecmd_output *handle) +{ + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); +} + /** * tracecmd_get_out_file_version - return the trace.dat file version * @handle: output handle for the trace.dat file From patchwork Mon Jun 14 07:50:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BFE9C48BE6 for ; Mon, 14 Jun 2021 07:51:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 884C061352 for ; Mon, 14 Jun 2021 07:51:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232589AbhFNHxL (ORCPT ); Mon, 14 Jun 2021 03:53:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232583AbhFNHxK (ORCPT ); Mon, 14 Jun 2021 03:53:10 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5693C061766 for ; Mon, 14 Jun 2021 00:51:07 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id f5so40212012eds.0 for ; Mon, 14 Jun 2021 00:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1WA9zC+hThGEZdfqPArjxMbh2DE3OSOtt2doRIwYfgI=; b=Sky/t7eEK5wIuIX+6fRW2H0asx88ijlI5b3B72uKBZ62eWF+sGSSXPdCirjZ4vGJhI VYd4C5o5qSn+Uhwt9gtMBiEbh8f8GmPQqV2qUTze1AkcZgvT02j1fX7mvzsVNZ4Mxmrn akgZqfBITqhPc2vJb+fMuuKs0g18mTuTHZXM0hfVjYqyzoud2FEs8BDI+Kcj2qO07H99 zGlLkz51E2/4lJiikHpW3nuNS9aTRBx1pXZKgxr9VqnzesWtJxue3ypSiXsdVQJg8rLs no6gCoOVBIIwd1D850DRb1xMrPH4HfTb7uBfwPAPf+cUMKGLoz5weAvocDrGur2iCnwh k7ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1WA9zC+hThGEZdfqPArjxMbh2DE3OSOtt2doRIwYfgI=; b=qHiS3bbeX1kCfBxnK1NyzP+2VGjAgRqB2xfyn5eA6HvWejLJ+M8SFjBPdjiRAnijcC xxahvGgWeZJj6ktd0QYLgijJISBb01mMmPNrkzX5izJ3mDwYGB95teIQNW8r6f8eGyxs F0MugkMaUgv0XeqAPhsJ/5dbqnzSX9rdF86EuNBZnI/uqbIJUu50UXahQjTNaijro8/e Su7E4hR8CeqGlZU5e21z+qnWvRX7IRYOnIDMMrM2jFtXRiISYk6yBSb1xVTefDqssnC6 35Kq2cGgUc/2iNq+3cfJB0HOLbePnveUDXj6+KB/rrRYcdMa7XyE0JZIAwd27bcvzE2E XCxw== X-Gm-Message-State: AOAM532Tnqv7ZSv7uDhGpaTveZGBNRTfwdK9fIcgLkgy7HcB6DjUdBrU J3U+ANImoVCkotoJ8dNb2u74MEG18G0P7w== X-Google-Smtp-Source: ABdhPJzbnmiNQg+CrziJ4u8D6us0Jy1ZvSn7tFDeLpaA+uo8ynkQSxoBMHy01/CKuE9pmGEggyIieQ== X-Received: by 2002:a05:6402:1c11:: with SMTP id ck17mr16273735edb.102.1623657066225; Mon, 14 Jun 2021 00:51:06 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:05 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 36/45] trace-cmd library: Refactor the logic for writing CPU trace data Date: Mon, 14 Jun 2021 10:50:20 +0300 Message-Id: <20210614075029.598048-37-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Refactored the internal library logic for writing trace CPU data in the trace file. The existing logic copies trace data from a temporary file into the trace file. In order to reuse the code, modify it to support copying the trace data not only from temporary file, but from given fd at given offset. These changes will be used in extending the tracecmd_copy API. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 13 +++ lib/trace-cmd/trace-output.c | 142 +++++++++++++++++------- 2 files changed, 114 insertions(+), 41 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 2e5682c0..970087c7 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -53,10 +53,23 @@ int in_uncompress_block(struct tracecmd_input *handle); void out_set_file_state(struct tracecmd_output *handle, int new_state); void out_save_options_offset(struct tracecmd_output *handle); +unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size); + int write_buffers_description_v7(struct tracecmd_output *handle); int write_buffers_description_v6(struct tracecmd_output *handle); long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); +struct cpu_data_source { + int fd; + int size; + off64_t offset; +}; + +int out_write_cpu_data(struct tracecmd_output *handle, int cpus, + struct cpu_data_source *data); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 95ca483c..e5da4cdb 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -286,18 +286,26 @@ static unsigned long get_size(const char *file) return size; } -static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd) +static tsize_t copy_file_fd(struct tracecmd_output *handle, int fd, unsigned long long max) { + tsize_t rsize = 0; tsize_t size = 0; char buf[BUFSIZ]; stsize_t r; do { - r = read(fd, buf, BUFSIZ); + if (max > 0 && (max - size) < BUFSIZ) + rsize = (max - size); + else + rsize = BUFSIZ; + + r = read(fd, buf, rsize); if (r > 0) { size += r; if (do_write_check(handle, buf, r)) return 0; + if (max > 0 && size >= max) + break; } } while (r > 0); @@ -315,47 +323,61 @@ static tsize_t copy_file(struct tracecmd_output *handle, tracecmd_warning("Can't read '%s'", file); return 0; } - size = copy_file_fd(handle, fd); + size = copy_file_fd(handle, fd, 0); close(fd); return size; } #define COMPRESS_CHUNK_SIZE (1*1024*1024) -static tsize_t copy_file_compress(struct tracecmd_output *handle, - const char *file, unsigned long long *write_size) +__hidden unsigned long long out_copy_fd_compress(struct tracecmd_output *handle, + int fd, unsigned long long max, + unsigned long long *write_size) { unsigned long long rsize = 0; unsigned long long wsize = 0; - tsize_t size; + unsigned long long size; int ret; - int fd; - - fd = open(file, O_RDONLY); - if (fd < 0) { - tracecmd_warning("Can't read '%s'", file); - return 0; - } if (handle->file_version >= 7) { + rsize = max; ret = tracecmd_compress_copy_from(handle->compress, fd, COMPRESS_CHUNK_SIZE, &rsize, &wsize); - if (ret < 0) { - tracecmd_warning("Can't compress '%s'", file); - close(fd); + if (ret < 0) return 0; - } + size = rsize; - *write_size = wsize; + if (write_size) + *write_size = wsize; } else { - size = copy_file_fd(handle, fd); - *write_size = size; + size = copy_file_fd(handle, fd, max); + if (write_size) + *write_size = size; } - close(fd); return size; } +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + ret = out_copy_fd_compress(handle, fd, 0, write_size); + if (!ret) + tracecmd_warning("Can't compress '%s'", file); + + close(fd); + return ret; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -467,7 +489,7 @@ static int read_header_files(struct tracecmd_output *handle) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s' size=%lld read=%lld", path, size, check_size); @@ -493,7 +515,7 @@ static int read_header_files(struct tracecmd_output *handle) endian8 = convert_endian_8(handle, size); if (do_write_check(handle, &endian8, 8)) goto out_close; - check_size = copy_file_fd(handle, fd); + check_size = copy_file_fd(handle, fd, 0); close(fd); if (size != check_size) { tracecmd_warning("wrong size for '%s'", path); @@ -1671,8 +1693,8 @@ struct data_file_write { off64_t doffset; }; -int tracecmd_write_cpu_data(struct tracecmd_output *handle, - int cpus, char * const *cpu_data_files) +__hidden int out_write_cpu_data(struct tracecmd_output *handle, + int cpus, struct cpu_data_source *data) { struct data_file_write *data_files = NULL; off64_t offset; @@ -1680,8 +1702,6 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, unsigned long long read_size; unsigned long long write_size; char *clock = NULL; - char *file; - struct stat st; int ret; int i; @@ -1705,14 +1725,7 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, /* Write 0 for trace data offset and size and store offsets of these fields */ for (i = 0; i < cpus; i++) { - file = cpu_data_files[i]; - ret = stat(file, &st); - if (ret < 0) { - tracecmd_warning("can not stat '%s'", file); - goto out_free; - } - data_files[i].file_size = st.st_size; - + data_files[i].file_size = data[i].size; endian8 = 0; data_files[i].doffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) @@ -1736,13 +1749,19 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (!tracecmd_get_quiet(handle)) fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", i, (unsigned long long) data_files[i].data_offset); - - read_size = copy_file_compress(handle, cpu_data_files[i], &write_size); - if (read_size != data_files[i].file_size) { - errno = EINVAL; - tracecmd_warning("did not match size of %lld to %lld", - read_size, data_files[i].file_size); + offset = lseek64(data[i].fd, data[i].offset, SEEK_SET); + if (offset == (off64_t)-1) goto out_free; + if (data[i].size) { + read_size = out_copy_fd_compress(handle, data[i].fd, data[i].size, &write_size); + if (read_size != data_files[i].file_size) { + errno = EINVAL; + tracecmd_warning("did not match size of %lld to %lld", + read_size, data_files[i].file_size); + goto out_free; + } + } else { + write_size = 0; } /* Write the real CPU data offset inthe file */ offset = lseek64(handle->fd, data_files[i].doffset, SEEK_SET); @@ -1775,6 +1794,47 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, return -1; } +int tracecmd_write_cpu_data(struct tracecmd_output *handle, + int cpus, char * const *cpu_data_files) +{ + struct cpu_data_source *data; + struct stat st; + int ret; + int i; + + data = calloc(cpus, sizeof(struct cpu_data_source)); + if (!data) + return -1; + for (i = 0; i < cpus; i++) + data[i].fd = -1; + for (i = 0; i < cpus; i++) { + ret = stat(cpu_data_files[i], &st); + if (ret < 0) { + tracecmd_warning("can not stat '%s'", cpu_data_files[i]); + break; + } + data[i].fd = open(cpu_data_files[i], O_RDONLY); + if (data[i].fd < 0) { + tracecmd_warning("Can't read '%s'", data[i].fd); + break; + } + + data[i].size = st.st_size; + data[i].offset = 0; + } + + if (i < cpus) + ret = -1; + else + ret = out_write_cpu_data(handle, cpus, data); + for (i = 0; i < cpus; i++) { + if (data[i].fd >= 0) + close(data[i].fd); + } + free(data); + return ret; +} + int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { From patchwork Mon Jun 14 07:50:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B7ADC49360 for ; Mon, 14 Jun 2021 07:52:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 748BE61289 for ; Mon, 14 Jun 2021 07:52:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232620AbhFNHyL (ORCPT ); Mon, 14 Jun 2021 03:54:11 -0400 Received: from mail-ed1-f46.google.com ([209.85.208.46]:34610 "EHLO mail-ed1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232622AbhFNHyK (ORCPT ); Mon, 14 Jun 2021 03:54:10 -0400 Received: by mail-ed1-f46.google.com with SMTP id cb9so45255782edb.1 for ; Mon, 14 Jun 2021 00:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CFwrGL/hfwhsvJgPKC75KzRj+oILJaqU4e0nSiGVm0U=; b=ejZiqxNMQQ3WGv3dI0LQ9egcL9arpVLuEupFVAOmfOYtBLXRzeAeS+1Aw0EPnattlS Vk9qwOB5AyHJ8ofIYucglsI45kUZDeQuqnAcqg7ZtGLy2Sn/WIa70M8uVeo3uHmLTIbO yX1S8UFeZC5hgrFBy+Wg5r0ORVGJP7fdk5wIOqzvJ/HtJNuyhlAX2k/5az61nqHypx/P AJFIpuUep2OG3+S2EyKV/r31wasab3B+zgN7I2fnxttEXVnXBjPNfVP6PgwXup4s5WL9 hHW0tyyarL3LHL22nlBuzbBBeJoaF0qBQz3ujNPp0j425giXd28Znub9tOZK6EGx7bjf cYVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CFwrGL/hfwhsvJgPKC75KzRj+oILJaqU4e0nSiGVm0U=; b=J/G9sa2KVkpJ99M8j5lHpS7qcyZjUggUPt517VuxI1gAP0ypnIZtjPLTPuug+9CrYf JgaUiOPfG8mcNPoJ5wtYK0BCPnsZIPZA2FvHawvyxvfDAU6md3fl6oLgt5GrxN4+yBpe Hymn9+u6MMfqbh8V7rA3W5592N/nXdsKltfn1/g8xJ3qTc0Z6JqSt3MpJbyV/hvDDEGo TCsYPQ+5DA7XreKA8XHBdqmBQPFj2Gm2g/QH4KtgjYDRS58/7nPLPRl3NGyr1t0ymwVk +eTu3ZRHGqQ5TtOyUZOppZ4QQ4eRujWJm2ZNDSA0nYOeKPZ/Rb08GEUsT++XJqsMU2dL ug2Q== X-Gm-Message-State: AOAM531wZWUN9VjIifgU1gtDGAemlhlG0VnKZGGeuFMeQCm0e/UZsGgE Uid+omYuHE/4FmEBo+HeBZ4= X-Google-Smtp-Source: ABdhPJzIjfM77LSEBETzL4LcqRtIUhZHvvkMcdSZ+D0HZSDyZbuX8NQiYZySAjPvUEhlNI714wAbJA== X-Received: by 2002:aa7:d785:: with SMTP id s5mr2117322edq.19.1623657067011; Mon, 14 Jun 2021 00:51:07 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:06 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 37/45] trace-cmd library: Refactor the logic for writing CPU instance trace data Date: Mon, 14 Jun 2021 10:50:21 +0300 Message-Id: <20210614075029.598048-38-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a helper function, internal to the library, for updating CPU instance offset in the trace file. This allows to reuse the logic inside the library, will be used in the tracecmd_copy API. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 ++ lib/trace-cmd/trace-output.c | 11 +++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 970087c7..3cb171ea 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -72,4 +72,6 @@ struct cpu_data_source { int out_write_cpu_data(struct tracecmd_output *handle, int cpus, struct cpu_data_source *data); +int out_update_buffer_cpu_offset(struct tracecmd_output *handle, const char *name); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e5da4cdb..54134d03 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1853,8 +1853,7 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, return tracecmd_write_cpu_data(handle, cpus, cpu_data_files); } -int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - const char *name, int cpus, char * const *cpu_data_files) +__hidden int out_update_buffer_cpu_offset(struct tracecmd_output *handle, const char *name) { tsize_t b_offset; tsize_t offset; @@ -1883,6 +1882,14 @@ int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, tracecmd_warning("could not seek to %lld\n", offset); return -1; } + return 0; +} + +int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, + const char *name, int cpus, char * const *cpu_data_files) +{ + if (out_update_buffer_cpu_offset(handle, name)) + return -1; return tracecmd_write_cpu_data(handle, cpus, cpu_data_files); } From patchwork Mon Jun 14 07:50:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2E46C49360 for ; Mon, 14 Jun 2021 07:52:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC2B961370 for ; Mon, 14 Jun 2021 07:52:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232562AbhFNHyZ (ORCPT ); Mon, 14 Jun 2021 03:54:25 -0400 Received: from mail-ed1-f50.google.com ([209.85.208.50]:33686 "EHLO mail-ed1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232640AbhFNHyY (ORCPT ); Mon, 14 Jun 2021 03:54:24 -0400 Received: by mail-ed1-f50.google.com with SMTP id f5so40212117eds.0 for ; Mon, 14 Jun 2021 00:52:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e0ixMsnikpyfSMZamlgZvIFRCDv7ZTHnXlFb3wlMRg0=; b=Ra3clube7+mk+HXaAWv52lwuZJZrAZykhgQE/ftcIxoGsmdLIjqykzjlQVgf5qyQIr M+GXsydGDX+Pk6cOcZQIajp+aC9+vBamHqbTGHGHtJ4RN/0E0AhjgsTa5ZZX7PLEQFwM i37z6LBzK64HCLCwLOUFw//EJUMau7BSw8Coh7tWS+lHH9jzv/I6dojVHH0KxX3hU8rc 97I9nvhT04N+yDNIX1yDtvRBIYtRJFRmgKDSGY1bhhwoFbfT4OjrAihMrG898IHwej/Q I3TaRBIuNN39JMk5VR4pLGA+6OyiCZx7nasz3Cn6ThNr/l5lVgaJD5AutIh7Xo0byFRV GznA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e0ixMsnikpyfSMZamlgZvIFRCDv7ZTHnXlFb3wlMRg0=; b=huThw74ilOKuLXkZq+r2wPqGmzzzTLFbUWatBOPHbWEm3OPisFY2bi6MaFuMs1lvUZ 2ZbqW/39l/oPy0iqUOXQA6gSFehGy1EGFuVlghrSR/QoZjSZj/d6itc5b1j5XyxFPUx8 WtQ7HBVzrI2EOnVnVdWjWsxc/qnX9bJ5qf6MxBXAteASfpk3j2O3WZpgraYzka76v7Oa je23pSmkr/Fv7BBcCYAlBUHBgGtNbTW1kfM4z54tEooQRUNLmWhEeD6jAcDEyuFkRpWu RQWGhNPqyigXiOdCgIogsi7hv0iY48Xlb1Csc3TDbmV7qTF8Z8n2whRFAQnX3OMcYyzJ +X+A== X-Gm-Message-State: AOAM532HZ/j/udE06p6om0QepzohAzxIT29kbRGz6wgR2RtSU+vUByvv Pk8/rRSeIEghntW8xcHWAdnZY1T0ZJeY6g== X-Google-Smtp-Source: ABdhPJwT5Vsuco+TsOVyC3vs3Ggo6TI5uF2gnPVDDz0P6hCJ6h2MkRVHLA7rXJ+4BuepjTOmqOXUaw== X-Received: by 2002:aa7:c41a:: with SMTP id j26mr15435223edq.133.1623657067937; Mon, 14 Jun 2021 00:51:07 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:07 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 38/45] trace-cmd: Copy trace data in tracecmd_copy Date: Mon, 14 Jun 2021 10:50:22 +0300 Message-Id: <20210614075029.598048-39-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Extend the tracecmd_copy() API to support copying of trace data from input to output trace handler. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 + lib/trace-cmd/trace-input.c | 101 ++++++++++++++++++ lib/trace-cmd/trace-output.c | 3 + 3 files changed, 106 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 595ba688..79fef884 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -178,6 +178,8 @@ int tracecmd_copy_buffer_descr(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); int tracecmd_copy_options(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle); +int tracecmd_copy_trace_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); void tracecmd_clear_flag(struct tracecmd_input *handle, int flag); unsigned long tracecmd_get_flags(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 794f8ea3..2544d825 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4291,6 +4291,107 @@ int tracecmd_copy_options(struct tracecmd_input *in_handle, return 0; } +static int copy_trace_latency(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + int fd; + + if (do_write_check(out_handle, "latency ", 10)) + return -1; + + if (in_handle->lat_cfd >= 0) + fd = in_handle->lat_cfd; + else + fd = in_handle->fd; + + if (!out_copy_fd_compress(out_handle, fd, 0, NULL)) + return -1; + + out_set_file_state(out_handle, TRACECMD_FILE_CPU_LATENCY); + return 0; +} + +static int copy_trace_flyrecord_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + struct cpu_data_source *data; + int ret; + int i; + + data = calloc(in_handle->cpus, sizeof(struct cpu_data_source)); + if (!data) + return -1; + for (i = 0; i < in_handle->cpus; i++) { + data[i].size = in_handle->cpu_data[i].file_size; + if (in_handle->cpu_data[i].cfd >= 0) { + data[i].fd = in_handle->cpu_data[i].cfd; + data[i].offset = 0; + } else { + data[i].fd = in_handle->fd; + data[i].offset = in_handle->cpu_data[i].file_offset; + } + } + ret = out_write_cpu_data(out_handle, in_handle->cpus, data); + + return ret; +} + +static int copy_trace_flyrecord(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + struct tracecmd_input *instance; + const char *name; + int ret; + int i; + + /* top instance */ + ret = copy_trace_flyrecord_data(in_handle, out_handle); + if (ret) + return ret; + + for (i = 0; i < in_handle->nr_buffers; i++) { + name = tracecmd_buffer_instance_name(in_handle, i); + if (!name) + continue; + instance = tracecmd_buffer_instance_handle(in_handle, i); + if (!instance) + continue; + if (!tracecmd_get_quiet(out_handle)) + fprintf(stderr, "\nBuffer: %s\n\n", name); + + if (!out_update_buffer_cpu_offset(out_handle, name)) + copy_trace_flyrecord_data(instance, out_handle); + + tracecmd_close(instance); + } + + return 0; +} + +int tracecmd_copy_trace_data(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) +{ + char buf[10]; + + if (!check_in_state(in_handle, TRACECMD_FILE_CPU_FLYRECORD) || + !check_out_state(out_handle, TRACECMD_FILE_CPU_FLYRECORD)) + return -1; + + tracecmd_set_out_clock(out_handle, in_handle->trace_clock); + + if (do_read_check(in_handle, buf, 10)) + return -1; + if (strncmp(buf, "latency", 7) == 0) { + in_handle->file_state = TRACECMD_FILE_CPU_LATENCY; + return copy_trace_latency(in_handle, out_handle); + } else if (strncmp(buf, "flyrecord", 9) == 0) { + in_handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; + return copy_trace_flyrecord(in_handle, out_handle); + } + + return -1; +} + /** * tracecmd_record_at_buffer_start - return true if record is first on subbuffer * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 54134d03..377c5863 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -2025,6 +2025,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (state >= TRACECMD_FILE_OPTIONS && tracecmd_copy_options(ihandle, handle) < 0) goto out_free; + if (state >= TRACECMD_FILE_CPU_LATENCY && + tracecmd_copy_trace_data(ihandle, handle) < 0) + goto out_free; /* The file is all ready to have cpu data attached */ return handle; From patchwork Mon Jun 14 07:50:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87259C48BE6 for ; Mon, 14 Jun 2021 07:51:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6EADF61352 for ; Mon, 14 Jun 2021 07:51:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232528AbhFNHxX (ORCPT ); Mon, 14 Jun 2021 03:53:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232592AbhFNHxX (ORCPT ); Mon, 14 Jun 2021 03:53:23 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44429C061766 for ; Mon, 14 Jun 2021 00:51:10 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id l1so14991544ejb.6 for ; Mon, 14 Jun 2021 00:51:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C9HHWdPfPmBqnujiBtCYBuxqmrxrwCuMAiL4us7iFZc=; b=pfCGKFXN6xLZXwr47qwqeuf1jDPXbsYIgj3BHpPC84OZ8JoVAtqCo8MMKfDAMp9btG lYliBoVRxVl5SX//f2c8AYa0yq3utJno+29uSaObAKmmjABoBR5/qnCq3awMGz4n5LXF A7sHTGNXcZQPlDL7aSFpRnRv5Rav4w+pVcxjiioOZmUGT+yF8StHqCXofPseLbae5hrR HYJTnjV2NISsVI/eMemzpmemgFO4zK85UDlzJdTNy8TdEvonWGwBsM1tLMbiuI8xbLq2 bhp2VPnIDGEebXdv0QPYE8+KlGTd1+rYTPVsev4tO7t77rjQWEAXrMnTz6yCguhwPOtg 3EuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C9HHWdPfPmBqnujiBtCYBuxqmrxrwCuMAiL4us7iFZc=; b=hYk5hpAqhZLTaZNTGrVla1jKh3980um+NZ4QaFM+mF4E5rRhR0GO9T+BGxgKS1nnUK rAf2v03goVdu0QcnZelMwDthGNZzrTkPbAHM7bGAiYmDJjV8puV5gVpI5ojeL3yLSn0v lhLVSSPTkZpqYeosHCLBb21J0rp2uOEXU2ll42lQV75fpz/6O3S5BYGtM0H82hfoqa03 R71pylzABWInYssmj1TYdYXgP6J+JinLAMf3TY3rgLdi+NdqzEzFMUjOHXrFXtmaMzSz 8ZNiWy/4jlWua0cU2PPX7uh3wzVRwkF4N5vk8vVaK5BsgDUDRM7bcpoWbSc3e2/0n9KQ KMyQ== X-Gm-Message-State: AOAM532J8Qyis4MyBk/kG1bl37jENauXzCfOQ+Yuhby/StOJkrkKQaCU duwtox3Kpgrjq86dplXf+HM= X-Google-Smtp-Source: ABdhPJxrMSmdGlm+Kzv+vvEAOtrN7EPy6i2WMpMSu1cHZ8W2FvxEvauQKPgqk7yJTpawdMYLDqS1Uw== X-Received: by 2002:a17:906:a20b:: with SMTP id r11mr7767202ejy.221.1623657068829; Mon, 14 Jun 2021 00:51:08 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:08 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 39/45] trace-cmd: Add compression parameter to tracecmd_copy Date: Mon, 14 Jun 2021 10:50:23 +0300 Message-Id: <20210614075029.598048-40-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added compression parameter to the tracecmd_copy() API, which allows to select the compression of the created output file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 +- lib/trace-cmd/trace-output.c | 11 +++++++---- tracecmd/trace-restore.c | 2 +- tracecmd/trace-split.c | 2 +- 4 files changed, 10 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 79fef884..c8f0ee93 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -316,7 +316,7 @@ int tracecmd_append_options(struct tracecmd_output *handle); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, - enum tracecmd_file_states state); + enum tracecmd_file_states state, const char *compression); int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 377c5863..dae11ed3 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1999,20 +1999,23 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * @ihandle: input handle of the trace.dat file to copy * @file: the trace.dat file to create * @state: what data will be copied from the source handle + * @compression: compression of the output file, can be one of: + * NULL - inherit compression from the input file + * "any" - compress the output file with the best available algorithm + * "none" - do not compress the output file + * algorithm_name - compress the output file with specified algorithm * * Reads the header information and creates a new trace data file * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file, - enum tracecmd_file_states state) + enum tracecmd_file_states state, const char *compression) { enum tracecmd_file_states fstate; struct tracecmd_output *handle; - const char *compr_name = NULL; - tracecmd_get_file_compress_proto(ihandle, &compr_name, NULL); - handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compr_name); + handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compression); if (!handle) return NULL; diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 7d439fd2..58913fa7 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -125,7 +125,7 @@ void trace_restore (int argc, char **argv) if (tracecmd_read_headers(ihandle, TRACECMD_FILE_CMD_LINES) < 0) die("error reading file %s headers", input); - handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES); + handle = tracecmd_copy(ihandle, output, TRACECMD_FILE_CMD_LINES, NULL); tracecmd_close(ihandle); } else handle = tracecmd_create_init_file(output, NULL); diff --git a/tracecmd/trace-split.c b/tracecmd/trace-split.c index 401e1d29..b98e6fc9 100644 --- a/tracecmd/trace-split.c +++ b/tracecmd/trace-split.c @@ -345,7 +345,7 @@ static double parse_file(struct tracecmd_input *handle, dir = dirname(output); base = basename(output); - ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES); + ohandle = tracecmd_copy(handle, output_file, TRACECMD_FILE_CMD_LINES, NULL); cpus = tracecmd_cpus(handle); cpu_data = malloc(sizeof(*cpu_data) * cpus); From patchwork Mon Jun 14 07:50:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A97FC48BE8 for ; Mon, 14 Jun 2021 07:52:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37E2361357 for ; Mon, 14 Jun 2021 07:52:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232642AbhFNHyY (ORCPT ); Mon, 14 Jun 2021 03:54:24 -0400 Received: from mail-ej1-f41.google.com ([209.85.218.41]:45707 "EHLO mail-ej1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232636AbhFNHyX (ORCPT ); Mon, 14 Jun 2021 03:54:23 -0400 Received: by mail-ej1-f41.google.com with SMTP id k7so15034156ejv.12 for ; Mon, 14 Jun 2021 00:52:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dpwXI5URK9HmN13gX7A6+hGktVkSdJlOwZaZ0ky6UkA=; b=lLLXmkpTIXzNnZo5TtaeBCEucOYurjv1vQa4UajGN/OzaWzH7pfHCWJU2tz5d4AdOs 2FGrx7F+BAPmuz3hnslLYGsCLZqoPmXS0y5B/4ogzs2R0Ka8u0vS2nZXYmbp8n1Qn+kr NzG4m06hJme3JUim96RNtKi0Qe+iAvv0JfTLaKPW1DXAoQdEyKGViQTDHQeaGdWw+yDa ZyMzUX/oZqBE6lXyMwHhYMpFiIHRt43phFJuMmkNldSteoopVygHaKL0hA44zjDn5x3t YigmWx0l66WWe6yL8lc+YNpiCXs7vCOoCOjWqow4jpqzUF/sqr8ZvJLDWkjYhJRkWcod bcGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dpwXI5URK9HmN13gX7A6+hGktVkSdJlOwZaZ0ky6UkA=; b=pNZeTYeq4G1rvGFcZpSMCYVKkXvjusfeoOPgyGUaI1hWOuKnmaqizfpBsbjO/gYDSX LfhT9PyZE8QsWitlllnrrQ1IbSfb3U625dvocnAQDI7lfewi52Gzq5PtZA8JjjQebvD3 TgOoBxGVh9debFDzlUFKK8Am9vIoCruqFEx6JiO+6Ne8JYpy6+GvNLbaLKdAydky6eVa uYIKTf4L6po88DjN5myAiui5jM5cGrM8owfZibzJdyJJMrWcMDz+eZBPuZZFWWrWCplD 5hpIlAolK8T19gpGk4I4uHp9bVih/9txVUlmL7YTRO3doBgSuHYnK10hGOxzaShb7ubO 7xkQ== X-Gm-Message-State: AOAM532dU6Teiopg+MyWPzBxY2jfgzSGoGK7moGzf53/jWrzAYyRcrUZ aot3KpnUqYaY0LsJXLkXQfM= X-Google-Smtp-Source: ABdhPJwPrOIJzelTAlZNmvrm15CxroEnOAT7wLQHEVvwBapa5IZ51H0iVSKGby+aQFGZIUP8BxkoPQ== X-Received: by 2002:a17:906:3d9:: with SMTP id c25mr14392535eja.63.1623657069842; Mon, 14 Jun 2021 00:51:09 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:09 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 40/45] trace-cmd: Add new command "trace-cmd convert" Date: Mon, 14 Jun 2021 10:50:24 +0300 Message-Id: <20210614075029.598048-41-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org A new trace-cmd subcommand is added, used to convert trace files: trace-cmd convert -i -o --compression The command reads the given input trace file and creates output trace file with given compression. The --compression parameter is optional: - no --compression parameter: the output file inherits the compression from the input file. --compression none - the output file is not compressed. --compression any - use the best available compression algorithm to compress the output file --compression - use the specified compression algorithm, if available, to compress the output file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/Makefile | 1 + tracecmd/include/trace-local.h | 2 + tracecmd/trace-cmd.c | 1 + tracecmd/trace-convert.c | 94 ++++++++++++++++++++++++++++++++++ tracecmd/trace-usage.c | 12 +++++ 5 files changed, 110 insertions(+) create mode 100644 tracecmd/trace-convert.c diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 80c69bbb..35086b71 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -36,6 +36,7 @@ TRACE_CMD_OBJS += trace-usage.o TRACE_CMD_OBJS += trace-dump.o TRACE_CMD_OBJS += trace-clear.o TRACE_CMD_OBJS += trace-vm.o +TRACE_CMD_OBJS += trace-convert.o ifeq ($(VSOCK_DEFINED), 1) TRACE_CMD_OBJS += trace-agent.o diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index e9a0aea8..13dab44c 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -110,6 +110,8 @@ void trace_usage(int argc, char **argv); void trace_dump(int argc, char **argv); +void trace_convert(int argc, char **argv); + int trace_record_agent(struct tracecmd_msg_handle *msg_handle, int cpus, int *fds, int argc, char **argv, bool use_fifos, diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 71c8f6d6..e6274c35 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -130,6 +130,7 @@ struct command commands[] = { {"list", trace_list}, {"help", trace_usage}, {"dump", trace_dump}, + {"convert", trace_convert}, {"-h", trace_usage}, }; diff --git a/tracecmd/trace-convert.c b/tracecmd/trace-convert.c new file mode 100644 index 00000000..555e0c0a --- /dev/null +++ b/tracecmd/trace-convert.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov + */ +#include +#include +#include +#include +#include + +#include "trace-local.h" +#include "trace-cmd.h" +#include "trace-cmd-private.h" + +static void convert_file(const char *in, const char *out, char *compr) +{ + struct tracecmd_input *ihandle; + struct tracecmd_output *ohandle; + + ihandle = tracecmd_open(in, 0); + if (!ihandle) + die("error reading %s", in); + + ohandle = tracecmd_copy(ihandle, out, TRACECMD_FILE_CPU_FLYRECORD, compr); + if (!ohandle) + die("error writing %s", out); + tracecmd_output_close(ohandle); + tracecmd_close(ihandle); +} + +enum { + OPT_comporession = 255, +}; + +void trace_convert(int argc, char **argv) +{ + char *input_file = NULL; + char *output_file = NULL; + char *compression = NULL; + int c; + + if (argc < 2) + usage(argv); + + if (strcmp(argv[1], "convert") != 0) + usage(argv); + for (;;) { + int option_index = 0; + static struct option long_options[] = { + {"compression", required_argument, NULL, OPT_comporession}, + {"help", no_argument, NULL, '?'}, + {NULL, 0, NULL, 0} + }; + + c = getopt_long (argc-1, argv+1, "+hi:o:", long_options, &option_index); + if (c == -1) + break; + switch (c) { + case 'i': + if (input_file) + die("Only one input file is supported, %s already set", input_file); + input_file = optarg; + break; + case 'o': + if (output_file) + die("Only one output file is supported, %s already set", output_file); + output_file = optarg; + break; + case OPT_comporession: + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + compression = optarg; + break; + case 'h': + case '?': + default: + usage(argv); + } + } + + if ((argc - optind) >= 2) { + if (output_file) + usage(argv); + output_file = argv[optind + 1]; + } + + if (!input_file) + input_file = DEFAULT_INPUT_FILE; + if (!output_file) + usage(argv); + + convert_file(input_file, output_file, compression); +} diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index c70f9919..d0d12e87 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -392,6 +392,18 @@ static struct usage_help usage_help[] = { " --clock trace clock, saved in the file\n" " -h, --help show usage information\n" }, + { + "convert", + "convert trace file to different version", + " %s convert [options]\n" + " -i input file, default is trace.dat\n" + " -o output file, mandatory parameter.\n" + " The output file can be specified also as last argument of the command\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { NULL, NULL, NULL } From patchwork Mon Jun 14 07:50:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85280C48BE8 for ; Mon, 14 Jun 2021 07:51:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6FB5961246 for ; Mon, 14 Jun 2021 07:51:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232583AbhFNHxe (ORCPT ); Mon, 14 Jun 2021 03:53:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232607AbhFNHxa (ORCPT ); Mon, 14 Jun 2021 03:53:30 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19748C061767 for ; Mon, 14 Jun 2021 00:51:12 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id ho18so15043735ejc.8 for ; Mon, 14 Jun 2021 00:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HiQsMk0wkHXTkHUtFtlVTetDCQMIW2DvIJ+jQNwbSjM=; b=STeJt63MgJdahmxKGQQ/ZH55vt8Shv0uli3rB5/3HhHhcPE41iisGeDq0bR4BULuJl hwErpJq9+aBE1rmDROuGnKGyacZM+hiwYIq0B5qSfV7+QsTWpfBZ80TUT+tmLJJBJiqU hd9P0JMcJ9+wpjCOXp7EgsGpb5iFT9mewZ8yaZ9yAF3J1lypHxB66tHHVhwChVdx5TcB CYVanMN69Zs3XSNU5jzyDkwpQFVNeR73wSzt90XuOPHhNigO6KhVHicvVVcdcCXvJ3/K /cAYO8MKuraDzzFa9arvE4lZCTrBYh/9G8qSEkfNRaOlJaVRyWjVr9VwCxR8y3eVmJep xsWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HiQsMk0wkHXTkHUtFtlVTetDCQMIW2DvIJ+jQNwbSjM=; b=Dg9Nhswj9d0MU4CtvOoVcXkDwcXz+aARAtcDEYUK1aO9DwLIRDqpHg53pb6nx66ZQD z3AzSblSwR+76q+BufYiPQcmUGbfue5b1jpOrx/e/6XobyPb7Uw+EIHRaaBx8nH5dEhd r5VYKo7RbCki41Z8wEoUMSGG5Epd4hZ4bnnYkVSf++YhsayPBMA21i9V67gkw38NibCC GPdue86DGqygVD6GGBUFACy5+3HnRrMZUQ4zOSDKYpOe47qx9boKU1pfmOtQl6uye5bg vIMpM6DaFvMcdcZqIR296qGC+UONiPquphKF6O7FCXAiDqNpaqT0dlvUCkj4tN5STcrF x3xg== X-Gm-Message-State: AOAM5318Q3WHuDamgGG3nczQRoDrtzkWL6Go3EEMbBEpSg8EE6BY+p+7 vc+yh8izQ7VXnMaY/x27EJc= X-Google-Smtp-Source: ABdhPJx+kgegdOORL684FJjGelyUxIqmtwedp5RGYHC4J3mvDXhZWafTEJOP9r84+5bb0NkLbz4hvA== X-Received: by 2002:a17:906:919:: with SMTP id i25mr13608046ejd.171.1623657070634; Mon, 14 Jun 2021 00:51:10 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:10 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 41/45] trace-cmd record: Update man page Date: Mon, 14 Jun 2021 10:50:25 +0300 Message-Id: <20210614075029.598048-42-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added documentation of the 'trace-cmd record --compression' parameter in the man page. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd/trace-cmd-record.1.txt | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/Documentation/trace-cmd/trace-cmd-record.1.txt b/Documentation/trace-cmd/trace-cmd-record.1.txt index 5d063cc2..c5a7fb75 100644 --- a/Documentation/trace-cmd/trace-cmd-record.1.txt +++ b/Documentation/trace-cmd/trace-cmd-record.1.txt @@ -367,6 +367,16 @@ OPTIONS Traces are extracted by busy waiting, which will hog the CPUs, so only use when really needed. +*--compression*:: + Compression of the trace output file, one of these strings can be passed: + + 'any' - auto select the best available compression algorithm + + 'none' - do not compress the trace file + + 'name' - the name of the desired compression algorithms. Available algorithms can be listed with + trace-cmd list -c + EXAMPLES -------- From patchwork Mon Jun 14 07:50:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D379EC48BE6 for ; Mon, 14 Jun 2021 07:52:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BADB561357 for ; Mon, 14 Jun 2021 07:52:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232520AbhFNHy2 (ORCPT ); Mon, 14 Jun 2021 03:54:28 -0400 Received: from mail-ed1-f45.google.com ([209.85.208.45]:42890 "EHLO mail-ed1-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232640AbhFNHy1 (ORCPT ); Mon, 14 Jun 2021 03:54:27 -0400 Received: by mail-ed1-f45.google.com with SMTP id i13so45213607edb.9 for ; Mon, 14 Jun 2021 00:52:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jHACQ9rJPj63PN8NI3v2/826PmbhUMZq2PCaQYXnCRE=; b=nFbH8tG/n69h1cVkLpq/gjIirJdJjDytxW1mq0yvNNY7HHvlksFIctR2dUuTZs4dEH DdhcvTKitJtD/27WpelnU9Gbjsh+35lSjiQ6UdCsfCEE+u27jKRO9V9DGzMmgTf+ifKG NBq7Vi9NVILa2ZgYNxwZyC2eATivlRALBfNu4nePNC9ckveB3Z4P0uaww1Y/LMApWw3X +P+RSQaI1ep3ese1gh3nAcUXqrMAqeYzOCq+fK4XMHdA0fdCawN/ShbfHJmF5kYKkSrB 9OWGky77e4bu6mL+IKw9nyfWQS2KKNH/j9KWkjfx377eWgLg5PDMXvYGbdS076b1M4Kt TAGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jHACQ9rJPj63PN8NI3v2/826PmbhUMZq2PCaQYXnCRE=; b=rJhvnn3om9gfe49VPEKURxbBvXxdKy80gbeFV1mfMpfLEu/YnuBG6UPJjdi2wuQKQO XvSfilpY+Gi4ux2lJWOIGzafv8L0BWbyumhdPDpOw96S4hiA4WXMFKW2v2nySE+Fi4UO yNAd+Mdl1SUEriz/fNd37SN83NjdUcuVVLSC/PApkUKXW3NK5P8nO+QUjd4OjuDqaG8o cgDJW5CLsCse01X0BQ7czGGn+4urtMSKy+dl8lL7/PAzHpIfbWIJ4OgMPZfLpufTLWSb fH7TbN2Tjw0/wwwWQtmjTCpsVmTSAqRj+3yjShhpOp+05rDu77FIcbR+58G5iGZvkdO/ DvVg== X-Gm-Message-State: AOAM530vemCVnJvH5jqiI4qAotB/FBV5jK/DBFFljD5U/1hJXyccpPkH zU6ycNGl/Q9jlxUN/c+Mcw0= X-Google-Smtp-Source: ABdhPJz5DVScUchxcHmyvvsuqUjZs4IShrnvFLQ55tZ4mHlItHNvc4rLe18f0phU0qslH1hKIdkJoA== X-Received: by 2002:aa7:d7cf:: with SMTP id e15mr15803588eds.114.1623657071646; Mon, 14 Jun 2021 00:51:11 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:11 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 42/45] trace-cmd: Add convert man page Date: Mon, 14 Jun 2021 10:50:26 +0300 Message-Id: <20210614075029.598048-43-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added new man page, documenting the "trace-cmd convert" command. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../trace-cmd/trace-cmd-convert.1.txt | 60 +++++++++++++++++++ Documentation/trace-cmd/trace-cmd.1.txt | 4 +- 2 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 Documentation/trace-cmd/trace-cmd-convert.1.txt diff --git a/Documentation/trace-cmd/trace-cmd-convert.1.txt b/Documentation/trace-cmd/trace-cmd-convert.1.txt new file mode 100644 index 00000000..e8d65843 --- /dev/null +++ b/Documentation/trace-cmd/trace-cmd-convert.1.txt @@ -0,0 +1,60 @@ +TRACE-CMD-CONVERT(1) +=================== + +NAME +---- +trace-cmd-convert - convert trace files + +SYNOPSIS +-------- +*trace-cmd convert* ['OPTIONS'] ['output-file'] + +DESCRIPTION +----------- +The trace-cmd(1) convert command converts trace file. It reads the input file and copies the data +into an output file. The output file may be in different format, depending on the command line +arguments. + +OPTIONS +------- +*-i* 'input-file':: + By default, trace-cmd convert will read the file 'trace.dat'. But the *-i* + option open up the given 'input-file' instead. +*-o* 'out-file':: + The name of the output file, this parameter is mandatory. Note, the output file may also be + specified as the last item on the command line. +*--compression*:: + Compression of the trace output file, one of these strings can be passed: + + 'any' - auto select the best available compression algorithm + + 'none' - do not compress the trace file + + 'name' - the name of the desired compression algorithms. Available algorithms can be listed with + trace-cmd list -c + +*--help*:: + Print usage information. + +EXAMPLES +-------- + +# trace-cmd convert --compression any trace_compress.dat + +SEE ALSO +-------- +trace-cmd(1), trace-cmd.dat(1) + +AUTHOR +------ +*Steven Rostedt* , author of *trace-cmd*. +*Tzvetomir Stoyanov* , author of this man page. + +RESOURCES +--------- +https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/ + +COPYING +------- +Copyright \(C) 2021 VMware. Free use of this software is granted under +the terms of the GNU Public License (GPL). diff --git a/Documentation/trace-cmd/trace-cmd.1.txt b/Documentation/trace-cmd/trace-cmd.1.txt index b330b4fe..27c6e973 100644 --- a/Documentation/trace-cmd/trace-cmd.1.txt +++ b/Documentation/trace-cmd/trace-cmd.1.txt @@ -58,6 +58,8 @@ COMMANDS check-events - parse format strings for all trace events and return whether all formats are parseable + convert - convert trace files + OPTIONS ------- @@ -70,7 +72,7 @@ SEE ALSO -------- trace-cmd-record(1), trace-cmd-report(1), trace-cmd-hist(1), trace-cmd-start(1), trace-cmd-stop(1), trace-cmd-extract(1), trace-cmd-reset(1), -trace-cmd-restore(1), trace-cmd-stack(1), +trace-cmd-restore(1), trace-cmd-stack(1), trace-cmd-convert(1), trace-cmd-split(1), trace-cmd-list(1), trace-cmd-listen(1), trace-cmd.dat(5), trace-cmd-check-events(1) trace-cmd-stat(1) From patchwork Mon Jun 14 07:50:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6CCEC48BE6 for ; Mon, 14 Jun 2021 07:51:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CB6A61289 for ; Mon, 14 Jun 2021 07:51:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232586AbhFNHxR (ORCPT ); Mon, 14 Jun 2021 03:53:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232583AbhFNHxQ (ORCPT ); Mon, 14 Jun 2021 03:53:16 -0400 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06FE2C0617AF for ; Mon, 14 Jun 2021 00:51:14 -0700 (PDT) Received: by mail-ed1-x532.google.com with SMTP id u24so45192311edy.11 for ; Mon, 14 Jun 2021 00:51:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pPf6AJRWjy/8bENp269gum0/POkaKV7VuSt+MbRzz0M=; b=Sc/WOEsGbV6RbZkOu0NUwsm62X6uezOjqoq/8kNRJ9PvAX3Q+JooFfcy3Bjmlq4gnP kLtZuoNeihouTur29ZNYTQIPbWzHX9Hysm3PJfbV0XofY/6KgnxJRtQEpcTuVP9Iic3t GbJ283KLY1IqgJDiKLfLkhnVj+ex75gupP8V+o4B01Zh60BUvOps1NJtwpDdYw72HsVP OW1tIG0zM/8quDPGMHiQy9nAsdu1Kf0H5Xa2+4sVlz9XLZLUMAsIrxkcfaRtz/eErLVT JjpsfZqzTwYYjX0fBaGJZaI9yo/1QJp/lcK4Z27n1dXGsy8Ve5Y51okVxYpgXlas4BIe x+uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pPf6AJRWjy/8bENp269gum0/POkaKV7VuSt+MbRzz0M=; b=f9obTdxo1vJRC/jADg3HyDx0uIQUOV3IwgrlVpqw4wM7D4oUyVHdXksES3u+50JFa9 sGjmt8pV4NIjjeI2nYBO18CZ9FEzzjoi0jFwmd7yOBt9e2w0zKm+J94vxkmndMCvUc9x nsfSchbTgf/ZuL+gQ2S1tKfdIxv31KS5IBhwNCASc5NemSsdYK0FfWRMsNVygHno64fR WhDBOzC7i4hIPNIRuvy7Bd7kdjdbpnMLgD3jS4iyXhVQERkEX0PnRU0SmrY69V1C4oaT nGtcDP4yuBfjCJd+nVl6aZuTxARIf1BjoADXCSSdXFMuMS9V7Mm+qJqhaJbHieTE1N0t Bj4Q== X-Gm-Message-State: AOAM532RlFmPjOvW7cE1Ihz3eqXpAo4OgMza3+aK3REjpxFODWaH468Q hypwL/f1WlXftGDKpFskDvg= X-Google-Smtp-Source: ABdhPJxBBwU1BhsKGTJBL57fQFVfsfs5jjUqVyYw/70XS8KTbPCMJCA5WkTIOp/gvwT5IT6Ayb0Ihw== X-Received: by 2002:a05:6402:26c7:: with SMTP id x7mr15585485edd.383.1623657072615; Mon, 14 Jun 2021 00:51:12 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:12 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 43/45] trace-cmd: Update bash completion Date: Mon, 14 Jun 2021 10:50:27 +0300 Message-Id: <20210614075029.598048-44-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added support for trace-cmd convert and compression parameters in bash completion. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-cmd.bash | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/tracecmd/trace-cmd.bash b/tracecmd/trace-cmd.bash index b01c7a07..6639c143 100644 --- a/tracecmd/trace-cmd.bash +++ b/tracecmd/trace-cmd.bash @@ -64,6 +64,13 @@ plugin_options() COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") ) } +compression_param() +{ + local opts=$(trace-cmd list -c | grep -v 'Supported' | cut -d "," -f1) + opts+=" any none " + COMPREPLY=( $(compgen -W "${opts}") ) +} + __trace_cmd_list_complete() { local prev=$1 @@ -181,6 +188,9 @@ __trace_cmd_record_complete() cmd_options record "$cur" fi ;; + --compression) + compression_param + ;; *) # stream start and profile do not show all options cmd_options record "$cur" @@ -222,6 +232,29 @@ __trace_cmd_dump_complete() esac } +__trace_cmd_convert_complete() +{ + local prev=$1 + local cur=$2 + shift 2 + local words=("$@") + + case "$prev" in + -i) + __show_files + ;; + -o) + __show_files + ;; + --compression) + compression_param + ;; + *) + cmd_options convert "$cur" + ;; + esac +} + __show_command_options() { local command="$1" @@ -298,6 +331,10 @@ _trace_cmd_complete() __trace_cmd_dump_complete "${prev}" "${cur}" ${words[@]} return 0 ;; + convert) + __trace_cmd_convert_complete "${prev}" "${cur}" ${words[@]} + return 0 + ;; *) __show_command_options "$w" "${prev}" "${cur}" ;; From patchwork Mon Jun 14 07:50:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 446D1C49360 for ; Mon, 14 Jun 2021 07:51:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D8AE610A1 for ; Mon, 14 Jun 2021 07:51:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232602AbhFNHxe (ORCPT ); Mon, 14 Jun 2021 03:53:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232608AbhFNHxa (ORCPT ); Mon, 14 Jun 2021 03:53:30 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9A3BC0613A2 for ; Mon, 14 Jun 2021 00:51:14 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id s6so45190070edu.10 for ; Mon, 14 Jun 2021 00:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+arG/TI83+6RRm6R28kM8v5MCinHpDND+5u8cLV9aC0=; b=de+Yq16lU+D5lLpd8UnrQURXa+1OxCLD6FZRP/3MsKEc8NdtbSwRkyPWGLBJdf0bYT Az2TqaRcV8UxNriQbAqYo5PyFonJETUGPBZbETuLKFfuyZpBFr6nILTKmFcoAwXfUqa9 J9xmWKRhRyNFTjmYhx7bVPww8CbvUj4kWQGQEj3nERMvyTY/SgqLpJGdxbePN1n5V7Y4 mJcs+CCOB+2ipcceRLFjYWM3co+Z6wNN3oUN4DxiEhdSiSg9lzD2tylHiHZz5OE8k5b4 trzcUk8aJYtvIXwmqoNr//kDpvWe+OG38Nc0mLn8/RbZRhXwFRhR+hQX8uv/tofwqgjM a+hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+arG/TI83+6RRm6R28kM8v5MCinHpDND+5u8cLV9aC0=; b=NPIOrvSqYdHAcaDr57JgXyimzipFBlfclH7fDPTgOodgZgrlRsxQNVWoTqMnHyujkZ YPzpZLTal6wqYBEMjy4avhndqZKsSTGVz/zD71EDgkCchxVjkUubefgerbFVR1npsPiQ sF/HhrqkMgwsIDjKFydTQTr8tCKWDGi22un2OJSSg9tzX4DF+N3Y4x/FWiqjA1klcPpq /fQmMPjBWBSek1iVSaV70grc+w7LEgxddODE0zrAoKS2dM2YfdLPT/nyWFx3TJP6qmpg fyvDOjvcLYvsaF0XY3emYGqhx4oDEldoq5wanyaBmGpGvGZmD1uaXKzLUAY7uNoPZvU6 unMA== X-Gm-Message-State: AOAM530YmesOTnexRtzqqt4dYIy08VDQm80gIxMcQmItuAe7yZiixSJk QstLbMQHrB0ztpoopGUtUKY= X-Google-Smtp-Source: ABdhPJwzMdpgagElqq9NpVLQC4ZEUB0079fiLvNC+Xf3TZM+tmoYq6zNiVykOWIpkLWVGHQFp+jVjQ== X-Received: by 2002:a05:6402:b5a:: with SMTP id bx26mr15163160edb.81.1623657073489; Mon, 14 Jun 2021 00:51:13 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:13 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 44/45] trace-cmd list: Update the man page Date: Mon, 14 Jun 2021 10:50:28 +0300 Message-Id: <20210614075029.598048-45-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added documentation of the 'trace-cmd list -c' parameter in the man page. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd/trace-cmd-list.1.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Documentation/trace-cmd/trace-cmd-list.1.txt b/Documentation/trace-cmd/trace-cmd-list.1.txt index a5c6b16c..b77e3460 100644 --- a/Documentation/trace-cmd/trace-cmd-list.1.txt +++ b/Documentation/trace-cmd/trace-cmd-list.1.txt @@ -71,6 +71,9 @@ OPTIONS List defined clocks that can be used with trace-cmd record -C. The one in brackets ([]) is the active clock. +*-c*:: + List the available trace file compression algorithms. + SEE ALSO -------- trace-cmd(1), trace-cmd-record(1), trace-cmd-report(1), trace-cmd-start(1), From patchwork Mon Jun 14 07:50:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12318273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68A5BC48BE6 for ; Mon, 14 Jun 2021 07:52:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 532F4613D3 for ; Mon, 14 Jun 2021 07:52:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232530AbhFNHyT (ORCPT ); Mon, 14 Jun 2021 03:54:19 -0400 Received: from mail-ej1-f42.google.com ([209.85.218.42]:37520 "EHLO mail-ej1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232632AbhFNHyR (ORCPT ); Mon, 14 Jun 2021 03:54:17 -0400 Received: by mail-ej1-f42.google.com with SMTP id ce15so15025882ejb.4 for ; Mon, 14 Jun 2021 00:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VbCjSkYiTayTZj1R7pN7UweSLyf1bgnBl/z0BvUhG2o=; b=fV9xtVDsZ4wObptp2M7zGdLeGh4gVaBVeAprPEg1eOPfHpeI6kXhcIAAxdVxy2vcl6 dU+a6BEzesIvqJarjzPZLmyAQDXYEaoIS1bdMPlSQOYsM8Y9zhccc7BmlPLYKqukfKUW 98WXX6M5hgs6VMznelbKOUUr3xmbMSG/9b6e0wD/NbzTwOEtV4ukvxO+tX899MNS2KJP nhrDnGAByty8QpK5mJXL40Zlqrgxv9kgFdPSj1gsDIJTp3jJlK4oNEbtey1+fECBX3Dw V4GB1fGdkHC9xTOxoFcrp8Or9t743BFUj6nNyvmRds++WK5nSf1IPuG+bgT0sqRraDiS L6Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VbCjSkYiTayTZj1R7pN7UweSLyf1bgnBl/z0BvUhG2o=; b=KmO3SrT+ZajN0GHsqJwI0qXmx8F+djB+nbtrDsWANQKTCRDsJ5UV6M27dGzjmpDH1J JVfVpyGSmI1cXprKEnWRuJxkTyRW68a3ndN9M2DAD/4uhATq4evrKWzjx13L0wnLYiod oCsQYBsrJ+A0Zg2rToLSIjYmkYndEvLtb4SGIE6E2pgaTl3o70Yk/jopOvuL1Lims7TV 9T1z7zq3ucztZsU8Z4f9MCrwsGnvIsIonmd7+3OdqzIJpi1vnDAV6mwpSpa872O4M2Hn NcswJrmYPmdP1fAC/Yt4PNu0pCq6JFNmkZUqoFewBBIBi9arRIe+VPlBj+UJNXwHgnA3 WNjQ== X-Gm-Message-State: AOAM530lRuGNq97yw96dEuZtwopQUffZMSaeiefilDOZQW63brhA51Om oF9S9xMbmW0Gbxv3cTR7xGI= X-Google-Smtp-Source: ABdhPJyBBOMO3/EDeFe+T27UJD/XtF85O8B6VsoK4yPzUDX9tWnbfbqOXYWXRJrB1qx/ZNwAQuI8oQ== X-Received: by 2002:a17:906:3888:: with SMTP id q8mr13970665ejd.15.1623657074508; Mon, 14 Jun 2021 00:51:14 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f6sm6536444eja.108.2021.06.14.00.51.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Jun 2021 00:51:14 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v6 45/45] trace-cmd: Update trace.dat man page Date: Mon, 14 Jun 2021 10:50:29 +0300 Message-Id: <20210614075029.598048-46-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210614075029.598048-1-tz.stoyanov@gmail.com> References: <20210614075029.598048-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Updated the trace.dat man page with the changes related to file version 7 and compression. Signed-off-by: Tzvetomir Stoyanov (VMware) --- Documentation/trace-cmd/trace-cmd.dat.5.txt | 56 ++++++++++++++++++--- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/Documentation/trace-cmd/trace-cmd.dat.5.txt b/Documentation/trace-cmd/trace-cmd.dat.5.txt index 8d285353..e80d460e 100644 --- a/Documentation/trace-cmd/trace-cmd.dat.5.txt +++ b/Documentation/trace-cmd/trace-cmd.dat.5.txt @@ -52,6 +52,23 @@ INITIAL FORMAT The next 4 bytes are a 32-bit word that defines what the traced host machine page size was. + If the file version is 7 or greater, the compression header is + written next: + "name version\0" + where "name" and "version" are strings, name and version of the + compression algorithm used to compress the trace file. + +COMPRESSION FORMAT OF THE HEADER SECTIONS +----------------------------------------- + If the file version is 7 or greater, some header sections are compressed + with the compression algorithm, specified in the compression header. + The format of these compressed sections is: + <4 bytes> unsigned int, size of compressed data in the next block. + <4 bytes> unsigned int, size of uncompressed data. + binary compressed data, with the specified size. + These sections must be uncompressed on reading. The described format of + the sections refers to the uncomperssed data. + HEADER INFO FORMAT ------------------ @@ -93,7 +110,8 @@ FTRACE EVENT FORMATS Directly after the header information comes the information about the Ftrace specific events. These are the events used by the Ftrace plugins - and are not enabled by the event tracing. + and are not enabled by the event tracing. If the file version is 7 or + greater, this section is compressed. The next 4 bytes contain a 32-bit word of the number of Ftrace event format files that are stored in the file. @@ -110,7 +128,8 @@ EVENT FORMATS ------------- Directly after the Ftrace formats comes the information about - the event layout. + the event layout. If the file version is 7 or greater, this section + is compressed. The next 4 bytes are a 32-bit word containing the number of event systems that are stored in the file. These are the @@ -137,7 +156,8 @@ KALLSYMS INFORMATION -------------------- Directly after the event formats comes the information of the mapping - of function addresses to the function names. + of function addresses to the function names. If the file version is 7 + or greater, this section is compressed. The next 4 bytes are a 32-bit word containing the size of the data holding the function mappings. @@ -154,6 +174,7 @@ TRACE_PRINTK INFORMATION store the format string outside the ring buffer. This information can be found in: debugfs/tracing/printk_formats + If the file version is 7 or greater, this section is compressed. The next 4 bytes are a 32-bit word containing the size of the data holding the printk formats. @@ -166,7 +187,8 @@ PROCESS INFORMATION ------------------- Directly after the trace_printk formats comes the information mapping - a PID to a process name. + a PID to a process name. If the file version is 7 or greater, this + section is compressed. The next 8 bytes contain a 64-bit word that holds the size of the data mapping the PID to a process name. @@ -193,10 +215,11 @@ REST OF TRACE-CMD HEADER "flyrecord\0" - If it is "options \0" then: + If it is "options \0" then follows a section with trace options. + If the file version is 7 or greater, this section is compressed. The next 2 bytes are a 16-bit word defining the current option. - If the the value is zero then there are no more options. + If the value is zero then there are no more options. Otherwise, the next 4 bytes contain a 32-bit word containing the option size. If the reader does not know how to handle the option @@ -206,6 +229,25 @@ REST OF TRACE-CMD HEADER The next option will be directly after the previous option, and the options ends with a zero in the option type field. +COMPRESSION FORMAT OF THE TRACE DATA +------------------------------------ + + If the file version is 7 or greater, the tarce data is compressed + with the compression algorithm, specified in the compression header. + The data is compressed in chunks. The size of one compression chunk + is defined when the file is written. The format of compressed trace + data is: + <4 bytes> unsigned int, count of chunks. + Follows the compressed chunks of givent count. For each chunk: + <4 bytes> unsigned int, size of compressed data in this chunk. + <4 bytes> unsigned int, size of uncompressed data. + binary compressed data, with the specified size. + These chunks must be uncompressed on reading. The described format of + trace data refers to the uncomperssed data. + +TRACE DATA +---------- + The next 10 bytes after the options are one of the following: "latency \0" @@ -217,6 +259,7 @@ REST OF TRACE-CMD HEADER If the value is "latency \0", then the rest of the file is simply ASCII text that was taken from the target's: debugfs/tracing/trace + If the file version is 7 or greater, the latency data is compressed. If the value is "flyrecord\0", the following is present: @@ -232,6 +275,7 @@ REST OF TRACE-CMD HEADER CPU DATA -------- + If the file version is 7 or greater, the CPU data is compressed. The CPU data is located in the part of the file that is specified in the end of the header. Padding is placed between the header and the CPU data, placing the CPU data at a page aligned (target page) position