From patchwork Thu May 20 03:19:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFAC8C433B4 for ; Thu, 20 May 2021 03:20:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9C080611AE for ; Thu, 20 May 2021 03:20:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229598AbhETDVZ (ORCPT ); Wed, 19 May 2021 23:21:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230062AbhETDVZ (ORCPT ); Wed, 19 May 2021 23:21:25 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74ED6C06175F for ; Wed, 19 May 2021 20:20:04 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id gb17so4920024ejc.8 for ; Wed, 19 May 2021 20:20:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0C6D0NjdPESwV62T7kxwWiilnywFWC5ZrocmTLNRsC4=; b=cxs4R6o3NU/8XjqhYJlMCtjkJhav2rY69OD+4+Vi5A2aVdFwsjDcBbsaG0ynz157Ig a+i9+Ui11nFh31GqueZawP9Z2IV11K+NxVLnKFK8E1/NQcAvuRkscCXaIPvd+7jKkHaw MlHYNRfTGUEmO8AaTfn/tpL8oaP1WvBZVW7Lpwr28jGZTempABROHtVCOBHfiPnxXqAN qAP4S3Bf7qn96Y3XmqPKx1G955QTBIowIOLAa2xP/O7CA12plyXI5NtRj4CYEvq0FqQW 7tv8y6OtgNb0z0seNU1mtgVk0efJ4XNbZAgE5q5khHgw3wIXnfqYs504rMFI3/qgXvFJ Yz7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0C6D0NjdPESwV62T7kxwWiilnywFWC5ZrocmTLNRsC4=; b=IbWOEX2a0yVaOWHRpHPKnw4qC5sIlJqq5ZBSbFo1My5MNJWS572SbenDUhuEVe2s8+ zlHBCW5lk1nOlqofiI4B3EFvjmdflGRpizyNjWY6dIsmKeg0FShVcwfoZbRcDFSA1nJy SGp6Wx/rvErCqBwaXBydBTDW/2xbEnl1WBcVZGWkUdxxRqQLbIafJDkbN0ykTDKAccA4 t/c0bVP+fgqm4BN398jHOrJAyQy4/MFqF5Kt6waou3j2ZyKWufgX5sW5dVONvmWE2Si4 aCrbixGlrEoneYvTuKWx9v2Rb9xH30vVIbpY0/Pn5gS1q1iznqN6U/q9xn4W5JCQ6Q9N RszA== X-Gm-Message-State: AOAM530jLRF/T96Y+sR3HMM34o2k56R+CVyG7OvVdqG5MzH+yRBxs/AJ 7mKFnqQVO0vprzSUREy0ToU= X-Google-Smtp-Source: ABdhPJwDADZ4O+2UZLpP9Rjm8lvktLYom+d5s2Rtf7nBf4C4EUorokHnxtRSKpuraMq17FSgI22nXw== X-Received: by 2002:a17:906:914d:: with SMTP id y13mr2404741ejw.489.1621480803112; Wed, 19 May 2021 20:20:03 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:02 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 01/29] trace-cmd library: Remove unused private APIs for creating trace files Date: Thu, 20 May 2021 06:19:31 +0300 Message-Id: <20210520031959.346165-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Remove these unused private trace-cmd library APIs: tracecmd_create_file() tracecmd_create_file_glob() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 6 ---- lib/trace-cmd/trace-output.c | 29 ------------------- 2 files changed, 35 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 42e739fa..8ec33ea2 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -263,12 +263,6 @@ struct tracecmd_option; struct tracecmd_msg_handle; struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files); -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *event_globs); struct tracecmd_output * tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a4a1eecc..9231c319 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1614,35 +1614,6 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *list) -{ - struct tracecmd_output *handle; - - handle = create_file(output_file, NULL, NULL, NULL, list); - if (!handle) - return NULL; - - if (tracecmd_write_cmdlines(handle)) - return NULL; - - if (tracecmd_append_cpu_data(handle, cpus, cpu_data_files) < 0) { - tracecmd_output_close(handle); - return NULL; - } - - return handle; -} - -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files) -{ - return tracecmd_create_file_glob(output_file, cpus, - cpu_data_files, &all_event_list); -} - struct tracecmd_output *tracecmd_create_init_fd(int fd) { return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); From patchwork Thu May 20 03:19:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7E2BC433ED for ; Thu, 20 May 2021 03:20:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B94BE61186 for ; Thu, 20 May 2021 03:20:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230062AbhETDV1 (ORCPT ); Wed, 19 May 2021 23:21:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229978AbhETDV1 (ORCPT ); Wed, 19 May 2021 23:21:27 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D080EC061574 for ; Wed, 19 May 2021 20:20:05 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id df21so17683371edb.3 for ; Wed, 19 May 2021 20:20:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dR9cwZyRjBzKaGdEXi13MgximQSnOMDkCL1jWfZl6fY=; b=AO/VJJDHradvS4IpLQcS2qED4FpfbC//OAmUzocsuX3yS50u7CAMD+SouvR/skL3aP hvb8k6t+GmthZsN1LqlNg/bsLnvqepnWZGHo9IY6NxIJSCOjXZcUYrlQiT84V5avt5HW 8IQRWU1hY1TUph5CvE1Qx2tir38+Wgi7WFlBnWbwgvDxIOmE7RR2nimxvL/lcyNmmTIQ bxsIY606AsfmU835Rs3hviaw9eKYzw48x8eF6a60YEBJvLamyzZPXp3/UwI5A3tyjQY4 O6VShD3rnjm9U3aJ6O3/rxVEW4vNI6ZKhMrdxV705iHeN0qR9J08vNt5UTRHOc7MvWEX aG+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dR9cwZyRjBzKaGdEXi13MgximQSnOMDkCL1jWfZl6fY=; b=TYvarKcUunmqsxeF0E1EOQLYU8oA/xxnuUyINg8pPmRnzWrC/KT0mfDZWuD1c5WCt3 xeEg/B0DSq9080t6kDVE60GDzsjJiTn59swsNdxcYcQmc3PHu0PRFiqjTmrbTcGrPMhB TBukKa1z5s/ygR6npdREAeyswQ9/Uf6PKuJt+Ai80I1GqSY97VrR5BG47uXokn/0mK4W b8tHD7JMYTyR0zHhP9abbtObbwg8x5JDf0o8dv2CUeAm5EbtRgTz9XnHN3ieux4lRTnj 7s2xT6zH9pInUcnA6wgAGPXmcr5epr4/n8U2raPx9A0BDR1+1YY+bLKR1dO7et5igsyf +dNQ== X-Gm-Message-State: AOAM533F+J9w3O897WyItV6Iv1eOu6WcdapEZokFD+sutN9sU41AICIP SmvSiH7v2UPZ8aspKACKAFyskEd04qJAtA== X-Google-Smtp-Source: ABdhPJzCcOkQsPjJeApdn5GgQkEpyFIxXGBk6KLbbCpLvjAPvFEVERMUR88YMZCJMcILP6XdTekliA== X-Received: by 2002:aa7:de1a:: with SMTP id h26mr2574617edv.240.1621480804411; Wed, 19 May 2021 20:20:04 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:03 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 02/29] trace-cmd library: Remove unused API tracecmd_update_option Date: Thu, 20 May 2021 06:19:32 +0300 Message-Id: <20210520031959.346165-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Removed not used trace-cmd library API for updating the options section of already written trace file: tracecmd_update_option() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 -- lib/trace-cmd/trace-output.c | 40 ------------------- 2 files changed, 43 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 8ec33ea2..ecd172ca 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -290,9 +290,6 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9231c319..c0aa58cf 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1229,46 +1229,6 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data) -{ - tsize_t offset; - stsize_t ret; - - if (size > option->size) { - tracecmd_warning("Can't update option with more data than allocated"); - return -1; - } - - if (handle->file_state < TRACECMD_FILE_OPTIONS) { - /* Hasn't been written yet. Just update current pointer */ - option->size = size; - memcpy(option->data, data, size); - return 0; - } - - /* Save current offset */ - offset = lseek64(handle->fd, 0, SEEK_CUR); - - ret = lseek64(handle->fd, option->offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); - return -1; - } - - if (do_write_check(handle, data, size)) - return -1; - - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offset); - return -1; - } - - return 0; -} - struct tracecmd_option * tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) From patchwork Thu May 20 03:19:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D096C433B4 for ; Thu, 20 May 2021 03:20:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 526E561184 for ; Thu, 20 May 2021 03:20:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230148AbhETDV3 (ORCPT ); Wed, 19 May 2021 23:21:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229978AbhETDV2 (ORCPT ); Wed, 19 May 2021 23:21:28 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1ED30C06175F for ; Wed, 19 May 2021 20:20:07 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id a25so17633660edr.12 for ; Wed, 19 May 2021 20:20:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=n4BM1BOF4S5lHeHaOEJIuOPiyiWwnSuhagC/nfywwWE=; b=onsieDlzy5j6PxXiH2lZb1B2H1RArntGf67hAyzkO/Lv7ZnO76IXviDvcSM0ihurpO wEo+4oIiWqur1KSlXfC2YEYSbEbdaYsc0pTeRNPxkGz8TNp5B5+m6qI8S1OP1eeKnIRe a3cA94xcjRc864FcM+erZAE02MPw545G8Mg/ALG+ZmRnCf5JBJWS23LrZQdxINf2UtcL vl1/9M/nGT4ndXhPpZBnTtjzFjHMEoDct37pI36PSklyFy1NQd8sSdbHvJljwZZX+Uc3 bER5SmxLLvDsKE3Cf72T3uy6pbtj7Y1rjkWj2+xy7couznywuktYKABAGfIU6DVLuctr iupA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n4BM1BOF4S5lHeHaOEJIuOPiyiWwnSuhagC/nfywwWE=; b=UiwdjOygLUFO9Z427q0eGtRTSVYtyh/K+7D7h6BEALgeWzZdIFAU0+PW8EhiG3zLpT A6Ja3l0olvdZMI1eQFdZXcMZf43DbYKUKxRyJ3ACFlpohi7dYYvTmYiFu0VtUxTOcawA /QjfIhU3dRvPds9bM/0m9tXXf49657m3Wh4WPb4IV97cSeRtKlCKAu3eoewFz29z/Wo1 UKFwAyzTlSJWAC3CcDUP3htQoWCFg1+u36Smuo18tQe7pTqRp8CQTdQEnGdblUdYwoQY +DsGMXkj0OyLTC78nJJEfqMxKfk+4YM+pDXhZd1mcpUn+C6m93NDCkmEwpwipT7yMM7g JcsQ== X-Gm-Message-State: AOAM532YfxdUggofU/mo4UdEJnhF7BYZnX4BOllIsxxMqneXhhaIyzCT QFOyDTKD3j9r5zd33xzLJ+P4RYYuOQ/r1Q== X-Google-Smtp-Source: ABdhPJy05fb2kAuCHmkvX+0JBEEvFdQBX2bIH3nvF/a4/yJpapYp5YOLO72o87oKtIQX+6veTPKNJg== X-Received: by 2002:aa7:db48:: with SMTP id n8mr2583170edt.11.1621480805758; Wed, 19 May 2021 20:20:05 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:04 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 03/29] trace-cmd: Check if file version is supported Date: Thu, 20 May 2021 06:19:33 +0300 Message-Id: <20210520031959.346165-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace file, version of the file is ignored. This could case problems when bumping the version number because of changes in in the structure of the file. The old code should detect unsupported file version and should not try to read it. A new trace-cmd library API is added to check if version is supported: tracecmd_is_version_supported() Checks are added in the code to ensure not trying to read trace file from unsupported version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 10 ++++++++++ lib/trace-cmd/trace-util.c | 7 +++++++ tracecmd/trace-dump.c | 7 +++++++ 4 files changed, 26 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ecd172ca..136057b4 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -42,6 +42,8 @@ void tracecmd_record_ref(struct tep_record *record); void tracecmd_set_debug(bool set_debug); bool tracecmd_get_debug(void); +bool tracecmd_is_version_supported(unsigned int version); + struct tracecmd_output; struct tracecmd_recorder; struct hook_list; diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 5ee69b14..97ad0a5d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -131,6 +131,7 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + int file_version; struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -3269,6 +3270,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) unsigned int page_size; char *version; char buf[BUFSIZ]; + unsigned long ver; handle = malloc(sizeof(*handle)); if (!handle) @@ -3293,6 +3295,14 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (!version) goto failed_read; tracecmd_info("version = %s\n", version); + ver = strtol(version, NULL, 10); + if (!ver && errno) + goto failed_read; + if (!tracecmd_is_version_supported(ver)) { + tracecmd_warning("Unsupported file version %lu", ver); + goto failed_read; + } + handle->file_version = ver; free(version); if (do_read_check(handle, buf, 1)) diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b65f9dec..b0c98c72 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -617,3 +617,10 @@ unsigned long long tracecmd_generate_traceid(void) free(str); return hash; } + +bool tracecmd_is_version_supported(unsigned int version) +{ + if (version <= FILE_VERSION) + return true; + return false; +} diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 98425b98..03cc82b4 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "trace-local.h" @@ -145,6 +146,7 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; + unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -166,6 +168,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); + ver = strtol(buf, NULL, 10); + if (!ver && errno) + die("Invalid file version string %s", buf); + if (!tracecmd_is_version_supported(ver)) + die("Unsupported file version %lu", ver); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) From patchwork Thu May 20 03:19:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A443C43460 for ; Thu, 20 May 2021 03:20:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CB87611AD for ; Thu, 20 May 2021 03:20:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229978AbhETDV3 (ORCPT ); Wed, 19 May 2021 23:21:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230113AbhETDV2 (ORCPT ); Wed, 19 May 2021 23:21:28 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4933EC061574 for ; Wed, 19 May 2021 20:20:08 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id c20so22976952ejm.3 for ; Wed, 19 May 2021 20:20:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dyNKoXfwKbcpwqBj3pYeS8+00BKdpmjvZYvw5dv9nuI=; b=i3Ka+M+hD3NMkC7NvgYzR/tfGc04A/UkOY2F5MF+Qp42kSCndOwjNQROYwO2TawMAo JLZlX5rH3sq02WbPbWSuCds/y3MGqnPLDaVfXoH7YexiSh2AUFnyli1pEnctWA5N2eTI 2nN52sVeEMiRLDQtmukRJ8l4kScoUXLxuAQFVWdzC7WpeMkIMbh4N4CKkkUzdJ3FoVJ+ SaQRPBl1MZZ3rX3XN6Gc/D4SDVPmc2E+GqWu+ZUgDLzpNgXphkDEnByWoL4Ej5JN4/jg YWtUQ6jc7BnVVGBu1lsksHUnRH/4C8OzhHgOIKiOEA8G24a0DICvrsZqVXAkdyKClll1 5QOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dyNKoXfwKbcpwqBj3pYeS8+00BKdpmjvZYvw5dv9nuI=; b=f3pA57280T+mhemNdGzW7AHzZrM+ZJOxOrrnKi/9YzRMZAk0ncFNSMzDkdSeeS1bwd qXZRZgKR/BBMbBUpgbae7u5K0lCcB71dHHP2Jy2QvRuj7pyg86lU6WTohRmqEMMRYHkt t1uqU1IUonBICzIcTKJIWtakv58JjFfwJiS0X6wZJTqVEwuyqFe6b9OF5aJMNW6nsm+A hCRPPH06Y461N5n6279dE+VjhgVz+MEdODz/bz1KQ1j2kpiyAT989P9rSuwWPCtS4ctr pGDHibWU56ETo7lzmspZZjx+wMHZTXlxiZY68siXBCcselF5O3spjDfMgYeROgQ3Ynx5 URTQ== X-Gm-Message-State: AOAM53169bjsVOoRDiMZp9EgvBwYZ0ancrXBGqCpl2ja0EH2twlHo5UE ovBrvbXLuguNwbxufYEGtOQ= X-Google-Smtp-Source: ABdhPJyKankNcLjPMoVf45I9F1qjhKevX3r1JRIH/dY9AySB+eH5i31nO/GYe3uHV8pw0g74GyZQUQ== X-Received: by 2002:a17:907:b09:: with SMTP id h9mr2388094ejl.430.1621480806939; Wed, 19 May 2021 20:20:06 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:06 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 04/29] trace-cmd library: Add new API to get file version of input handler Date: Thu, 20 May 2021 06:19:34 +0300 Message-Id: <20210520031959.346165-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added an API to get the version of the trace file, associated with given input file handler. tracecmd_get_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 136057b4..6a211333 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -232,6 +232,8 @@ int tracecmd_set_cursor(struct tracecmd_input *handle, unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); +unsigned long tracecmd_get_file_version(struct tracecmd_input *handle); + int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); tracecmd_show_data_func diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 97ad0a5d..67e73c4c 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4046,6 +4046,15 @@ struct tep_handle *tracecmd_get_tep(struct tracecmd_input *handle) return handle->pevent; } +/** + * tracecmd_get_file_version - return the trace.dat file version + * @handle: input handle for the trace.dat file + */ +unsigned long tracecmd_get_file_version(struct tracecmd_input *handle) +{ + return handle->file_version; +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Thu May 20 03:19:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B6CDC433ED for ; Thu, 20 May 2021 03:20:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4E8D3611AB for ; Thu, 20 May 2021 03:20:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230185AbhETDVb (ORCPT ); Wed, 19 May 2021 23:21:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230113AbhETDVb (ORCPT ); Wed, 19 May 2021 23:21:31 -0400 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62ADAC061574 for ; Wed, 19 May 2021 20:20:09 -0700 (PDT) Received: by mail-ej1-x62f.google.com with SMTP id l1so22954518ejb.6 for ; Wed, 19 May 2021 20:20:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9HuGT59fpnbaV/Ei2fhVY2rS3y5IDn6cNc88pcWTIj8=; b=bTgUXz3t1IavBA1SuDARQhcVJeV9uR/FOsYIAVxPKQ18qKAlfH8r1AS11wMI07cLWA nu2BjaMHS7A2hC34HhxFXHkwi+/HwfZnN0Can98dTXEFWqdY+8OTMwnpJScyM59Ebvs4 a7LcPf0u3sXq2sWKnG3ox9fWiMeCVmTugSji+h2EqHMtj+pnPG5aosePljjQC0FD/u62 hvLrNa9xiDZphN28sJXo6NhX0bONhzuuZojvaBHKV5J42S1UEwJOWq+LpV3LC5457dEw qRzjbgghY0g+JYd9E0wuWKcfLzsNzmCPpHFKYslk4+fMUTiX5L5wSdc7oWnqOWKr6SGl N5xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9HuGT59fpnbaV/Ei2fhVY2rS3y5IDn6cNc88pcWTIj8=; b=kxUCAimYLlYwYfBmBvX/X7BzEVWFUnOiY3CYNZxdoee/5/XoUCNDbiHiC/LX/Kqh4+ ZtOhssum4TW6nkn3hzrkxNHYmzWHzIi58k2B20YDkEsb3RrGp4Ofoe8mHxTG1bD92vxF 7BYNOpf+UObDJMHdErosvdavZn1miQxFpCe+ORj2wODsrGlL8JGOl/gzQaBEot3pat9F 9i8ty5URIRgZDhV7eYODdcCUjROySsRsxvUu958J5MyPavYnzykClVhvYocRqU96VtNs iHfKM3FhPsh4Jqu8kpwdRjzRkFgK2zO9BfYru6j6ItsGnRHIIPhKsfO0SOZ+VT+Cju7F ML1g== X-Gm-Message-State: AOAM530x3jSwOCwJJCvfEs5n/2DqQuvoxRyQWX2qzD2v4PpWPzdnaS8R Wq33WByben2qSHTITrTYO3Y= X-Google-Smtp-Source: ABdhPJxkrl40oqlmmVnoodyrkBJ0yTfuZEmndw6sgYwnTkQyICESdosHKL3Gha0cR954XSTMa7H6Xg== X-Received: by 2002:a17:906:2bd0:: with SMTP id n16mr2530788ejg.110.1621480808073; Wed, 19 May 2021 20:20:08 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:07 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 05/29] trace-cmd library: Select the file version when writing trace file Date: Thu, 20 May 2021 06:19:35 +0300 Message-Id: <20210520031959.346165-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new output handler to trace file is allocated, select the proper file version. If this output handler is based on an existing input trace file handler, inherit the trace file version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c0aa58cf..3c30edc1 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -57,6 +57,7 @@ struct tracecmd_output { int nr_options; bool quiet; unsigned long file_state; + unsigned long file_version; struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -907,6 +908,17 @@ out_free: return ret; } +static int select_file_version(struct tracecmd_output *handle, + struct tracecmd_input *ihandle) +{ + if (ihandle) + handle->file_version = tracecmd_get_file_version(ihandle); + else + handle->file_version = FILE_VERSION; + + return 0; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -933,6 +945,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; + if (select_file_version(handle, ihandle)) + goto out_free; + list_head_init(&handle->options); buf[0] = 23; @@ -943,7 +958,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, if (do_write_check(handle, buf, 10)) goto out_free; - if (do_write_check(handle, FILE_VERSION_STRING, strlen(FILE_VERSION_STRING) + 1)) + sprintf(buf, "%lu", handle->file_version); + if (do_write_check(handle, buf, strlen(buf) + 1)) goto out_free; /* get endian and page size */ @@ -1562,6 +1578,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->pevent = tracecmd_get_tep(ihandle); tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); + handle->file_version = tracecmd_get_file_version(ihandle); list_head_init(&handle->options); tracecmd_close(ihandle); From patchwork Thu May 20 03:19:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F335C433B4 for ; Thu, 20 May 2021 03:20:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2056D611AB for ; Thu, 20 May 2021 03:20:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230197AbhETDVc (ORCPT ); Wed, 19 May 2021 23:21:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230113AbhETDVc (ORCPT ); Wed, 19 May 2021 23:21:32 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A2F8C06175F for ; Wed, 19 May 2021 20:20:10 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id s22so22733982ejv.12 for ; Wed, 19 May 2021 20:20:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PQUYB1qHIWalGZLN0HLBYN4T8dtYe86QUofDMP/GfVo=; b=CBP2F1GH/144cIg334+dCmhdBXkWn9MQjOg1/jurvtpVWd9q7659P9G4f27vrXBoCT FNlPTlhCQrom35jCxyVX1QSg3K3kEtKay5sY2fMKuykxbF3WOI0kKiBzDW96+Pj9qI5b GzARPULoemixDD1BCaVEgJo9dL2P9EWh5cecwkekeyUyX5NfRvPtmh1UY31R/ebqacd+ xcR3bfWNwJscq6KN3fcrZA81BD1gkxFZLCZZKWIILm/ECThQJ1pZX3GXJxR5HwKgmvwD iUljmdTZpjW/2zCrnhpgW+Gr8H44b+xUVvToGOP+ZHo8yqdN1au8dg4IXg6PV1f/s9FI UjmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PQUYB1qHIWalGZLN0HLBYN4T8dtYe86QUofDMP/GfVo=; b=bBmX8nYl5Njl/UTBclJxLJzvC/KIpYolbgExeRjVp+LUGkA6s7H8kwz8Ehf2MaKn5g RXai56SrUA+Le2EzTkn2Dg/mUMyzPSWu377WoDCYKskJ6DecerTXr+RSo34prG/s+jd9 NEO8ruKJI6mvruZYNKJIT4RwoGvkNWYOz+eB5m4EcEc7/pCxo48Wzk/Ozuq/EBDrURus 69Xmv2E9DwrYKzf+7WwMykvVUl9wnxcZfhpJM4uEe/zcIOMHmSN4TGndifWvb7ADdvfR JSIqR0RWLkDRMonFpZh9zPEFq06S7+tmgq3i+/6T6yEfoat3TSFcCvurbicdEeUwszEE o7FQ== X-Gm-Message-State: AOAM532WPhX6Wvo5otcA4C0Eij/CLgpjFZUSpk1axASKwN2n20+xySIr ko+J+tgGFkYt3mTzJ+KcjVw= X-Google-Smtp-Source: ABdhPJxTs/kd3UA94Gl71Scf72zHSzL342zNeZPZLXjJ4+iTXyrHfZ1Fu9U+H4f+acmBWdACyODXZQ== X-Received: by 2002:a17:906:bb0e:: with SMTP id jz14mr2394865ejb.285.1621480809014; Wed, 19 May 2021 20:20:09 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:08 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 06/29] trace-cmd: Add APIs for library initialization and free Date: Thu, 20 May 2021 06:19:36 +0300 Message-Id: <20210520031959.346165-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The trace-cmd library has no APIs for initialization and free of the whole library. Added these new APIs: tracecmd_lib_init() tracecmd_lib_free() and call them in trace-cmd main function. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 3 +++ lib/trace-cmd/trace-util.c | 9 +++++++++ tracecmd/trace-cmd.c | 11 ++++++++--- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 6a211333..e410a6f1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -29,6 +29,9 @@ struct tep_plugin_list *trace_load_plugins(struct tep_handle *tep, int flags); +int tracecmd_lib_init(void); +void tracecmd_lib_free(void); + int *tracecmd_add_id(int *list, int id, int len); enum { diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index b0c98c72..61054ad2 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -624,3 +624,12 @@ bool tracecmd_is_version_supported(unsigned int version) return true; return false; } + +int tracecmd_lib_init(void) +{ + return 0; +} + +void tracecmd_lib_free(void) +{ +} diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 00cdaa37..71c8f6d6 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -142,15 +142,20 @@ int main (int argc, char **argv) if (argc < 2) trace_usage(argc, argv); + tracecmd_lib_init(); + for (i = 0; i < ARRAY_SIZE(commands); ++i) { if (strcmp(argv[1], commands[i].name) == 0 ){ commands[i].run(argc, argv); - goto out; + break; } } + tracecmd_lib_free(); + /* No valid command found, show help */ - trace_usage(argc, argv); -out: + if (i == ARRAY_SIZE(commands)) + trace_usage(argc, argv); + exit(0); } From patchwork Thu May 20 03:19:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE0EEC43461 for ; Thu, 20 May 2021 03:20:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F0C361186 for ; Thu, 20 May 2021 03:20:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230224AbhETDVc (ORCPT ); Wed, 19 May 2021 23:21:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230113AbhETDVc (ORCPT ); Wed, 19 May 2021 23:21:32 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DD24C061574 for ; Wed, 19 May 2021 20:20:11 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id et19so15888640ejc.4 for ; Wed, 19 May 2021 20:20:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pfZuCMiQ2WMv1xVe5I++MHkWQrdasI1qZ5dyffsdFdY=; b=cjqJntIklQu703JJ7XSFHUHvexBL5OiKDXnqf5DdyDsY3OO2e2IKnAaBAXcdNyw1T8 rTmayFMYzJPd8UjDb01f53WWtXxazr8q8fargmfAPf/G85Vut7vEiephxntGcGMvfm4q EQyg68NCCIh+P+mAGFB1kipOnoT3VDJV0DUHIRRjzjpPHItJQwic2k3qwiFlac2ui0Yi wNEGrJHMXn9H93Qx2d2H0pApYi2y/aXwCDboaTB3OK5VJc1D11Fgj4IcgOg3B2jLuMLe yC86F/xx4rAhjsyo/qI395xE2FRA0bhJq/R4GJz45m6WzSo6r1jBpNep+OMQWjf4FVdh vJKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pfZuCMiQ2WMv1xVe5I++MHkWQrdasI1qZ5dyffsdFdY=; b=adXCfchkwHLHMd4h/8AYa2FS+i4W2x4uBAepe/O6wuNs1j+pTNXe/u8Hdowvg5Kkqc CMFspxA93V8sX5QSV7CND0Kd2M1xUDtAmSce6641cce+5alFeP8Tq8TZn+C3sgtGhIrk uo8AYk17ogWtPXuT1lMJMvP4Z3pN20/jMjcJAgbKtYzXTgLs5VVNLitE90K+meClgt4c kFpwVPv84J600pgUm4iPzA6J4zFJbQgLoYboTNfyaLDXjo7l/tFAKO3jkJkoEEgohUWE Hbj13DWlVtFck1zRSxzJKWxdGWQBDvPOMd5yvqkZJXFLnhQAnZ0Va526W1wxx7iyZpen j94w== X-Gm-Message-State: AOAM531lwJH3WnP9qPh10z1rZjHipz+FiW7kWHUeokgHT9mNXq9MY1MX KwbK9JGgh368S6HYMVTmhW0= X-Google-Smtp-Source: ABdhPJyBrpumtIONNWoG7enmGAJzC2efse9JqJvcLJFONRSsPsVxR8uJPi8iagEtLX72M4q1XuliXw== X-Received: by 2002:a17:906:1e44:: with SMTP id i4mr2379740ejj.61.1621480810121; Wed, 19 May 2021 20:20:10 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:09 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 07/29] trace-cmd library: Add support for compression algorithms Date: Thu, 20 May 2021 06:19:37 +0300 Message-Id: <20210520031959.346165-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added infrastructure to trace-cmd library for compression. Introduced various new APIs to work with this new functionality: struct tracecmd_compression tracecmd_compress_init() tracecmd_compress_free() tracecmd_compress_alloc() tracecmd_compress_destroy() tracecmd_compress_block() tracecmd_uncompress_block() tracecmd_compress_reset() tracecmd_compress_read() tracecmd_compress_pread() tracecmd_compress_write() tracecmd_compress_lseek() tracecmd_compress_proto_get_name() tracecmd_compress_is_supported() tracecmd_compress_protos_get() tracecmd_compress_proto_register() The compression algorithms are not part of this patch. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 1 + .../include/private/trace-cmd-private.h | 26 + lib/trace-cmd/include/trace-cmd-local.h | 2 + lib/trace-cmd/trace-compress.c | 588 ++++++++++++++++++ lib/trace-cmd/trace-util.c | 3 + 5 files changed, 620 insertions(+) create mode 100644 lib/trace-cmd/trace-compress.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index 17600318..bab4322d 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -25,6 +25,7 @@ ifeq ($(VSOCK_DEFINED), 1) OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif +OBJS += trace-compress.o # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index e410a6f1..602817cb 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -463,6 +463,32 @@ void tracecmd_tsync_free(struct tracecmd_time_sync *tsync); int tracecmd_write_guest_time_shift(struct tracecmd_output *handle, struct tracecmd_time_sync *tsync); +/* --- Compression --- */ +struct tracecmd_compression; +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle); +void tracecmd_compress_destroy(struct tracecmd_compression *handle); +int tracecmd_compress_block(struct tracecmd_compression *handle); +int tracecmd_uncompress_block(struct tracecmd_compression *handle); +void tracecmd_compress_reset(struct tracecmd_compression *handle); +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len); +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset); +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size); +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence); +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version); +bool tracecmd_compress_is_supported(const char *name, const char *version); +int tracecmd_compress_protos_get(char ***names, char ***versions); +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)); + /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 821b5cdb..041582c6 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,5 +31,7 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +int tracecmd_compress_init(void); +void tracecmd_compress_free(void); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c new file mode 100644 index 00000000..ff742226 --- /dev/null +++ b/lib/trace-cmd/trace-compress.c @@ -0,0 +1,588 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include +#include + +#include "trace-cmd-private.h" +#include "trace-cmd-local.h" + +struct compress_proto { + struct compress_proto *next; + char *proto_name; + char *proto_version; + int weight; + + int (*comress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + int (*uncompress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + unsigned int (*comress_size)(unsigned int bytes); + bool (*is_supported)(const char *name, const char *version); +}; + +static struct compress_proto *proto_list; + +struct tracecmd_compression { + int fd; + unsigned int capacity; + unsigned long pointer; + char *buffer; + struct compress_proto *proto; + struct tep_handle *tep; + struct tracecmd_msg_handle *msg_handle; +}; + +static int read_fd(int fd, char *dst, int len) +{ + size_t size = 0; + int r; + + do { + r = read(fd, dst+size, len); + if (r > 0) { + size += r; + len -= r; + } else + break; + } while (r > 0); + + if (len) + return -1; + return size; +} + +static long long write_fd(int fd, const void *data, size_t size) +{ + long long tot = 0; + long long w; + + do { + w = write(fd, data + tot, size - tot); + tot += w; + + if (!w) + break; + if (w < 0) + return w; + } while (tot != size); + + return tot; +} + +static long long do_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + int ret; + + if (handle->msg_handle) { + ret = tracecmd_msg_data_send(handle->msg_handle, data, size); + if (ret) + return -1; + return size; + } + return write_fd(handle->fd, data, size); +} + +/** + * tracecmd_compress_lseek - Move the read/write pointer into the compression buffer + * @handle: compression handler + * @offset: number of bytes to move the pointer, can be negative or positive + * @whence: the starting position of the pointer movement, + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns the new file pointer on success, or -1 in case of an error. + */ +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence) +{ + unsigned long p, extend; + char *buf; + + if (!handle || !handle->buffer) + return (off_t)-1; + + switch (whence) { + case SEEK_CUR: + p = handle->pointer + offset; + break; + case SEEK_END: + p = handle->capacity + offset; + break; + case SEEK_SET: + p = offset; + break; + default: + return (off_t)-1; + } + + if (p <= handle->capacity) { + handle->pointer = p; + } else { + extend = p - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return (off_t)-1; + handle->buffer = buf; + handle->capacity += extend; + handle->pointer = p; + } + + return p; +} + +static int compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int s; + + if (handle->pointer + len > handle->capacity) + s = handle->capacity - handle->pointer; + else + s = len; + memcpy(dst, handle->buffer + handle->pointer, s); + + return s; +} + +/** + * tracecmd_compress_pread - pread() on compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * @offset: offset in the buffer of data to be read + * + * Read a @len of data from the compression buffer at given @offset, + * without updating the buffer pointer. + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset) +{ + int ret; + + if (!handle || !handle->buffer || offset > handle->capacity) + return -1; + + ret = tracecmd_compress_lseek(handle, offset, SEEK_SET); + if (ret < 0) + return ret; + return compress_read(handle, dst, len); +} + +/** + * tracecmd_compress_read - read() from compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * + * Read a @len of data from the compression buffer + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int ret; + + if (!handle || !handle->buffer) + return -1; + + ret = compress_read(handle, dst, len); + if (ret > 0) + handle->pointer += ret; + + return ret; +} + +/** + * tracecmd_compress_reset - Reset the compression buffer + * @handle: compression handler + * + * Reset the compression buffer, any data currently in the buffer will be destroyed. + * + */ +void tracecmd_compress_reset(struct tracecmd_compression *handle) +{ + if (handle) { + free(handle->buffer); + handle->buffer = NULL; + handle->pointer = 0; + handle->capacity = 0; + } +} + +/** + * tracecmd_uncompress_block - uncompress a memory block + * @handle: compression handler + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_uncompress_block(struct tracecmd_compression *handle) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + char buf[4]; + char *bytes = NULL; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + tracecmd_compress_reset(handle); + + if (read(handle->fd, buf, 4) != 4) + return -1; + s_compressed = tep_read_number(handle->tep, buf, 4); + if (read(handle->fd, buf, 4) != 4) + return -1; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + + handle->buffer = malloc(s_uncompressed); + if (!handle->buffer) + return -1; + bytes = malloc(s_compressed); + if (!bytes) + goto error; + + if (read_fd(handle->fd, bytes, s_compressed) < 0) + goto error; + ret = handle->proto->uncompress_block(bytes, s_compressed, + handle->buffer, &s_uncompressed); + if (ret) + goto error; + free(bytes); + handle->pointer = 0; + handle->capacity = s_uncompressed; + return 0; +error: + tracecmd_compress_reset(handle); + free(bytes); + return -1; +} + +/** + * tracecmd_compress_block - compress a memory block + * @handle: compression handler + * + * Compress the content of the internal memory buffer and write the compressed data in the file + * The tracecmd_compress_write() can be used to write data into the internal memory buffer, before + * calling this API. + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_compress_block(struct tracecmd_compression *handle) +{ + unsigned int size; + char *buf; + int endian4; + int ret; + + size = handle->proto->comress_size(handle->pointer); + buf = malloc(size); + if (!buf) + return -1; + ret = handle->proto->comress_block(handle->buffer, handle->pointer, buf, &size); + if (ret < 0) + goto out; + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &handle->pointer, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write the uncompressed data */ + ret = do_write(handle, buf, size); + ret = ((ret == size) ? 0 : -1); +out: + tracecmd_compress_reset(handle); + free(buf); + return ret; +} + +/** + * tracecmd_compress_write - write() to compression buffer + * @handle: compression handler + * @data: data to be written + * @size: size of @data + * + * Write @data of @size in the compression buffer + * + * Returns 0 on success, or -1 on failure. + */ +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + char *buf; + int extend; + + if (handle->capacity < handle->pointer + size) { + extend = (handle->pointer + size) - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return -1; + handle->buffer = buf; + handle->capacity += extend; + } + memcpy(&handle->buffer[handle->pointer], data, size); + handle->pointer += size; + return 0; +} + +/** + * tracecmd_compress_init - initialize the library with available compression algorithms + * + * Returns 0. If no compression algorithms are available, a warning is printed. + */ +int tracecmd_compress_init(void) +{ + struct timeval time; + + gettimeofday(&time, NULL); + srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); + + return 0; +} + +static struct compress_proto *compress_proto_select(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *selected = NULL; + + while (proto) { + if (!selected || selected->weight > proto->weight) + selected = proto; + proto = proto->next; + } + + return selected; +} + +/** + * tracecmd_compress_alloc - Allocate a new compression context + * @name: name of the compression algorithm + * @version: version of the compression algorithm + * @fd: file descriptor for reading / writing data + * @tep: tep handler, used to encode the data + * @msg_handle: message handler, use it for reading / writing data instead of @fd + * + * Returns NULL on failure or pointer to allocated compression context. + * The returned context must be freed by tracecmd_compress_destroy() + */ +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle) +{ + struct compress_proto *proto; + struct tracecmd_compression *new; + + if (name) { + proto = proto_list; + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + break; + proto = proto->next; + } + } else { + proto = compress_proto_select(); + } + if (!proto) + return NULL; + + new = calloc(1, sizeof(*new)); + if (!new) + return NULL; + new->fd = fd; + new->tep = tep; + new->msg_handle = msg_handle; + new->proto = proto; + return new; +} + +/** + * tracecmd_compress_destroy - Free a compression context + * @handle: handle to the compression context that will be freed + */ +void tracecmd_compress_destroy(struct tracecmd_compression *handle) +{ + tracecmd_compress_reset(handle); + free(handle); +} + +/** + * tracecmd_compress_is_supported - check if compression algorithm with given name and + * version is supported + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * + * Returns true if the algorithm with given name and version is supported or false if it is not. + */ +bool tracecmd_compress_is_supported(const char *name, const char *version) +{ + struct compress_proto *proto = proto_list; + + if (!name) + return NULL; + + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + return true; + proto = proto->next; + } + return false; +} + +/** + * tracecmd_compress_proto_get_name - get name and version of compression algorithm + * @compress: compression handler. + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version) +{ + if (!compress || !compress->proto) + return -1; + if (name) + *name = compress->proto->proto_name; + if (version) + *version = compress->proto->proto_version; + return 0; +} + +/** + * tracecmd_compress_proto_register - register a new compression algorithm + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * @weight: weight of the compression algorithm, lower is better. + * @compress: compression hook, called to compress a memory block. + * @uncompress: uncompression hook, called to uncompress a memory block. + * @comress_size: hook, called to get the required minimum size of the buffer for compression + * given number of bytes. + * @is_supported: check hook, called to check if compression with given name and version is + * supported by this plugin. + * + * Returns 0 on success, or -1 in case of an error. If algorithm with given name and version is + * already registered, -1 is returned. + */ +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)) +{ + struct compress_proto *new; + + if (!name || !compress || !uncompress) + return -1; + if (tracecmd_compress_is_supported(name, version)) + return -1; + + new = calloc(1, sizeof(*new)); + if (!new) + return -1; + + new->proto_name = strdup(name); + if (!new->proto_name) + goto error; + new->proto_version = strdup(version); + if (!new->proto_version) + goto error; + new->comress_block = compress; + new->uncompress_block = uncompress; + new->comress_size = comress_size; + new->is_supported = is_supported; + new->weight = weight; + new->next = proto_list; + proto_list = new; + return 0; + +error: + free(new->proto_name); + free(new->proto_version); + free(new); + return -1; +} + +/** + * tracecmd_compress_free - free the library resources, related to available compression algorithms + * + */ +void tracecmd_compress_free(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *del; + + while (proto) { + del = proto; + proto = proto->next; + free(del->proto_name); + free(del->proto_version); + free(del); + } + proto_list = NULL; +} + +/** + * tracecmd_compress_protos_get - get a list of all supported compression algorithms and versions + * @names: return, array with names of all supported compression algorithms + * @versions: return, array with versions of all supported compression algorithms + * + * On success, the size of @names and @versions arrays is returned. Those arrays are allocated by + * the API and must be freed with free() by the caller. Both arrays are with same size, each name + * from @names corresponds to a version from @versions. + * On error -1 is returned and @names and @versions arrays are not allocated. + */ +int tracecmd_compress_protos_get(char ***names, char ***versions) +{ + struct compress_proto *proto = proto_list; + char **n = NULL; + char **v = NULL; + int c, i; + + for (c = 0; proto; proto = proto->next) + c++; + + if (c < 1) + return c; + + n = calloc(c, sizeof(char *)); + if (!n) + goto error; + v = calloc(c, sizeof(char *)); + if (!v) + goto error; + + proto = proto_list; + for (i = 0; i < c && proto; i++) { + n[i] = proto->proto_name; + v[i] = proto->proto_version; + proto = proto->next; + } + + *names = n; + *versions = v; + return c; + +error: + free(n); + free(v); + return -1; +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 61054ad2..136260c6 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -627,9 +627,12 @@ bool tracecmd_is_version_supported(unsigned int version) int tracecmd_lib_init(void) { + + tracecmd_compress_init(); return 0; } void tracecmd_lib_free(void) { + tracecmd_compress_free(); } From patchwork Thu May 20 03:19:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A811C433ED for ; Thu, 20 May 2021 03:20:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B6AC611AB for ; Thu, 20 May 2021 03:20:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230228AbhETDVe (ORCPT ); Wed, 19 May 2021 23:21:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230113AbhETDVd (ORCPT ); Wed, 19 May 2021 23:21:33 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14098C061574 for ; Wed, 19 May 2021 20:20:13 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id i7so5122398ejc.5 for ; Wed, 19 May 2021 20:20:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L4cIrri7H50G2QgorkNc4Lv9OxhhIH4oM+gSM061M5w=; b=dwilI5zfe+DRCahxffccEXZ63u3XCRnUVbs5aiL6vrh8prH3m3rRhkTmITD1LYEyw7 HOTwniOMR0zAliKuzqCNybS22frgNznbyN+E8VdUgsgdXK4Fk0yQapyNRBW/XlbMa71V idX47P6MFv20mOJPsqLIMBjZC3YriWccUyvtQ4ov1+C6EK+7T1AeA9QOiV9eRjeBNpv9 PuxDwbgimPhIUCpRop3xuAUOccaRi67VdmPyHuh+tcDQRG3BVBbMuyLWqjyCFSgYvTOr yds+0vOWLZJBGr78MG6SOy9Vlw69mK+gVryh0Wzhh8GSFQbtxFwUDx834/t9rubPosbq hW7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L4cIrri7H50G2QgorkNc4Lv9OxhhIH4oM+gSM061M5w=; b=YuzdIEw886r2v2ZlRJMG/qqf5M+08n/fZF4QXqi/KterikrJ9L/NrA/b0VVfx56Sry 8sWWJoO4DpF/Wsx7VtWTRCVAdv1xL8LjDYFovLUxHjjlVZsyCPC6AY+A5yO5G5t0vsaq KNJNiLbCK0KIHgw/Cv+Uqtki/rNgD6r9X/D4JPfuS/2mufLdaffFzYcy1lXqJJHyKyaE k895iyB8ZPzj76Xsoc0o51q4WUIFwh4Ty9Qz+HbI2v3FvxXNnis1yVQhVpZhlGLJTlaO L3dylZhiyPVLGVIehWmGMITP6gOF825qr1cMQvVPh3CNSsW/xHoS7/MOM8Dn+lRS88A0 H+TA== X-Gm-Message-State: AOAM532UOr9jLhWypRm2qJx18M6ajtpXKkbqcaWMBQRJFCVAlFvdmXTb KXtraudL/d6ZF/GRbJH47Y0= X-Google-Smtp-Source: ABdhPJyYp1m2Hayn9M94CH379YdlJkLpg6XvOnDIH6mPRvyJdndoB+gDp0doN8rbMr5B+YXZz60ENA== X-Received: by 2002:a17:907:105e:: with SMTP id oy30mr2441566ejb.258.1621480811653; Wed, 19 May 2021 20:20:11 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:10 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 08/29] trace-cmd list: Show supported compression algorithms Date: Thu, 20 May 2021 06:19:38 +0300 Message-Id: <20210520031959.346165-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add new parameter "trace-cmd list -c" to show supported compression algorithms. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-list.c | 26 ++++++++++++++++++++++++++ tracecmd/trace-usage.c | 1 + 2 files changed, 27 insertions(+) diff --git a/tracecmd/trace-list.c b/tracecmd/trace-list.c index d060c810..900da73b 100644 --- a/tracecmd/trace-list.c +++ b/tracecmd/trace-list.c @@ -549,6 +549,24 @@ static void show_plugins(void) tep_free(pevent); } +static void show_compression(void) +{ + char **versions, **names; + int c, i; + + c = tracecmd_compress_protos_get(&names, &versions); + if (c <= 0) { + printf("No compression algorithms are supported\n"); + return; + } + printf("Supported compression algorithms:\n"); + for (i = 0; i < c; i++) + printf("\t%s, %s\n", names[i], versions[i]); + + free(names); + free(versions); +} + void trace_list(int argc, char **argv) { int events = 0; @@ -562,6 +580,7 @@ void trace_list(int argc, char **argv) int flags = 0; int systems = 0; int show_all = 1; + int compression = 0; int i; const char *arg; const char *funcre = NULL; @@ -626,6 +645,10 @@ void trace_list(int argc, char **argv) systems = 1; show_all = 0; break; + case 'c': + compression = 1; + show_all = 0; + break; case '-': if (strcmp(argv[i], "--debug") == 0) { tracecmd_set_debug(true); @@ -670,6 +693,8 @@ void trace_list(int argc, char **argv) show_clocks(); if (systems) show_systems(); + if (compression) + show_compression(); if (show_all) { printf("event systems:\n"); show_systems(); @@ -679,6 +704,7 @@ void trace_list(int argc, char **argv) show_tracers(); printf("\noptions:\n"); show_options(); + show_compression(); } return; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 998f5a24..c388eccd 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -330,6 +330,7 @@ static struct usage_help usage_help[] = { " -O list plugin options\n" " -B list defined buffer instances\n" " -C list the defined clocks (and active one)\n" + " -c list the supported trace file compression algorithms\n" }, { "restore", From patchwork Thu May 20 03:19:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2759BC433B4 for ; Thu, 20 May 2021 03:20:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0CDE361184 for ; Thu, 20 May 2021 03:20:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230251AbhETDVh (ORCPT ); Wed, 19 May 2021 23:21:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230113AbhETDVf (ORCPT ); Wed, 19 May 2021 23:21:35 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BF52C061574 for ; Wed, 19 May 2021 20:20:14 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id l1so22954665ejb.6 for ; Wed, 19 May 2021 20:20:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cp2xjlaw4uvJnn+MOUAkFZQ+eOSnxvlwG7WEJsSGUdQ=; b=YRHZbSVE5pv3lMh0W1IqVJXLYQAEoGd21woEm6KqzEdsli9AsaMzkJU7jEMCwhfiQX DzKApOiVZThDhE7IyMOQmTPZBepU1Mkjmr1oIV2QA8De4DsTS8+haIaWvkDeGHUM613e fV0gMDssch5O67K/dEZ6dfF0P5fwoIUcbRL53Xt2jPKfXKBUbqEBiWg0ptSacuP1AwFY HEGc+05m/YiozWroM1DfVCJVTnNljWXnfWAN8YqZQBwMyyxO7NFgvgWAfVFh/qQRZ8/b Y80Vxy23pNfUob5qP48N+IRB3oRc6uy02jt0aLpLYf3sad80Dnf2TcmZAj83G7YLhxrD liMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cp2xjlaw4uvJnn+MOUAkFZQ+eOSnxvlwG7WEJsSGUdQ=; b=C/0akrn0kaXIJ0cNVkcpn0M1ctZr5R4UOG5j7fydrtDy89J7QdagKncySQRe9GczE+ i4qb8EhiFfH+e5uiZgUeU52vnEClXuPvdlrqt/tqwOxlQNhkQxPGDFXe1jBubWOYQVBc S8+aVYTLbF0Ig4nDC9X9ZxgDGp1dHjaJM+bxkWU2NLKcW/D2GNeo+9xoir1vK+eJBl1o H82wDAO0Beu60kym062vMF8DQOhPzDBCJrIC+6q9lAYhrKcoWx91wVOppC+j3MLjVYbx nYvxsB9bafdYvm7jxv6t7DO38FMavQ7FX473XH2G5WV8qeY5uFz+9QG4nFv1xxzh4Vbv SsEQ== X-Gm-Message-State: AOAM532HjWLI8vapv9CgFFUYY15DO6usMlfHkY4XuJ8QjZB9qfOatWRv ZBjqKAbVqf10lmUtbLF1hBQlttKCc4lbFQ== X-Google-Smtp-Source: ABdhPJz47slKBZPlPbrzqEU8DLM2rCKZmL/346hjpTfQKxLpFJkau+nvC9m8WV+awo8mFd+HFXfK/Q== X-Received: by 2002:a17:906:40d1:: with SMTP id a17mr2417042ejk.43.1621480812714; Wed, 19 May 2021 20:20:12 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:12 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 09/29] trace-cmd library: Bump the trace file version to 7 Date: Thu, 20 May 2021 06:19:39 +0300 Message-Id: <20210520031959.346165-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Adding a compression of the trace.dat file will change its structure. These changes are not backward compatible, the old trace-cmd binaries will not be able to read compressed trace files. Bumping the version to 7 will prevent old trace-cmd to read such files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 041582c6..3332572b 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -15,7 +15,7 @@ void tracecmd_critical(const char *fmt, ...); void tracecmd_info(const char *fmt, ...); /* trace.dat file format version */ -#define FILE_VERSION 6 +#define FILE_VERSION 7 #define _STR(x) #x #define STR(x) _STR(x) From patchwork Thu May 20 03:19:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 597E5C433ED for ; Thu, 20 May 2021 03:20:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 376C661184 for ; Thu, 20 May 2021 03:20:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230113AbhETDVh (ORCPT ); Wed, 19 May 2021 23:21:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230242AbhETDVg (ORCPT ); Wed, 19 May 2021 23:21:36 -0400 Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26F5AC06175F for ; Wed, 19 May 2021 20:20:15 -0700 (PDT) Received: by mail-ej1-x632.google.com with SMTP id l1so22954695ejb.6 for ; Wed, 19 May 2021 20:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aCymDVV1Bv4LwtKc97CwHsv2WaQxip8pkkh6KpnmeOw=; b=nm9aBrknMCdlSXBYmdgSE4tb4opi/lJ/31u3hk3Dpgbf+18B6cJJDnSvGl69s6+3W+ QNudShi16IpAKg5nksc2DM/kSdxBvCW78sln4xqrLfis1n1W5wCy98k/WwVdH6NMD0NG 3MBgF24qEq0vKMWVbiidbLlbo7YXXgUv95Dg4roFfKtkUCAsXN6W9+fKfaTY0pQMRHHD vEqqv93bydXN05bPqIqD2vk0DA70oETKQpOHeLa49fPyU8rJIbimKAy1jjQlDTcWjLbU vs3JHtu0TKjzvb1UPtxyj3fax13BnMG4WDkhplF4STx63zhfC7i3Etrf2dvaYHErkRWK 0++g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aCymDVV1Bv4LwtKc97CwHsv2WaQxip8pkkh6KpnmeOw=; b=RzqLe8MoOasIMWKph+9n9CZfStlhiFNP4no+5IumEjeevDveRWYcA2Q2bBmXtKTQcH T8ZQPMmso78XFvRISUSQ3a1vmXcXqJM9zTkxmxPOa0ZP4Ge4GFrLrpjvLKWM4g5V65e2 Sm/1lRbEjE/UhoORWBdKOSUGRjA1FbCB7j62bSljqj7baeW4g1Q+cyd6Y4famhNovGDc h+v6g5K1n/5trjAQkGGpU/0Qwqv4T8Hk4ngbc/G0CnzvYxPpc9lnBjlLH9WppNffUL56 qzEhrasm/i2R9u1ugVzkJBkLlLHlAQu1jBIfffp4BiZjqYB3/MZcoWYWEUMVoy18ocEC EfSg== X-Gm-Message-State: AOAM532cHhe6sQBhUQJhFIMyE3HEti8hCs5AxWbs850uFQLDLznxEj9i Hz83LHVYT6XLabKqL5ugjLAIAodpKHu0/A== X-Google-Smtp-Source: ABdhPJz1bDCfD2zDU9EXPKVPHJRUzY6H8Ju/IhZNoOcFQd3dyi0BQGSjKkKZ/IgfNjuPztHkGiVvwQ== X-Received: by 2002:a17:906:22c6:: with SMTP id q6mr2423405eja.275.1621480813757; Wed, 19 May 2021 20:20:13 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:13 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 10/29] trace-cmd library: Compress part of the trace file Date: Thu, 20 May 2021 06:19:40 +0300 Message-Id: <20210520031959.346165-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Compress part of the trace.dat file metadata. If there is compression support, chose file version 7 and compress these parts of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name A new compression header is added in the file, right after the page size information. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 + lib/trace-cmd/trace-output.c | 145 ++++++++++++++++++++---- 2 files changed, 127 insertions(+), 22 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 3332572b..bf90eae8 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -34,4 +34,8 @@ void tracecmd_info(const char *fmt, ...); int tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_compression_start(struct tracecmd_output *handle); +int out_compression_end(struct tracecmd_output *handle); +void out_compression_reset(struct tracecmd_output *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 3c30edc1..2846f7f1 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -58,6 +58,10 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + + bool do_compress; + struct tracecmd_compression *compress; + struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -78,12 +82,31 @@ struct list_event_system { static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { + if (handle->do_compress && handle->compress) + return tracecmd_compress_write(handle->compress, data, size); + if (handle->msg_handle) return tracecmd_msg_data_send(handle->msg_handle, data, size); return __do_write_check(handle->fd, data, size); } +static inline int do_lseek(struct tracecmd_output *handle, off_t offset, int whence) +{ + if (handle->do_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek64(handle->fd, offset, whence); +} + +static inline int do_preed(struct tracecmd_output *handle, void *dst, int len, off_t offset) +{ + if (handle->do_compress) + return tracecmd_compress_pread(handle->compress, dst, len, offset); + else + return pread(handle->fd, dst, len, offset); +} + static short convert_endian_2(struct tracecmd_output *handle, short val) { if (!handle->pevent) @@ -109,6 +132,31 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +__hidden void out_compression_reset(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return; + tracecmd_compress_reset(handle->compress); + handle->do_compress = false; +} + +__hidden int out_compression_start(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + tracecmd_compress_reset(handle->compress); + handle->do_compress = true; + return 0; +} + +__hidden int out_compression_end(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + handle->do_compress = false; + return tracecmd_compress_block(handle->compress); +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen @@ -159,6 +207,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) free(option); } free(handle->trace_clock); + tracecmd_compress_destroy(handle->compress); free(handle); } @@ -659,13 +708,17 @@ static int read_ftrace_files(struct tracecmd_output *handle) } create_event_list_item(handle, &systems, &list); - + out_compression_start(handle); ret = copy_event_system(handle, systems); + if (!ret) + ret = out_compression_end(handle); + else + out_compression_reset(handle); free_list_events(systems); - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - + if (!ret) + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; return ret; } @@ -715,6 +768,7 @@ static int read_event_files(struct tracecmd_output *handle, for (slist = systems; slist; slist = slist->next) count++; + out_compression_start(handle); ret = -1; endian4 = convert_endian_4(handle, count); if (do_write_check(handle, &endian4, 4)) @@ -730,8 +784,14 @@ static int read_event_files(struct tracecmd_output *handle, ret = copy_event_system(handle, slist); } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + if (!ret) + ret = out_compression_end(handle); out_free: + if (!ret) + handle->file_state = TRACECMD_FILE_ALL_EVENTS; + else + out_compression_reset(handle); + free_list_events(systems); return ret; @@ -793,20 +853,20 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, if (kallsyms) path = kallsyms; - + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ size = 0; endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; - return 0; + ret = do_write_check(handle, &endian4, 4); + goto out; } size = get_size(path); endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; + ret = do_write_check(handle, &endian4, 4); + if (ret) + goto out; set_proc_kptr_restrict(0); check_size = copy_file(handle, path); @@ -814,13 +874,18 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, errno = EINVAL; tracecmd_warning("error in size of file '%s'", path); set_proc_kptr_restrict(1); - return -1; + ret = -1; + goto out; } set_proc_kptr_restrict(1); - handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = out_compression_end(handle); +out: + if (!ret) + handle->file_state = TRACECMD_FILE_KALLSYMS; + else + out_compression_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_output *handle) @@ -840,6 +905,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + out_compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -861,11 +927,14 @@ static int read_ftrace_printk(struct tracecmd_output *handle) } out: - handle->file_state = TRACECMD_FILE_PRINTK; put_tracing_file(path); + if (out_compression_end(handle)) + return -1; + handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); + out_compression_reset(handle); return -1; } @@ -911,14 +980,39 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { - if (ihandle) + if (ihandle) { handle->file_version = tracecmd_get_file_version(ihandle); - else - handle->file_version = FILE_VERSION; + } else { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + } return 0; } +static int write_compression_header(struct tracecmd_output *handle) +{ + const char *name = NULL; + const char *ver = NULL; + char *buf; + int ret; + + ret = tracecmd_compress_proto_get_name(handle->compress, &name, &ver); + if (ret < 0 || !name || !ver) + return -1; + ret = asprintf(&buf, "%s %s", name, ver); + if (ret < 0) + return -1; + ret = do_write_check(handle, buf, strlen(buf) + 1); + + free(buf); + return ret; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -992,8 +1086,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, endian4 = convert_endian_4(handle, handle->page_size); if (do_write_check(handle, &endian4, 4)) goto out_free; + if (handle->file_version >= 7 && write_compression_header(handle)) + goto out_free; handle->file_state = TRACECMD_FILE_INIT; - if (ihandle) return handle; @@ -1285,11 +1380,17 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) handle->file_state); return ret; } + out_compression_start(handle); + ret = save_tracing_file_data(handle, "saved_cmdlines"); - if (ret < 0) + if (ret < 0) { + out_compression_reset(handle); return ret; - handle->file_state = TRACECMD_FILE_CMD_LINES; - return 0; + } + ret = out_compression_end(handle); + if (!ret) + handle->file_state = TRACECMD_FILE_CMD_LINES; + return ret; } struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) From patchwork Thu May 20 03:19:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30D67C43460 for ; Thu, 20 May 2021 03:20:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14DD861163 for ; Thu, 20 May 2021 03:20:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230242AbhETDVh (ORCPT ); Wed, 19 May 2021 23:21:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230246AbhETDVg (ORCPT ); Wed, 19 May 2021 23:21:36 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29027C061760 for ; Wed, 19 May 2021 20:20:16 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id z12so21498775ejw.0 for ; Wed, 19 May 2021 20:20:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fX9PwmoOIG/Y2zlDYh63vTtFa6rvc9Fi7btreGZA+4E=; b=hMQul0kJOdKDIyvvPskKOwtBAucyqv3cVZYzd2e/g8c5OTXjeVdjfwZZjslEvndffd pl38n8WayLkSfl735kAaLqD9lEqCrtbtDt0XJ98BTBdGzY3dzLMegjxpOA9WPVH8hMbB R8CvB+o6EafZ5Uxn2Zrxth3P9P6MDu7frQj8AgPCakU/kKW0Tth6UdHqTlru/8yu0JDV wxkwenTWRG9dufs7RzLuuG/ALMQAB4MV5wNeG2JDGthEeIQYVk3PlBtxEtQiN7rGEjaD a749DAc0ksPPzI4gX/3T/g3x3LFMirxoozjhrZ0Q09qkSt+IZy5xJqHqJDWBNuamkaYe /nAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fX9PwmoOIG/Y2zlDYh63vTtFa6rvc9Fi7btreGZA+4E=; b=k4fvuzQbzYxrh+SrMDS6nWrDM39o5LJuzJwevEpxMySJxLxJ4M750BajrCtmSjjES5 0BztsOYOms6Ulph8wom0ylhvc1thMDnBaI5dS+zuxV6paaLtBklhv19Do8yEDKgvMo2R QrxcIxTiGVlP0BM0uXYyJKpa5zxFYDNhtRUbo7we09Gj4fjSThahfYDTQy9tLhE3Z1qL BePyDzJSYB9FnAUq1KjSk7IwDTAAHcn5qkyTa26rmE0j+3AgGmhVG8jyqWIBBU3Rth4w qSEdEx3dfO09e5oXqLYcG+T6bSo8c4m7bXil/qjRo7laPLVozxAjoVbpN3YNz1H6At57 gIPQ== X-Gm-Message-State: AOAM533Slvm+sl2sswm7FS8mRir1jT7NHZzeRTX2yA7xJiX78bBE51bD eKpMr//Jfjy0YVT1OuWy0Nk= X-Google-Smtp-Source: ABdhPJx6l4IfICSgP88+B7nQBTtkgzs4aYoPhRl/ptguuVuGMUMRafx3TCGMMnF2xITaAv1JjzdsFw== X-Received: by 2002:a17:907:2d0c:: with SMTP id gs12mr2447863ejc.173.1621480814658; Wed, 19 May 2021 20:20:14 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:14 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 11/29] trace-cmd library: Read compressed trace file Date: Thu, 20 May 2021 06:19:41 +0300 Message-Id: <20210520031959.346165-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If a trace file version 7 is detected, read compressed sections of it. Read the compression header, check if the compression algorithm and version, used to compress the file, are supported and use it to uncompress these sections of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 3 + lib/trace-cmd/trace-input.c | 257 ++++++++++++++++-------- 2 files changed, 178 insertions(+), 82 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index bf90eae8..24228d8f 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -38,4 +38,7 @@ int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); +void in_uncompress_reset(struct tracecmd_input *handle); +int in_uncompress_block(struct tracecmd_input *handle); + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 67e73c4c..1bfa26d7 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -136,6 +136,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + bool read_compress; + struct tracecmd_compression *compress; + struct host_trace_info host; double ts2secs; char * cpustats; @@ -255,12 +258,28 @@ static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) return tot; } +static inline int do_lseek(struct tracecmd_input *handle, int offset, int whence) +{ + if (handle->read_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek(handle->fd, offset, whence); +} + +static inline ssize_t do_read_compressed(struct tracecmd_input *handle, void *data, size_t size) +{ + if (handle->read_compress) + return tracecmd_compress_read(handle->compress, data, size); + else + return do_read(handle, data, size); +} + static ssize_t do_read_check(struct tracecmd_input *handle, void *data, size_t size) { ssize_t ret; - ret = do_read(handle, data, size); + ret = do_read_compressed(handle, data, size); if (ret < 0) return ret; if (ret != size) @@ -278,10 +297,8 @@ static char *read_string(struct tracecmd_input *handle) ssize_t r; for (;;) { - r = do_read(handle, buf, BUFSIZ); - if (r < 0) - goto fail; - if (!r) + r = do_read_compressed(handle, buf, BUFSIZ); + if (r <= 0) goto fail; for (i = 0; i < r; i++) { @@ -307,7 +324,7 @@ static char *read_string(struct tracecmd_input *handle) } /* move the file descriptor to the end of the string */ - r = lseek(handle->fd, -(r - (i+1)), SEEK_CUR); + r = do_lseek(handle, -(r - (i+1)), SEEK_CUR); if (r < 0) goto fail; @@ -359,6 +376,26 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +__hidden void in_uncompress_reset(struct tracecmd_input *handle) +{ + if (handle->compress && handle->file_version >= 7) { + handle->read_compress = false; + tracecmd_compress_reset(handle->compress); + } +} + +__hidden int in_uncompress_block(struct tracecmd_input *handle) +{ + int ret = 0; + + if (handle->compress && handle->file_version >= 7) { + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->read_compress = true; + } + return ret; +} + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -602,34 +639,40 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) + if (in_uncompress_block(handle)) return -1; + ret = read4(handle, &count); + if (ret < 0) + goto out; + for (i = 0; i < count; i++) { - if (read8(handle, &size) < 0) - return -1; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_ftrace_file(handle, size, print_all, ereg); if (ret < 0) - return -1; + goto out; } handle->event_files_start = lseek64(handle->fd, 0, SEEK_CUR); + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + ret = 0; +out: if (sreg) { regfree(sreg); regfree(ereg); } - - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - - return 0; + in_uncompress_reset(handle); + return ret; } static int read_event_files(struct tracecmd_input *handle, const char *regex) { unsigned long long size; - char *system; + char *system = NULL; regex_t spreg; regex_t epreg; regex_t *sreg = NULL; @@ -654,13 +697,19 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) return -1; } - if (read4(handle, &systems) < 0) + if (in_uncompress_block(handle)) return -1; + ret = read4(handle, &systems); + if (ret < 0) + goto out; + for (i = 0; i < systems; i++) { system = read_string(handle); - if (!system) - return -1; + if (!system) { + ret = -1; + goto out; + } sys_printed = 0; print_all = 0; @@ -687,103 +736,117 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) - goto failed; + ret = read4(handle, &count); + if (ret < 0) + goto out; for (x=0; x < count; x++) { - if (read8(handle, &size) < 0) - goto failed; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_event_file(handle, system, size, print_all, &sys_printed, reg); if (ret < 0) - goto failed; + goto out; } free(system); - } - - if (sreg) { - regfree(sreg); - regfree(ereg); + system = NULL; } handle->file_state = TRACECMD_FILE_ALL_EVENTS; - - return 0; - - failed: + ret = 0; + out: + in_uncompress_reset(handle); if (sreg) { regfree(sreg); regfree(ereg); } free(system); - return -1; + return ret; } static int read_proc_kallsyms(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; + struct tep_handle *tep = handle->pevent; unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_KALLSYMS) return 0; - if (read4(handle, &size) < 0) + if (in_uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ - buf = malloc(size+1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)){ - free(buf); - return -1; + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_KALLSYMS; + goto out; /* OK? */ } - buf[size] = 0; - - tep_parse_kallsyms(pevent, buf); - free(buf); + buf = malloc(size+1); + if (!buf) { + ret = -1; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; + buf[size] = 0; + tep_parse_kallsyms(tep, buf); handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = 0; +out: + free(buf); + in_uncompress_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_input *handle) { unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_PRINTK) return 0; - if (read4(handle, &size) < 0) + if (in_uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ + + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_PRINTK; + goto out; /* OK? */ + } buf = malloc(size + 1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)) { - free(buf); - return -1; + if (!buf) { + ret = -1; + goto out; } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; buf[size] = 0; tep_parse_printk_formats(handle->pevent, buf); - - free(buf); - handle->file_state = TRACECMD_FILE_PRINTK; + ret = 0; - return 0; +out: + free(buf); + in_uncompress_reset(handle); + return ret; } static int read_and_parse_cmdlines(struct tracecmd_input *handle); @@ -2974,20 +3037,30 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; unsigned long long size; - char *cmdlines; + char *cmdlines = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_CMD_LINES) return 0; - if (read_data_and_size(handle, &cmdlines, &size) < 0) + if (in_uncompress_block(handle)) return -1; + + ret = read_data_and_size(handle, &cmdlines, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_CMD_LINES; + goto out; + } cmdlines[size] = 0; tep_parse_saved_cmdlines(pevent, cmdlines); - free(cmdlines); - handle->file_state = TRACECMD_FILE_CMD_LINES; - - return 0; + ret = 0; +out: + free(cmdlines); + in_uncompress_reset(handle); + return ret; } static void extract_trace_clock(struct tracecmd_input *handle, char *line) @@ -3268,7 +3341,9 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) struct tracecmd_input *handle; char test[] = TRACECMD_MAGIC; unsigned int page_size; - char *version; + size_t offset; + char *str = NULL; + char *zver; char buf[BUFSIZ]; unsigned long ver; @@ -3291,11 +3366,11 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (memcmp(buf, "tracing", 7) != 0) goto failed_read; - version = read_string(handle); - if (!version) + str = read_string(handle); + if (!str) goto failed_read; - tracecmd_info("version = %s\n", version); - ver = strtol(version, NULL, 10); + tracecmd_info("version = %s\n", str); + ver = strtol(str, NULL, 10); if (!ver && errno) goto failed_read; if (!tracecmd_is_version_supported(ver)) { @@ -3303,7 +3378,8 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) goto failed_read; } handle->file_version = ver; - free(version); + free(str); + str = NULL; if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3328,20 +3404,36 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) read4(handle, &page_size); handle->page_size = page_size; - handle->header_files_start = - lseek64(handle->fd, 0, SEEK_CUR); + offset = lseek64(handle->fd, 0, SEEK_CUR); + handle->total_file_size = lseek64(handle->fd, 0, SEEK_END); + lseek64(handle->fd, offset, SEEK_SET); - handle->total_file_size = - lseek64(handle->fd, 0, SEEK_END); + if (handle->file_version >= 7) { + str = read_string(handle); + if (!str) + goto failed_read; + zver = strchr(str, ' '); + if (!zver) + goto failed_read; + *zver = '\0'; + handle->compress = tracecmd_compress_alloc(str, zver + 1, + handle->fd, handle->pevent, NULL); + if (!handle->compress) { + tracecmd_warning("Unsupported file compression %s %s", str, zver + 1); + goto failed_read; + } + free(str); + str = NULL; + } handle->header_files_start = - lseek64(handle->fd, handle->header_files_start, SEEK_SET); - + lseek64(handle->fd, 0, SEEK_CUR); handle->file_state = TRACECMD_FILE_INIT; return handle; failed_read: + free(str); free(handle); return NULL; @@ -3519,7 +3611,8 @@ void tracecmd_close(struct tracecmd_input *handle) if (handle->flags & TRACECMD_FL_BUFFER_INSTANCE) tracecmd_close(handle->parent); else { - /* Only main handle frees plugins and pevent */ + /* Only main handle frees plugins, pevent and compression context */ + tracecmd_compress_destroy(handle->compress); tep_unload_plugins(handle->plugin_list, handle->pevent); tep_free(handle->pevent); } From patchwork Thu May 20 03:19:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45DDBC433B4 for ; Thu, 20 May 2021 03:20:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 29B90611AB for ; Thu, 20 May 2021 03:20:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230270AbhETDVk (ORCPT ); Wed, 19 May 2021 23:21:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230246AbhETDVj (ORCPT ); Wed, 19 May 2021 23:21:39 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36ED5C061574 for ; Wed, 19 May 2021 20:20:17 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id n2so22938781ejy.7 for ; Wed, 19 May 2021 20:20:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9zzLZoywnemuYA+f+nKTzbbPz0GVFLrD2sFUUhvin54=; b=Y6eQVbtBtAYn0j/PdInkvot4mro6+9KX4HFucC3Ml/WolnhzTuiVJtqnlxy9rZOtkp CD00KW9/phZq4CJkcPyu5s6SNI8+QV88F1OufTTEr3zaFGnwQbDuE3DJX4+jdbafAkRA krmWI9ZlVsvsuPbP7893Y3GVSJcEjl4EwDw5ddMg/bs5xUXbVrGewxiut1LbF/58dwJ6 xgjCm2ICcTCGRlsPxfVURJ43nRfrnbkcbPSNiqVhn6xOe8mON6kKb80eIvLhX2SEt3tW FbhCQX94Tzec4B8EhewMkQ9txxI2N5xKZqkAYxeVVqOHClwQzKTK6SlQYawqyErOCRwE hsoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9zzLZoywnemuYA+f+nKTzbbPz0GVFLrD2sFUUhvin54=; b=Y3X/H5yWqkId3g8X0OKopwm2SGq3VkgruzbM6JlMcwzTYb+CqV+Et3jVokWqEcIRui c7uAtC6KsQhyi/MYxtF8UFBzSOQEhG+jGwnWYgF/97iR8PFxd5dwKGYRflQH3Mk6vIrR M+JRelqPQzmEjBpML+tGr9xO7J+Lacxyqh2sBFcz7YOxss9XPJ3aLP7P0FVwowlxYUuf uWIQ/BGLuuS0aIyqy4ZWfJRtYk9cpA5Hexft7P2k9pYxAYlequIz0DucnADr7bRMh/Na MGw4yieYGhvUQvG6lDzCOF1nBOulTWLaItHj/iELzJgyIDQFm/Fr7O1u9ZrafcixhHwl 7aYw== X-Gm-Message-State: AOAM531sHACx5zhQShIfKCxCMxaj/6FlV0HCmD2BtddDBFjY6/DOiw6x fXrHjhAPIVUGDSe8s8uaBKtCBb8hkM0klw== X-Google-Smtp-Source: ABdhPJygI8Xv+9UoOMoar1vVEHsO6CkRwd7V3gwlTgB5WfY2xfvaSxqfjD7ACtRNeKIAseK6fyuBbg== X-Received: by 2002:a17:906:6c96:: with SMTP id s22mr2480591ejr.468.1621480815876; Wed, 19 May 2021 20:20:15 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:15 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 12/29] trace-cmd library: Add new API to get compression of input handler Date: Thu, 20 May 2021 06:19:42 +0300 Message-Id: <20210520031959.346165-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The new API returns name and version of the compression algorithm, used to compress the trace file associated with given input file handler: tracecmd_get_file_compress_proto() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 16 ++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 602817cb..4265e37a 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -236,6 +236,8 @@ unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_file_version(struct tracecmd_input *handle); +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 1bfa26d7..e32789a8 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4148,6 +4148,22 @@ unsigned long tracecmd_get_file_version(struct tracecmd_input *handle) return handle->file_version; } +/** + * tracecmd_get_file_compress_proto - get name and version of compression algorithm, + * used to compress the trace file + * @handle: input handle for the trace.dat file + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version) +{ + return tracecmd_compress_proto_get_name(handle->compress, name, version); +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Thu May 20 03:19:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 923B3C433ED for ; Thu, 20 May 2021 03:20:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 74D0C611AD for ; Thu, 20 May 2021 03:20:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230246AbhETDVk (ORCPT ); Wed, 19 May 2021 23:21:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230262AbhETDVj (ORCPT ); Wed, 19 May 2021 23:21:39 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EDD5C06175F for ; Wed, 19 May 2021 20:20:18 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id s22so22734273ejv.12 for ; Wed, 19 May 2021 20:20:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hQYYC2TnA8NLtdWvLkwQiO+hUHTcdaTSemyO9tb4ixc=; b=t00L184Lp9VnxIqgopI1w4lnYZb2H2jlYcu17ChpMRe6KE4RBuqEWpWyTz1YQembDo BvCZCOWh6hJG3dWGaqglixAFbLvDIaEvUu3uorCJa4A4/6pG1NDu6XTTLKNXyQCAU9nl rRJRKOvKcIoB6KeAaOHHAicHCsNKHnm58D6qRhLsyBaG/vChRQOGoiQ7B/p4ejOtToRP gVVnW8UNQuu/4YNpjp3g5tTJ1P9Z5/N05P0HwXhPAgqhvOOFA1l9qrcRlex2VDC4Dn1c Nm4YU3/3ZMqebYq+hdJTZsQT1eR/mBb5umg9tdIDIZGep9qsBdMjvxKPXRgSKw8wELqF HdeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hQYYC2TnA8NLtdWvLkwQiO+hUHTcdaTSemyO9tb4ixc=; b=KQ9KesQXdCRaXnGkGVflmvch2QUwzSA4bmxaV2jNaJRrTUSiQe+wlD3XTYXxWl5Js/ +I3hA+F2Jte1DlZkO155cp62OY8WQDNhZASKXC0K+QDDRmq7ezfeEea2L9xdMa1m25Cn OfV8oeWoFnd4YhfB8rlCwTktcpA2Yptdda5um0NhwdRTFBeHF5qPUPyc5pRdioMNPCGS y/9OpFJAqCE/U7F8DS6NITHQz0rnEel4CDjwXIt/nme3svTQ+mAvSkYdNfFLQGhGm0D8 bLz8B/HOLRxHHjWtuigLlqasrfdohQDqwTtkTT48or2mwcqrLhVlddld+fSNbjrf+8v4 H0Zw== X-Gm-Message-State: AOAM532XE1Bp8T4+4CYZVJpAclAzad7lPesoM4+tqn6jHQLAU/CNwAhr LeVUMeYBIm5sW5oStdrJBf3FR4HworsK4g== X-Google-Smtp-Source: ABdhPJx0QBqMkDar2SxDPwA7VQImc7q5X61jpI2xAyl0g6n2HF6sKFQTJ8jhIXkeXbdLW7WBl0pTfQ== X-Received: by 2002:a17:907:10c7:: with SMTP id rv7mr2426996ejb.125.1621480817024; Wed, 19 May 2021 20:20:17 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:16 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 13/29] trace-cmd library: Inherit compression algorithm from input file Date: Thu, 20 May 2021 06:19:43 +0300 Message-Id: <20210520031959.346165-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace file output handler is allocated, based on given trace file input handler - use the same compression algorithm. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 2846f7f1..f0ebc1af 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -980,17 +980,27 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { + const char *cname = NULL; + const char *cver = NULL; + if (ihandle) { handle->file_version = tracecmd_get_file_version(ihandle); - } else { - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); - if (handle->compress) - handle->file_version = 7; - else - handle->file_version = 6; + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + return -1; + } + return 0; } + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + return 0; } @@ -1645,6 +1655,8 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) { struct tracecmd_output *handle = NULL; struct tracecmd_input *ihandle; + const char *cname = NULL; + const char *cver = NULL; int fd2; /* Move the file descriptor to the beginning */ @@ -1682,6 +1694,12 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->file_version = tracecmd_get_file_version(ihandle); list_head_init(&handle->options); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + goto out_free; + } tracecmd_close(ihandle); return handle; From patchwork Thu May 20 03:19:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B71B4C43461 for ; Thu, 20 May 2021 03:20:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A19661186 for ; Thu, 20 May 2021 03:20:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230262AbhETDVk (ORCPT ); Wed, 19 May 2021 23:21:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVk (ORCPT ); Wed, 19 May 2021 23:21:40 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C36AC061760 for ; Wed, 19 May 2021 20:20:19 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id s22so22734311ejv.12 for ; Wed, 19 May 2021 20:20:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ngRSmrkPXG+9QYQDdUwaQ5lXucdX3JKxISx3/qDcUgM=; b=jFMdonHqQDbPvtm7yTp7s2rkwYlJ/8DhJzFdwb/Ng4j+sDcaFPHOEwkKzbsDyAWVsS FIvVOeyypCqpxwGefAXzB9O3MmFY5IzQskPVUnXwRfeUGmnwf/Ekpm+mYg1qPtTqT8D8 yyErxHvNkhcS1huqi+3rZTaRLS8KufZkRg0vCEymXTxd5ZyQXuLttj/DaJpWHSGw9+iR VACZbm56ElE98G6ViHhNzgr/GacUedlMVutpfAAq37g8BR1NCTqkc2r/C3e2Vy13S7Nv KVj560IZaf1M73DAomOJFaNujHG9do9LE7GJo3Zt5L0uM3FpuFQi3S3rKuYcl8rjn1no v/MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ngRSmrkPXG+9QYQDdUwaQ5lXucdX3JKxISx3/qDcUgM=; b=WtrZdRpIpQij34u/WzQrdgO7EU+PqOpdmlK6swScOIyE8T5ZKA+yboJzhrxMrZRoK4 /VTlXzsTJ9y9+KROQew08i03H+oIfVYqxNBTzIu+h+zn5BZylY3+Pq146BS2P/ncShfq GH73tsLo38RPOVxMQE1r8LqsHJS/C4sflvm4SQ68IpvfMK19clL6XqSWVDF4bJ65kXdC kiIdVimmoRAnGqYS/8i8f68Rd1gLDs4UTFErSGmTinlFsuhI4C/Vfu+VuTZ/CMM+wbQM KbAMyPIvmi6XCTYXGX82gvQ0a35hIKwbogFWFXDAklUarmHybepbJgYM74NavWLB77Ys P/Xg== X-Gm-Message-State: AOAM531cNnCY1Uc/dMgx+BUufxQ5PaJ23DEI1W0sB+DuDQXS0ZqVAzRN bcg1phLMNe0HhrrhH93X8t8= X-Google-Smtp-Source: ABdhPJzs9ydEnMQI9jpC/PAemKGCmNDhHEg7RTEnAecEWn3y1jAosncwA4OLKCSr0MAWSz4xK3nngQ== X-Received: by 2002:a17:906:4a81:: with SMTP id x1mr2399430eju.508.1621480818055; Wed, 19 May 2021 20:20:18 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:17 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 14/29] trace-cmd library: Extend the create file APIs to support different compression Date: Thu, 20 May 2021 06:19:44 +0300 Message-Id: <20210520031959.346165-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added additional parameter for file compression to all trace-cmd library APIs for creating a new trace file. The caller could specify what compression algorithm should be used when creating a new trace file: tracecmd_create_file_latency tracecmd_create_init_file_glob tracecmd_create_init_fd_glob tracecmd_create_init_fd_msg tracecmd_create_init_file tracecmd_create_init_file_override tracecmd_create_init_fd The new parameter is of type string and can be one of: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 19 +++-- lib/trace-cmd/trace-output.c | 77 +++++++++++++------ tracecmd/trace-record.c | 12 ++- tracecmd/trace-restore.c | 4 +- tracecmd/trace-stream.c | 2 +- 5 files changed, 71 insertions(+), 43 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4265e37a..ebd633e1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -271,20 +271,23 @@ struct tracecmd_event_list { struct tracecmd_option; struct tracecmd_msg_handle; -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression); struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_fd(int fd); +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_fd(int fd, const char *compression); struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list); +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression); struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_file(const char *output_file); + struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression); struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms); + const char *kallsyms, + const char *compression); struct tracecmd_option *tracecmd_add_option(struct tracecmd_output *handle, unsigned short id, int size, const void *data); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index f0ebc1af..d575ea3b 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -978,7 +978,8 @@ out_free: } static int select_file_version(struct tracecmd_output *handle, - struct tracecmd_input *ihandle) + struct tracecmd_input *ihandle, + const char *compression) { const char *cname = NULL; const char *cver = NULL; @@ -994,8 +995,28 @@ static int select_file_version(struct tracecmd_output *handle, return 0; } - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); + if (compression) { + if (!strcmp(compression, "any")) { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("No compression algorithms are not supported"); + return -1; + } + } else if (strcmp(compression, "none")) { + handle->compress = tracecmd_compress_alloc(compression, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("Compression algorithm %s is not supported", + compression); + return -1; + } + } + } else { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + } + if (handle->compress) handle->file_version = 7; else @@ -1028,7 +1049,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, struct tracecmd_event_list *list, - struct tracecmd_msg_handle *msg_handle) + struct tracecmd_msg_handle *msg_handle, + const char *compression) { struct tracecmd_output *handle; struct tep_handle *pevent; @@ -1049,7 +1071,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; - if (select_file_version(handle, ihandle)) + if (select_file_version(handle, ihandle, compression)) goto out_free; list_head_init(&handle->options); @@ -1128,7 +1150,8 @@ static struct tracecmd_output *create_file(const char *output_file, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, + const char *compression) { struct tracecmd_output *handle; int fd; @@ -1137,7 +1160,7 @@ static struct tracecmd_output *create_file(const char *output_file, if (fd < 0) return NULL; - handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL); + handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL, compression); if (!handle) { close(fd); unlink(output_file); @@ -1403,13 +1426,14 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return ret; } -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression) { struct tracecmd_output *handle; char *path; int ret; - handle = create_file(output_file, NULL, NULL, NULL, &all_event_list); + handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) return NULL; @@ -1710,41 +1734,43 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output *tracecmd_create_init_fd(int fd) +struct tracecmd_output *tracecmd_create_init_fd(int fd, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL, compression); } struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(msg_handle->fd, NULL, NULL, NULL, list, msg_handle); + return create_file_fd(msg_handle->fd, NULL, NULL, NULL, + list, msg_handle, compression); } struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list) +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, list, NULL, compression); } struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list) +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression) { - return create_file(output_file, NULL, NULL, NULL, list); + return create_file(output_file, NULL, NULL, NULL, list, compression); } -struct tracecmd_output *tracecmd_create_init_file(const char *output_file) +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression) { - return create_file(output_file, NULL, NULL, NULL, &all_event_list); + return create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); } struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms) + const char *kallsyms, + const char *compression) { - return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list); + return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list, compression); } /** @@ -1756,12 +1782,13 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) { struct tracecmd_output *handle; + const char *compr_name = NULL; - handle = create_file(file, ihandle, NULL, NULL, &all_event_list); + tracecmd_get_file_compress_proto(ihandle, &compr_name, NULL); + handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compr_name); if (!handle) return NULL; diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index bf911895..03d82377 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3700,7 +3700,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events); + network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3718,8 +3718,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, - listed_events); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4067,8 +4066,7 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, - listed_events); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4460,7 +4458,7 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count); + handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4491,7 +4489,7 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events); + handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 280a37f0..96b4fb5d 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -91,7 +91,7 @@ void trace_restore (int argc, char **argv) } handle = tracecmd_create_init_file_override(output, tracing_dir, - kallsyms); + kallsyms, NULL); if (!handle) die("Unabled to create output file %s", output); if (tracecmd_write_cmdlines(handle) < 0) @@ -128,7 +128,7 @@ void trace_restore (int argc, char **argv) handle = tracecmd_copy(ihandle, output); tracecmd_close(ihandle); } else - handle = tracecmd_create_init_file(output); + handle = tracecmd_create_init_file(output, NULL); if (!handle) die("error writing to %s", output); diff --git a/tracecmd/trace-stream.c b/tracecmd/trace-stream.c index b47b208c..317de1b2 100644 --- a/tracecmd/trace-stream.c +++ b/tracecmd/trace-stream.c @@ -43,7 +43,7 @@ trace_stream_init(struct buffer_instance *instance, int cpu, int fd, int cpus, tfd = fileno(fp); ofd = dup(tfd); - trace_output = tracecmd_create_init_fd(ofd); + trace_output = tracecmd_create_init_fd(ofd, NULL); if (!trace_output) { fclose(fp); return NULL; From patchwork Thu May 20 03:19:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C98E4C433B4 for ; Thu, 20 May 2021 03:20:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE178611AB for ; Thu, 20 May 2021 03:20:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230271AbhETDVm (ORCPT ); Wed, 19 May 2021 23:21:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVm (ORCPT ); Wed, 19 May 2021 23:21:42 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5FAB2C061574 for ; Wed, 19 May 2021 20:20:20 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id u21so22884825ejo.13 for ; Wed, 19 May 2021 20:20:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3rk++3Mb4YdRNFVxgxBrd/q2IYK7fq8ChT6DACQoplM=; b=KPLDVTCj24l7U/RkOMqieZF8TyNgtnQbl4K7KeBjE2zVkhavdaQTdqCtSNDiKnBkm1 ZBamxl393FEEj4saXTM2CrHzcKueKHP1DWjjVVoenbalh7vY8dlLCpbeojhN5gav/A2/ lmcM8LEkzoVLbkE9CkGI3D+lxVYWT+SGKmucd9iR2O2EUXs/hFekVtmrMCRzFJwXl0ia FBMzqmaHkcX9z65+eU50jRu0MIDMjiQwGZx7p9QC8IjhuV4DgX4EoCe5BpAfcpj9TxAz GfYj6hqbIsi1lD9+B64/mhJCPWYJ8/dJ3YU0e0nJyIsWlvooCP0hpTSbfXdGBNZ0N78v LH+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3rk++3Mb4YdRNFVxgxBrd/q2IYK7fq8ChT6DACQoplM=; b=gIVBxCc2d4263Ohz1aQlKpEt3OCCnxj/fZUpxtusSI4Njahwx34j7vFMQ3I7vXD6B/ M5+aCr8mPhTC7bhl9WNGmr65CkPTF9QpC4yzaf6H3v5y17WQ45nF9RtOvdgDS1nEUD8P HbiiUEaVsTboiUKoLt8EnF2S87ouj8Fa60hoLp43ddGSKzoM4w2L7AY6UMb5WuzrKKfW Tw2opBXOBm/TMcBZY7bd7w/bj4u9/M/UqO/U+n43lSuPAtK3SNxQ2fmia5Smsl6Xm9aQ a4nURArl+pramqQ54i/3l3j8ejgm30kGskEWJEEt62MuRYPVNAfTZAZmTXkEpCcw9BoC K8Aw== X-Gm-Message-State: AOAM530qyqTLGyO4OhKvVZKNdnvCLwz6SwEtWOfI5XOqGmott+NigHh/ egL4gyZdQ4NYzRBkjct7npM= X-Google-Smtp-Source: ABdhPJzn1U9rW0SVp+/VdsAfllOK2CZohazgzLwq6czp2p5iOH4TV2MiWkDSPGEe9SxJTTERZDA4aQ== X-Received: by 2002:a17:906:a245:: with SMTP id bi5mr2551277ejb.316.1621480818957; Wed, 19 May 2021 20:20:18 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:18 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 15/29] trace-cmd record: Add new parameter --compression Date: Thu, 20 May 2021 06:19:45 +0300 Message-Id: <20210520031959.346165-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a new parameter "trace-cmd record --compression", can be used to select the desired compression algorithm for the trace output file. One of these strings can be passed: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms, available algorithms can be listed with "trace-cmd list -c" If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 24 +++++++++++++++++++----- tracecmd/trace-usage.c | 5 +++++ 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 03d82377..b71c9608 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -199,6 +199,7 @@ struct common_record_context { char *date2ts; char *user; const char *clock; + const char *compression; struct tsc_nsec tsc2nsec; int data_flags; int tsync_loop_interval; @@ -3700,7 +3701,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(msg_handle, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3718,7 +3720,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -4066,7 +4069,8 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, + listed_events, ctx->compression); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4458,7 +4462,8 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); + handle = tracecmd_create_file_latency(ctx->output, + local_cpu_count, ctx->compression); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4489,7 +4494,8 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); + handle = tracecmd_create_init_file_glob(ctx->output, + listed_events, ctx->compression); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); @@ -5736,6 +5742,7 @@ void init_top_instance(void) } enum { + OPT_comporession = 239, OPT_tsc2nsec = 240, OPT_fork = 241, OPT_tsyncinterval = 242, @@ -6170,6 +6177,7 @@ static void parse_record_options(int argc, {"tsync-interval", required_argument, NULL, OPT_tsyncinterval}, {"fork", no_argument, NULL, OPT_fork}, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, + {"compression", required_argument, NULL, OPT_comporession}, {NULL, 0, NULL, 0} }; @@ -6591,6 +6599,12 @@ static void parse_record_options(int argc, die("TSC to nanosecond is not supported"); ctx->instance->flags |= BUFFER_FL_TSC2NSEC; break; + case OPT_comporession: + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + ctx->compression = strdup(optarg); + break; case OPT_quiet: case 'q': quiet = true; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index c388eccd..a1d7799a 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -68,6 +68,11 @@ static struct usage_help usage_help[] = { " If a negative number is specified, timestamps synchronization is disabled" " If 0 is specified, no loop is performed - timestamps offset is calculated only twice," " at the beginnig and at the end of the trace\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { "set", From patchwork Thu May 20 03:19:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65F4DC43460 for ; Thu, 20 May 2021 03:20:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4DF6C611AB for ; Thu, 20 May 2021 03:20:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230273AbhETDVn (ORCPT ); Wed, 19 May 2021 23:21:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVn (ORCPT ); Wed, 19 May 2021 23:21:43 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78EF4C06175F for ; Wed, 19 May 2021 20:20:21 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id t15so17654587edr.11 for ; Wed, 19 May 2021 20:20:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=ToqPb8AXJJdZtXtkL8eSJy6JuQoK9gH/+3Ysm+idK3JsNSrzGOWKibk3Fz4w+F013n aQ6hfsYw0oT1ZNzq9XY5yOMvJhCkqT5g9WL9QQ3R00fTuSQHU4ddQh3jsIrnkWoSSn3d kqAed0cPz0ylkWdWus6AcZg0zG41Bv98JKtKBizvWNaxuT1tA+vrExHFThggX5Htaa63 mWLZozAfiVj+CnfKK1cA0WB+hxtkW/xyiFHdzT0fhZxLS5K7Q+6v52cLFobuz1yOB9mS wAc3ry+2xD219WzLlc0sG/9CX1PKlcWnQh5Fe5FPkfWX1aPPaANH6GO4qaJqpkhqQOAU tFZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=DjP5B25SwxkTKr8UOqZnWyLconr0ghlXq3KmqdquGP7S7J0vLMtdfFzOANaN2pUOJQ prMTXtfUW9AbfOm+U7TsRj44tzvGBT9ER1YwtAJHQ6+LYJ3u6ujFMg+wzW8EDLm/9br1 vegJjFPIUs4Uxaj3nDx1DxH2Y2bQUru35g2uItGy8o+jOEiE0vbm3j3GYdRfEyxfFj7P x8u7YTVWftb4F0CvOvlJsIQkVzk/oNuYTcVsHQE/KQXES6Jp0pqmm2zSM3v/dj7Eu5qL q17MxG/MASjnl2XS5Ysvw8wimaGAk31BgShvFcWh1HGhudGN1CpUy9tAzSlxprVsbXJE PBrQ== X-Gm-Message-State: AOAM533kM/bhyLkj+qfS8eWuSdmUgBiR9MTMap6v2FLUlbLjPx5pdgGp cSrEn2fP+BLTTWvGR+txCOwokxlRKRS/oA== X-Google-Smtp-Source: ABdhPJwdENnhmsljCNvydUT06/9Z8w+BUJ8ljWoY3Bx+xHzJNkV+u8IBtZHlGLzGm78P5kum5rqo3Q== X-Received: by 2002:a50:afa3:: with SMTP id h32mr2541218edd.202.1621480820086; Wed, 19 May 2021 20:20:20 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:19 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 16/29] trace-cmd dump: Add support for trace files version 7 Date: Thu, 20 May 2021 06:19:46 +0300 Message-Id: <20210520031959.346165-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added support to read and dump information of trace files version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 125 +++++++++++++++++++++++++++++++++++------- 1 file changed, 105 insertions(+), 20 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 03cc82b4..ed82104b 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -26,6 +26,9 @@ static struct tep_handle *tep; static unsigned int trace_cpus; static int has_clock; +static unsigned long file_version; +static bool read_compress; +static struct tracecmd_compression *compress; enum dump_items { SUMMARY = (1 << 0), @@ -52,46 +55,62 @@ enum dump_items verbosity; tracecmd_plog(fmt, ##__VA_ARGS__); \ } while (0) -static int read_file_string(int fd, char *dst, int len) +static int read_fd(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, 1); + r = read(fd, dst+size, len); if (r > 0) { - size++; - len--; + size += r; + len -= r; } else break; - if (!dst[size - 1]) - break; - } while (r > 0 && len); + } while (r > 0); - if (!size || dst[size - 1]) + if (len) return -1; - return 0; + return size; } -static int read_file_bytes(int fd, char *dst, int len) +static int read_compressed(int fd, char *dst, int len) +{ + + if (read_compress) + return tracecmd_compress_read(compress, dst, len); + return read_fd(fd, dst, len); +} + +static int read_file_string(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, len); + r = read_compressed(fd, dst+size, 1); if (r > 0) { - size += r; - len -= r; + size++; + len--; } else break; - } while (r > 0); + if (!dst[size - 1]) + break; + } while (r > 0 && len); - if (len) + if (!size || dst[size - 1]) return -1; return 0; } +static int read_file_bytes(int fd, char *dst, int len) +{ + int ret; + + ret = read_compressed(fd, dst, len); + return ret < 0 ? ret : 0; +} + static void read_dump_string(int fd, int size, enum dump_items id) { char buf[DUMP_SIZE]; @@ -146,7 +165,6 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; - unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -168,11 +186,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); - ver = strtol(buf, NULL, 10); - if (!ver && errno) + file_version = strtol(buf, NULL, 10); + if (!file_version && errno) die("Invalid file version string %s", buf); - if (!tracecmd_is_version_supported(ver)) - die("Unsupported file version %lu", ver); + if (!tracecmd_is_version_supported(file_version)) + die("Unsupported file version %lu", file_version); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) @@ -192,6 +210,29 @@ static void dump_initial_format(int fd) do_print(SUMMARY, "\t\t%d\t[Page size, bytes]\n", val4); } +static void dump_compress(int fd) +{ + char compr[DUMP_SIZE]; + char *ver = NULL; + + if (file_version < 7) + return; + + /* get compression header */ + if (read_file_string(fd, compr, DUMP_SIZE)) + die("no compression header"); + ver = strchr(compr, ' '); + if (!ver) + die("no compression version"); + *ver = '\0'; + do_print((SUMMARY), "\t\t%s\t[Compression algorithm]\n", compr); + do_print((SUMMARY), "\t\t%s\t[Compression version]\n", ver + 1); + + compress = tracecmd_compress_alloc(compr, ver + 1, fd, tep, NULL); + if (!compress) + die("cannot uncomress the file"); +} + static void dump_header_page(int fd) { unsigned long long size; @@ -234,11 +275,34 @@ static void dump_header_event(int fd) read_dump_string(fd, size, HEAD_EVENT); } +static void uncompress_reset(void) +{ + if (compress && file_version >= 7) { + read_compress = false; + tracecmd_compress_reset(compress); + } +} + +static int uncompress_block(void) +{ + int ret = 0; + + if (compress && file_version >= 7) { + ret = tracecmd_uncompress_block(compress); + if (!ret) + read_compress = true; + + } + return ret; +} + static void dump_ftrace_events_format(int fd) { unsigned long long size; unsigned int count; + if (uncompress_block()) + die("cannot uncompress file block"); do_print((SUMMARY | FTRACE_FORMAT), "\t[Ftrace format, "); if (read_file_number(fd, &count, 4)) die("cannot read the count of the ftrace events"); @@ -251,6 +315,7 @@ static void dump_ftrace_events_format(int fd) read_dump_string(fd, size, FTRACE_FORMAT); count--; } + uncompress_reset(); } static void dump_events_format(int fd) @@ -262,6 +327,9 @@ static void dump_events_format(int fd) do_print((SUMMARY | EVENT_FORMAT | EVENT_SYSTEMS), "\t[Events format, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &systems, 4)) die("cannot read the count of the event systems"); @@ -284,6 +352,7 @@ static void dump_events_format(int fd) } systems--; } + uncompress_reset(); } static void dump_kallsyms(int fd) @@ -292,12 +361,17 @@ static void dump_kallsyms(int fd) do_print((SUMMARY | KALLSYMS), "\t[Kallsyms, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the kallsyms"); do_print((SUMMARY | KALLSYMS), "%d bytes]\n", size); read_dump_string(fd, size, KALLSYMS); + + uncompress_reset(); } static void dump_printk(int fd) @@ -306,12 +380,17 @@ static void dump_printk(int fd) do_print((SUMMARY | TRACE_PRINTK), "\t[Trace printk, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the trace printk"); do_print((SUMMARY | TRACE_PRINTK), "%d bytes]\n", size); read_dump_string(fd, size, TRACE_PRINTK); + + uncompress_reset(); } static void dump_cmdlines(int fd) @@ -320,12 +399,17 @@ static void dump_cmdlines(int fd) do_print((SUMMARY | CMDLINES), "\t[Saved command lines, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 8)) die("cannot read the size of the saved command lines"); do_print((SUMMARY | CMDLINES), "%d bytes]\n", size); read_dump_string(fd, size, CMDLINES); + + uncompress_reset(); } static void dump_cpus_count(int fd) @@ -669,6 +753,7 @@ static void dump_file(const char *file) do_print(SUMMARY, "\n Tracing meta data in file %s:\n", file); dump_initial_format(fd); + dump_compress(fd); dump_header_page(fd); dump_header_event(fd); dump_ftrace_events_format(fd); From patchwork Thu May 20 03:19:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLACK,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B739FC433ED for ; Thu, 20 May 2021 03:20:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9548E61186 for ; Thu, 20 May 2021 03:20:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230284AbhETDVo (ORCPT ); Wed, 19 May 2021 23:21:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVo (ORCPT ); Wed, 19 May 2021 23:21:44 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 63E03C061574 for ; Wed, 19 May 2021 20:20:22 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id s22so22734424ejv.12 for ; Wed, 19 May 2021 20:20:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KF+feKjMyKpX88QmGJlWIg3wMmDI0ahJDLjuPSqTtL0=; b=K9Vx88ZRakGDmVAD+xO72gCBs5L9KeIpNhbKYY3bqcEqV3RTEVA18m55ozHGDrk9iQ xRcQTLIDG2jZVpxpt2nnW43Xim0vZTrCe73LR4QmZaU1dSHXbMxCfh8DrpwHPbguzjCz 0sq1zFbxsgCG4phImoKq10AG+PaTz1T8FsYijRYVt7Rudmsz9TDSXqGdMZyvaNSKq1IV Nj8DxAKnYEDdf2Jk4qLF8C1my3Rfu44XrDEgtOUn1Z6Wk0tOuPLeqObdAUQ5txdzma3S DB4H/A4b7jdIY9yOafKHSgM6ethiEdoodWdBgOSUxBWEDb+lAh4ufWgj73ZMxDdwhP1O bRWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KF+feKjMyKpX88QmGJlWIg3wMmDI0ahJDLjuPSqTtL0=; b=QgJazrzLfYlb6G3CIBrk9jlFrx9iSShw+RRMILlEHkrikEBRC0tdYrp5Esihs2tGjx KhfAL+IxKxHjQoC2/eaLHuc5otHUYUjtIw/9vuFcsSeQHpLu2G8n4QsDI/EaT7ryeby7 ZeVO4S5MD3HpnzHD0N0E11wW+PP+agg1qKnl1YPgA1e/gCVPw6ASlo0h68BgMBHUXA3p rNvkcqUG05l/309IrWaLDybKAYxf61g5DPdRssRwa9hHPaDekvzEHUAN7z4iAkhXDmlU R5RiIlZKRCa9PsrSDd7qeHlKksL3pG8Ovw/yOUmC1DNlIrUfYtiMTFW0OvQeR66YNylI cxig== X-Gm-Message-State: AOAM5327CFl0P5gat1pKj+MY/HtdbgTRa14anATilQDTB124jTOy2d2b ddV9dDWM+QAOc369VfkeMYU= X-Google-Smtp-Source: ABdhPJwTlVWhwlGWPrnM/mkmMa8xm3T92kVn0dkzgwXMCW9IDFKbymGo2Pfy0TkSpBZqSPEFCLbPiQ== X-Received: by 2002:a17:906:2b4b:: with SMTP id b11mr2464712ejg.379.1621480821033; Wed, 19 May 2021 20:20:21 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:20 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 17/29] trace-cmd library: Add support for zlib compression library Date: Thu, 20 May 2021 06:19:47 +0300 Message-Id: <20210520031959.346165-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If libz is available, use that library to provide trace file compression support. The library is detected runtime. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 10 ++ lib/trace-cmd/include/trace-cmd-local.h | 5 + lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++++++++++++++++ lib/trace-cmd/trace-compress.c | 8 ++ 4 files changed, 195 insertions(+) create mode 100644 lib/trace-cmd/trace-compress-zlib.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index bab4322d..83ba7016 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -7,6 +7,13 @@ ldir:=$(src)/lib/trace-cmd DEFAULT_TARGET = $(LIBTRACECMD_STATIC) +pound := \# +ZLIB_INSTALLED := $(shell if (printf "$(pound)include \n void main(){deflateInit(NULL, Z_BEST_COMPRESSION);}" | $(CC) -o /dev/null -x c - -lz >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) +ifeq ($(ZLIB_INSTALLED), 1) +CFLAGS += -DHAVE_ZLIB +$(info Have zlib compression support) +endif + OBJS = OBJS += trace-hash.o OBJS += trace-hooks.o @@ -26,6 +33,9 @@ OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif OBJS += trace-compress.o +ifeq ($(ZLIB_INSTALLED), 1) +OBJS += trace-compress-zlib.o +endif # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 24228d8f..80304141 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -31,6 +31,11 @@ void tracecmd_info(const char *fmt, ...); #endif #endif +#ifdef HAVE_ZLIB +int tracecmd_zlib_init(void); +void tracecmd_zlib_free(void); +#endif + int tracecmd_compress_init(void); void tracecmd_compress_free(void); diff --git a/lib/trace-cmd/trace-compress-zlib.c b/lib/trace-cmd/trace-compress-zlib.c new file mode 100644 index 00000000..3208d57b --- /dev/null +++ b/lib/trace-cmd/trace-compress-zlib.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include + +#include "trace-cmd-private.h" + +#define __ZLIB_NAME "zlib" +#define __ZLIB_WEIGTH 10 +#define __ZLIB_FILE "libz.so" +#define ZLIB_FUNC_COMPRESS "compress2" +#define ZLIB_FUNC_DECOMOPRESS "uncompress" +#define ZLIB_FUNC_SIZE "compressBound" +#define ZLIB_FUNC_VERSION "zlibVersion" + +static void *zlib_handle; +static int (*_lib_compress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes, int level); +static int (*_libz_decompress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes); +static unsigned long (*_libz_compress_bound)(unsigned long in_bytes); +static const char *(*_libz_version)(void); + +static int zlib_compress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_lib_compress) + return -1; + + ret = _lib_compress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes, Z_BEST_COMPRESSION); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_STREAM_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static int zlib_decompress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_libz_decompress) + return -1; + + ret = _libz_decompress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_DATA_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static unsigned int zlib_compress_bound(unsigned int in_bytes) +{ + if (!_libz_compress_bound) + return 0; + return _libz_compress_bound(in_bytes); +} + +static bool zlib_is_supported(const char *name, const char *version) +{ + const char *zver; + + if (!name) + return false; + if (strlen(name) != strlen(__ZLIB_NAME) || strcmp(name, __ZLIB_NAME)) + return false; + + if (!version) + return true; + + if (!_libz_version) + return false; + zver = _libz_version(); + if (!zver) + return false; + + /* Compare the major version number */ + if (atoi(version) <= atoi(zver)) + return true; + + return false; +} + +int tracecmd_zlib_init(void) +{ + if (zlib_handle) + return 0; + + zlib_handle = dlopen(__ZLIB_FILE, RTLD_NOW | RTLD_GLOBAL); + if (!zlib_handle) + return -1; + _lib_compress = dlsym(zlib_handle, ZLIB_FUNC_COMPRESS); + if (!_lib_compress) + goto error; + _libz_decompress = dlsym(zlib_handle, ZLIB_FUNC_DECOMOPRESS); + if (!_libz_decompress) + goto error; + _libz_compress_bound = dlsym(zlib_handle, ZLIB_FUNC_SIZE); + if (!_libz_compress_bound) + goto error; + _libz_version = dlsym(zlib_handle, ZLIB_FUNC_VERSION); + if (!_libz_version) + goto error; + + return tracecmd_compress_proto_register(__ZLIB_NAME, _libz_version(), __ZLIB_WEIGTH, + zlib_compress, zlib_decompress, + zlib_compress_bound, zlib_is_supported); + +error: + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + dlclose(zlib_handle); + zlib_handle = NULL; + return -1; +} + +void tracecmd_zlib_free(void) +{ + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + + if (zlib_handle) { + dlclose(zlib_handle); + zlib_handle = NULL; + } + +} diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index ff742226..2b329d00 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -353,6 +353,10 @@ int tracecmd_compress_init(void) gettimeofday(&time, NULL); srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); +#ifdef HAVE_ZLIB + tracecmd_zlib_init(); +#endif + return 0; } @@ -530,6 +534,10 @@ void tracecmd_compress_free(void) struct compress_proto *proto = proto_list; struct compress_proto *del; +#ifdef HAVE_ZLIB + tracecmd_zlib_free(); +#endif + while (proto) { del = proto; proto = proto->next; From patchwork Thu May 20 03:19:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C620C433B4 for ; Thu, 20 May 2021 03:20:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5FD52611AB for ; Thu, 20 May 2021 03:20:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230289AbhETDVp (ORCPT ); Wed, 19 May 2021 23:21:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVp (ORCPT ); Wed, 19 May 2021 23:21:45 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53002C06175F for ; Wed, 19 May 2021 20:20:23 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id i7so5122795ejc.5 for ; Wed, 19 May 2021 20:20:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jog/mo/n8mLItmRaoO92yw0+0oR9z7o/2pPf0jRM2mI=; b=IFWmvdGact1/Yew21eUhejhiHrtxnTKlAjCoDOmCSyrjg9NJ2rcTdNvVnvEDxrNuxN 9qSsxFopymEXDJNqlGhef1O3slOgfStZ7mUrzkKMMJxQ3jzhUIfiSDbjxmw+ZHwecxKL vhX3gnJS2f60zUEJMhJjblPGASozrWfYAcf1d5HZQjrwqUJ8FGsQDML0UaTEkNCEpK4V 4tN/KOPDT1wNY2oX7p+tTsmE+0Ms9AvPYViBvBhn0CT+RjS1ROhAkLTjDjtP/ZTKsPQe RxeNlxPsOUsWtfE+AZU6wDr8ITyJkiNx7Fsdv59ez5vSYQSLT198l4t+JglVWe5AP0z1 zAQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Jog/mo/n8mLItmRaoO92yw0+0oR9z7o/2pPf0jRM2mI=; b=rhPXZGViNQrVqgqM6OPPPwXeLUci2dsUnV3o3eRSZC2cG2dGsABhMc5q/mE7cZnrWU DpfwNOU6Go37v/JOrsJ7MgQnahqc7qiMYlCAcW7m1BKOEsigGY6a9/fJOE1UI5udqcDm ev/oocyL9nMHoGHOZDw8J448+k0u2go96NDKdmlxhqHCjwGywmvJiiDfZpV0mO7Sfdgk GIcQZWedWRmP5YPGhZz667UtV/XxumZaSZvwBFLASbvax41qxTvxZHsblrOwAFjSr2IE ApCt/Joj1sEXQQm3KNnDEnHG3OEOhv0zBZXYfeyZUyPRf1+jHHNANpa6Bon/g9+EVrS7 8Qkg== X-Gm-Message-State: AOAM530exXfWyhaKwGmrMmYvqJ0+BUkL4C61xTCxEqyhL37I2q4VDZR4 rZaZf4Sw6B8502dYaaLD81s= X-Google-Smtp-Source: ABdhPJxyWE9jQXtjHe89WkutnynYvyEtmqP18JQOgBzOI8OpVxs5RprJQwYpDFRqs8YQ+dIjSUZFDA== X-Received: by 2002:a17:906:3b0f:: with SMTP id g15mr2420436ejf.308.1621480821961; Wed, 19 May 2021 20:20:21 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:21 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 18/29] trace-cmd library: Hide the logic for updating buffer offset Date: Thu, 20 May 2021 06:19:48 +0300 Message-Id: <20210520031959.346165-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a trace buffer data are written in the trace file, the buffer option in the file metadata is updated with the file offset of the tracing data. Hide this logic into the trace-cmd library. Added new APIs: tracecmd_add_buffer_description() tracecmd_write_buffers_description() Changed APIs: tracecmd_append_buffer_cpu_data() Removed APIs: tracecmd_add_buffer_option() This internal refactoring is needed for changes, related to compression of the options sections of the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 8 +- lib/trace-cmd/trace-output.c | 83 +++++++++++++++++-- tracecmd/trace-record.c | 16 ++-- 3 files changed, 84 insertions(+), 23 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ebd633e1..58f77524 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -295,8 +295,8 @@ struct tracecmd_option * tracecmd_add_option_v(struct tracecmd_output *handle, unsigned short id, const struct iovec *vector, int count); -struct tracecmd_option *tracecmd_add_buffer_option(struct tracecmd_output *handle, - const char *name, int cpus); +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus); +int tracecmd_write_buffers_description(struct tracecmd_output *handle); int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); @@ -312,9 +312,7 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files); - + const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index d575ea3b..7ec85e71 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -44,6 +44,14 @@ struct tracecmd_option { struct list_head list; }; +struct tracecmd_buffer { + int cpus; + void *name; + tsize_t offset; + struct tracecmd_option *option; + struct list_head list; +}; + enum { OUTPUT_FL_SEND_META = (1 << 0), }; @@ -63,6 +71,7 @@ struct tracecmd_output { struct tracecmd_compression *compress; struct list_head options; + struct list_head buffers; struct tracecmd_msg_handle *msg_handle; char *trace_clock; }; @@ -189,6 +198,7 @@ bool tracecmd_get_quiet(struct tracecmd_output *handle) void tracecmd_output_free(struct tracecmd_output *handle) { struct tracecmd_option *option; + struct tracecmd_buffer *buffer; if (!handle) return; @@ -199,6 +209,13 @@ void tracecmd_output_free(struct tracecmd_output *handle) if (handle->pevent) tep_unref(handle->pevent); + while (!list_empty(&handle->buffers)) { + buffer = container_of(handle->buffers.next, + struct tracecmd_buffer, list); + list_del(&buffer->list); + free(buffer->name); + free(buffer); + } while (!list_empty(&handle->options)) { option = container_of(handle->options.next, struct tracecmd_option, list); @@ -1075,6 +1092,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, goto out_free; list_head_init(&handle->options); + list_head_init(&handle->buffers); buf[0] = 23; buf[1] = 8; @@ -1373,9 +1391,8 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -struct tracecmd_option * -tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, - int cpus) +static struct tracecmd_option * +add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; @@ -1403,6 +1420,52 @@ tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, return option; } +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus) +{ + struct tracecmd_buffer *buf; + + buf = calloc(1, sizeof(struct tracecmd_buffer)); + if (!buf) + return -1; + buf->name = strdup(name); + if (!buf->name) { + free(buf); + return -1; + } + list_add_tail(&buf->list, &handle->buffers); + return 0; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + struct tracecmd_option *option; + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + option = add_buffer_option(handle, buf->name, buf->cpus); + if (!option) + return -1; + buf->option = option; + } + return 0; +} + +static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) +{ + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + if (strlen(name) == strlen(buf->name) && !strcmp(name, buf->name)) { + if (handle->file_version >= 7) + return buf->offset; + if (!buf->option) + break; + return buf->option->offset; + } + } + return 0; +} + int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; @@ -1647,18 +1710,23 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, } int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files) + const char *name, int cpus, char * const *cpu_data_files) { + tsize_t b_offset; tsize_t offset; stsize_t ret; + b_offset = get_buffer_file_offset(handle, name); + if (!b_offset) { + tracecmd_warning("Cannot find description for buffer %s\n", name); + return -1; + } offset = lseek64(handle->fd, 0, SEEK_CUR); /* Go to the option data, where will write the offest */ - ret = lseek64(handle->fd, option->offset, SEEK_SET); + ret = lseek64(handle->fd, b_offset, SEEK_SET); if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); + tracecmd_warning("could not seek to %lld\n", b_offset); return -1; } @@ -1717,6 +1785,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_file_version(ihandle); list_head_init(&handle->options); + list_head_init(&handle->buffers); if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index b71c9608..011e7b55 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -4152,7 +4152,6 @@ static void touch_file(const char *file) } static void append_buffer(struct tracecmd_output *handle, - struct tracecmd_option *buffer_option, struct buffer_instance *instance, char **temp_files) { @@ -4180,7 +4179,7 @@ static void append_buffer(struct tracecmd_output *handle, touch_file(temp_files[i]); } - tracecmd_append_buffer_cpu_data(handle, buffer_option, + tracecmd_append_buffer_cpu_data(handle, tracefs_instance_get_name(instance->tracefs), cpu_count, temp_files); for (i = 0; i < instance->cpu_count; i++) { @@ -4441,7 +4440,6 @@ static void write_guest_file(struct buffer_instance *instance) static void record_data(struct common_record_context *ctx) { - struct tracecmd_option **buffer_options; struct tracecmd_output *handle; struct buffer_instance *instance; bool local = false; @@ -4512,9 +4510,6 @@ static void record_data(struct common_record_context *ctx) } if (buffers) { - buffer_options = malloc(sizeof(*buffer_options) * buffers); - if (!buffer_options) - die("Failed to allocate buffer options"); i = 0; for_each_instance(instance) { int cpus = instance->cpu_count != local_cpu_count ? @@ -4522,10 +4517,9 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; - - buffer_options[i++] = tracecmd_add_buffer_option(handle, - tracefs_instance_get_name(instance->tracefs), - cpus); + tracecmd_add_buffer_description(handle, + tracefs_instance_get_name(instance->tracefs), + cpus); add_buffer_stat(handle, instance); } } @@ -4560,7 +4554,7 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; print_stat(instance); - append_buffer(handle, buffer_options[i++], instance, temp_files); + append_buffer(handle, instance, temp_files); } } From patchwork Thu May 20 03:19:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA333C43460 for ; Thu, 20 May 2021 03:20:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 90964611AB for ; Thu, 20 May 2021 03:20:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230295AbhETDVq (ORCPT ); Wed, 19 May 2021 23:21:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVq (ORCPT ); Wed, 19 May 2021 23:21:46 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8D78C061574 for ; Wed, 19 May 2021 20:20:24 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id t15so17654702edr.11 for ; Wed, 19 May 2021 20:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QlkQ7voEOme6xdAikBIIiDGC6PqFSEaNWoSrQB8V/78=; b=bbFu0eSnN2bCBIEUwddvo8tR1GMFVOHC8UqLUphWtK/0JJR6hNRDQQPOO859iYEamt Ah8lMqvA7IE6piVcjdU7P2lgCwBhEgzAmr49WJWNXuhNtIQ2NWcIoa8PQ8dUQZatwDWn lz0bXSrUQKmDOjD3+u6tQLztWVt+/u7RpaCLhhcoNTYYksrh9w8Ue0oZfBJDz9g5qRjs 2gRYnMkEtWk5wH1hOjV5X+4Myd64lGsv1n7vo82EfPTNK9piDQWwCOQoPHEfqxja7J5W nPvMaUr9fF14s2kamI3PkpitGdpHLJLv+v2B+ldPI4pIH1z1oxR40yagI/PBc4I7h1UZ JB9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QlkQ7voEOme6xdAikBIIiDGC6PqFSEaNWoSrQB8V/78=; b=Y2U3RRuapmlLpyA0o7PR2vHcmcECXrBq/itacRqovxqtgnqPJdx5qhKoRet0uEAgG9 RYcX1zEA5PPq0bJIg3h93z3LWxKh0qpkaBqvjbL0KDRykiApb3ajYWo5Zmlctn7jIrq2 ID0kBxjSD26xCA3YPrPlIRtqYRFNpT3iqcxAntbNRrizLwamrizNU+1EHzJA/bfn46Si kJ85pLK8XDjBk82z8tJvNplPhwd+xaYliLAwbtwx7HE7Ywpfd4ncfmRIyy9ZGu7fMqlI yKL/vDM2xKpLyNF6fnkehCMmge4QqjZ+g7CPIMTtZvlOAE5HtrqjMB8n03Umfg1o0IeD pLDg== X-Gm-Message-State: AOAM533F1LkbRMhqoDCUCPn/WUqYaPnoA4WwI/TAXG/mJU0BmRYAFN8j otZBUvxnm5Jg7vqWKxUuO3w= X-Google-Smtp-Source: ABdhPJxKdwtvGn3s0Tp0PpJ401qoSXEiUuJeai8uA+wG/i/WfGs9DP2dXQgcVU0aCghMsN5jGoBTHg== X-Received: by 2002:a05:6402:1046:: with SMTP id e6mr2559499edu.218.1621480823344; Wed, 19 May 2021 20:20:23 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:22 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 19/29] trace-cmd: Move buffers description outside of options Date: Thu, 20 May 2021 06:19:49 +0300 Message-Id: <20210520031959.346165-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new buffer data are written in the trace file, the file offset is stored in the buffer description. As the buffer description is part of the trace options, which are already written in the file, the offset field in this option is updated with the new value. This causes a serious problem when option section of the file is compressed. Updating a value in already compressed section may change the size of this section, which can make the calculated offset of the buffer trace data invalid. To solve that problem and to enable the compression of the option section of the file, the structure of the trace file is changed: - In trace file v7, the description of the buffers is moved to a separate section in the trace file medatata. It is stored after the CPU count section and before the option section. The format of this new buffers description is: 4 bytes - count of the buffers, stored in the file. The top buffer is not counted. For each buffer: 8 bytes, offset of the trace data for this buffer in the file. null-terminated string, name of the buffer. The new section is mandatory, in case of no buffers - the count 0 is stored in the first 4 bytes. - In trace file v6, the description of the buffers is not changed. These changes are needed for compression of the options sections from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 40 ++++++++++ lib/trace-cmd/trace-output.c | 78 ++++++++++++++++++- tracecmd/trace-dump.c | 24 ++++++ tracecmd/trace-listen.c | 3 + tracecmd/trace-record.c | 4 + 6 files changed, 146 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 58f77524..d67ec86e 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -112,6 +112,7 @@ enum tracecmd_file_states { TRACECMD_FILE_PRINTK, TRACECMD_FILE_CMD_LINES, TRACECMD_FILE_CPU_COUNT, + TRACECMD_FILE_BUFERS, TRACECMD_FILE_OPTIONS, TRACECMD_FILE_CPU_LATENCY, TRACECMD_FILE_CPU_FLYRECORD, diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index e32789a8..ec83ab09 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -881,6 +881,39 @@ static int read_cpus(struct tracecmd_input *handle) return 0; } +static int read_buffers(struct tracecmd_input *handle) +{ + struct input_buffer_instance *buffer; + unsigned long long offset; + unsigned int count; + int i; + + if (handle->file_version < 7 || handle->file_state >= TRACECMD_FILE_BUFERS) + return 0; + + if (read4(handle, &count) < 0) + return -1; + i = handle->nr_buffers; + handle->nr_buffers += count; + handle->buffers = realloc(handle->buffers, + sizeof(*handle->buffers) * handle->nr_buffers); + if (!handle->buffers) + return -1; + + for (; i < handle->nr_buffers; i++) { + buffer = &handle->buffers[i]; + if (read8(handle, &offset) < 0) + return -1; + buffer->offset = offset; + buffer->name = read_string(handle); + if (!buffer->name) + return -1; + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +} + /** * tracecmd_read_headers - read the header information from trace.dat * @handle: input handle for the trace.dat file @@ -955,6 +988,13 @@ int tracecmd_read_headers(struct tracecmd_input *handle, if (state <= handle->file_state) return 0; + if (read_buffers(handle) < 0) + return -1; + + if (state <= handle->file_state) + return 0; + + if (read_options_type(handle) < 0) return -1; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 7ec85e71..580882eb 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -376,6 +376,17 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) if (!handle) return -1; + if (handle->file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, handle->file_version); + return -1; + default: + break; + } + } + switch (new_state) { case TRACECMD_FILE_HEADERS: case TRACECMD_FILE_FTRACE_EVENTS: @@ -384,10 +395,16 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) case TRACECMD_FILE_PRINTK: case TRACECMD_FILE_CMD_LINES: case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_OPTIONS: + case TRACECMD_FILE_BUFERS: if (handle->file_state == (new_state - 1)) return 0; break; + case TRACECMD_FILE_OPTIONS: + if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) + return 0; + if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) + return 0; + break; case TRACECMD_FILE_CPU_LATENCY: case TRACECMD_FILE_CPU_FLYRECORD: if (handle->file_state == TRACECMD_FILE_OPTIONS) @@ -1436,7 +1453,7 @@ int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char * return 0; } -int tracecmd_write_buffers_description(struct tracecmd_output *handle) +static int write_buffers_description_v6(struct tracecmd_output *handle) { struct tracecmd_option *option; struct tracecmd_buffer *buf; @@ -1450,6 +1467,56 @@ int tracecmd_write_buffers_description(struct tracecmd_output *handle) return 0; } +static int write_buffers_description_v7(struct tracecmd_output *handle) +{ + struct tracecmd_buffer *buf; + char *data = NULL; + int count = 0; + int size; + int ret; + + ret = check_out_state(handle, TRACECMD_FILE_BUFERS); + if (ret < 0) { + tracecmd_warning("Cannot write buffers descriptions"); + return ret; + } + + list_for_each_entry(buf, &handle->buffers, list) { + count++; + } + + count = convert_endian_4(handle, count); + ret = do_write_check(handle, &count, 4); + if (ret < 0) + return ret; + + list_for_each_entry(buf, &handle->buffers, list) { + size = 8 + strlen(buf->name) + 1; + data = malloc(size); + if (!data) + goto error; + *(tsize_t *)data = 0; + strcpy(data + 8, buf->name); + /* Save the data location in case it needs to be updated */ + buf->offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, data, size)) + goto error; + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +error: + return -1; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + if (handle->file_version >= 7) + return write_buffers_description_v7(handle); + + return write_buffers_description_v6(handle); +} + static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) { struct tracecmd_buffer *buf; @@ -1508,7 +1575,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_cpus(handle, cpus) < 0) goto out_free; - + if (tracecmd_write_buffers_description(handle) < 0) + goto out_free; if (tracecmd_write_options(handle) < 0) goto out_free; @@ -1701,7 +1769,9 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, ret = tracecmd_write_cpus(handle, cpus); if (ret) return ret; - + ret = tracecmd_write_buffers_description(handle); + if (ret) + return ret; ret = tracecmd_write_options(handle); if (ret) return ret; diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index ed82104b..56acb01d 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -420,6 +420,29 @@ static void dump_cpus_count(int fd) do_print(SUMMARY, "\t%d [CPUs with tracing data]\n", trace_cpus); } +static void dump_buffers(int fd) +{ + unsigned long long offset; + char name[DUMP_SIZE]; + int count; + + if (file_version < 7) + return; + + if (read_file_number(fd, &count, 4)) + die("cannot read the buffers count"); + do_print(SUMMARY, "\t%d [buffers with tracing data]\n", count); + + while (count) { + if (read_file_number(fd, &offset, 8)) + die("cannot read the buffer offset"); + if (read_file_string(fd, name, DUMP_SIZE)) + die("cannot read the buffer name"); + do_print(SUMMARY, "\t\t 0x%llx %s[data offset, buffer name]\n", offset, name); + count--; + } +} + static void dump_option_string(int fd, int size, char *desc) { do_print(OPTIONS, "\t\t[Option %s, %d bytes]\n", desc, size); @@ -762,6 +785,7 @@ static void dump_file(const char *file) dump_printk(fd); dump_cmdlines(fd); dump_cpus_count(fd); + dump_buffers(fd); dump_therest(fd); tep_free(tep); diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index 0ae1c948..62debdb6 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -604,6 +604,9 @@ static int put_together_file(int cpus, int ofd, const char *node, if (write_options) { ret = tracecmd_write_cpus(handle, cpus); + if (ret) + goto out; + ret = tracecmd_write_buffers_description(handle); if (ret) goto out; ret = tracecmd_write_options(handle); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 011e7b55..6d1b45b5 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3711,6 +3711,9 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; ret = tracecmd_write_cpus(network_handle, instance->cpu_count); + if (ret) + goto error; + ret = tracecmd_write_buffers_description(network_handle); if (ret) goto error; ret = tracecmd_write_options(network_handle); @@ -4074,6 +4077,7 @@ static void setup_agent(struct buffer_instance *instance, add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_buffers_description(network_handle); tracecmd_write_options(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); instance->network_handle = network_handle; From patchwork Thu May 20 03:19:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F969C433ED for ; Thu, 20 May 2021 03:20:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 04DD461186 for ; Thu, 20 May 2021 03:20:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230298AbhETDVr (ORCPT ); Wed, 19 May 2021 23:21:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230267AbhETDVr (ORCPT ); Wed, 19 May 2021 23:21:47 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B165C06175F for ; Wed, 19 May 2021 20:20:25 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id lz27so22915136ejb.11 for ; Wed, 19 May 2021 20:20:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gMo8KC9sNILqewOKWuCmRF3qHF+EaLVS8tEPPLTTOtM=; b=YGgP7yLtM6N2qkK66XPuL2e7jzAjL+UNqImN6112FJP2giotWztPSiHmfJF4QTU09a nZAWCqorR5ov6JKbiPXp7gQiDdS1Vb43zFdaldYT6XEL7GG55m6+SuW0BOPoZ7vMDmtj plx4vI7Sms4Vk3Tl9KV3IcKsjd06lljVuTMKawcBzMKh0Sl6Vmvoia7CQH1ynR1iyHXi mm4c7evEA8tA4wQjXveLF91RQ1aWesn0vV1/R5riEFDk1O1tD0Rmb2f39Fg9orsXyWuY O+ry2B79AncpHYDTwaAWYK6UKWrH29lMqNGCtca0ce862Ate6FlsGOCoDSnrAN1nfcfG Yofg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gMo8KC9sNILqewOKWuCmRF3qHF+EaLVS8tEPPLTTOtM=; b=ju/kbGSuGzIR0oIrLz7+Ou3LUmgR71KsQZBsnahnAseU/kHTUiqAG691nTJMTnJlab sgT5jWGo+Ux2T695qUuJsFcSJrnEJ+gYDZu/cjA436yJV9wX0cHbvAhrjoJuILce8/eR iwWQ19yTdE+fH5Z4a5/jakd4cOYBi8vFcNKSmjPF5BvLo+4E8ueJTOXorP1YEJ7m/acE lhCLu65E7QX6pZjvX/ottq/4ZCHEcQjsavFVc448ZqJNpVads4WDspCtto+hEcq1pDek +WLAnVEhsq4M6xgDwBBK0zvZlg+nmAAG8E78WftbeLrZZ8V91e+62+ned+EOs+Y/sx/y VwRw== X-Gm-Message-State: AOAM530BXRto1WDcifw0kb1np5Us9eP2znUDvu/0ojElX9fAaitBMeYr c9Y481fpDknXKvtJD5tXBBg= X-Google-Smtp-Source: ABdhPJyyEdxDtFkX9kJGZXDfF/mUy+NX2/kWUDLONekfdf2HQKMJh+bE7iEbXrmhm0yXDIXtJ1bd2Q== X-Received: by 2002:a17:906:458:: with SMTP id e24mr2497270eja.199.1621480824206; Wed, 19 May 2021 20:20:24 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:23 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 20/29] trace-cmd library: Track the offset in the option section in the trace file Date: Thu, 20 May 2021 06:19:50 +0300 Message-Id: <20210520031959.346165-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading / writing trace file, store the offset of the option section in the file. A new internal API is added, to get the offset of the option section from an input file handler: tracecmd_get_options_offset() These changes are needed for compression of the option section from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 11 +++++++++++ lib/trace-cmd/trace-output.c | 2 ++ 3 files changed, 14 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index d67ec86e..2c28ee04 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -239,6 +239,7 @@ tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_file_version(struct tracecmd_input *handle); int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, const char **name, const char **version); +size_t tracecmd_get_options_offset(struct tracecmd_input *handle); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index ec83ab09..ee3c23f2 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -157,6 +157,7 @@ struct tracecmd_input { size_t header_files_start; size_t ftrace_files_start; size_t event_files_start; + size_t options_start; size_t total_file_size; /* For custom profilers. */ @@ -2759,6 +2760,7 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); for (;;) { if (do_read_check(handle, &option, 2)) @@ -4213,6 +4215,15 @@ bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle) return handle->use_trace_clock; } +/** + * tracecmd_get_options_offset - get offset of the options sections in the file + * @handle: input handle for the trace.dat file + */ +size_t tracecmd_get_options_offset(struct tracecmd_input *handle) +{ + return handle->options_start; +} + /** * tracecmd_get_show_data_func - return the show data func * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 580882eb..fd899c45 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -66,6 +66,7 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + size_t options_start; bool do_compress; struct tracecmd_compression *compress; @@ -1854,6 +1855,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_file_version(ihandle); + handle->options_start = tracecmd_get_options_offset(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Thu May 20 03:19:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E548C433B4 for ; Thu, 20 May 2021 03:20:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DB45611AD for ; Thu, 20 May 2021 03:20:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230267AbhETDVs (ORCPT ); Wed, 19 May 2021 23:21:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230312AbhETDVr (ORCPT ); Wed, 19 May 2021 23:21:47 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9E39C061574 for ; Wed, 19 May 2021 20:20:26 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id p24so21697466ejb.1 for ; Wed, 19 May 2021 20:20:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eVP19EAu60UTBvEyGCDlP/ZJjeTqckM+eu5aYBDPDWY=; b=ComKlmhJ704R7bhNix7eZTnqDC/TF/BWuXqTlBJlkDGOFwsAJ+/Wx4ZbzuOhf+mVqo /NED5yZJ85TSXBgeUEiTwBMGQemjIrIZmkApk+PsKKqKnFb7KKy1N8rS//bwnhXRaaeo iUXEfBFcLLxTxpJ/FNSs8i+SaKGv94FJp4FzsKFMs8zFLoaQPPV0kFofiQdkapnXcvu5 FW2TMhDZQNctpPxAQZwr6AiAyVLP8UvQmM7NRXTc8baQ4ldcSJxzZk7lNxN0qkcIUJ+f mEPFI1NHflYZbUherUpK5s/+sC6CCuY72f3eI42K9Vdf8by7FZvV79G1eoT47tVHZddT yhig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eVP19EAu60UTBvEyGCDlP/ZJjeTqckM+eu5aYBDPDWY=; b=ZkeeaGqTG/MAjm9NjgtkIOMA5cLe1Nxnt8ueozbhQSGuuv3tFZ6IxOKz1kAUhF8mrL 3pmG79agg7TxpWmWtnnAecVhiTqMaWC+wwJhbsWEFr6aUg8k7Iyu3IAu/s99uW/avoGP pE3atpN5WJhKvyleGaUQFvRLSssc0LqEa54A71rntnkUzlZ4PeK3De5RaIUeJ8WLmPFZ 8nHf9y7nSNvVhrS5i084fh67UAMBQk/2N9D9/EOAP/Z9bbZ4ZsXs+ehTBA6Wajs1E3bh c13Wf6zLPYL+LHIZ01wmJ5p8yWxjH1t6WjTi1nHBsDEhiRFNsRTKSLBZ/7aQ7ydPNFwA NieA== X-Gm-Message-State: AOAM533LJMzLsHpit/owgIMqP7ZhmlMuaDvg7kRDSy1K61JuqBdP9oMK hXImpIJ7wkUNiryN5J/UtDk= X-Google-Smtp-Source: ABdhPJx7EaIE/+DTnh/7cAi8bKs/djTFXrcTDK/COiBJMMeMxp73QxfAU8LmypFSeTdK9WDV4kofrg== X-Received: by 2002:a17:906:5211:: with SMTP id g17mr2417582ejm.281.1621480825513; Wed, 19 May 2021 20:20:25 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:24 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 21/29] trace-cmd library: Add compression of the option section of the trace file Date: Thu, 20 May 2021 06:19:51 +0300 Message-Id: <20210520031959.346165-22-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Comperss the option section of the trace file. This section is not big currently and compressing it does not reduce significantly the size of the file. This could be useful in the future as new options can be added, storing a potentially huge amount of data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 46 +++++++++++------ lib/trace-cmd/trace-output.c | 68 +++++++++++++++++-------- tracecmd/trace-dump.c | 16 ++++-- 4 files changed, 90 insertions(+), 41 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 80304141..e579a669 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,7 @@ void tracecmd_zlib_free(void); int tracecmd_compress_init(void); void tracecmd_compress_free(void); +int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); void out_compression_reset(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index ee3c23f2..f94ceab4 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2761,23 +2761,29 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); - + if (in_uncompress_block(handle)) + return -1; for (;;) { - if (do_read_check(handle, &option, 2)) - return -1; + ret = do_read_check(handle, &option, 2); + if (ret) + goto out; if (option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (do_read_check(handle, &size, 4)) - return -1; + ret = do_read_check(handle, &size, 4); + if (ret) + goto out; size = tep_read_number(handle->pevent, &size, 4); buf = malloc(size); - if (!buf) - return -ENOMEM; - if (do_read_check(handle, buf, size)) - return -1; + if (!buf) { + ret = -ENOMEM; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret) + goto out; switch (option) { case TRACECMD_OPTION_DATE: @@ -2827,14 +2833,16 @@ static int handle_options(struct tracecmd_input *handle) buf + 8, 4); ret = tsync_cpu_offsets_load(handle, buf + 12, size - 12); if (ret < 0) - return ret; + goto out; tracecmd_enable_tsync(handle, true); break; case TRACECMD_OPTION_CPUSTAT: buf[size-1] = '\n'; cpustats = realloc(cpustats, cpustats_size + size + 1); - if (!cpustats) - return -ENOMEM; + if (!cpustats) { + ret = -ENOMEM; + goto out; + } memcpy(cpustats + cpustats_size, buf, size); cpustats_size += size; cpustats[cpustats_size] = 0; @@ -2844,14 +2852,17 @@ static int handle_options(struct tracecmd_input *handle) handle->nr_buffers++; handle->buffers = realloc(handle->buffers, sizeof(*handle->buffers) * handle->nr_buffers); - if (!handle->buffers) - return -ENOMEM; + if (!handle->buffers) { + ret = -ENOMEM; + goto out; + } buffer = &handle->buffers[handle->nr_buffers - 1]; buffer->name = strdup(buf + 8); if (!buffer->name) { free(handle->buffers); handle->buffers = NULL; - return -ENOMEM; + ret = -ENOMEM; + goto out; } offset = *(unsigned long long *)buf; buffer->offset = tep_read_number(handle->pevent, &offset, 8); @@ -2908,8 +2919,11 @@ static int handle_options(struct tracecmd_input *handle) } handle->cpustats = cpustats; + ret = 0; - return 0; +out: + in_uncompress_reset(handle); + return ret; } static int read_options_type(struct tracecmd_input *handle) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index fd899c45..c1d86478 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -150,6 +150,18 @@ __hidden void out_compression_reset(struct tracecmd_output *handle) handle->do_compress = false; } +__hidden int out_uncompress_block(struct tracecmd_output *handle) +{ + int ret = 0; + + if (handle->file_version < 7 || !handle->compress) + return 0; + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->do_compress = true; + return ret; +} + __hidden int out_compression_start(struct tracecmd_output *handle) { if (handle->file_version < 7 || !handle->compress) @@ -1330,32 +1342,38 @@ int tracecmd_write_options(struct tracecmd_output *handle) if (do_write_check(handle, "options ", 10)) return -1; - + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + out_compression_start(handle); list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; + goto error; + if (out_compression_end(handle)) + goto error; handle->file_state = TRACECMD_FILE_OPTIONS; return 0; +error: + out_compression_reset(handle); + return -1; } int tracecmd_append_options(struct tracecmd_output *handle) @@ -1371,42 +1389,48 @@ int tracecmd_append_options(struct tracecmd_output *handle) * We can append only if options are already written and tracing data * is not yet written */ - if (handle->file_state != TRACECMD_FILE_OPTIONS) + if (handle->file_state != TRACECMD_FILE_OPTIONS || !handle->options_start) return -1; - - if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) return -1; - offset = lseek64(handle->fd, -2, SEEK_CUR); - if (offset == (off_t)-1) + if (out_uncompress_block(handle)) return -1; - - r = pread(handle->fd, &option, 2, offset); + if (do_lseek(handle, 0, SEEK_END) == -1) + goto error; + offset = do_lseek(handle, -2, SEEK_CUR); + if (offset == (off_t)-1) + goto error; + r = do_preed(handle, &option, 2, offset); if (r != 2 || option != TRACECMD_OPTION_DONE) - return -1; - + goto error; list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } - option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; - + goto error; + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) + goto error; + if (out_compression_end(handle)) + goto error; return 0; +error: + out_compression_reset(handle); + return -1; } static struct tracecmd_option * diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 56acb01d..8d0f2251 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -82,6 +82,13 @@ static int read_compressed(int fd, char *dst, int len) return read_fd(fd, dst, len); } +static int do_lseek(int fd, int offset, int whence) +{ + if (read_compress) + return tracecmd_compress_lseek(compress, offset, whence); + return lseek64(fd, offset, whence); +} + static int read_file_string(int fd, char *dst, int len) { size_t size = 0; @@ -625,6 +632,9 @@ static void dump_options(int fd) unsigned int size; int count = 0; + if (uncompress_block()) + die("cannot uncompress file block"); + for (;;) { if (read_file_number(fd, &option, 2)) die("cannot read the option id"); @@ -635,7 +645,7 @@ static void dump_options(int fd) count++; if (!DUMP_CHECK(OPTIONS) && !DUMP_CHECK(CLOCK) && !DUMP_CHECK(SUMMARY)) { - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); continue; } switch (option) { @@ -685,12 +695,12 @@ static void dump_options(int fd) default: do_print(OPTIONS, " %d %d\t[Unknown option, size - skipping]\n", option, size); - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); break; } } do_print(SUMMARY, "\t[%d options]\n", count); - + uncompress_reset(); } static void dump_latency(int fd) From patchwork Thu May 20 03:19:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 357FFC433ED for ; Thu, 20 May 2021 03:20:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DA8A611AD for ; Thu, 20 May 2021 03:20:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230312AbhETDVu (ORCPT ); Wed, 19 May 2021 23:21:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhETDVt (ORCPT ); Wed, 19 May 2021 23:21:49 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3D73C061574 for ; Wed, 19 May 2021 20:20:27 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id l1so22955184ejb.6 for ; Wed, 19 May 2021 20:20:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DT2olDRkYdkyAh5O82MnbG1y8BJPXtJuXNqmEuufDK0=; b=XEN8Ux6+CMqkKzqHeqIvPZpxWJIFCWGlS7eWmWTVzsDVfxQ6SdgUdsuscP0InK6zCW 0+Kj32MZ57rYRFlgP/hfWgnYqcR2JWBoUab+EtANvX8Lg7Smui1cvhtjb5DXt1hb0wsl UCqr3i2dZGkHhsb+WegjIUOuC6tye2fSRYMhE8qPlrYf0770BIFbtMerfJnFThaUEmPK R/Ekf6t584QcFaJPkP+gMzm1VfzMKty+uKVGWQx6Kx7CJQilBGWewO6+1D+acvVy6CCs lLwqbJpNuWqqVEwEY5btQHBDuyMnGqpn8RowFsVEtc0uiQ0oKS0P3WKi78hg2S22JxbD KjkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DT2olDRkYdkyAh5O82MnbG1y8BJPXtJuXNqmEuufDK0=; b=ekATVpR369AxBdnpypZDyGcd85F0ryG82ToyAekcC7aRpuqU1jhvHPnOaJZ8mUo/CZ /S6sIl8PVKfsNjt4bf0esgcwOFp1g90nEKCyVI1lZ0FT1/QbV9wPiDrCQH3eFdQ6SCTX L+Oa2kmokRmA7qtUOmIrwvydk0fjH0ePv8lsWil5myi1sK/HJPX8ZabX+7SjG/8CjdAH GumR9H7MlLSAsED3UQxySzpUGYsR4ngWqkJCsPsNMswsabLIYXcC08Tqi6+Q1hkSTH5q ELlc8keZ3yRxIK8kieWpIhRYSO8vWaahbMWwL2sRO0MqMdCmtiPP78InDJuYT2tpat35 o40A== X-Gm-Message-State: AOAM533YUiRMVaIJYNJbqYsi6nn48i6cTxmfIQMCJLPN84I5ONgK6hTR DbQeL5Oc82exzzMldlWy6pk= X-Google-Smtp-Source: ABdhPJx3vwU0z/HfNGVkDJwp5IgKxNLj+JaJu9Qk/pfpjWWvv4+B46V8r435GhNpUOOPPqtVQSYf6g== X-Received: by 2002:a17:907:980c:: with SMTP id ji12mr2398683ejc.152.1621480826425; Wed, 19 May 2021 20:20:26 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:25 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 22/29] trace-cmd library: Refactor the logic for writing trace data in the file Date: Thu, 20 May 2021 06:19:52 +0300 Message-Id: <20210520031959.346165-23-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Refactored the internal logic of tracecmd_write_cpu_data() API to be suitable for adding trace data compression. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 94 +++++++++++++++++------------------- 1 file changed, 44 insertions(+), 50 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c1d86478..046fa2d7 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1672,11 +1672,17 @@ static char *get_clock(struct tracecmd_output *handle) return handle->trace_clock; } +struct data_file_write { + int file_size; + off64_t soffset; + off64_t data_offset; + off64_t doffset; +}; + int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { - off64_t *offsets = NULL; - unsigned long long *sizes = NULL; + struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; char *clock = NULL; @@ -1698,37 +1704,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (do_write_check(handle, "flyrecord", 10)) goto out_free; - offsets = malloc(sizeof(*offsets) * cpus); - if (!offsets) - goto out_free; - sizes = malloc(sizeof(*sizes) * cpus); - if (!sizes) - goto out_free; - - offset = lseek64(handle->fd, 0, SEEK_CUR); - - /* hold any extra data for data */ - offset += cpus * (16); - - /* - * Unfortunately, the trace_clock data was placed after the - * cpu data, and wasn't accounted for with the offsets. - * We need to save room for the trace_clock file. This means - * we need to find the size of it before we define the final - * offsets. - */ - clock = get_clock(handle); - if (!clock) + data_files = calloc(cpus, sizeof(struct data_file_write)); + if (!data_files) goto out_free; - /* Save room for storing the size */ - offset += 8; - offset += strlen(clock); - /* 2 bytes for [] around the clock */ - offset += 2; - - /* Page align offset */ - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + /* Write 0 for trace data offset and size and store offsets of these fields */ for (i = 0; i < cpus; i++) { file = cpu_data_files[i]; ret = stat(file, &st); @@ -1736,53 +1716,67 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, tracecmd_warning("can not stat '%s'", file); goto out_free; } - offsets[i] = offset; - sizes[i] = st.st_size; - offset += st.st_size; - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].file_size = st.st_size; - endian8 = convert_endian_8(handle, offsets[i]); + endian8 = 0; + data_files[i].doffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; - endian8 = convert_endian_8(handle, sizes[i]); + data_files[i].soffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; } - if (save_clock(handle, clock)) + clock = get_clock(handle); + if (clock && save_clock(handle, clock)) goto out_free; for (i = 0; i < cpus; i++) { + data_files[i].data_offset = lseek64(handle->fd, 0, SEEK_CUR); + /* Page align offset */ + data_files[i].data_offset = (data_files[i].data_offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].data_offset = lseek64(handle->fd, data_files[i].data_offset, SEEK_SET); + if (data_files[i].data_offset == (off64_t)-1) + goto out_free; if (!tracecmd_get_quiet(handle)) fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", - i, (unsigned long long) offsets[i]); - offset = lseek64(handle->fd, offsets[i], SEEK_SET); - if (offset == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offsets[i]); - goto out_free; - } + i, (unsigned long long) data_files[i].data_offset); + check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != sizes[i]) { + if (check_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, sizes[i]); + check_size, data_files[i].file_size); goto out_free; } + /* Write the real CPU data offset inthe file */ + offset = lseek64(handle->fd, data_files[i].doffset, SEEK_SET); + endian8 = convert_endian_8(handle, data_files[i].data_offset); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + /* Write the real CPU data size in the file */ + offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); + endian8 = convert_endian_8(handle, check_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + check_size; + offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", (unsigned long long)check_size); } - free(offsets); - free(sizes); + if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) + goto out_free; + free(data_files); handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; return 0; out_free: - free(offsets); - free(sizes); + lseek64(handle->fd, 0, SEEK_END); + free(data_files); return -1; } From patchwork Thu May 20 03:19:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F36F1C43460 for ; Thu, 20 May 2021 03:20:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9AFE61186 for ; Thu, 20 May 2021 03:20:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230325AbhETDVu (ORCPT ); Wed, 19 May 2021 23:21:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230312AbhETDVt (ORCPT ); Wed, 19 May 2021 23:21:49 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9416C06175F for ; Wed, 19 May 2021 20:20:28 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id lg14so22930606ejb.9 for ; Wed, 19 May 2021 20:20:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3SuOgqaLVIFlZcKjsmKN3FpRA1xd7G9h6G1W6+B2JeA=; b=PZUp5UKNFH995leNxVsPFNvP2oo541nnHi9aRVIwPYpSoLgOF/CajHhchnziS1f9ve AyPejXhbj24FiQnIlzo3gG8vwdA+bWKfKZyfPOJ50xQ1S10cLnTi9KGWSDAnda3Au/kr 70JckuRmVbQ0805anlXYjgokOVLnzhGgel/fx9XLZzds5d69Afg3Dug8HGfWvGLUolQ9 u2zyTBJYpaUgFqaktB4hh54IOL/tCOEamZ2hog5pNFnTrYNsMoxdPXneKmIQEEpbpdc+ lwcyVa0ASoHVlKfltrKFNkZC+BZ8r9pjZ7A+VB2r822YXl41yWV32ancwDG60JnO9QYv 1UDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3SuOgqaLVIFlZcKjsmKN3FpRA1xd7G9h6G1W6+B2JeA=; b=IstKBIuC5BbCq3vNZ0ADjhOqFGD/7Ic6iHRHX0Vb5bAZktsu1N9wro5aaQqXdk3FCN L++HtrYXsD+kE6G3EOLf2884KezeJBcegQyhDGdl6syBICsZyOnMqWml42ZLfhQT7qIA jE7XN3eRQ4szR+y/lx2J1Udjgh+RyJNyC3+258mTygf1BUJlSEeDA1Ue3KcYLi+lOkLn hrFmhPN2C8debnLZdVe9kCk6v2aQpxeqLl0mWoMFVmK8DjpC1RzIZY9wmxzc54kXd/n+ 0zeGpE1cNKAiinZgjktQ1Ix9zEy8HvAZLix62Dt2E6TFKQKpRoaAigc1gtG499pTbi4J nv0g== X-Gm-Message-State: AOAM531gIsp1aX5dIKBAZXejX6sU9QlsFnq6MKghRKYjXtvaNFx/FSJL KU5VzO2Gdjp0xqyjqXq0EO4= X-Google-Smtp-Source: ABdhPJxdOVK7YdU78RvIWtCgFrpRxfZ+raVGK9IIzqYEmtY/IcIncuteKKI0gWRrWEhvZ2deNo4GbA== X-Received: by 2002:a17:906:4c8c:: with SMTP id q12mr2459440eju.254.1621480827468; Wed, 19 May 2021 20:20:27 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:26 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 23/29] trace-cmd library: Add APIs for read and write compressed data in chunks Date: Thu, 20 May 2021 06:19:53 +0300 Message-Id: <20210520031959.346165-24-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New compression APIs are added for writing and reading compressed data from / to files in chunks: tracecmd_compress_copy_from() tracecmd_uncompress_copy_to() Format of the compressed data, used by these APIs, is: - 4 bytes, chunks count - for each chunk: - 4 bytes, size of compressed data in this chunk - 4 bytes, uncompressed size of the data in this chunk - data, bytes of Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 +- lib/trace-cmd/trace-compress.c | 191 ++++++++++++++++++ 2 files changed, 195 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 2c28ee04..19dc9e05 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -493,7 +493,10 @@ int tracecmd_compress_proto_register(const char *name, const char *version, int char *out, unsigned int *out_bytes), unsigned int (*comress_size)(unsigned int bytes), bool (*is_supported)(const char *name, const char *version)); - +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size); /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index 2b329d00..3a7accc2 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -594,3 +594,194 @@ error: free(v); return -1; } + +/** + * tracecmd_compress_copy_from - Copy and compress data from a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data + * @chunk_size: size of one compression chunk + * @read_size: return, size of the uncompressed data read from @fd + * @write_size: return, size of the compressed data written into @handle + * + * This function reads uncompressed data from given @fd until EOF is reached, compresses + * the data using the @handle compression context and writes the compressed data into the fd + * associated with the @handle. The data is compressed on chunks with given @chunk_size size. + * The compressed data is written in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int chunks = 0; + unsigned int rsize = 0; + unsigned int wsize = 0; + unsigned int csize; + unsigned int size; + unsigned int r; + off64_t offset; + char *buf_from; + char *buf_to; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->comress_block || !handle->proto->comress_size) + return 0; + + csize = handle->proto->comress_size(chunk_size); + buf_from = malloc(chunk_size); + if (!buf_from) + return -1; + buf_to = malloc(csize); + if (!buf_to) + return -1; + /* save the initial offset and write 0 chunks */ + offset = lseek64(handle->fd, 0, SEEK_CUR); + write_fd(handle->fd, &chunks, 4); + + do { + r = read(fd, buf_from, chunk_size); + if (r < 0) + break; + rsize += r; + size = csize; + if (r > 0) { + ret = handle->proto->comress_block(buf_from, r, buf_to, &size); + if (ret < 0) { + if (errno == EINTR) + continue; + break; + } + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &r, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write the compressed data */ + ret = write_fd(handle->fd, buf_to, size); + if (ret != size) + break; + /* data + compress header */ + wsize += (size + 8); + chunks++; + } + } while (r > 0); + free(buf_from); + free(buf_to); + if (r) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + endian4 = tep_read_number(handle->tep, &chunks, 4); + /* write chunks count*/ + write_fd(handle->fd, &chunks, 4); + lseek64(handle->fd, offset, SEEK_SET); + if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} + +/** + * tracecmd_uncompress_copy_to - Uncompress data and copy to a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data + * @read_size: return, size of the compressed data read from @handle + * @write_size: return, size of the uncompressed data written into @fd + * + * This function reads compressed data from the fd, associated with @handle, uncompresses it + * using the @handle compression context and writes the uncompressed data into the fd. + * The compressed data must be in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + unsigned int rsize = 0; + unsigned int wsize = 0; + char *bytes_out = NULL; + char *bytes_in = NULL; + int size_out; + int size_in; + int chunks; + char buf[4]; + char *tmp; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + chunks = tep_read_number(handle->tep, buf, 4); + rsize += 4; + while (chunks) { + if (read(handle->fd, buf, 4) != 4) + break; + s_compressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (read(handle->fd, buf, 4) != 4) + break; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (!bytes_in || size_in < s_compressed) { + tmp = realloc(bytes_in, s_compressed); + if (!tmp) + break; + bytes_in = tmp; + size_in = s_compressed; + } + + if (!bytes_out || size_out < s_uncompressed) { + tmp = realloc(bytes_out, s_uncompressed); + if (!tmp) + break; + bytes_out = tmp; + size_out = s_uncompressed; + } + + if (read_fd(handle->fd, bytes_in, s_compressed) < 0) + break; + rsize += s_compressed; + ret = handle->proto->uncompress_block(bytes_in, s_compressed, + bytes_out, &s_uncompressed); + if (ret) + break; + write_fd(fd, bytes_out, s_uncompressed); + wsize += s_uncompressed; + chunks--; + } + free(bytes_in); + free(bytes_out); + if (chunks) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} From patchwork Thu May 20 03:19:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A39B3C433B4 for ; Thu, 20 May 2021 03:20:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8908B611AB for ; Thu, 20 May 2021 03:20:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230333AbhETDVv (ORCPT ); Wed, 19 May 2021 23:21:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230330AbhETDVv (ORCPT ); Wed, 19 May 2021 23:21:51 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2D53C061574 for ; Wed, 19 May 2021 20:20:29 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id l1so22955244ejb.6 for ; Wed, 19 May 2021 20:20:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eOA7n18lW1M1FakMU6/vCmTAO3qDfgKSz00bVksdClA=; b=h9O/7iQPFiMN1Jw00q4T0jDbwojvpHfzAsxlDntxtCFs84e1zrMtNIKvGyxdsOBFS+ xfc8+e7IGdrok1wasiKK4IES9TgRBwK/4JNQfgBLiGGibjTz7eyduk27Iz4Pk7GBNiwH owyjm55NS3wjYBRBZ07mxOBnEMXTc/emQi+qL4rwtSUUwwoItI47KqfSDfw96DdlHNBC BRT1lk/o81+eK2SluObIVAoznDTIw5H3WYWTaOy989R3D8sp0vYprEnGcDAupctNAk40 HIYMuQkLRULyZhUt4+ZVdsVpf8iHB3ydlzgUHbWEhDwqKrWZmbaIAruLVGCmNPC1hOum G7Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eOA7n18lW1M1FakMU6/vCmTAO3qDfgKSz00bVksdClA=; b=KITTeWpKupbJsuvUJkAy7AgmP0FRg6dPu9ut4BFt2nXLC5xIWBZ6MJnNrhCdLqOP1W 4jGYX8yWMWQ0m7yzTbL7W5bdUZMI1MloTSJdqueV3boZmcZHtsy4QoMC1V+ql/2NFlE1 AVS78TYQvRaHXU8a7XDGQ9lM8eiIDBzIXFqeRMTfTCS+HBHXbKy9nqDpDv0xqXDd/IHN G0bqItQFbQTPNQBKHzTFiybedNnIVfBKqO2CdWwvo0f3MdMH+BwAECif+wwpAOPwU6fp R69njRBmuVJaE4JPeDuMaG44s+TPrcDCZ1OPacsZjHfTSv9ODKyUrSMHgTxLP0McirAF dr6w== X-Gm-Message-State: AOAM533DnM64mP04G7Ew0vUG+KGtzdImA/u/jDi97XWIwG3kJGYrk8zN pqDg+4r9QRioBRersKTcT5Q= X-Google-Smtp-Source: ABdhPJzbS4IahR8udAd1DZSOPnQMvvPfwwbXJjRuXHy1+XfHDTfWsNMH58MpW6E4F42wGqYxbVCSnQ== X-Received: by 2002:a17:906:606:: with SMTP id s6mr2448293ejb.206.1621480828358; Wed, 19 May 2021 20:20:28 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:27 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 24/29] trace-cmd: Compress trace data Date: Thu, 20 May 2021 06:19:54 +0300 Message-Id: <20210520031959.346165-25-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a trace.dat file of version 7, compress the trace data. The data is compressed in chunks. The chunk size is a compile time parameter, set by default to 1MB. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 50 +++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 046fa2d7..dbc956d4 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -321,6 +321,41 @@ static tsize_t copy_file(struct tracecmd_output *handle, return size; } +#define COMPRESS_CHUNK_SIZE (1*1024*1024) +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + unsigned long long rsize = 0; + unsigned long long wsize = 0; + tsize_t size; + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + if (handle->file_version >= 7) { + ret = tracecmd_compress_copy_from(handle->compress, fd, + COMPRESS_CHUNK_SIZE, &rsize, &wsize); + if (ret < 0) { + tracecmd_warning("Can't compress '%s'", file); + close(fd); + return 0; + } + size = rsize; + *write_size = wsize; + } else { + size = copy_file_fd(handle, fd); + *write_size = size; + } + + close(fd); + return size; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -1685,8 +1720,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; + unsigned long long read_size; + unsigned long long write_size; char *clock = NULL; - off64_t check_size; char *file; struct stat st; int ret; @@ -1742,11 +1778,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", i, (unsigned long long) data_files[i].data_offset); - check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != data_files[i].file_size) { + read_size = copy_file_compress(handle, cpu_data_files[i], &write_size); + if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, data_files[i].file_size); + read_size, data_files[i].file_size); goto out_free; } /* Write the real CPU data offset inthe file */ @@ -1756,14 +1792,14 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, goto out_free; /* Write the real CPU data size in the file */ offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); - endian8 = convert_endian_8(handle, check_size); + endian8 = convert_endian_8(handle, write_size); if (do_write_check(handle, &endian8, 8)) goto out_free; - offset = data_files[i].data_offset + check_size; + offset = data_files[i].data_offset + write_size; offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", - (unsigned long long)check_size); + (unsigned long long)write_size); } if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) From patchwork Thu May 20 03:19:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE573C43461 for ; Thu, 20 May 2021 03:20:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B80C3611AD for ; Thu, 20 May 2021 03:20:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230330AbhETDVv (ORCPT ); Wed, 19 May 2021 23:21:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhETDVv (ORCPT ); Wed, 19 May 2021 23:21:51 -0400 Received: from mail-ej1-x62f.google.com (mail-ej1-x62f.google.com [IPv6:2a00:1450:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF827C06175F for ; Wed, 19 May 2021 20:20:30 -0700 (PDT) Received: by mail-ej1-x62f.google.com with SMTP id lg14so22930681ejb.9 for ; Wed, 19 May 2021 20:20:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KAi7sMVpNBuzvhj06aaJOwX5EJDMkuTTmz1SyghfmaA=; b=PNSqlTcdnaI8z1ZiVO/NVPKlVdE7Z0vDYcWH3ibrVh4gtc6sQQhxMqEyOWCHSo3VyU +sOxNgmg9SbO0/XZ4JvJv0N8No4WnfbLfy7ACJTYw+tWiHgt4DNloIiL+wTuzXncXCzk 5w8BWKsh61DOH/+u/ZowNt5LQGrM8UhNyFzs6FRrdh7K643KU7/6+NkAm7fSdCcuqX3T iC2CR9ejzXy83Wm1T98lStl/0SlXF2yBQh6VGgWjVL//0n7YVvf7pdCT7q6K0XmTt0WN Frj8M7yfhaSdlMaOZ6B1xBT3qfRjRWWl+MGHX5iwEsKsxYnxctJcRDq/5ZldqJLL/Cua YC2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KAi7sMVpNBuzvhj06aaJOwX5EJDMkuTTmz1SyghfmaA=; b=fJPG7dlW5DCyAMZXQJTODT95p/7EaR/fBspRSma+B1N0CXXB0wkNBILuOgJMpIe6uh /cLDKXI1UOuM/vNBFZ15JZtGmxlqCvE1iMbsRSxz7Nf1IpLlivVnd4hobEjGaar7xlNk Lc+BLnU+MmoJTGNNQmgv6M1pvKo0HK9R7POhcMaTUbDqFQzNsAIz1ZRwqmopMz6khZu5 YhsEwWhv8PN1MJNqzwsFJFqmOT39sCsRfndW3JCXiY+Leu37um6QD30NnF06d/p0vwJf XNrhgiLkHsVg5+2DiMUsGcgV9yvEQp5epcAPRINYgV96sCozflmhmIlPIDVAqAQ06FB8 doWA== X-Gm-Message-State: AOAM5334bWwFpFOgqjvNUQOMnhqntxfDJYK+JFhW/uN/oR0nF6LJ9drm zvU/3SkJDul+0jKTR3p4U/8= X-Google-Smtp-Source: ABdhPJySnYZPnFPYOZMc//vL49CqB9lmsioCEbz9fnP7ou5obyCrG4/jA3aUQze0j1Ogda3cVpPAHw== X-Received: by 2002:a17:906:fc0d:: with SMTP id ov13mr2407996ejb.504.1621480829412; Wed, 19 May 2021 20:20:29 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:28 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 25/29] trace-cmd: Read compressed trace data Date: Thu, 20 May 2021 06:19:55 +0300 Message-Id: <20210520031959.346165-26-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace.dat file of version 7, uncompress the trace data. The trace data for each CPU is uncompressed in a temporary file, located in /tmp directory with prefix "trace_cpu_data". Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 75 +++++++++++++++++++++++++++++-------- tracecmd/trace-read.c | 8 ++++ 2 files changed, 68 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index f94ceab4..c2b64ea9 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,7 @@ struct page { #endif }; +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +73,10 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + + /* temporary file for uncompressed cpu data */ + int cfd; + char cfile[26]; /* strlen(COMPR_TEMP_FILE) */ }; struct input_buffer_instance { @@ -1080,6 +1085,7 @@ static void *allocate_page_map(struct tracecmd_input *handle, off64_t map_offset; void *map; int ret; + int fd; if (handle->read_page) { map = malloc(handle->page_size); @@ -1119,12 +1125,15 @@ static void *allocate_page_map(struct tracecmd_input *handle, map_size -= map_offset + map_size - (cpu_data->file_offset + cpu_data->file_size); + if (cpu_data->cfd >= 0) + fd = cpu_data->cfd; + else + fd = handle->fd; again: page_map->size = map_size; page_map->offset = map_offset; - page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, - handle->fd, map_offset); + page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, fd, map_offset); if (page_map->map == MAP_FAILED) { /* Try a smaller map */ @@ -2316,13 +2325,42 @@ tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record) /* Not reached */ } +static int cpu_data_uncompress(struct tracecmd_input *handle, int cpu, unsigned long long *size) +{ + struct cpu_data *cpu_data; + + cpu_data = &handle->cpu_data[cpu]; + strcpy(cpu_data->cfile, COMPR_TEMP_FILE); + cpu_data->cfd = mkstemp(cpu_data->cfile); + if (cpu_data->cfd < 0) + return -1; + return tracecmd_uncompress_copy_to(handle->compress, cpu_data->cfd, NULL, size); +} + static int init_cpu(struct tracecmd_input *handle, int cpu) { struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + unsigned long long size; + off64_t offset; int i; - cpu_data->offset = cpu_data->file_offset; - cpu_data->size = cpu_data->file_size; + /* When creating a pipe, file_size is set to -1 to indicate no data in the file */ + if (handle->file_version >= 7 && cpu_data->file_size != -1) { + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + if (cpu_data_uncompress(handle, cpu, &size) < 0) + return -1; + cpu_data->offset = 0; + cpu_data->file_offset = 0; + cpu_data->file_size = size; + cpu_data->size = size; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + } else { + cpu_data->offset = cpu_data->file_offset; + cpu_data->size = cpu_data->file_size; + } cpu_data->timestamp = 0; list_head_init(&cpu_data->page_maps); @@ -3015,6 +3053,7 @@ static int read_cpu_data(struct tracecmd_input *handle) handle->cpu_data[cpu].file_offset = offset; handle->cpu_data[cpu].file_size = size; + handle->cpu_data[cpu].cfd = -1; if (size > max_size) max_size = size; @@ -3635,17 +3674,23 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].cfd >= 0) { + close(handle->cpu_data[cpu].cfd); + unlink(handle->cpu_data[cpu].cfile); + } } } diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index 0cf6e773..d605d05a 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1363,7 +1363,14 @@ struct tracecmd_input *read_trace_header(const char *file, int flags) static void sig_end(int sig) { + struct handle_list *handles; + fprintf(stderr, "trace-cmd: Received SIGINT\n"); + + list_for_each_entry(handles, &handle_list, list) { + tracecmd_close(handles->handle); + } + exit(0); } @@ -1924,6 +1931,7 @@ void trace_report (int argc, char **argv) /* and version overrides uname! */ if (show_version) otype = OUTPUT_VERSION_ONLY; + read_data_info(&handle_list, otype, global, align_ts); list_for_each_entry(handles, &handle_list, list) { From patchwork Thu May 20 03:19:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C2CDC433ED for ; Thu, 20 May 2021 03:20:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5B2E611AB for ; Thu, 20 May 2021 03:20:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230335AbhETDVw (ORCPT ); Wed, 19 May 2021 23:21:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhETDVw (ORCPT ); Wed, 19 May 2021 23:21:52 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B87E9C061574 for ; Wed, 19 May 2021 20:20:31 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id s22so22734820ejv.12 for ; Wed, 19 May 2021 20:20:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fkZxC2X7hEb0oYvidiu3AmDabaLvi7WZgsLSTr3tHio=; b=RO1k6abBtbfIQcLPD2BlWfixDJQWtZfyZI+xqYfnCsGHp1RuQVARwwwwvc72FOOyIK Nq/LdvQYXzOvBMEM7Cg0pAJdaKjXpjfCa74V6NnKSJLBTRqDXQU0ZjDoSOwzVpIZWt/d rVl6QEUSeMK4znCDFfJeSFwEGkElY169IuKemH9SSee/y89+FHWLwZDw8kud2+Opy5ma cYZgZhNtrLTA018OIKpdur9aLMWQqmfQ3+qe+h+2JGuVvrpgX9lMZ5LmSaY4ckw2xcV2 12ibncP11s7qr/9kD9oybqUyVp22wlov//oi02KjqNry0fwtzfrow78/VpFt4eX4FtLY 8kaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fkZxC2X7hEb0oYvidiu3AmDabaLvi7WZgsLSTr3tHio=; b=VIcJqgYGuPn/lx8sFfC3digGhtFllm0lD0JI62GxkYDiSknCZoOXYvv1J7galmJen4 xDnsP8pp2cMMFsWPnBJ7pjjGT0WaMZ2TWr7Q0o99slChakO7XgZiRCHTdas3YfKMwZdp MsOaz/pQ3Es6g9PJ4wxWWqZbUPWKWeovvA39pLGJl5DHc5PNkfNWvaFWy+BNWcF3mc1g mHoTvc3+B02F5+LWNGss0eWSe8dqYbdw7mLbR4elNE+xcVjNDqb74uYjLCJqYquvK9k9 710KgnouezvZTenow/m3iT+pCU7U48KPKu5CTjoi0Jo3A0vFh5GomqQK6OAYKgTQFLLt /32g== X-Gm-Message-State: AOAM530AEYcAeE7e7rEYDWEXQ5My9DUWNKfRXpPMDOio599fMiXZUjC5 b6Tf3rBkod/yU2drkJWRdvI= X-Google-Smtp-Source: ABdhPJwIW/LaaNkUyzKeT7ON9/JEidQu+s6p7uyrhe+othlRMUTihdcS5I/YitcO201I3Kb9ibwMDw== X-Received: by 2002:a17:906:914d:: with SMTP id y13mr2406183ejw.489.1621480830305; Wed, 19 May 2021 20:20:30 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:29 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 26/29] trace-cmd library: Reuse within the library the function that checks file state. Date: Thu, 20 May 2021 06:19:56 +0300 Message-Id: <20210520031959.346165-27-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Make the function, that checks if the next file state is valid, global for the tracmd-library, so it can be reused. It is important the same check logic to be used in the whole library. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 4 ++ lib/trace-cmd/trace-input.c | 5 ++ lib/trace-cmd/trace-output.c | 88 +++++++------------------ lib/trace-cmd/trace-util.c | 41 ++++++++++++ 4 files changed, 72 insertions(+), 66 deletions(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index e579a669..48a01d43 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -39,6 +39,10 @@ void tracecmd_zlib_free(void); int tracecmd_compress_init(void); void tracecmd_compress_free(void); +bool check_file_state(unsigned long file_version, int current_state, int new_state); +bool check_in_state(struct tracecmd_input *handle, int new_state); +bool check_out_state(struct tracecmd_output *handle, int new_state); + int out_uncompress_block(struct tracecmd_output *handle); int out_compression_start(struct tracecmd_output *handle); int out_compression_end(struct tracecmd_output *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index c2b64ea9..a1248f26 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4397,3 +4397,8 @@ int tracecmd_enable_tsync(struct tracecmd_input *handle, bool enable) return 0; } + +__hidden bool check_in_state(struct tracecmd_input *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index dbc956d4..e7370798 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -419,50 +419,6 @@ int tracecmd_ftrace_enable(int set) return ret; } -static int check_out_state(struct tracecmd_output *handle, int new_state) -{ - if (!handle) - return -1; - - if (handle->file_version < 7) { - switch (new_state) { - case TRACECMD_FILE_BUFERS: - tracecmd_warning("State %d is not supported in trace file version %d", - new_state, handle->file_version); - return -1; - default: - break; - } - } - - switch (new_state) { - case TRACECMD_FILE_HEADERS: - case TRACECMD_FILE_FTRACE_EVENTS: - case TRACECMD_FILE_ALL_EVENTS: - case TRACECMD_FILE_KALLSYMS: - case TRACECMD_FILE_PRINTK: - case TRACECMD_FILE_CMD_LINES: - case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_BUFERS: - if (handle->file_state == (new_state - 1)) - return 0; - break; - case TRACECMD_FILE_OPTIONS: - if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) - return 0; - if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) - return 0; - break; - case TRACECMD_FILE_CPU_LATENCY: - case TRACECMD_FILE_CPU_FLYRECORD: - if (handle->file_state == TRACECMD_FILE_OPTIONS) - return 0; - break; - } - - return -1; -} - static int read_header_files(struct tracecmd_output *handle) { tsize_t size, check_size, endian8; @@ -471,7 +427,7 @@ static int read_header_files(struct tracecmd_output *handle) int fd; int ret; - if (check_out_state(handle, TRACECMD_FILE_HEADERS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_HEADERS)) { tracecmd_warning("Cannot read header files, unexpected state 0x%X", handle->file_state); return -1; @@ -783,7 +739,7 @@ static int read_ftrace_files(struct tracecmd_output *handle) struct tracecmd_event_list list = { .glob = "ftrace/*" }; int ret; - if (check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_FTRACE_EVENTS)) { tracecmd_warning("Cannot read ftrace files, unexpected state 0x%X", handle->file_state); return -1; @@ -828,7 +784,7 @@ static int read_event_files(struct tracecmd_output *handle, int endian4; int ret; - if (check_out_state(handle, TRACECMD_FILE_ALL_EVENTS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_ALL_EVENTS)) { tracecmd_warning("Cannot read event files, unexpected state 0x%X", handle->file_state); return -1; @@ -927,7 +883,7 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, struct stat st; int ret; - if (check_out_state(handle, TRACECMD_FILE_KALLSYMS) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_KALLSYMS)) { tracecmd_warning("Cannot read kallsyms, unexpected state 0x%X", handle->file_state); return -1; @@ -977,7 +933,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) char *path; int ret; - if (check_out_state(handle, TRACECMD_FILE_PRINTK) < 0) { + if (!check_out_state(handle, TRACECMD_FILE_PRINTK)) { tracecmd_warning("Cannot read printk, unexpected state 0x%X", handle->file_state); return -1; @@ -1343,11 +1299,10 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CPU_COUNT); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_COUNT)) { tracecmd_warning("Cannot write CPU count into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } cpus = convert_endian_4(handle, cpus); ret = do_write_check(handle, &cpus, 4); @@ -1363,16 +1318,14 @@ int tracecmd_write_options(struct tracecmd_output *handle) unsigned short option; unsigned short endian2; unsigned int endian4; - int ret; /* If already written, ignore */ if (handle->file_state == TRACECMD_FILE_OPTIONS) return 0; - ret = check_out_state(handle, TRACECMD_FILE_OPTIONS); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_OPTIONS)) { tracecmd_warning("Cannot write options into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } if (do_write_check(handle, "options ", 10)) @@ -1535,10 +1488,9 @@ static int write_buffers_description_v7(struct tracecmd_output *handle) int size; int ret; - ret = check_out_state(handle, TRACECMD_FILE_BUFERS); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_BUFERS)) { tracecmd_warning("Cannot write buffers descriptions"); - return ret; + return -1; } list_for_each_entry(buf, &handle->buffers, list) { @@ -1597,11 +1549,10 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; - ret = check_out_state(handle, TRACECMD_FILE_CMD_LINES); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CMD_LINES)) { tracecmd_warning("Cannot write command lines into the file, unexpected state 0x%X", handle->file_state); - return ret; + return -1; } out_compression_start(handle); @@ -1621,7 +1572,6 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in { struct tracecmd_output *handle; char *path; - int ret; handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) @@ -1640,8 +1590,7 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_options(handle) < 0) goto out_free; - ret = check_out_state(handle, TRACECMD_FILE_CPU_LATENCY); - if (ret < 0) { + if (!check_out_state(handle, TRACECMD_FILE_CPU_LATENCY)) { tracecmd_warning("Cannot write latency data into the file, unexpected state 0x%X", handle->file_state); goto out_free; @@ -1730,7 +1679,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, /* This can be called multiple times (when recording instances) */ ret = handle->file_state == TRACECMD_FILE_CPU_FLYRECORD ? 0 : - check_out_state(handle, TRACECMD_FILE_CPU_FLYRECORD); + check_file_state(handle->file_version, + handle->file_state, + TRACECMD_FILE_CPU_FLYRECORD); if (ret < 0) { tracecmd_warning("Cannot write trace data into the file, unexpected state 0x%X", handle->file_state); @@ -1999,3 +1950,8 @@ out_free: tracecmd_output_close(handle); return NULL; } + +__hidden bool check_out_state(struct tracecmd_output *handle, int new_state) +{ + return check_file_state(handle->file_version, handle->file_state, new_state); +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 136260c6..087d37ab 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -636,3 +636,44 @@ void tracecmd_lib_free(void) { tracecmd_compress_free(); } + +__hidden bool check_file_state(unsigned long file_version, int current_state, int new_state) +{ + if (file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, file_version); + return false; + default: + break; + } + } + + switch (new_state) { + case TRACECMD_FILE_HEADERS: + case TRACECMD_FILE_FTRACE_EVENTS: + case TRACECMD_FILE_ALL_EVENTS: + case TRACECMD_FILE_KALLSYMS: + case TRACECMD_FILE_PRINTK: + case TRACECMD_FILE_CMD_LINES: + case TRACECMD_FILE_CPU_COUNT: + case TRACECMD_FILE_BUFERS: + if (current_state == (new_state - 1)) + return true; + break; + case TRACECMD_FILE_OPTIONS: + if (file_version < 7 && current_state == TRACECMD_FILE_CPU_COUNT) + return true; + if (file_version >= 7 && current_state == TRACECMD_FILE_BUFERS) + return true; + break; + case TRACECMD_FILE_CPU_LATENCY: + case TRACECMD_FILE_CPU_FLYRECORD: + if (current_state == TRACECMD_FILE_OPTIONS) + return true; + break; + } + + return false; +} From patchwork Thu May 20 03:19:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C9E4C433B4 for ; Thu, 20 May 2021 03:20:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 452A161163 for ; Thu, 20 May 2021 03:20:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230253AbhETDVy (ORCPT ); Wed, 19 May 2021 23:21:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhETDVy (ORCPT ); Wed, 19 May 2021 23:21:54 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3363C061574 for ; Wed, 19 May 2021 20:20:32 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id l1so22955386ejb.6 for ; Wed, 19 May 2021 20:20:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Vmi5AjbJuXgF72hp4cDxxO6svoQZ6g9atW7rqDFkATg=; b=C34kszP+1igmfAnvDQFCstOCVzB4Kt9AmTz8cZXHa4/GCHgTMseYrIy7RIZpRh//3b PP9wdZRgBbV4ZkPh0+Vt7PBz3ql3bTEvvOOnCFhM5WhjUP574BiklNdP3D1YaUi1dnNY aWRUvakJZB667Cf0EB5G7Z8wEVu0PLS9S6mtjerbEk7TedYwHfT1z1nlRMCKH/QVsm3o FP5mPnlckUJ+0+IRkGe3gbhCJREU1hS2EsYy571mUuEoh4Qz5iD6Y5KJ9C8Ssvd4tjL4 SIVMwvGTmWmYBjPtrYZ+PpqHcIrTQpt6v0GpYEwtbilxx6IKtjuRw0ncYNDmkyE/U8Cv F6kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Vmi5AjbJuXgF72hp4cDxxO6svoQZ6g9atW7rqDFkATg=; b=aqZ7oHIwv2WPk6KIY3EeYYDpKOfyNPbPIkPNS1bJN0kBXlEFBypK1eue8VS9WdA0W8 Tt8+coXT8IEXuM7HFz5TI8rDsx7gMRdU9fNJO3SnRPUeWSOWMScu9HrwXHIDxrJmp/rk lbZ5kA7CHe1Ds7dUIEHp35xWuoFfTCsi8h5Mzbec70DOC4JiRiRZs3KBpXxlbNcoOiS2 9N1EgVye4t1Sbq+kpzre2Svf2V+aWBNujab5eIbwPlbBM8Cx+C4nAkgZW/aAna8+FwLn V/orl4EMWhuDOTf8bNKBnEEmGbTWFqQwd+8dyrydMkHJNa2sK7e9W+iu+qNWSX233mEi plJg== X-Gm-Message-State: AOAM5307SESVw9g5xtmOyrv9polg2PXe3o4RkUBE9kJ0qyk++splKrIk KqPoAUwi7ySXBa7dTkwipQpVcNj+YrQ1QQ== X-Google-Smtp-Source: ABdhPJz0L5BPeXtUyTjyJB0T/dW03k6dAEln297RrwjLtnspRHP4sqHAGMlqsCCJ9erlXzq91wwmYQ== X-Received: by 2002:a17:906:b2c1:: with SMTP id cf1mr2390619ejb.544.1621480831463; Wed, 19 May 2021 20:20:31 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:30 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 27/29] trace-cmd library: New internal API to set file state of output handler Date: Thu, 20 May 2021 06:19:57 +0300 Message-Id: <20210520031959.346165-28-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a new library internal API to set the state of an output file handler: set_out_file_state() This new API will be used internaly in library for split and convert functionality. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 3 +++ lib/trace-cmd/trace-output.c | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 48a01d43..830bbb1a 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -51,4 +51,7 @@ void out_compression_reset(struct tracecmd_output *handle); void in_uncompress_reset(struct tracecmd_input *handle); int in_uncompress_block(struct tracecmd_input *handle); +void set_out_file_state(struct tracecmd_output *handle, int new_state); + + #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index e7370798..24c260c4 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1951,6 +1951,11 @@ out_free: return NULL; } +__hidden void set_out_file_state(struct tracecmd_output *handle, int new_state) +{ + handle->file_state = new_state; +} + __hidden bool check_out_state(struct tracecmd_output *handle, int new_state) { return check_file_state(handle->file_version, handle->file_state, new_state); From patchwork Thu May 20 03:19:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2AD8C43461 for ; Thu, 20 May 2021 03:20:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8859361186 for ; Thu, 20 May 2021 03:20:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230338AbhETDVz (ORCPT ); Wed, 19 May 2021 23:21:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhETDVz (ORCPT ); Wed, 19 May 2021 23:21:55 -0400 Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C91F2C061574 for ; Wed, 19 May 2021 20:20:33 -0700 (PDT) Received: by mail-ed1-x531.google.com with SMTP id g7so5494280edm.4 for ; Wed, 19 May 2021 20:20:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Bq5fzextg6YcUVF0n0Qs3Sqdfiq/13flAPjTIE2WE08=; b=j9dB8gdELlmaldf1SqDirkvTTUC7/5McGJ/5w//synqaNncljRi91PNppX16Vdilb4 2xL6NRCn4F2NAhRzOhh4yenDkMZ36Eke0fV8d3WWa68YSYRA7gY7Hip5FZc/0si+Bjbo WwJzSmIKPvQCYvOpZVWHGh5jq9yKiI6egMPUEEjPus0tyYQlnrw9DWm0nwIWCHQU0EHx 8GqAkU6+MHlkjP6a+LUpnNWmJD/Mqrh14wDVbAbUWqpz5g5GmLiEOs1JkyNBbPMa71oU zLHyhiwe83WlTzEpJooXkRc11ptz8FehajWi5EK1qz9vhjYJXZkf0OY0eHU9xZ1J9KyP fU6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Bq5fzextg6YcUVF0n0Qs3Sqdfiq/13flAPjTIE2WE08=; b=eqqDqVBMae4j5IRsIPFsq8xrr6RlsghXI7BqJYLqzGH0aTxW4c/Sg06aNVUISlHAOd XrzxmAqbQnYr1M+O9Mtam50JUMzpmw6g2j+So0Lj/tVQZzQX0SRWfwUbj4KTLaR5lCVP 9HmoccDmh0XxWSXYLnZMrkd/tBTtoL+u2Vepc7CyaIM7+5DI7RxYmP46ICSde8pvSUA8 KGbE0ZU+xcUVYwjjXmgFqNNbhGYiiKtpppVDvL7WIc5zxvQt6MHil3HkheAbvkueQQE5 LpfetWQhtXMkCstp+ARyccJeX+/JlXGsmDCsAbZnWno/0eqUB/Hyvp7V+H8JtzrU3wiz lT4w== X-Gm-Message-State: AOAM530Jvc8WzjMCkKUtpIjt79W5qhnjpdpZyngelRqY6txgLrRkAK3/ fBG6xp7b6LYbz6WR6ebjNeGOIv/b+99AhQ== X-Google-Smtp-Source: ABdhPJwfAIq5LbgfXP5iZRvB2BUYyfEY7ujbkEd65T/5iYoyZXV9pVXHvZha2HZIdgDMDe7yd65nDA== X-Received: by 2002:a05:6402:17d9:: with SMTP id s25mr2520511edy.337.1621480832425; Wed, 19 May 2021 20:20:32 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:31 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 28/29] trace-cmd library: Make tracecmd_copy_headers() to work with output handler Date: Thu, 20 May 2021 06:19:58 +0300 Message-Id: <20210520031959.346165-29-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When copying headers between two trace files, use output handler context instead of just a fd. Using output handler will allow to use file compression, if needed. This change fixes "trace-cmd split" command to work with trace files v7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 +- lib/trace-cmd/include/trace-cmd-local.h | 1 + lib/trace-cmd/trace-input.c | 244 ++++++++++++------ lib/trace-cmd/trace-output.c | 8 +- 4 files changed, 166 insertions(+), 90 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 19dc9e05..975a11b1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -170,7 +170,8 @@ int tracecmd_get_parsing_failures(struct tracecmd_input *handle); int tracecmd_long_size(struct tracecmd_input *handle); int tracecmd_page_size(struct tracecmd_input *handle); int tracecmd_cpus(struct tracecmd_input *handle); -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state); void tracecmd_set_flag(struct tracecmd_input *handle, int flag); diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 830bbb1a..06556c8d 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -53,5 +53,6 @@ int in_uncompress_block(struct tracecmd_input *handle); void set_out_file_state(struct tracecmd_output *handle, int new_state); +long long do_write_check(struct tracecmd_output *handle, const void *data, long long size); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index a1248f26..0c697a22 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -3720,44 +3720,47 @@ void tracecmd_close(struct tracecmd_input *handle) free(handle); } -static int read_copy_size8(struct tracecmd_input *handle, int fd, unsigned long long *size) +static int read_copy_size8(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, unsigned long long *size) { /* read size */ - if (do_read_check(handle, size, 8)) + if (do_read_check(in_handle, size, 8)) return -1; - if (__do_write_check(fd, size, 8)) + if (do_write_check(out_handle, size, 8)) return -1; - *size = tep_read_number(handle->pevent, size, 8); + *size = tep_read_number(in_handle->pevent, size, 8); return 0; } -static int read_copy_size4(struct tracecmd_input *handle, int fd, unsigned int *size) +static int read_copy_size4(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle, + unsigned int *size) { /* read size */ - if (do_read_check(handle, size, 4)) + if (do_read_check(in_handle, size, 4)) return -1; - if (__do_write_check(fd, size, 4)) + if (do_write_check(out_handle, size, 4)) return -1; - *size = tep_read_number(handle->pevent, size, 4); + *size = tep_read_number(in_handle->pevent, size, 4); return 0; } -static int read_copy_data(struct tracecmd_input *handle, - unsigned long long size, int fd) +static int read_copy_data(struct tracecmd_input *in_handle, + unsigned long long size, + struct tracecmd_output *out_handle) { char *buf; buf = malloc(size); if (!buf) return -1; - if (do_read_check(handle, buf, size)) + if (do_read_check(in_handle, buf, size)) goto failed_read; - if (__do_write_check(fd, buf, size)) + if (do_write_check(out_handle, buf, size)) goto failed_read; free(buf); @@ -3769,65 +3772,82 @@ static int read_copy_data(struct tracecmd_input *handle, return -1; } -static int copy_header_files(struct tracecmd_input *handle, int fd) +static int copy_header_files(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_HEADERS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_HEADERS) || + !check_out_state(out_handle, TRACECMD_FILE_HEADERS)) return -1; /* "header_page" */ - if (read_copy_data(handle, 12, fd) < 0) + if (read_copy_data(in_handle, 12, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; /* "header_event" */ - if (read_copy_data(handle, 13, fd) < 0) + if (read_copy_data(in_handle, 13, out_handle) < 0) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (read_copy_size8(in_handle, out_handle, &size) < 0) return -1; - if (read_copy_data(handle, size, fd) < 0) + if (read_copy_data(in_handle, size, out_handle) < 0) return -1; - handle->file_state = TRACECMD_FILE_HEADERS; + in_handle->file_state = TRACECMD_FILE_HEADERS; + set_out_file_state(out_handle, in_handle->file_state); return 0; } -static int copy_ftrace_files(struct tracecmd_input *handle, int fd) +static int copy_ftrace_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; unsigned int count; unsigned int i; - if (handle->file_state != TRACECMD_FILE_FTRACE_EVENTS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_FTRACE_EVENTS) || + !check_out_state(out_handle, TRACECMD_FILE_FTRACE_EVENTS)) return -1; - if (read_copy_size4(handle, fd, &count) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (i = 0; i < count; i++) { - if (read_copy_size8(handle, fd, &size) < 0) - return -1; + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; } - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + + if (out_compression_end(out_handle)) + goto error; + set_out_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_event_files(struct tracecmd_input *handle, int fd) +static int copy_event_files(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; char *system; @@ -3835,103 +3855,159 @@ static int copy_event_files(struct tracecmd_input *handle, int fd) unsigned int count; unsigned int i,x; - if (handle->file_state != TRACECMD_FILE_ALL_EVENTS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_ALL_EVENTS) || + !check_out_state(out_handle, TRACECMD_FILE_ALL_EVENTS)) return -1; - if (read_copy_size4(handle, fd, &systems) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &systems) < 0) + goto error; for (i = 0; i < systems; i++) { - system = read_string(handle); + system = read_string(in_handle); if (!system) - return -1; - if (__do_write_check(fd, system, strlen(system) + 1)) { + goto error; + if (do_write_check(out_handle, system, strlen(system) + 1)) { free(system); - return -1; + goto error; } free(system); - if (read_copy_size4(handle, fd, &count) < 0) - return -1; + if (read_copy_size4(in_handle, out_handle, &count) < 0) + goto error; for (x=0; x < count; x++) { - if (read_copy_size8(handle, fd, &size) < 0) - return -1; + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; } } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_ALL_EVENTS; + + if (out_compression_end(out_handle)) + goto error; + set_out_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_proc_kallsyms(struct tracecmd_input *handle, int fd) +static int copy_proc_kallsyms(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_KALLSYMS - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_KALLSYMS) || + !check_out_state(out_handle, TRACECMD_FILE_KALLSYMS)) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; - handle->file_state = TRACECMD_FILE_KALLSYMS; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_KALLSYMS; + + if (out_compression_end(out_handle)) + goto error; + set_out_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_ftrace_printk(struct tracecmd_input *handle, int fd) +static int copy_ftrace_printk(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned int size; - if (handle->file_state != TRACECMD_FILE_PRINTK - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_PRINTK) || + !check_out_state(out_handle, TRACECMD_FILE_PRINTK)) return -1; - if (read_copy_size4(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; - if (!size) - return 0; /* OK? */ + out_compression_start(out_handle); - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_size4(in_handle, out_handle, &size) < 0) + goto error; + if (!size) + goto out; /* OK? */ - handle->file_state = TRACECMD_FILE_PRINTK; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_PRINTK; + if (out_compression_end(out_handle)) + goto error; + set_out_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } -static int copy_command_lines(struct tracecmd_input *handle, int fd) +static int copy_command_lines(struct tracecmd_input *in_handle, struct tracecmd_output *out_handle) { unsigned long long size; - if (handle->file_state != TRACECMD_FILE_CMD_LINES - 1) + if (!check_in_state(in_handle, TRACECMD_FILE_CMD_LINES) || + !check_out_state(out_handle, TRACECMD_FILE_CMD_LINES)) return -1; - if (read_copy_size8(handle, fd, &size) < 0) + if (in_uncompress_block(in_handle)) return -1; + out_compression_start(out_handle); + + if (read_copy_size8(in_handle, out_handle, &size) < 0) + goto error; if (!size) - return 0; /* OK? */ + goto out; /* OK? */ - if (read_copy_data(handle, size, fd) < 0) - return -1; + if (read_copy_data(in_handle, size, out_handle) < 0) + goto error; - handle->file_state = TRACECMD_FILE_CMD_LINES; +out: + in_uncompress_reset(in_handle); + in_handle->file_state = TRACECMD_FILE_CMD_LINES; + if (out_compression_end(out_handle)) + goto error; + set_out_file_state(out_handle, in_handle->file_state); return 0; +error: + out_compression_reset(out_handle); + in_uncompress_reset(in_handle); + return -1; } /** * tracecmd_copy_headers - Copy headers from a tracecmd_input handle to a file descriptor - * @handle: input handle for the trace.dat file to copy from. - * @fd: The file descriptor to copy to. + * @in_handle: input handle for the trace.dat file to copy from. + * @out_handle: output handle to the trace.dat file to copy to. * @start_state: The file state to start copying from (zero for the beginnig) * @end_state: The file state to stop at (zero for up to cmdlines) * @@ -3942,7 +4018,8 @@ static int copy_command_lines(struct tracecmd_input *handle, int fd) * NOTE: The input handle is also modified, and ends at the end * state as well. */ -int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, +int tracecmd_copy_headers(struct tracecmd_input *in_handle, + struct tracecmd_output *out_handle, enum tracecmd_file_states start_state, enum tracecmd_file_states end_state) { @@ -3958,68 +4035,67 @@ int tracecmd_copy_headers(struct tracecmd_input *handle, int fd, if (end_state < TRACECMD_FILE_HEADERS) return 0; - - if (handle->file_state >= start_state) { + if (in_handle->file_state >= start_state) { /* Set the handle to just before the start state */ - lseek64(handle->fd, handle->header_files_start, SEEK_SET); + lseek64(in_handle->fd, in_handle->header_files_start, SEEK_SET); /* Now that the file handle has moved, change its state */ - handle->file_state = TRACECMD_FILE_INIT; + in_handle->file_state = TRACECMD_FILE_INIT; } /* Try to bring the input up to the start state - 1 */ - ret = tracecmd_read_headers(handle, start_state - 1); + ret = tracecmd_read_headers(in_handle, start_state - 1); if (ret < 0) goto out; switch (start_state) { case TRACECMD_FILE_HEADERS: - ret = copy_header_files(handle, fd); + ret = copy_header_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_FTRACE_EVENTS: /* handle's state is now updating with the copies */ - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_files(handle, fd); + ret = copy_ftrace_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_ALL_EVENTS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_event_files(handle, fd); + ret = copy_event_files(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_KALLSYMS: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_proc_kallsyms(handle, fd); + ret = copy_proc_kallsyms(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_PRINTK: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_ftrace_printk(handle, fd); + ret = copy_ftrace_printk(in_handle, out_handle); if (ret < 0) goto out; /* fallthrough */ case TRACECMD_FILE_CMD_LINES: - if (end_state <= handle->file_state) + if (end_state <= in_handle->file_state) return 0; - ret = copy_command_lines(handle, fd); + ret = copy_command_lines(in_handle, out_handle); default: break; } diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 24c260c4..e27bb45c 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -89,8 +89,8 @@ struct list_event_system { char *name; }; -static stsize_t -do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) +__hidden long long +do_write_check(struct tracecmd_output *handle, const void *data, long long size) { if (handle->do_compress && handle->compress) return tracecmd_compress_write(handle->compress, data, size); @@ -1938,11 +1938,9 @@ struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char if (!handle) return NULL; - if (tracecmd_copy_headers(ihandle, handle->fd, 0, 0) < 0) + if (tracecmd_copy_headers(ihandle, handle, 0, 0) < 0) goto out_free; - handle->file_state = tracecmd_get_file_state(ihandle); - /* The file is all ready to have cpu data attached */ return handle; From patchwork Thu May 20 03:19:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12268885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3147C43462 for ; Thu, 20 May 2021 03:20:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8CB361184 for ; Thu, 20 May 2021 03:20:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230339AbhETDV5 (ORCPT ); Wed, 19 May 2021 23:21:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbhETDV4 (ORCPT ); Wed, 19 May 2021 23:21:56 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFD1DC06175F for ; Wed, 19 May 2021 20:20:34 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id u21so22885379ejo.13 for ; Wed, 19 May 2021 20:20:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BTU1iFO2V4ZzMjExOiVLwCGbuFnGNHp5D2Mj4ZKntEk=; b=W440AGhxE9s4eP2ja0m0sAEk1PUnS2ZqH7cDk9g+PNHcXKPZzJzsfV3ZxxfKPMxR4Z UO21Ph22yLLjLy9IdMAKNXzk4FMl+Og//g4fFvHeg7VAZ9gKA6yFi1gG9i7KYGu+Z5eT n4ZsBcqCwxD1yaCj1w7ee6102WJ5bTxezgGWyhOA2gAQXVQk2h/Yf///2PXivw8Xeyiu y6ojX1s6klwXXnUHEoxSHIrSB6n3Apl+kvY7CbskVHVAqGQvKdoTnXaUfwUw/IAGhkxE NtydpHw5rFPpSrpylWuiK6TVtuSJ0IhRsgd7jsPnfOx3U517WyHLoZ+zN/OznayaiHd+ ZrJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BTU1iFO2V4ZzMjExOiVLwCGbuFnGNHp5D2Mj4ZKntEk=; b=HIqUfbP49PIU3r5NzA+WVKMuZrkJc2PHn1PoM4dr4rwiezd93y6c4LN7RdD1s/OZPt /AOPP9O2TOazIVA9XHmsTNf28/5CTB0QBsxHt9CwG9uGmT316o7IkfHtKYvLjtCOTp/S RZL6mT++Uqglu5x1MJeY/t8ozpIRhZ3qauqAHOQlg9usfFCc6B9Q8V7g4ACANrJP4ECK MZvAoQWbKYBHXVq+TC4Idsa9KvKsaT+d36WTOj5BT7zBFbMT5lQij6bn2ec9Z96KEjkX omXULtZlYqZDATsoV4j+8vh0v5U8aN64j8JgOM2aM0P+ioo20RniySzFe69mChM8oVzG w52w== X-Gm-Message-State: AOAM531tyFeBDHoA7RCm0z61wAUn3ll0Ne2lM1iw30BXKx+LdMdBF5rN 2Yx9CwTWyBo8SzbU34FDP2k= X-Google-Smtp-Source: ABdhPJwXHqYJQhkCBuLPS7viA9X6IhbPZjBGnQn8x5b+1eYekUWubFO2NtvUihHdHfHwv6CJr8vM9Q== X-Received: by 2002:a17:906:5052:: with SMTP id e18mr2512218ejk.112.1621480833318; Wed, 19 May 2021 20:20:33 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id f5sm763280eds.55.2021.05.19.20.20.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 20:20:32 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 29/29] trace-cmd: Do not use trace file compression with streams Date: Thu, 20 May 2021 06:19:59 +0300 Message-Id: <20210520031959.346165-30-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520031959.346165-1-tz.stoyanov@gmail.com> References: <20210520031959.346165-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a temporary output handler for use with a stream, force it not not use trace file compression. As no data are saved in this file, there is no need to have a compression overhead. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-stream.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tracecmd/trace-stream.c b/tracecmd/trace-stream.c index 317de1b2..1ca3a79d 100644 --- a/tracecmd/trace-stream.c +++ b/tracecmd/trace-stream.c @@ -43,7 +43,7 @@ trace_stream_init(struct buffer_instance *instance, int cpu, int fd, int cpus, tfd = fileno(fp); ofd = dup(tfd); - trace_output = tracecmd_create_init_fd(ofd, NULL); + trace_output = tracecmd_create_init_fd(ofd, "none"); if (!trace_output) { fclose(fp); return NULL;