From patchwork Wed May 12 06:37:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66EACC433B4 for ; Wed, 12 May 2021 06:38:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 277F9613BA for ; Wed, 12 May 2021 06:38:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229994AbhELGjm (ORCPT ); Wed, 12 May 2021 02:39:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229626AbhELGjg (ORCPT ); Wed, 12 May 2021 02:39:36 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D017C06174A for ; Tue, 11 May 2021 23:38:28 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id l7so25742162edb.1 for ; Tue, 11 May 2021 23:38:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0C6D0NjdPESwV62T7kxwWiilnywFWC5ZrocmTLNRsC4=; b=FRIO3pJvmFNoBVgmcWnsuKsVZ6cHQNPUCEsLPP5XN4fQwQJ5aCKf6O53c3HX7fwIyF yPt04cK6eVleBfC1EUM4rZgi6ThiN+WLeX6ia8hkb3QQJCwD8C1qs4AIOM3vCHn7CqkX VRNiqE//sSLRZSBg61S3SeooJmq9QsROOfMVlQWwCXIW560hguULFvEBgi7R5Ry0jayZ ta6L7yPlKzXpNwXV2QC1rvudzPRVRwGwK9DZfPieb5d5mmEiOeXKNPnvRUCN2E2WGT3C 6KNUG3p4N7SwUGMLMRTGB0aTskPjXJk8oPZjp+6eLCYtm3v52ZIrrTxcfPfmhtWBptqB P9fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0C6D0NjdPESwV62T7kxwWiilnywFWC5ZrocmTLNRsC4=; b=QtrHhguVpdPKvAyJVDtiyhlewXIelq4vUQOppVqO7jAjZ2tmATYL0VK4Jc5Ca7xhva NjkqdL7FNucg+iJbPAAkRo1145ijXUXUp9VJxbVeINxF7hBWmc9x5QTiqCzbzesvyiHX nBbj65tgOEoxfiUyhatmpQ/XGau4fe0gQopZx+7lBLXje0SNCs+p1n3cJ4HEHa3yZ1ds nnsjrvNTjuappBxScaJUK3WT+2wivyFr3GxNZ7SRgwe1hlJdmqPgNdmmjmzGEihg5oJF MYKg97gl4Elm6hDxHcCUCr1C6lB3Y8ynriXudsBNodPLZcReVMv+ndGlIFSFBOOBdRQi haFg== X-Gm-Message-State: AOAM5328rK7uaqjV/PVn42cQg7faN3SOrw+iLYAMsisIgMeUJ6QfUa/L iq3mnKNL9F2HdPh7zIEir40= X-Google-Smtp-Source: ABdhPJx31c5tX8JRLoJt74kXPGXRqiWms8V3mU85EBcrijChVJ92QqWKvyD/rLY1q0a3sKsoi5ndlw== X-Received: by 2002:a05:6402:50d2:: with SMTP id h18mr6498654edb.10.1620801506840; Tue, 11 May 2021 23:38:26 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:26 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 01/25] trace-cmd library: Remove unused private APIs for creating trace files Date: Wed, 12 May 2021 09:37:59 +0300 Message-Id: <20210512063823.324610-2-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Remove these unused private trace-cmd library APIs: tracecmd_create_file() tracecmd_create_file_glob() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 6 ---- lib/trace-cmd/trace-output.c | 29 ------------------- 2 files changed, 35 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 42e739fa..8ec33ea2 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -263,12 +263,6 @@ struct tracecmd_option; struct tracecmd_msg_handle; struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files); -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *event_globs); struct tracecmd_output * tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index a4a1eecc..9231c319 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1614,35 +1614,6 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) return NULL; } -struct tracecmd_output * -tracecmd_create_file_glob(const char *output_file, - int cpus, char * const *cpu_data_files, - struct tracecmd_event_list *list) -{ - struct tracecmd_output *handle; - - handle = create_file(output_file, NULL, NULL, NULL, list); - if (!handle) - return NULL; - - if (tracecmd_write_cmdlines(handle)) - return NULL; - - if (tracecmd_append_cpu_data(handle, cpus, cpu_data_files) < 0) { - tracecmd_output_close(handle); - return NULL; - } - - return handle; -} - -struct tracecmd_output *tracecmd_create_file(const char *output_file, - int cpus, char * const *cpu_data_files) -{ - return tracecmd_create_file_glob(output_file, cpus, - cpu_data_files, &all_event_list); -} - struct tracecmd_output *tracecmd_create_init_fd(int fd) { return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); From patchwork Wed May 12 06:38:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84B0CC433ED for ; Wed, 12 May 2021 06:38:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 54801613B4 for ; Wed, 12 May 2021 06:38:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229626AbhELGjn (ORCPT ); Wed, 12 May 2021 02:39:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229627AbhELGji (ORCPT ); Wed, 12 May 2021 02:39:38 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1594AC061574 for ; Tue, 11 May 2021 23:38:29 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id c22so5194360ejd.12 for ; Tue, 11 May 2021 23:38:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dR9cwZyRjBzKaGdEXi13MgximQSnOMDkCL1jWfZl6fY=; b=iUA1Uot74D7MZK8VSpnRZVoYrV8gBoxTJNvTnYpi3lfw2J2Pu6bvEPLz7cz5JcLNs4 P/R9/ipJgk21XLWz8nax0NEOJXubkRSALDxk3RNXdglBIXb+b6Z0zr7QptE9PttYkJXi 7VrQSDDWtJYNU9K6C+hA72c65vOa01RFxf0fwAu7i9yd0fUuwdpGg3nPgGx/bToM0qRt S9uqCPsmip+LB6LJBDXGfAbCY8WdS8kvzKpeiSbXZZG1zsIXgO1s0/zgaIR0fBjNJCpa Na2IGt2M/DSYyiVf2wlCJ9zmfVQVOObFrCpZrS20FqDZq7qZ7xO+X7aUCb7NOPl9yg2a t4Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dR9cwZyRjBzKaGdEXi13MgximQSnOMDkCL1jWfZl6fY=; b=OhvPfJJ9cQm40LEHo4mLu0QPQd5qFBYc30Qmdh+BHTev0o15wq4QQfVHaQvfSfUkkS naJhYZjza3pCvev//2r8bnfnd2/aqp//+D/eGm3VD3DZ9g5tMDmQWVvLJC5LlhruMVau NK9oioE8h3su0jthWobjxU2MbjpEAkbUYvZYnYLzzsy44by82W/vc5d7dkyv6yW/+Mm6 jVK5VeRE93VLGK2yXVSOn9UI1uDTGMvbuKVXi2n4c7x2JGycR9FzjJkcacDWjYorZn5O qBaGQkFMtueMjXBTaGHwdbMTWwrooJx1Jnr/BTp5oiAFwXcf8LiTMsaX6/gvSWmFWIjs GqYw== X-Gm-Message-State: AOAM530N1LnxryoNyr5ziUx8vmwFueYUNygEtwnBR15M5bPoMgEveZDi Hxz/hf/l1vCyay8pwWTKMFmLXQ7eXuBILg== X-Google-Smtp-Source: ABdhPJysXv4Fz9pFZh2lnfRT7LqU99HOpm2G4Pm3Q08v562U2QM12LH5YX5T4FUOLkBMAOPWOHSSGA== X-Received: by 2002:a17:906:7ac9:: with SMTP id k9mr36607812ejo.229.1620801507889; Tue, 11 May 2021 23:38:27 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:27 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 02/25] trace-cmd library: Remove unused API tracecmd_update_option Date: Wed, 12 May 2021 09:38:00 +0300 Message-Id: <20210512063823.324610-3-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Removed not used trace-cmd library API for updating the options section of already written trace file: tracecmd_update_option() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 3 -- lib/trace-cmd/trace-output.c | 40 ------------------- 2 files changed, 43 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 8ec33ea2..ecd172ca 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -290,9 +290,6 @@ int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); int tracecmd_write_options(struct tracecmd_output *handle); int tracecmd_append_options(struct tracecmd_output *handle); -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data); void tracecmd_output_close(struct tracecmd_output *handle); void tracecmd_output_free(struct tracecmd_output *handle); struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 9231c319..c0aa58cf 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -1229,46 +1229,6 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -int tracecmd_update_option(struct tracecmd_output *handle, - struct tracecmd_option *option, int size, - const void *data) -{ - tsize_t offset; - stsize_t ret; - - if (size > option->size) { - tracecmd_warning("Can't update option with more data than allocated"); - return -1; - } - - if (handle->file_state < TRACECMD_FILE_OPTIONS) { - /* Hasn't been written yet. Just update current pointer */ - option->size = size; - memcpy(option->data, data, size); - return 0; - } - - /* Save current offset */ - offset = lseek64(handle->fd, 0, SEEK_CUR); - - ret = lseek64(handle->fd, option->offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); - return -1; - } - - if (do_write_check(handle, data, size)) - return -1; - - ret = lseek64(handle->fd, offset, SEEK_SET); - if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offset); - return -1; - } - - return 0; -} - struct tracecmd_option * tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) From patchwork Wed May 12 06:38:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52D68C43460 for ; Wed, 12 May 2021 06:38:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 191C8613B4 for ; Wed, 12 May 2021 06:38:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229627AbhELGjn (ORCPT ); Wed, 12 May 2021 02:39:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229850AbhELGjm (ORCPT ); Wed, 12 May 2021 02:39:42 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DCB5C06175F for ; Tue, 11 May 2021 23:38:30 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id w3so33333555ejc.4 for ; Tue, 11 May 2021 23:38:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nRqZS1BvJvd5nWIx62PAgM57YEudBnyFXReBCf0NtS0=; b=X+wFw+m1trTZy8zQoykhPywhIVrW3HYBp10ZN3JA4i+aXCdrSqmeSFFpSQS1lD+5yq AhJI26p//mHK0UNWic0/gqrrhhrJ/DERodCBxJRFBLNGnqiMgkOEs8sZ5O9NNqyfo541 p9I+8+fKczDDgViwgNiMbniZQFAFhGJnN2QK367uNbLcQaayezxjCHsiO4GL/nUqr5Ju L/HuTx0jC+I3GFMTVQnM4cOklIWnI6++JNZuVeR3yJi3F6OweudChJh9QAXqQXa3oS3m h2Wnl3PcqrpNCP7FLH0x/LdV4oqstuwrccMGPZ+7MxDTpewS3SuFYD9S4D79XsdIbNzm zbrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nRqZS1BvJvd5nWIx62PAgM57YEudBnyFXReBCf0NtS0=; b=g86rg6DofesN8qvdQ/pgT4ghX/Ev2jsV8hkYZhYyzLA7Xr8INbzyp8H8/p5kXA6l+1 fUZFA4mc7DMSe44k481NjYOZeHGpXLjpBL5TIpKJ89mhOhrQCmB2Hhf9TqF4P1e91q8X VKUETDTiuKTFqjS5wWLcNjY6dAd5T3z0P8wSINAu9vSF34KTRfwydIcV1fBhX373ea+e NNiK2hncxq1IGrbFFiIvIJJMxzdLEabrbyzMbm5We7jzOXbNCbcRO2Y+Fzmu15WN1jtO 8lVL8wcY3njm9D4bXglfP+8LdoWslCiktdl+Dceb5YkrRNbEzdgVk+My0fShMuT2IU/g buYg== X-Gm-Message-State: AOAM533HfpiON9Xs//99lsVGbJa9Y8pRNGM6vSq2Hl72NOzuzJqluhJ9 1riJqlyS1F0Vcp+ObnXKRKspCmM6szABbA== X-Google-Smtp-Source: ABdhPJxK1VFdg+3XVnliG3EK2ZxnewTK/eSwFCOASyfVpyZxC+q+7c/H/WSgcQPkHOagyAztW0AJWw== X-Received: by 2002:a17:907:204b:: with SMTP id pg11mr3937147ejb.433.1620801508799; Tue, 11 May 2021 23:38:28 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:28 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 03/25] trace-cmd: Check if file version is supported Date: Wed, 12 May 2021 09:38:01 +0300 Message-Id: <20210512063823.324610-4-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace file, version of the file is ignored. This could case problems when bumping the version number because of changes in in the structure of the file. The old code should detect unsupported file version and should not try to read it. A new trace-cmd library API is added to check if version is supported: tracecmd_is_version_supported() Checks are added in the code to ensure not trying to read trace file from unsupported version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 10 ++++++++++ lib/trace-cmd/trace-util.c | 7 +++++++ tracecmd/trace-dump.c | 7 +++++++ 4 files changed, 26 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index ecd172ca..136057b4 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -42,6 +42,8 @@ void tracecmd_record_ref(struct tep_record *record); void tracecmd_set_debug(bool set_debug); bool tracecmd_get_debug(void); +bool tracecmd_is_version_supported(unsigned int version); + struct tracecmd_output; struct tracecmd_recorder; struct hook_list; diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 9726e994..b095004d 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -131,6 +131,7 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + int file_version; struct cpu_data *cpu_data; long long ts_offset; struct tsc2nsec tsc_calc; @@ -3269,6 +3270,7 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) unsigned int page_size; char *version; char buf[BUFSIZ]; + unsigned long ver; handle = malloc(sizeof(*handle)); if (!handle) @@ -3293,6 +3295,14 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (!version) goto failed_read; pr_stat("version = %s\n", version); + ver = strtol(version, NULL, 10); + if (!ver && errno) + goto failed_read; + if (!tracecmd_is_version_supported(ver)) { + tracecmd_warning("Unsupported file version %lu", ver); + goto failed_read; + } + handle->file_version = ver; free(version); if (do_read_check(handle, buf, 1)) diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 2d3bc741..a6fc64c8 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -582,3 +582,10 @@ unsigned long long tracecmd_generate_traceid(void) free(str); return hash; } + +bool tracecmd_is_version_supported(unsigned int version) +{ + if (version <= FILE_VERSION) + return true; + return false; +} diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 98425b98..03cc82b4 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "trace-local.h" @@ -145,6 +146,7 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; + unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -166,6 +168,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); + ver = strtol(buf, NULL, 10); + if (!ver && errno) + die("Invalid file version string %s", buf); + if (!tracecmd_is_version_supported(ver)) + die("Unsupported file version %lu", ver); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) From patchwork Wed May 12 06:38:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFE82C43462 for ; Wed, 12 May 2021 06:38:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A89D961927 for ; Wed, 12 May 2021 06:38:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229803AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229654AbhELGjm (ORCPT ); Wed, 12 May 2021 02:39:42 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBF43C061760 for ; Tue, 11 May 2021 23:38:30 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id j26so22038132edf.9 for ; Tue, 11 May 2021 23:38:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=T9eIxjxaxhocZQRyjD0lWh0WnJn5g8jbP9shwgVRU1Y=; b=BCAzbXotzSF+7a10JtQdwFugKMdiYntY/WD3dnyET743+O8h1HEoPVcAOywSkidasK IC5M2kf7PvEt+XQiVKTNWbEyFCfQWjQHm7YjDjnqor1Hk1Awq2fWbwmLmbV1EKA9uSJ7 VCopdxZy4i42zPSZAD8ToIa4sRE9EHg3OanoUoeBZjUPn0uE72X9OFwiH1T8cI09zQ0J hKnDLsZMiHeV+2pKyk/M3w+7MRTMVAn6ARWfO1AMvfRPyWzZgQDHM+OjW25sB5wlvUfL p5uNW3cfmCdOqJapRQ7LcZEAVAvGL1W0Ocss4GWbyQ/Kd0FkzbHffOKFCR5immybQTuf J1Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T9eIxjxaxhocZQRyjD0lWh0WnJn5g8jbP9shwgVRU1Y=; b=TecYd9sY3Jp7xqK1IBpW6/G5phF4U4CaaOxMOv+t47POdytWPKF5wUdFwQOOxnkIi3 Roz3rESs8RFtaewacNQHBujyY3ilHCQlAWGQVnmG9WWAS9Vrw5bWbYGQgDczW9aCbt99 dcRTcgOSb59nrHoiHlrVWJ8wGBctqAea/cFcUfRyp9PxffKatCsjp/e1q2/y4cfi2hxW L4Wo7i5EWnBLleheawqMn9PDeyfnuxTN1xJsR+H6Z78xGodQyWgkG400bq5O2I+2RvJD eAqw5jnl/Ywqxa8RdgSoTFzjlmB6rOYDy7bVyAjEza02S81mVUI2YeqP3dblWWmOJqwW utlg== X-Gm-Message-State: AOAM530uqqfshMlkWL4hv3Sqc0Obe1vs0GP0+xrIcW03CIyMcntI3IhO F4qzm65SFRUBCMoiiENzRtY= X-Google-Smtp-Source: ABdhPJzzRD1ypwM9X6nNUIACR7uw5bLZFrI6mIhv+wESdU9YNNFuxEj/8jUUOw+NqPEygdGpXbkHkA== X-Received: by 2002:aa7:c4d0:: with SMTP id p16mr42547532edr.102.1620801509638; Tue, 11 May 2021 23:38:29 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:29 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 04/25] trace-cmd library: Add new API to get file version of input handler Date: Wed, 12 May 2021 09:38:02 +0300 Message-Id: <20210512063823.324610-5-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added an API to get the version of the trace file, associated with given input file handler. tracecmd_get_file_version() Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 136057b4..6a211333 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -232,6 +232,8 @@ int tracecmd_set_cursor(struct tracecmd_input *handle, unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); +unsigned long tracecmd_get_file_version(struct tracecmd_input *handle); + int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); tracecmd_show_data_func diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index b095004d..9c9180d6 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4046,6 +4046,15 @@ struct tep_handle *tracecmd_get_tep(struct tracecmd_input *handle) return handle->pevent; } +/** + * tracecmd_get_file_version - return the trace.dat file version + * @handle: input handle for the trace.dat file + */ +unsigned long tracecmd_get_file_version(struct tracecmd_input *handle) +{ + return handle->file_version; +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Wed May 12 06:38:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 984DEC43461 for ; Wed, 12 May 2021 06:38:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5BA91613B4 for ; Wed, 12 May 2021 06:38:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229850AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229803AbhELGjm (ORCPT ); Wed, 12 May 2021 02:39:42 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD97BC06174A for ; Tue, 11 May 2021 23:38:31 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id s20so27793068ejr.9 for ; Tue, 11 May 2021 23:38:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9HuGT59fpnbaV/Ei2fhVY2rS3y5IDn6cNc88pcWTIj8=; b=ZLuIGyOdAt8FdvNI3yGRkN4RE2pLPsv5DTlhA4LIRmExVrroJ9IA97P/ODES9dFsfE Up5Sp6Iib2M+L/HySCi4lqw3KjAzIaaWU/wUuILWWXN/L9mCeJBCEbUh53GzzH/XriTK 4T6/8DN/uHya5sfFbKKCuXl8yXKxmHVBJD1hEKzArBP2NgJycxHJW5tri1sRQqLXqAJ8 kB7o+pssPBlVlu8mBA+d8OkjqPpq0xbo2/iPWYONI4E5/Jyd7NblCWRRm7fERSooy6Al 0E1J+7HQPwfZZi3u/ceU1UCfe2xvnuS9ITApZlrpRCd4OUF7BAc6tr16+df7E1teH5NI SBzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9HuGT59fpnbaV/Ei2fhVY2rS3y5IDn6cNc88pcWTIj8=; b=KiS53aMV0kOesRHLyyCjxCe0R6eRzetNaDgOX5bOGrQMevWmZKXNETTECime68/L30 wLYewkePMZDMAofFN8fD/gFjpXNpf28hksH9RKXNIvYqxzduyxwN/vnoI3eIi4S3TQ4T Z380n8hdVpliDBX7/Uc/x/gNRlryVi3L6aUJYzgIioLf6IsJ08SMFMX6W7v4jYiblkvA OXE9Mha+v01XQ4epYGXYb5egBxpVlFisKe8Zn3BHnjxNvJEQ7avdwa85nlSC1AXBD9Xo SUtTRBuagTJGzs3LOrIQ1DnmGR/Yea8xUtTEf2Y3khTdQ2fVuXc1jEoNNkhyH5L2t8he yP1g== X-Gm-Message-State: AOAM531v0ZZT34j8ak8h/egcdQjBed3sG4HTxf+8Ilb/3jSXD/gHLVjO X4D48sIcKWOVrkzHqj9upv4= X-Google-Smtp-Source: ABdhPJx/As7VUcIiPTJxSHyfQ9rq8oJlakGLd4mjmaQrEDALwHUToflIIG9FGre10TjxOtdasuR4ZQ== X-Received: by 2002:a17:907:10cb:: with SMTP id rv11mr36221198ejb.379.1620801510583; Tue, 11 May 2021 23:38:30 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:30 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 05/25] trace-cmd library: Select the file version when writing trace file Date: Wed, 12 May 2021 09:38:03 +0300 Message-Id: <20210512063823.324610-6-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new output handler to trace file is allocated, select the proper file version. If this output handler is based on an existing input trace file handler, inherit the trace file version. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index c0aa58cf..3c30edc1 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -57,6 +57,7 @@ struct tracecmd_output { int nr_options; bool quiet; unsigned long file_state; + unsigned long file_version; struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -907,6 +908,17 @@ out_free: return ret; } +static int select_file_version(struct tracecmd_output *handle, + struct tracecmd_input *ihandle) +{ + if (ihandle) + handle->file_version = tracecmd_get_file_version(ihandle); + else + handle->file_version = FILE_VERSION; + + return 0; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -933,6 +945,9 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; + if (select_file_version(handle, ihandle)) + goto out_free; + list_head_init(&handle->options); buf[0] = 23; @@ -943,7 +958,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, if (do_write_check(handle, buf, 10)) goto out_free; - if (do_write_check(handle, FILE_VERSION_STRING, strlen(FILE_VERSION_STRING) + 1)) + sprintf(buf, "%lu", handle->file_version); + if (do_write_check(handle, buf, strlen(buf) + 1)) goto out_free; /* get endian and page size */ @@ -1562,6 +1578,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->pevent = tracecmd_get_tep(ihandle); tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); + handle->file_version = tracecmd_get_file_version(ihandle); list_head_init(&handle->options); tracecmd_close(ihandle); From patchwork Wed May 12 06:38:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 372FBC43470 for ; Wed, 12 May 2021 06:38:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 06274613B4 for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230019AbhELGjp (ORCPT ); Wed, 12 May 2021 02:39:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230018AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E465C061761 for ; Tue, 11 May 2021 23:38:32 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id f24so33314576ejc.6 for ; Tue, 11 May 2021 23:38:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TgrEkpK0n5ohS6Kr+I4Byc4/t1KCKAcwzKJLo6TD8vA=; b=k9QGHgpi5eRQSvm6g49vW1v5f03PpfKwlX5ucbERAzxM4YoOlCHaAnlaGLpxSEhY6D nzH1Be5+Kqp4zNrE63k1E7i3x3VNUfbcZbzwWsPx9mV3Im4M9kUph5G4ZTC9NenPrQ1T Gs/7eXSFcRBuTCYDq9R3TyPEhZrk/dlM0JurffoxQoB5DxYbFgFGF8rDkO3/8BgHMKA0 PCZ9fOgu5tPzJjVoqRXnh6ZGS8f8/BAhuo3RimjD+SwYGI9wFmCCizPRkcnCSr4wboQU ehBxE1wuT9DLRRqABmWPeTAOpTBiuTnJErieSnuDmW0SDlcbtQOQsmSdMCVQfM/RCYXt vEdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TgrEkpK0n5ohS6Kr+I4Byc4/t1KCKAcwzKJLo6TD8vA=; b=Fl2/Ftv1nv/VElvuFXH/2eM4LuWmWds19dqKtN3Np4dSqGlGcQTLqUt6oi0vibPrX1 WG4Cm72O04CaAnCB9Ymlbj0OpLQlZyU97TUMSEilbG+M2gH7Ur0QVWQXAe9pjGa8YSJC 55Z+KOoMcG/fEEgjtyyw62jJLfhXs6qJP1CFWL1KB1Vr6WrIlmvh4jpvfc5zPBqiWK2h lUj1/5MOl+5QKIJEZLQjdDid7Fl6QyKjJU0DRn77ZiEAOJBaenk4AVvSKH1d0nD3LPSo BOXLifhlZ1EuLV9sAi38ec5pr0qNsKTDBHnPPsaFW5lK4RFDBveH+9BCrylIRuzdeMU4 Q3cg== X-Gm-Message-State: AOAM531aIVGOQe2zMPtCvvtgQp5/46zZ7wCEhS5pdrwI8GtqKQVY64Ng eGZlyQwK9YbpJD436KIHE0c= X-Google-Smtp-Source: ABdhPJznGmqDQTtpXBJ5yzuxaEfPw7FR9IS2O2K88etdDrQFdt6KDIq5jZEmcCYRdLAMzUSdOqLQBQ== X-Received: by 2002:a17:906:3458:: with SMTP id d24mr36777318ejb.54.1620801511588; Tue, 11 May 2021 23:38:31 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:31 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 06/25] trace-cmd: Add APIs for library initialization and free Date: Wed, 12 May 2021 09:38:04 +0300 Message-Id: <20210512063823.324610-7-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The trace-cmd library has no APIs for initialization and free of the whole library. Added these new APIs: tracecmd_lib_init() tracecmd_lib_free() and call them in trace-cmd main function. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 3 +++ lib/trace-cmd/trace-util.c | 9 +++++++++ tracecmd/trace-cmd.c | 11 ++++++++--- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 6a211333..e410a6f1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -29,6 +29,9 @@ struct tep_plugin_list *trace_load_plugins(struct tep_handle *tep, int flags); +int tracecmd_lib_init(void); +void tracecmd_lib_free(void); + int *tracecmd_add_id(int *list, int id, int len); enum { diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index a6fc64c8..3ef10eae 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -589,3 +589,12 @@ bool tracecmd_is_version_supported(unsigned int version) return true; return false; } + +int tracecmd_lib_init(void) +{ + return 0; +} + +void tracecmd_lib_free(void) +{ +} diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 7376c5a5..aad8e0a7 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -116,15 +116,20 @@ int main (int argc, char **argv) if (argc < 2) trace_usage(argc, argv); + tracecmd_lib_init(); + for (i = 0; i < ARRAY_SIZE(commands); ++i) { if (strcmp(argv[1], commands[i].name) == 0 ){ commands[i].run(argc, argv); - goto out; + break; } } + tracecmd_lib_free(); + /* No valid command found, show help */ - trace_usage(argc, argv); -out: + if (i == ARRAY_SIZE(commands)) + trace_usage(argc, argv); + exit(0); } From patchwork Wed May 12 06:38:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 518DAC433B4 for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1AA8761927 for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229654AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229996AbhELGjn (ORCPT ); Wed, 12 May 2021 02:39:43 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E565C061763 for ; Tue, 11 May 2021 23:38:33 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id s20so27793152ejr.9 for ; Tue, 11 May 2021 23:38:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=byTazfY4wZgpqpJmxbO4eOCl9GOTkMpmiaOEg/AhcAo=; b=SKmBCeOju239bPqQjh//uIujOXxp43B+ZDuB1S2QkqIAKFh5MVxbCrBcD5bf4Hjya7 Dzgortfmx9vxiQx1nKehSpzuGV1HWiN41Hi+zd4a5gW3TGhyUUK9qYeeFY8G+FgBmY0W QBo+NkK7Mdy7obnaBrEH9DfKnrMIE9ohPLqxYN+08iw6Dj1XPFMEFzOIg2qvoLiqjPW3 4GmqnWlw6NBMkdOXkUlPUJnzzo8hQbBRNqIhf2SltVhfKTqMyN/TKPMOBdmFRtJK/uUq /GTz2mOD7RBoFJXNB0GbJGDUBxlHoow+VcH1hd7v4dzpEyujbFScbEk8wGY99koHX6SS A5YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=byTazfY4wZgpqpJmxbO4eOCl9GOTkMpmiaOEg/AhcAo=; b=HJRW1Fe+IlZ4ctm/JrTXxpMaTqQZOaRYNLBtoQibZyA3/qk0Y4PAbJ9wJ4GXX2C68I vaR60X6nMAYg/w3PDQeTeTEgKSY1iNjspWEg749XgLblpZgDrqapFnaISuBmW6kS7xu8 pRBWkjK418Ot8h6FIGxKXZVHbGLn3jb7/MYGg1qc5g/9Uvyow0L/bFmiu9x9gWpyjIPj x0WxVOstY9vikbqhaJGTfcmhdk3NA/PNrjv1wYxiw2L2VLmcs9mIqaQEy1qu0F1OvpGc /eKcZ/nM/ER+f8e/6pF+kq98iJfPIjwZa95gpqRSwfwW7bpjceDnyx82e4FP1rxvC3VK w2Lg== X-Gm-Message-State: AOAM530jchtz4C88MIUABxB7cqguqu6lvvr33Wjg+P8iwSSq0iaJIbLz Hf49+63XXA8vEG70FVQyLXg= X-Google-Smtp-Source: ABdhPJxbZbj59GE7rbe7wt6zMmalOp0SUkXfRfcTokxwpLvz2WWIH61cXMU7SgtiOvX6HLljjBzvLA== X-Received: by 2002:a17:906:5211:: with SMTP id g17mr8260602ejm.281.1620801512578; Tue, 11 May 2021 23:38:32 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:32 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 07/25] trace-cmd library: Add support for compression algorithms Date: Wed, 12 May 2021 09:38:05 +0300 Message-Id: <20210512063823.324610-8-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added infrastructure to trace-cmd library for compression. Introduced various new APIs to work with this new functionality: struct tracecmd_compression tracecmd_compress_init() tracecmd_compress_free() tracecmd_compress_alloc() tracecmd_compress_destroy() tracecmd_compress_block() tracecmd_uncompress_block() tracecmd_compress_reset() tracecmd_compress_read() tracecmd_compress_pread() tracecmd_compress_write() tracecmd_compress_lseek() tracecmd_compress_proto_get_name() tracecmd_compress_is_supported() tracecmd_compress_protos_get() tracecmd_compress_proto_register() The compression algorithms are not part of this patch. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 1 + .../include/private/trace-cmd-private.h | 26 + lib/trace-cmd/include/trace-cmd-local.h | 2 + lib/trace-cmd/trace-compress.c | 588 ++++++++++++++++++ lib/trace-cmd/trace-util.c | 3 + 5 files changed, 620 insertions(+) create mode 100644 lib/trace-cmd/trace-compress.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index 17600318..bab4322d 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -25,6 +25,7 @@ ifeq ($(VSOCK_DEFINED), 1) OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif +OBJS += trace-compress.o # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index e410a6f1..602817cb 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -463,6 +463,32 @@ void tracecmd_tsync_free(struct tracecmd_time_sync *tsync); int tracecmd_write_guest_time_shift(struct tracecmd_output *handle, struct tracecmd_time_sync *tsync); +/* --- Compression --- */ +struct tracecmd_compression; +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle); +void tracecmd_compress_destroy(struct tracecmd_compression *handle); +int tracecmd_compress_block(struct tracecmd_compression *handle); +int tracecmd_uncompress_block(struct tracecmd_compression *handle); +void tracecmd_compress_reset(struct tracecmd_compression *handle); +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len); +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset); +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size); +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence); +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version); +bool tracecmd_compress_is_supported(const char *name, const char *version); +int tracecmd_compress_protos_get(char ***names, char ***versions); +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)); + /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index e8533b22..862283fc 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -30,5 +30,7 @@ void tracecmd_fatal(const char *fmt, ...); #endif #endif +int tracecmd_compress_init(void); +void tracecmd_compress_free(void); #endif /* _TRACE_CMD_LOCAL_H */ diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c new file mode 100644 index 00000000..ff742226 --- /dev/null +++ b/lib/trace-cmd/trace-compress.c @@ -0,0 +1,588 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include +#include + +#include "trace-cmd-private.h" +#include "trace-cmd-local.h" + +struct compress_proto { + struct compress_proto *next; + char *proto_name; + char *proto_version; + int weight; + + int (*comress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + int (*uncompress_block)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes); + unsigned int (*comress_size)(unsigned int bytes); + bool (*is_supported)(const char *name, const char *version); +}; + +static struct compress_proto *proto_list; + +struct tracecmd_compression { + int fd; + unsigned int capacity; + unsigned long pointer; + char *buffer; + struct compress_proto *proto; + struct tep_handle *tep; + struct tracecmd_msg_handle *msg_handle; +}; + +static int read_fd(int fd, char *dst, int len) +{ + size_t size = 0; + int r; + + do { + r = read(fd, dst+size, len); + if (r > 0) { + size += r; + len -= r; + } else + break; + } while (r > 0); + + if (len) + return -1; + return size; +} + +static long long write_fd(int fd, const void *data, size_t size) +{ + long long tot = 0; + long long w; + + do { + w = write(fd, data + tot, size - tot); + tot += w; + + if (!w) + break; + if (w < 0) + return w; + } while (tot != size); + + return tot; +} + +static long long do_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + int ret; + + if (handle->msg_handle) { + ret = tracecmd_msg_data_send(handle->msg_handle, data, size); + if (ret) + return -1; + return size; + } + return write_fd(handle->fd, data, size); +} + +/** + * tracecmd_compress_lseek - Move the read/write pointer into the compression buffer + * @handle: compression handler + * @offset: number of bytes to move the pointer, can be negative or positive + * @whence: the starting position of the pointer movement, + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns the new file pointer on success, or -1 in case of an error. + */ +off_t tracecmd_compress_lseek(struct tracecmd_compression *handle, off_t offset, int whence) +{ + unsigned long p, extend; + char *buf; + + if (!handle || !handle->buffer) + return (off_t)-1; + + switch (whence) { + case SEEK_CUR: + p = handle->pointer + offset; + break; + case SEEK_END: + p = handle->capacity + offset; + break; + case SEEK_SET: + p = offset; + break; + default: + return (off_t)-1; + } + + if (p <= handle->capacity) { + handle->pointer = p; + } else { + extend = p - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return (off_t)-1; + handle->buffer = buf; + handle->capacity += extend; + handle->pointer = p; + } + + return p; +} + +static int compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int s; + + if (handle->pointer + len > handle->capacity) + s = handle->capacity - handle->pointer; + else + s = len; + memcpy(dst, handle->buffer + handle->pointer, s); + + return s; +} + +/** + * tracecmd_compress_pread - pread() on compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * @offset: offset in the buffer of data to be read + * + * Read a @len of data from the compression buffer at given @offset, + * without updating the buffer pointer. + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_pread(struct tracecmd_compression *handle, char *dst, int len, off_t offset) +{ + int ret; + + if (!handle || !handle->buffer || offset > handle->capacity) + return -1; + + ret = tracecmd_compress_lseek(handle, offset, SEEK_SET); + if (ret < 0) + return ret; + return compress_read(handle, dst, len); +} + +/** + * tracecmd_compress_read - read() from compression buffer + * @handle: compression handler + * @dst: return, store the read data + * @len: length of data to be read + * + * Read a @len of data from the compression buffer + * + * On success returns the number of bytes read, or -1 on failure. + */ +int tracecmd_compress_read(struct tracecmd_compression *handle, char *dst, int len) +{ + int ret; + + if (!handle || !handle->buffer) + return -1; + + ret = compress_read(handle, dst, len); + if (ret > 0) + handle->pointer += ret; + + return ret; +} + +/** + * tracecmd_compress_reset - Reset the compression buffer + * @handle: compression handler + * + * Reset the compression buffer, any data currently in the buffer will be destroyed. + * + */ +void tracecmd_compress_reset(struct tracecmd_compression *handle) +{ + if (handle) { + free(handle->buffer); + handle->buffer = NULL; + handle->pointer = 0; + handle->capacity = 0; + } +} + +/** + * tracecmd_uncompress_block - uncompress a memory block + * @handle: compression handler + * + * Read compressed memory block from the file and uncompress it into internal buffer. + * The tracecmd_compress_read() can be used to read the uncompressed data from the buffer + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_uncompress_block(struct tracecmd_compression *handle) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + char buf[4]; + char *bytes = NULL; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + tracecmd_compress_reset(handle); + + if (read(handle->fd, buf, 4) != 4) + return -1; + s_compressed = tep_read_number(handle->tep, buf, 4); + if (read(handle->fd, buf, 4) != 4) + return -1; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + + handle->buffer = malloc(s_uncompressed); + if (!handle->buffer) + return -1; + bytes = malloc(s_compressed); + if (!bytes) + goto error; + + if (read_fd(handle->fd, bytes, s_compressed) < 0) + goto error; + ret = handle->proto->uncompress_block(bytes, s_compressed, + handle->buffer, &s_uncompressed); + if (ret) + goto error; + free(bytes); + handle->pointer = 0; + handle->capacity = s_uncompressed; + return 0; +error: + tracecmd_compress_reset(handle); + free(bytes); + return -1; +} + +/** + * tracecmd_compress_block - compress a memory block + * @handle: compression handler + * + * Compress the content of the internal memory buffer and write the compressed data in the file + * The tracecmd_compress_write() can be used to write data into the internal memory buffer, before + * calling this API. + * + * Returns 0 on success, or -1 in case of an error. + */ +int tracecmd_compress_block(struct tracecmd_compression *handle) +{ + unsigned int size; + char *buf; + int endian4; + int ret; + + size = handle->proto->comress_size(handle->pointer); + buf = malloc(size); + if (!buf) + return -1; + ret = handle->proto->comress_block(handle->buffer, handle->pointer, buf, &size); + if (ret < 0) + goto out; + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &handle->pointer, 4); + ret = do_write(handle, &endian4, 4); + if (ret != 4) + goto out; + /* Write the uncompressed data */ + ret = do_write(handle, buf, size); + ret = ((ret == size) ? 0 : -1); +out: + tracecmd_compress_reset(handle); + free(buf); + return ret; +} + +/** + * tracecmd_compress_write - write() to compression buffer + * @handle: compression handler + * @data: data to be written + * @size: size of @data + * + * Write @data of @size in the compression buffer + * + * Returns 0 on success, or -1 on failure. + */ +int tracecmd_compress_write(struct tracecmd_compression *handle, + const void *data, unsigned long long size) +{ + char *buf; + int extend; + + if (handle->capacity < handle->pointer + size) { + extend = (handle->pointer + size) - handle->capacity; + extend = extend < BUFSIZ ? BUFSIZ : extend; + buf = realloc(handle->buffer, handle->capacity + extend); + if (!buf) + return -1; + handle->buffer = buf; + handle->capacity += extend; + } + memcpy(&handle->buffer[handle->pointer], data, size); + handle->pointer += size; + return 0; +} + +/** + * tracecmd_compress_init - initialize the library with available compression algorithms + * + * Returns 0. If no compression algorithms are available, a warning is printed. + */ +int tracecmd_compress_init(void) +{ + struct timeval time; + + gettimeofday(&time, NULL); + srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); + + return 0; +} + +static struct compress_proto *compress_proto_select(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *selected = NULL; + + while (proto) { + if (!selected || selected->weight > proto->weight) + selected = proto; + proto = proto->next; + } + + return selected; +} + +/** + * tracecmd_compress_alloc - Allocate a new compression context + * @name: name of the compression algorithm + * @version: version of the compression algorithm + * @fd: file descriptor for reading / writing data + * @tep: tep handler, used to encode the data + * @msg_handle: message handler, use it for reading / writing data instead of @fd + * + * Returns NULL on failure or pointer to allocated compression context. + * The returned context must be freed by tracecmd_compress_destroy() + */ +struct tracecmd_compression *tracecmd_compress_alloc(const char *name, const char *version, + int fd, struct tep_handle *tep, + struct tracecmd_msg_handle *msg_handle) +{ + struct compress_proto *proto; + struct tracecmd_compression *new; + + if (name) { + proto = proto_list; + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + break; + proto = proto->next; + } + } else { + proto = compress_proto_select(); + } + if (!proto) + return NULL; + + new = calloc(1, sizeof(*new)); + if (!new) + return NULL; + new->fd = fd; + new->tep = tep; + new->msg_handle = msg_handle; + new->proto = proto; + return new; +} + +/** + * tracecmd_compress_destroy - Free a compression context + * @handle: handle to the compression context that will be freed + */ +void tracecmd_compress_destroy(struct tracecmd_compression *handle) +{ + tracecmd_compress_reset(handle); + free(handle); +} + +/** + * tracecmd_compress_is_supported - check if compression algorithm with given name and + * version is supported + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * + * Returns true if the algorithm with given name and version is supported or false if it is not. + */ +bool tracecmd_compress_is_supported(const char *name, const char *version) +{ + struct compress_proto *proto = proto_list; + + if (!name) + return NULL; + + while (proto) { + if (proto->is_supported && proto->is_supported(name, version)) + return true; + proto = proto->next; + } + return false; +} + +/** + * tracecmd_compress_proto_get_name - get name and version of compression algorithm + * @compress: compression handler. + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_compress_proto_get_name(struct tracecmd_compression *compress, + const char **name, const char **version) +{ + if (!compress || !compress->proto) + return -1; + if (name) + *name = compress->proto->proto_name; + if (version) + *version = compress->proto->proto_version; + return 0; +} + +/** + * tracecmd_compress_proto_register - register a new compression algorithm + * @name: name of the compression algorithm. + * @version: version of the compression algorithm. + * @weight: weight of the compression algorithm, lower is better. + * @compress: compression hook, called to compress a memory block. + * @uncompress: uncompression hook, called to uncompress a memory block. + * @comress_size: hook, called to get the required minimum size of the buffer for compression + * given number of bytes. + * @is_supported: check hook, called to check if compression with given name and version is + * supported by this plugin. + * + * Returns 0 on success, or -1 in case of an error. If algorithm with given name and version is + * already registered, -1 is returned. + */ +int tracecmd_compress_proto_register(const char *name, const char *version, int weight, + int (*compress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + int (*uncompress)(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes), + unsigned int (*comress_size)(unsigned int bytes), + bool (*is_supported)(const char *name, const char *version)) +{ + struct compress_proto *new; + + if (!name || !compress || !uncompress) + return -1; + if (tracecmd_compress_is_supported(name, version)) + return -1; + + new = calloc(1, sizeof(*new)); + if (!new) + return -1; + + new->proto_name = strdup(name); + if (!new->proto_name) + goto error; + new->proto_version = strdup(version); + if (!new->proto_version) + goto error; + new->comress_block = compress; + new->uncompress_block = uncompress; + new->comress_size = comress_size; + new->is_supported = is_supported; + new->weight = weight; + new->next = proto_list; + proto_list = new; + return 0; + +error: + free(new->proto_name); + free(new->proto_version); + free(new); + return -1; +} + +/** + * tracecmd_compress_free - free the library resources, related to available compression algorithms + * + */ +void tracecmd_compress_free(void) +{ + struct compress_proto *proto = proto_list; + struct compress_proto *del; + + while (proto) { + del = proto; + proto = proto->next; + free(del->proto_name); + free(del->proto_version); + free(del); + } + proto_list = NULL; +} + +/** + * tracecmd_compress_protos_get - get a list of all supported compression algorithms and versions + * @names: return, array with names of all supported compression algorithms + * @versions: return, array with versions of all supported compression algorithms + * + * On success, the size of @names and @versions arrays is returned. Those arrays are allocated by + * the API and must be freed with free() by the caller. Both arrays are with same size, each name + * from @names corresponds to a version from @versions. + * On error -1 is returned and @names and @versions arrays are not allocated. + */ +int tracecmd_compress_protos_get(char ***names, char ***versions) +{ + struct compress_proto *proto = proto_list; + char **n = NULL; + char **v = NULL; + int c, i; + + for (c = 0; proto; proto = proto->next) + c++; + + if (c < 1) + return c; + + n = calloc(c, sizeof(char *)); + if (!n) + goto error; + v = calloc(c, sizeof(char *)); + if (!v) + goto error; + + proto = proto_list; + for (i = 0; i < c && proto; i++) { + n[i] = proto->proto_name; + v[i] = proto->proto_version; + proto = proto->next; + } + + *names = n; + *versions = v; + return c; + +error: + free(n); + free(v); + return -1; +} diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c index 3ef10eae..a42499fe 100644 --- a/lib/trace-cmd/trace-util.c +++ b/lib/trace-cmd/trace-util.c @@ -592,9 +592,12 @@ bool tracecmd_is_version_supported(unsigned int version) int tracecmd_lib_init(void) { + + tracecmd_compress_init(); return 0; } void tracecmd_lib_free(void) { + tracecmd_compress_free(); } From patchwork Wed May 12 06:38:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7350C433ED for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 90A5F613BA for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230011AbhELGjp (ORCPT ); Wed, 12 May 2021 02:39:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230019AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DDFFC0613ED for ; Tue, 11 May 2021 23:38:34 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id t4so33379570ejo.0 for ; Tue, 11 May 2021 23:38:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L4cIrri7H50G2QgorkNc4Lv9OxhhIH4oM+gSM061M5w=; b=eYWSRnquTBWx/aPXFpdR6b74vv1lV0ak4ogCfYTrfWCyXFtuegwmkRQNe3HDEtiwCF kOrroY4kJ36ItyRUHyUnaDeTbWECj4rOFzXLVhIK0/O4zQo/wTYgd8b4V5Sc+i7FEY7R M1oljOu6wPKEOZ3qW9L6CQrFEqhoLeuSn4OYc2x8jrdPyBaFyEagbkEiB8KdIbMh+/R/ OrmI/ob9GFVDrsNoLuZdeSbuTf5n3Bm6pXK8CJtovyJpY47FxN9SijMvabmXX3GmPKWS ZXfB1ew3diyXHej8QcYi9AoZ3tTGgsAyz1DONc0hp7izmlFoRfv7SmaEqMjj6VkAYHZ6 urww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L4cIrri7H50G2QgorkNc4Lv9OxhhIH4oM+gSM061M5w=; b=VbKFuRfeuhgKAk/Jo8xIFDjKqK1I8sSUb2FmRl7ikFGcHoPNBcoRWxi//J3D+44/xT aFUf7jJauMa8lcPQ0QdAGIHGh1BA7JVF70yV9YBke/tEH7mL+wR16VVKx8rdcDDvxqXW J3jmqBTL4I+SpcWLQkbApcfnE4ELx9yTdTJ0SGtwStKlxPP8sR1Kvh230QHLOg4xKvrM Hi7z3tsnTH3fjnYEJOHgaSUbSCh1lkOaRWdHJQsuzxEBIgHSIr2WZ0fH1jTG+uQOZHKs e58gpgFLtWQN6yg7A7/mPv5w3brGxZ59UCL7LcwQa+ux3WrdjV5sochdIQaZL0KjFtsp RONw== X-Gm-Message-State: AOAM533mIthrzm8nHepOjCKhgwX/n1ogcs0skQaU9ejb8K4t6kJMIewD /ytqwmz8wNHg79fRfNgn0oQ= X-Google-Smtp-Source: ABdhPJw9FMF4eYVXDrd8APEyiNk3+sctmb6gUbrNzDlsBnfjMwNXDzEQ2VMHGfLSdPjzpd+mTSZL3w== X-Received: by 2002:a17:906:d159:: with SMTP id br25mr8593953ejb.248.1620801513391; Tue, 11 May 2021 23:38:33 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:32 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 08/25] trace-cmd list: Show supported compression algorithms Date: Wed, 12 May 2021 09:38:06 +0300 Message-Id: <20210512063823.324610-9-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Add new parameter "trace-cmd list -c" to show supported compression algorithms. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-list.c | 26 ++++++++++++++++++++++++++ tracecmd/trace-usage.c | 1 + 2 files changed, 27 insertions(+) diff --git a/tracecmd/trace-list.c b/tracecmd/trace-list.c index d060c810..900da73b 100644 --- a/tracecmd/trace-list.c +++ b/tracecmd/trace-list.c @@ -549,6 +549,24 @@ static void show_plugins(void) tep_free(pevent); } +static void show_compression(void) +{ + char **versions, **names; + int c, i; + + c = tracecmd_compress_protos_get(&names, &versions); + if (c <= 0) { + printf("No compression algorithms are supported\n"); + return; + } + printf("Supported compression algorithms:\n"); + for (i = 0; i < c; i++) + printf("\t%s, %s\n", names[i], versions[i]); + + free(names); + free(versions); +} + void trace_list(int argc, char **argv) { int events = 0; @@ -562,6 +580,7 @@ void trace_list(int argc, char **argv) int flags = 0; int systems = 0; int show_all = 1; + int compression = 0; int i; const char *arg; const char *funcre = NULL; @@ -626,6 +645,10 @@ void trace_list(int argc, char **argv) systems = 1; show_all = 0; break; + case 'c': + compression = 1; + show_all = 0; + break; case '-': if (strcmp(argv[i], "--debug") == 0) { tracecmd_set_debug(true); @@ -670,6 +693,8 @@ void trace_list(int argc, char **argv) show_clocks(); if (systems) show_systems(); + if (compression) + show_compression(); if (show_all) { printf("event systems:\n"); show_systems(); @@ -679,6 +704,7 @@ void trace_list(int argc, char **argv) show_tracers(); printf("\noptions:\n"); show_options(); + show_compression(); } return; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 998f5a24..c388eccd 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -330,6 +330,7 @@ static struct usage_help usage_help[] = { " -O list plugin options\n" " -B list defined buffer instances\n" " -C list the defined clocks (and active one)\n" + " -c list the supported trace file compression algorithms\n" }, { "restore", From patchwork Wed May 12 06:38:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8078DC43600 for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48D8C613BA for ; Wed, 12 May 2021 06:38:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229996AbhELGjp (ORCPT ); Wed, 12 May 2021 02:39:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230011AbhELGjn (ORCPT ); Wed, 12 May 2021 02:39:43 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AF4EC061574 for ; Tue, 11 May 2021 23:38:35 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id a4so33346232ejk.1 for ; Tue, 11 May 2021 23:38:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=asuuAM/DbFHwoKgAFTzA5hM0EAmoCA+ufY1mLOt9v2I=; b=gBIVRftNEQK6KQXU731sb2OR9VjL47FNWWptyluy07GKb0sama5bG8Evk1ReGQfKga YdDa9k1d1RXqK9QlFFxWVtrNBWoFh3XRh5kaxT6LlXy3BQ9qGdW1U8VK80sS/f/X9hOH NT22yS0cexieVbjsKaqG4oeZ+Xg/sCn6ZlA5TVDymZcJ7s+k9wiG+xD+p5qxYKMSTknD M5P1PfFX3LYxlt0wGDEELjvTGaiLB6m66W+vF4aSh5VzNVhq/iLzcH614PPZ5XftB5wi 857x81hs/eALdC6Gj+//45n/0p8ZIPBE+TbXLVYULuh3rCh4ZC8PzhSmsoJH02uFrwyt RfDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=asuuAM/DbFHwoKgAFTzA5hM0EAmoCA+ufY1mLOt9v2I=; b=qGmskjyaqaEo1InJtIFpFY+n6floBex+4YtFV8C2pUU1DWOktW17Ky0CiSO6uTptBy 6oj70vhEV0VOVsaMzL8FIepLC7aGrT6UIcqfCbq7noNejxpyyF8NYYQk9Ptwo8MzZjah ovT6DPPQCMEpJcQPyoim8hGMO1NTA6jAnLL7uDneIgvr1bhxGe/eHxclFVEjMOXmGOqJ k+HHVNLZdD+V2rC/MB/G5k4oGk6TP7xVZVxGE2owfKBYlDBmIFOXzHs/CQawq8iPDiIK /zBeH9a9e/wOhSWYdGLNMGrDG5FrTPBRKNaw3yh01a9/d0FNE7Ia//kwJorAY0cBe5q6 VcnA== X-Gm-Message-State: AOAM530gTfY0PO/bcLKhvrMbmpJXdmmkajSxhqhK7cfuL/X277iCQCuM P6G/AXLbletWAurLChPpaPk= X-Google-Smtp-Source: ABdhPJyHUAci9ZsUmY6k8Yr+H7rZ0A9+pKRR0R9kHSHNb0v3Q6i+Q8ijQ/a7pQnwFTvePsWwrRJQWQ== X-Received: by 2002:a17:906:fa19:: with SMTP id lo25mr35239957ejb.394.1620801514325; Tue, 11 May 2021 23:38:34 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:33 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 09/25] trace-cmd library: Bump the trace file version to 7 Date: Wed, 12 May 2021 09:38:07 +0300 Message-Id: <20210512063823.324610-10-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Adding a compression of the trace.dat file will change its structure. These changes are not backward compatible, the old trace-cmd binaries will not be able to read compressed trace files. Bumping the version to 7 will prevent old trace-cmd to read such files. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/trace-cmd-local.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index 862283fc..c8c51ff0 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -14,7 +14,7 @@ void tracecmd_warning(const char *fmt, ...); void tracecmd_fatal(const char *fmt, ...); /* trace.dat file format version */ -#define FILE_VERSION 6 +#define FILE_VERSION 7 #define _STR(x) #x #define STR(x) _STR(x) From patchwork Wed May 12 06:38:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAEFAC43603 for ; Wed, 12 May 2021 06:38:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9434E613B4 for ; Wed, 12 May 2021 06:38:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230018AbhELGjq (ORCPT ); Wed, 12 May 2021 02:39:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230026AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D860C06174A for ; Tue, 11 May 2021 23:38:36 -0700 (PDT) Received: by mail-ej1-x62d.google.com with SMTP id k10so2087919ejj.8 for ; Tue, 11 May 2021 23:38:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PEr253Mm0RmbxPJKfbLv9EYOe3AtPhgRSq8+c7MGg64=; b=JaeS4nxg0kZ8Cha+6ThQiU2v9HrRjkhGC76U85AtYUjtJlCeudyc4WZ1e1Cg/QpJla cqpeIBVMBXgsV5D45ZH8KfEOd7LgbsH7e7vDx39WEmwVu2VjX6lmCAguwAgP5mq+DcxU BM+cnG/W5TSzHQ2fnnTEXtPmXgKdwkv/BofF61hvE30ygj/0MFNGHVjUA6HpqPKXHMnl YlKP4hMUm/ikS1x5L62j2veLlBUZsyM0sxc/hYVqK3G4DdeNXh2i6lVlmtXzXghEiLIG maXQ0X3b/nD5gSz8bB7l3kcaH5/7wpLO3G4dcTCcbYUDoWPrtJi8yeE5DNa+0+FatEpw Wj8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PEr253Mm0RmbxPJKfbLv9EYOe3AtPhgRSq8+c7MGg64=; b=pq2/pFZfQrV37UIxeuXH62kn4MFAc8RjwDMYlii+3AjeHIYVfxP9PF2yRb6sPLe3BN XIN006nuwDkJeaYlczfqQu6NV5STIM7XhHK5oryY9KYjwlmMsu8LVMrYoaPBiejqnxjk 9ao7GDlIlYs+Oswc9ETq6LXCIwv+l+vuxpKqJfjxkHqzOfqxZWvcsyShqbMMhgZDFuxn OsNCij7BbybfqCMxc6zexH40okyR8hnOw0kw/uVz6Rq3W4/zD+aut1/TnQ71qXqkyvHZ 9K5HcT1OQFHREKfDewfe2+sNAyk0Accfie8KYqc3dyCAgM3KDB2v+XC39PvizLOAhY/v XNJQ== X-Gm-Message-State: AOAM533AzqOHl4729m71iTtqZKWHubtgh9CKQoXEXKzKIHVNpIgcCq3P gfjVdojzuCaO3XkzFn5k/o0= X-Google-Smtp-Source: ABdhPJzvIGuKuQN0P0cdlLoDkzecDbny6cf2Ipw/9IEq/f4KxOjS1HOHyhBtGXR9VAMj/c//JJh9OA== X-Received: by 2002:a17:907:c04:: with SMTP id ga4mr23960442ejc.413.1620801515219; Tue, 11 May 2021 23:38:35 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:34 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 10/25] trace-cmd library: Compress part of the trace file Date: Wed, 12 May 2021 09:38:08 +0300 Message-Id: <20210512063823.324610-11-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Compress part of the trace.dat file metadata. If there is compression support, chose file version 7 and compress these parts of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name A new compression header is added in the file, right after the page size information. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 145 +++++++++++++++++++++++++++++------ 1 file changed, 123 insertions(+), 22 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 3c30edc1..627cea42 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -58,6 +58,10 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + + bool do_compress; + struct tracecmd_compression *compress; + struct list_head options; struct tracecmd_msg_handle *msg_handle; char *trace_clock; @@ -78,12 +82,31 @@ struct list_event_system { static stsize_t do_write_check(struct tracecmd_output *handle, const void *data, tsize_t size) { + if (handle->do_compress && handle->compress) + return tracecmd_compress_write(handle->compress, data, size); + if (handle->msg_handle) return tracecmd_msg_data_send(handle->msg_handle, data, size); return __do_write_check(handle->fd, data, size); } +static inline int do_lseek(struct tracecmd_output *handle, off_t offset, int whence) +{ + if (handle->do_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek64(handle->fd, offset, whence); +} + +static inline int do_preed(struct tracecmd_output *handle, void *dst, int len, off_t offset) +{ + if (handle->do_compress) + return tracecmd_compress_pread(handle->compress, dst, len, offset); + else + return pread(handle->fd, dst, len, offset); +} + static short convert_endian_2(struct tracecmd_output *handle, short val) { if (!handle->pevent) @@ -109,6 +132,31 @@ static unsigned long long convert_endian_8(struct tracecmd_output *handle, return tep_read_number(handle->pevent, &val, 8); } +static inline void compression_reset(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return; + tracecmd_compress_reset(handle->compress); + handle->do_compress = false; +} + +static inline int compression_start(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + tracecmd_compress_reset(handle->compress); + handle->do_compress = true; + return 0; +} + +static inline int compression_end(struct tracecmd_output *handle) +{ + if (handle->file_version < 7 || !handle->compress) + return 0; + handle->do_compress = false; + return tracecmd_compress_block(handle->compress); +} + /** * tracecmd_set_quiet - Set if to print output to the screen * @quiet: If non zero, print no output to the screen @@ -159,6 +207,7 @@ void tracecmd_output_free(struct tracecmd_output *handle) free(option); } free(handle->trace_clock); + tracecmd_compress_destroy(handle->compress); free(handle); } @@ -659,13 +708,17 @@ static int read_ftrace_files(struct tracecmd_output *handle) } create_event_list_item(handle, &systems, &list); - + compression_start(handle); ret = copy_event_system(handle, systems); + if (!ret) + ret = compression_end(handle); + else + compression_reset(handle); free_list_events(systems); - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - + if (!ret) + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; return ret; } @@ -715,6 +768,7 @@ static int read_event_files(struct tracecmd_output *handle, for (slist = systems; slist; slist = slist->next) count++; + compression_start(handle); ret = -1; endian4 = convert_endian_4(handle, count); if (do_write_check(handle, &endian4, 4)) @@ -730,8 +784,14 @@ static int read_event_files(struct tracecmd_output *handle, ret = copy_event_system(handle, slist); } - handle->file_state = TRACECMD_FILE_ALL_EVENTS; + if (!ret) + ret = compression_end(handle); out_free: + if (!ret) + handle->file_state = TRACECMD_FILE_ALL_EVENTS; + else + compression_reset(handle); + free_list_events(systems); return ret; @@ -793,20 +853,20 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, if (kallsyms) path = kallsyms; - + compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ size = 0; endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; - return 0; + ret = do_write_check(handle, &endian4, 4); + goto out; } size = get_size(path); endian4 = convert_endian_4(handle, size); - if (do_write_check(handle, &endian4, 4)) - return -1; + ret = do_write_check(handle, &endian4, 4); + if (ret) + goto out; set_proc_kptr_restrict(0); check_size = copy_file(handle, path); @@ -814,13 +874,18 @@ static int read_proc_kallsyms(struct tracecmd_output *handle, errno = EINVAL; tracecmd_warning("error in size of file '%s'", path); set_proc_kptr_restrict(1); - return -1; + ret = -1; + goto out; } set_proc_kptr_restrict(1); - handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = compression_end(handle); +out: + if (!ret) + handle->file_state = TRACECMD_FILE_KALLSYMS; + else + compression_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_output *handle) @@ -840,6 +905,7 @@ static int read_ftrace_printk(struct tracecmd_output *handle) if (!path) return -1; + compression_start(handle); ret = stat(path, &st); if (ret < 0) { /* not found */ @@ -861,11 +927,14 @@ static int read_ftrace_printk(struct tracecmd_output *handle) } out: - handle->file_state = TRACECMD_FILE_PRINTK; put_tracing_file(path); + if (compression_end(handle)) + return -1; + handle->file_state = TRACECMD_FILE_PRINTK; return 0; fail: put_tracing_file(path); + compression_reset(handle); return -1; } @@ -911,14 +980,39 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { - if (ihandle) + if (ihandle) { handle->file_version = tracecmd_get_file_version(ihandle); - else - handle->file_version = FILE_VERSION; + } else { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + } return 0; } +static int write_compression_header(struct tracecmd_output *handle) +{ + const char *name = NULL; + const char *ver = NULL; + char *buf; + int ret; + + ret = tracecmd_compress_proto_get_name(handle->compress, &name, &ver); + if (ret < 0 || !name || !ver) + return -1; + ret = asprintf(&buf, "%s %s", name, ver); + if (ret < 0) + return -1; + ret = do_write_check(handle, buf, strlen(buf) + 1); + + free(buf); + return ret; +} + static struct tracecmd_output * create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, @@ -993,7 +1087,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, if (do_write_check(handle, &endian4, 4)) goto out_free; handle->file_state = TRACECMD_FILE_INIT; - + if (handle->file_version >= 7 && write_compression_header(handle)) + goto out_free; if (ihandle) return handle; @@ -1285,11 +1380,17 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) handle->file_state); return ret; } + compression_start(handle); + ret = save_tracing_file_data(handle, "saved_cmdlines"); - if (ret < 0) + if (ret < 0) { + compression_reset(handle); return ret; - handle->file_state = TRACECMD_FILE_CMD_LINES; - return 0; + } + ret = compression_end(handle); + if (!ret) + handle->file_state = TRACECMD_FILE_CMD_LINES; + return ret; } struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) From patchwork Wed May 12 06:38:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19F9AC43611 for ; Wed, 12 May 2021 06:38:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD619617C9 for ; Wed, 12 May 2021 06:38:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230026AbhELGjq (ORCPT ); Wed, 12 May 2021 02:39:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40732 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230075AbhELGjo (ORCPT ); Wed, 12 May 2021 02:39:44 -0400 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C679C061574 for ; Tue, 11 May 2021 23:38:37 -0700 (PDT) Received: by mail-ej1-x636.google.com with SMTP id a4so33346323ejk.1 for ; Tue, 11 May 2021 23:38:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Boy9Dx9PB3F1HURcqneksD9K9DTX55V0wboIMImbeNA=; b=WVLRCwrvIygmlylQzvLfc7ampCLyzSe970X6m3G+kkFJNYxBmtwdZS0F2ViykWPX9c soHdWwT22mCmVgr0Mwl/eSkt7q/OO/u0OtO7BfGjF37+1FMJIJ7laNI9BHdBdYdEy4HD puJui+PrbQ68jHBluLQzy1HNDOM2j8TeCYV0waQmyf2CgIePcvf59VaBsFDAuSKbRchI OzJVZWjV4sP4BU36jWLfZeThJOEvhtokUSmeEuW0qyPGf37LPQxfiToFrHle6lcbbNC0 DXIqVX11OdjU5GCwJH55Bmh7a2J50j9nVqv4JTLICC7EVkUOcXRLKGqCBFdU5wIygkdF 1hzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Boy9Dx9PB3F1HURcqneksD9K9DTX55V0wboIMImbeNA=; b=HvWx6dZJP2GatoTUpiDcaW+veoqqaUwR5owQAJMJd/LnfwE72TPPLo+tZz++zyxc6P egIP1q6LF8ttHNhFXfVabshqoae7IzKFc2S2pBZJWIRrtD80FSowaML2esmfqkhR8fJ5 h6Tb92P0LfUQHS4G/0n+f8bBIJoc4CWt9BXrbsKDl4yC7JS0gshYU6gZtGL3d5BBME6j aDpC2y1PMs+QUHiZrIZsW2uRfNzIRPyKNAjbTWNppkWfCTEdpsVbn1QtkHxMMqnC79PG jGl95hvFqbrMt1gKKKooid30vigK6ji8Z/Dqqih3depwLduuz9FRHw4vHYX1KUisMOrm LJyQ== X-Gm-Message-State: AOAM53287MXtv+hrArgyHsqGlL3bQCPrF6aYTEHYr1GFZN5IozVDREyE REqumkwkW7phuKSmnUcgABQ= X-Google-Smtp-Source: ABdhPJzoTWNrBrAQUl7lmdt26/q8SAQM5D4RxxU9vRRA38AMHkKw3PEtSHnxizYvkmw30jK46c0DRg== X-Received: by 2002:a17:907:2050:: with SMTP id pg16mr13605480ejb.255.1620801516057; Tue, 11 May 2021 23:38:36 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:35 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 11/25] trace-cmd library: Read compressed trace file Date: Wed, 12 May 2021 09:38:09 +0300 Message-Id: <20210512063823.324610-12-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If a trace file version 7 is detected, read compressed sections of it. Read the compression header, check if the compression algorithm and version, used to compress the file, are supported and use it to uncompress these sections of the file: - ftrace events format - format of recorded events - information of the mapping of function addresses to the function names - trace_printk() format strings - information of the mapping a PID to a process name Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 247 +++++++++++++++++++++++++----------- 1 file changed, 171 insertions(+), 76 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 9c9180d6..a23c26bd 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -136,6 +136,9 @@ struct tracecmd_input { long long ts_offset; struct tsc2nsec tsc_calc; + bool read_compress; + struct tracecmd_compression *compress; + struct host_trace_info host; double ts2secs; char * cpustats; @@ -255,12 +258,28 @@ static ssize_t do_read(struct tracecmd_input *handle, void *data, size_t size) return tot; } +static inline int do_lseek(struct tracecmd_input *handle, int offset, int whence) +{ + if (handle->read_compress) + return tracecmd_compress_lseek(handle->compress, offset, whence); + else + return lseek(handle->fd, offset, whence); +} + +static inline ssize_t do_read_compressed(struct tracecmd_input *handle, void *data, size_t size) +{ + if (handle->read_compress) + return tracecmd_compress_read(handle->compress, data, size); + else + return do_read(handle, data, size); +} + static ssize_t do_read_check(struct tracecmd_input *handle, void *data, size_t size) { ssize_t ret; - ret = do_read(handle, data, size); + ret = do_read_compressed(handle, data, size); if (ret < 0) return ret; if (ret != size) @@ -278,10 +297,8 @@ static char *read_string(struct tracecmd_input *handle) ssize_t r; for (;;) { - r = do_read(handle, buf, BUFSIZ); - if (r < 0) - goto fail; - if (!r) + r = do_read_compressed(handle, buf, BUFSIZ); + if (r <= 0) goto fail; for (i = 0; i < r; i++) { @@ -307,7 +324,7 @@ static char *read_string(struct tracecmd_input *handle) } /* move the file descriptor to the end of the string */ - r = lseek(handle->fd, -(r - (i+1)), SEEK_CUR); + r = do_lseek(handle, -(r - (i+1)), SEEK_CUR); if (r < 0) goto fail; @@ -359,6 +376,26 @@ static int read8(struct tracecmd_input *handle, unsigned long long *size) return 0; } +static void uncompress_reset(struct tracecmd_input *handle) +{ + if (handle->compress && handle->file_version >= 7) { + handle->read_compress = false; + tracecmd_compress_reset(handle->compress); + } +} + +static int uncompress_block(struct tracecmd_input *handle) +{ + int ret = 0; + + if (handle->compress && handle->file_version >= 7) { + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->read_compress = true; + } + return ret; +} + static int read_header_files(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; @@ -602,34 +639,40 @@ static int read_ftrace_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) + if (uncompress_block(handle)) return -1; + ret = read4(handle, &count); + if (ret < 0) + goto out; + for (i = 0; i < count; i++) { - if (read8(handle, &size) < 0) - return -1; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_ftrace_file(handle, size, print_all, ereg); if (ret < 0) - return -1; + goto out; } handle->event_files_start = lseek64(handle->fd, 0, SEEK_CUR); + handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; + ret = 0; +out: if (sreg) { regfree(sreg); regfree(ereg); } - - handle->file_state = TRACECMD_FILE_FTRACE_EVENTS; - - return 0; + uncompress_reset(handle); + return ret; } static int read_event_files(struct tracecmd_input *handle, const char *regex) { unsigned long long size; - char *system; + char *system = NULL; regex_t spreg; regex_t epreg; regex_t *sreg = NULL; @@ -654,13 +697,19 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) return -1; } - if (read4(handle, &systems) < 0) + if (uncompress_block(handle)) return -1; + ret = read4(handle, &systems); + if (ret < 0) + goto out; + for (i = 0; i < systems; i++) { system = read_string(handle); - if (!system) - return -1; + if (!system) { + ret = -1; + goto out; + } sys_printed = 0; print_all = 0; @@ -687,103 +736,117 @@ static int read_event_files(struct tracecmd_input *handle, const char *regex) } } - if (read4(handle, &count) < 0) - goto failed; + ret = read4(handle, &count); + if (ret < 0) + goto out; for (x=0; x < count; x++) { - if (read8(handle, &size) < 0) - goto failed; + ret = read8(handle, &size); + if (ret < 0) + goto out; ret = read_event_file(handle, system, size, print_all, &sys_printed, reg); if (ret < 0) - goto failed; + goto out; } free(system); - } - - if (sreg) { - regfree(sreg); - regfree(ereg); + system = NULL; } handle->file_state = TRACECMD_FILE_ALL_EVENTS; - - return 0; - - failed: + ret = 0; + out: + uncompress_reset(handle); if (sreg) { regfree(sreg); regfree(ereg); } free(system); - return -1; + return ret; } static int read_proc_kallsyms(struct tracecmd_input *handle) { - struct tep_handle *pevent = handle->pevent; + struct tep_handle *tep = handle->pevent; unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_KALLSYMS) return 0; - if (read4(handle, &size) < 0) + if (uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ - buf = malloc(size+1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)){ - free(buf); - return -1; + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_KALLSYMS; + goto out; /* OK? */ } - buf[size] = 0; - - tep_parse_kallsyms(pevent, buf); - free(buf); + buf = malloc(size+1); + if (!buf) { + ret = -1; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; + buf[size] = 0; + tep_parse_kallsyms(tep, buf); handle->file_state = TRACECMD_FILE_KALLSYMS; - - return 0; + ret = 0; +out: + free(buf); + uncompress_reset(handle); + return ret; } static int read_ftrace_printk(struct tracecmd_input *handle) { unsigned int size; - char *buf; + char *buf = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_PRINTK) return 0; - if (read4(handle, &size) < 0) + if (uncompress_block(handle)) return -1; - if (!size) - return 0; /* OK? */ + + ret = read4(handle, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_PRINTK; + goto out; /* OK? */ + } buf = malloc(size + 1); - if (!buf) - return -1; - if (do_read_check(handle, buf, size)) { - free(buf); - return -1; + if (!buf) { + ret = -1; + goto out; } + ret = do_read_check(handle, buf, size); + if (ret < 0) + goto out; buf[size] = 0; tep_parse_printk_formats(handle->pevent, buf); - - free(buf); - handle->file_state = TRACECMD_FILE_PRINTK; + ret = 0; - return 0; +out: + free(buf); + uncompress_reset(handle); + return ret; } static int read_and_parse_cmdlines(struct tracecmd_input *handle); @@ -2974,20 +3037,30 @@ static int read_and_parse_cmdlines(struct tracecmd_input *handle) { struct tep_handle *pevent = handle->pevent; unsigned long long size; - char *cmdlines; + char *cmdlines = NULL; + int ret; if (handle->file_state >= TRACECMD_FILE_CMD_LINES) return 0; - if (read_data_and_size(handle, &cmdlines, &size) < 0) + if (uncompress_block(handle)) return -1; + + ret = read_data_and_size(handle, &cmdlines, &size); + if (ret < 0) + goto out; + if (!size) { + handle->file_state = TRACECMD_FILE_CMD_LINES; + goto out; + } cmdlines[size] = 0; tep_parse_saved_cmdlines(pevent, cmdlines); - free(cmdlines); - handle->file_state = TRACECMD_FILE_CMD_LINES; - - return 0; + ret = 0; +out: + free(cmdlines); + uncompress_reset(handle); + return ret; } static void extract_trace_clock(struct tracecmd_input *handle, char *line) @@ -3268,7 +3341,8 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) struct tracecmd_input *handle; char test[] = TRACECMD_MAGIC; unsigned int page_size; - char *version; + char *str = NULL; + char *zver; char buf[BUFSIZ]; unsigned long ver; @@ -3291,11 +3365,11 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) if (memcmp(buf, "tracing", 7) != 0) goto failed_read; - version = read_string(handle); - if (!version) + str = read_string(handle); + if (!str) goto failed_read; - pr_stat("version = %s\n", version); - ver = strtol(version, NULL, 10); + pr_stat("version = %s\n", str); + ver = strtol(str, NULL, 10); if (!ver && errno) goto failed_read; if (!tracecmd_is_version_supported(ver)) { @@ -3303,7 +3377,8 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) goto failed_read; } handle->file_version = ver; - free(version); + free(str); + str = NULL; if (do_read_check(handle, buf, 1)) goto failed_read; @@ -3337,11 +3412,30 @@ struct tracecmd_input *tracecmd_alloc_fd(int fd, int flags) handle->header_files_start = lseek64(handle->fd, handle->header_files_start, SEEK_SET); + if (handle->file_version >= 7) { + str = read_string(handle); + if (!str) + goto failed_read; + zver = strchr(str, ' '); + if (!zver) + goto failed_read; + *zver = '\0'; + handle->compress = tracecmd_compress_alloc(str, zver + 1, + handle->fd, handle->pevent, NULL); + if (!handle->compress) { + tracecmd_warning("Unsupported file compression %s %s", str, zver + 1); + goto failed_read; + } + free(str); + str = NULL; + } + handle->file_state = TRACECMD_FILE_INIT; return handle; failed_read: + free(str); free(handle); return NULL; @@ -3519,7 +3613,8 @@ void tracecmd_close(struct tracecmd_input *handle) if (handle->flags & TRACECMD_FL_BUFFER_INSTANCE) tracecmd_close(handle->parent); else { - /* Only main handle frees plugins and pevent */ + /* Only main handle frees plugins, pevent and compression context */ + tracecmd_compress_destroy(handle->compress); tep_unload_plugins(handle->plugin_list, handle->pevent); tep_free(handle->pevent); } From patchwork Wed May 12 06:38:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 425BEC43462 for ; Wed, 12 May 2021 06:38:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 14935613B4 for ; Wed, 12 May 2021 06:38:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230075AbhELGjr (ORCPT ); Wed, 12 May 2021 02:39:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230109AbhELGjq (ORCPT ); Wed, 12 May 2021 02:39:46 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27129C061574 for ; Tue, 11 May 2021 23:38:38 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id c22so5194800ejd.12 for ; Tue, 11 May 2021 23:38:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xlVP7vcl2IIrsv9q322xpUVzXj6w983u8QgNphroLKE=; b=jKvPhVkIJDsQoGWr4ONKnJ78M7fgAZLnB5yIs50YmYeAQX60TiCeN4mYAg22hlRNRG t0xhEoDUnl4jAFIEfBa/4u1P2P0urfVH1NAiGqT84vDa+46cTm1SqWOPiCd8D1O7TQnX nKu2E+s9f8jcbzPxYlslzZb5iqYvFiMKKkAJ1XYf8dPaIHjJZTMM98Og+7HBEZi88Ila N63l5JMa8kkQIRiLajCBwOVvsMxRJaQPWfk2a4dMUVIps5sW7M0+Qw4cHBTpEaN2fKwb YuYFsEUkLRBYojGq0Uhjtoji3xtKJsNRGrGKPazUTOQfF7keovQ3zd/glReShs1a3C7i pTGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xlVP7vcl2IIrsv9q322xpUVzXj6w983u8QgNphroLKE=; b=ss0FFYzvvX4gakOWiY6HMha+Trbd2LtzNgAROou0e58U6Bg79PLcfud1LhaWAKrenP mI0GRQyFJ+mNMk791zwZI6H2E4joDpQCxR5YVP92uOSurPi7VyqYuo450anYyi6E88Kx KSkz51DmmFlcJ05tihR1LZ4BIEWp7Lff8gKx9C60RDmSb3VaE1NE6brhN3lHIQRVw3SG p8noqZ4/6UxBnCHy5RA5RmUGDvm8uHWJT9gAp3W/CF86PrFSjaEswNFGVjoMKz0z3fx9 iH8IcrNky5RQqS6ot8WlTSYbCIrSo6rN5NP/Az1iqDOwZUvFv3L+FyY2U4A1wq2YIWmt 4FVA== X-Gm-Message-State: AOAM533R5+8xe0YUIUyCQZsCoqjhhP96X3uMxcorNuuTvbr+50RZKVFp 4cyC8ApoyGkeNKH+BSsTBTo= X-Google-Smtp-Source: ABdhPJzu8JSH6sOJuN6DFOHjbCnc1BQwzmeWS2EnpnK5PNAywxJtfJbssNfLeDLRH3K0/rAvmX6I7A== X-Received: by 2002:a17:906:a0a:: with SMTP id w10mr35555729ejf.416.1620801516959; Tue, 11 May 2021 23:38:36 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:36 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 12/25] trace-cmd library: Add new API to get compression of input handler Date: Wed, 12 May 2021 09:38:10 +0300 Message-Id: <20210512063823.324610-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The new API returns name and version of the compression algorithm, used to compress the trace file associated with given input file handler: tracecmd_get_file_compress_proto() Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 2 ++ lib/trace-cmd/trace-input.c | 16 ++++++++++++++++ 2 files changed, 18 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 602817cb..4265e37a 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -236,6 +236,8 @@ unsigned long long tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_file_version(struct tracecmd_input *handle); +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index a23c26bd..9dff2188 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -4150,6 +4150,22 @@ unsigned long tracecmd_get_file_version(struct tracecmd_input *handle) return handle->file_version; } +/** + * tracecmd_get_file_compress_proto - get name and version of compression algorithm, + * used to compress the trace file + * @handle: input handle for the trace.dat file + * @name: return, name of the compression algorithm. + * @version: return, version of the compression algorithm. + * + * Returns 0 on success, or -1 in case of an error. If 0 is returned, the name and version of the + * algorithm are stored in @name and @version. The returned strings must *not* be freed. + */ +int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, + const char **name, const char **version) +{ + return tracecmd_compress_proto_get_name(handle->compress, name, version); +} + /** * tracecmd_get_use_trace_clock - return use_trace_clock * @handle: input handle for the trace.dat file From patchwork Wed May 12 06:38:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9E99C43460 for ; Wed, 12 May 2021 06:38:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD5C561628 for ; Wed, 12 May 2021 06:38:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230115AbhELGjs (ORCPT ); Wed, 12 May 2021 02:39:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230109AbhELGjr (ORCPT ); Wed, 12 May 2021 02:39:47 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C3CCC06174A for ; Tue, 11 May 2021 23:38:39 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id n25so25710896edr.5 for ; Tue, 11 May 2021 23:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7YQ5q9T+DM6JDItU8RS9aWYpqQte60zQDz1p2bIZnzM=; b=pwoDwCSB0r5YSEw2gjAIWPOCQBlhheAjPw+3BDJdQsaPKntUaEVpSmhCzQm+CdJiLk wREZbvqZCRSs7r08Adnbm+hQMocSEfVWCbvpqgjcvf3qzhIFNMfF4fLUAbhfojfEUF5G pPjOwKDLAnfezxG1gPl0nqBPaELVuUaAPIwf81tXuhnSeJi1Kd+45wEr/s9zA9iQnCmW S5UxLqvpKjJvqKL+xuK8g98+g/neEGWYtPSXGEVA2BkIcegnTqaTk6FycwIsoszst6qL NXHkcAvbdLfvxcJ89KyPzVdNsBp8A9iAJQKLtD2H7HOYXlwp9vZ5kvqHBf/xGS5ZzK81 7XbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7YQ5q9T+DM6JDItU8RS9aWYpqQte60zQDz1p2bIZnzM=; b=WWeLQV8PU+qRcK3tDkCkfeBdBfJTslRbu8OoqRl1PyF55h84Fs1FeLPjMA/RQz9v3H RZVyxhKO1j2RxIgq4myKSuXCmnI17nnza8nLvtw+gNuPI6e6cMVJw6/zsGR9rDlrB67d 0DebhVjVEKeT0XHckQ0oDrZYdbxtwygemFMEJgIuA830zrsb4y1nr7CCBroWMVes5uns TFiVmMFSg+kZiJ/SXNIkrA2152Kn35xGqnnQeVpGenWiGrBXxhEqm46aRsYYCVWobmLo 5Wwi5Kf+4TH8d+CyODjoi8DpIKrweaJ7bI0TzMD2nkLtqkJqZY/WadM17aE8zKEfI+r9 lIMQ== X-Gm-Message-State: AOAM532g52K+i1E9Ba/1c6tIhw70GqS8CDU/BvlF5IYpsss6QnyCIQBd +nb3Y61p0KTnlTDM3NbfItJ6Pc2Q5XHvJw== X-Google-Smtp-Source: ABdhPJwOtZnmfzPPGFCcCP9K8neKkbTKPT7koQwTJTyRmgqqX81i8ru3qk/ze9dgTWZ6I9sTqLGUcA== X-Received: by 2002:a05:6402:1281:: with SMTP id w1mr37232606edv.148.1620801517990; Tue, 11 May 2021 23:38:37 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:37 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 13/25] trace-cmd library: Inherit compression algorithm from input file Date: Wed, 12 May 2021 09:38:11 +0300 Message-Id: <20210512063823.324610-14-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new trace file output handler is allocated, based on given trace file input handler - use the same compression algorithm. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 627cea42..ea90ef75 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -980,17 +980,27 @@ out_free: static int select_file_version(struct tracecmd_output *handle, struct tracecmd_input *ihandle) { + const char *cname = NULL; + const char *cver = NULL; + if (ihandle) { handle->file_version = tracecmd_get_file_version(ihandle); - } else { - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); - if (handle->compress) - handle->file_version = 7; - else - handle->file_version = 6; + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + return -1; + } + return 0; } + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (handle->compress) + handle->file_version = 7; + else + handle->file_version = 6; + return 0; } @@ -1645,6 +1655,8 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) { struct tracecmd_output *handle = NULL; struct tracecmd_input *ihandle; + const char *cname = NULL; + const char *cver = NULL; int fd2; /* Move the file descriptor to the beginning */ @@ -1682,6 +1694,12 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->file_version = tracecmd_get_file_version(ihandle); list_head_init(&handle->options); + if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { + handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) + goto out_free; + } tracecmd_close(ihandle); return handle; From patchwork Wed May 12 06:38:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E822C433B4 for ; Wed, 12 May 2021 06:38:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0A4D561628 for ; Wed, 12 May 2021 06:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230118AbhELGjt (ORCPT ); Wed, 12 May 2021 02:39:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230109AbhELGjt (ORCPT ); Wed, 12 May 2021 02:39:49 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25A92C061574 for ; Tue, 11 May 2021 23:38:40 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id s20so27793457ejr.9 for ; Tue, 11 May 2021 23:38:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wsSxvmAbhnZAf8Sh83EpfX3LIKRCrfjGaaxzWubMLz0=; b=gmp7pCYoKNaMGfCBvG7Wr35uFnA0zSbkLxmOnI8L7WBJjoARQ/Vn2eJ1Maq3mOTgOr ts7Qa1ueeeycabpX4H5cuH//G0YduAuev83UlNjWWibc5H5NBoYqdwJWbJlojgJHrhTy mP8ngBDGmYdqjA5P32BaZJYilcwmUCI7lcKeJdea4HHhgBnBLpwhlyfn51rYzXZ8xFPq fJ8gJLtQu5AKpPbXMEYh//TncqvL8PmxnfHW0JYcDkgEpNZq7UfMhKo6/eq9jytT7PP1 xem4lBWxe+CgMGC21/Ia3pMd/yH3QAujOiLfC1OiVBfUh7v0GwbefmbJ4xeNiER65t4b Kibw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wsSxvmAbhnZAf8Sh83EpfX3LIKRCrfjGaaxzWubMLz0=; b=ZbHBn2MWCQ3d+lnBWOOu+gXeWxIERN5vSTr+9y1uewC6tClZX59gSiN/OmxviZ3nf7 UlIriohjeHkAo/dYFeM8hl+tlmlInCTLxaCCB2DORiF/m0w6sBvgN5wdextGVcoksraQ 0JnGjLvWXtPlL5PLwR5gVSH9tGcDCgvBvTTSe9/SzCfXpUsVtlOi69dhgkHArM3tQr/g /fa6hhHgTe4+k10F/2a9d4Q+HAH7tdKLMD6SDfkVzJXdtSjyw4yZqWylX2hGj5ah/bIK bjvrNDk2qzFhnR9MD4ca6xjTfRI0VNj1bFulzqrbTeh04/PbjDzkufFC7HwfF9TmnOpR +ogA== X-Gm-Message-State: AOAM530N0xU1Jc44O3d2qwjFvlZIAn0NFt1To1ms1cEqeJ6HkvjMmsks qHTKCkMbjL9BxE/TsQT2mnUL5zWkNdRgtg== X-Google-Smtp-Source: ABdhPJxrm4sV/w7AC+KMhv7HM13ihg7lnmFK1dLgSPEkNFyzEsIkYmKDjkfw5p/SSRl4eQweztHs8Q== X-Received: by 2002:a17:906:c9d8:: with SMTP id hk24mr36276823ejb.480.1620801518907; Tue, 11 May 2021 23:38:38 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:38 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 14/25] trace-cmd library: Extend the create file APIs to support different compression Date: Wed, 12 May 2021 09:38:12 +0300 Message-Id: <20210512063823.324610-15-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added additional parameter for file compression to all trace-cmd library APIs for creating a new trace file. The caller could specify what compression algorithm should be used when creating a new trace file: tracecmd_create_file_latency tracecmd_create_init_file_glob tracecmd_create_init_fd_glob tracecmd_create_init_fd_msg tracecmd_create_init_file tracecmd_create_init_file_override The new parameter is of type string and can be one of: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 17 +++-- lib/trace-cmd/trace-output.c | 75 +++++++++++++------ tracecmd/trace-record.c | 12 ++- tracecmd/trace-restore.c | 4 +- 4 files changed, 68 insertions(+), 40 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 4265e37a..7c5e3fe1 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -271,20 +271,23 @@ struct tracecmd_event_list { struct tracecmd_option; struct tracecmd_msg_handle; -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus); +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression); struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list); +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression); struct tracecmd_output *tracecmd_create_init_fd(int fd); struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list); +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression); struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list); -struct tracecmd_output *tracecmd_create_init_file(const char *output_file); + struct tracecmd_event_list *list, + const char *compression); +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression); struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms); + const char *kallsyms, + const char *compression); struct tracecmd_option *tracecmd_add_option(struct tracecmd_output *handle, unsigned short id, int size, const void *data); diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index ea90ef75..56b68edd 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -978,7 +978,8 @@ out_free: } static int select_file_version(struct tracecmd_output *handle, - struct tracecmd_input *ihandle) + struct tracecmd_input *ihandle, + const char *compression) { const char *cname = NULL; const char *cver = NULL; @@ -994,8 +995,28 @@ static int select_file_version(struct tracecmd_output *handle, return 0; } - handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, - handle->pevent, handle->msg_handle); + if (compression) { + if (!strcmp(compression, "any")) { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("No compression algorithms are not supported"); + return -1; + } + } else if (strcmp(compression, "none")) { + handle->compress = tracecmd_compress_alloc(compression, NULL, handle->fd, + handle->pevent, handle->msg_handle); + if (!handle->compress) { + tracecmd_warning("Compression algorithm %s is not supported", + compression); + return -1; + } + } + } else { + handle->compress = tracecmd_compress_alloc(NULL, NULL, handle->fd, + handle->pevent, handle->msg_handle); + } + if (handle->compress) handle->file_version = 7; else @@ -1028,7 +1049,8 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, struct tracecmd_event_list *list, - struct tracecmd_msg_handle *msg_handle) + struct tracecmd_msg_handle *msg_handle, + const char *compression) { struct tracecmd_output *handle; struct tep_handle *pevent; @@ -1049,7 +1071,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, handle->msg_handle = msg_handle; - if (select_file_version(handle, ihandle)) + if (select_file_version(handle, ihandle, compression)) goto out_free; list_head_init(&handle->options); @@ -1128,7 +1150,8 @@ static struct tracecmd_output *create_file(const char *output_file, struct tracecmd_input *ihandle, const char *tracing_dir, const char *kallsyms, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, + const char *compression) { struct tracecmd_output *handle; int fd; @@ -1137,7 +1160,7 @@ static struct tracecmd_output *create_file(const char *output_file, if (fd < 0) return NULL; - handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL); + handle = create_file_fd(fd, ihandle, tracing_dir, kallsyms, list, NULL, compression); if (!handle) { close(fd); unlink(output_file); @@ -1403,13 +1426,14 @@ int tracecmd_write_cmdlines(struct tracecmd_output *handle) return ret; } -struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus) +struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, int cpus, + const char *compression) { struct tracecmd_output *handle; char *path; int ret; - handle = create_file(output_file, NULL, NULL, NULL, &all_event_list); + handle = create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); if (!handle) return NULL; @@ -1712,39 +1736,41 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) struct tracecmd_output *tracecmd_create_init_fd(int fd) { - return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, &all_event_list, NULL, NULL); } struct tracecmd_output * tracecmd_create_init_fd_msg(struct tracecmd_msg_handle *msg_handle, - struct tracecmd_event_list *list) + struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(msg_handle->fd, NULL, NULL, NULL, list, msg_handle); + return create_file_fd(msg_handle->fd, NULL, NULL, NULL, + list, msg_handle, compression); } struct tracecmd_output * -tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list) +tracecmd_create_init_fd_glob(int fd, struct tracecmd_event_list *list, const char *compression) { - return create_file_fd(fd, NULL, NULL, NULL, list, NULL); + return create_file_fd(fd, NULL, NULL, NULL, list, NULL, compression); } struct tracecmd_output * -tracecmd_create_init_file_glob(const char *output_file, - struct tracecmd_event_list *list) +tracecmd_create_init_file_glob(const char *output_file, struct tracecmd_event_list *list, + const char *compression) { - return create_file(output_file, NULL, NULL, NULL, list); + return create_file(output_file, NULL, NULL, NULL, list, compression); } -struct tracecmd_output *tracecmd_create_init_file(const char *output_file) +struct tracecmd_output *tracecmd_create_init_file(const char *output_file, const char *compression) { - return create_file(output_file, NULL, NULL, NULL, &all_event_list); + return create_file(output_file, NULL, NULL, NULL, &all_event_list, compression); } struct tracecmd_output *tracecmd_create_init_file_override(const char *output_file, const char *tracing_dir, - const char *kallsyms) + const char *kallsyms, + const char *compression) { - return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list); + return create_file(output_file, NULL, tracing_dir, kallsyms, &all_event_list, compression); } /** @@ -1756,12 +1782,13 @@ struct tracecmd_output *tracecmd_create_init_file_override(const char *output_fi * with the same characteristics (events and all) and returns * tracecmd_output handle to this new file. */ -struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, - const char *file) +struct tracecmd_output *tracecmd_copy(struct tracecmd_input *ihandle, const char *file) { struct tracecmd_output *handle; + const char *compr_name = NULL; - handle = create_file(file, ihandle, NULL, NULL, &all_event_list); + tracecmd_get_file_compress_proto(ihandle, &compr_name, NULL); + handle = create_file(file, ihandle, NULL, NULL, &all_event_list, compr_name); if (!handle) return NULL; diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 5dd8be4a..e7a360a4 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3642,7 +3642,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events); + network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3660,8 +3660,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, - listed_events); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3849,8 +3848,7 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, - listed_events); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4242,7 +4240,7 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count); + handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4273,7 +4271,7 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events); + handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); diff --git a/tracecmd/trace-restore.c b/tracecmd/trace-restore.c index 280a37f0..96b4fb5d 100644 --- a/tracecmd/trace-restore.c +++ b/tracecmd/trace-restore.c @@ -91,7 +91,7 @@ void trace_restore (int argc, char **argv) } handle = tracecmd_create_init_file_override(output, tracing_dir, - kallsyms); + kallsyms, NULL); if (!handle) die("Unabled to create output file %s", output); if (tracecmd_write_cmdlines(handle) < 0) @@ -128,7 +128,7 @@ void trace_restore (int argc, char **argv) handle = tracecmd_copy(ihandle, output); tracecmd_close(ihandle); } else - handle = tracecmd_create_init_file(output); + handle = tracecmd_create_init_file(output, NULL); if (!handle) die("error writing to %s", output); From patchwork Wed May 12 06:38:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA475C43462 for ; Wed, 12 May 2021 06:38:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C285D617C9 for ; Wed, 12 May 2021 06:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230109AbhELGju (ORCPT ); Wed, 12 May 2021 02:39:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230129AbhELGjt (ORCPT ); Wed, 12 May 2021 02:39:49 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0AEECC06174A for ; Tue, 11 May 2021 23:38:41 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id a25so1571398edr.12 for ; Tue, 11 May 2021 23:38:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/BGJ1mbLjfC7/Y5pZ3u3hKYzrbdXuGnvDfbI2PrCQDs=; b=pJ618sSZs1oKF5+hv5yehOqWnwBQMf1ErfELjceRZPw57Jh0tdJ30o/huoQpR/UPPr 6vp2RFuhCT32pIUj1BebkW+QyyFfQ+a5wPAFSqoyhAv5j0St7Sq/tJM5teqymCcSHMO1 SpYj5a+6fSRDGwNESZ/fAyhXzxuKwnX+aN+FyTALo4Gy8k9vuKKhjItJ18NQIUVaCLy9 vZHeiFfmDpKhwzgaYhZWYa6lpo7kLqiSPnyUf3eDLtfiFNrAux5K16ACsohFl//h88xY eW9fbwuVE7aKWkfOrDKL5mhxoGbkU7lrxg4iQS2dVxPy84kzUsr/NzhXEkgfYVbNPZhT Kwtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/BGJ1mbLjfC7/Y5pZ3u3hKYzrbdXuGnvDfbI2PrCQDs=; b=VwzblpMQzb5KKlmYUE+5JkRoNum9/hPpESP61afSlpDbZBe3Jl+DOT73c2+Vzn//0j OV/jeJ5GAA/SOzrx6jEsWC9qyLJMEWiZmfJwoNsu+ZUTzg7mWysJoKG4iwm+zemh0JRc 7pA9Bc8NHwFmj9CS55hwrZKW/lbnG+I1NzNo5f2w5EsZxbu7if9PGpQXLpZVaNgWMbuQ 9Gs7xTEbpy4b3KUxMdQw3jYpwhJ+Fsb8uLKVbbCfEx5Q8+aN2DTCIyDJTTAFp/M/T2jX crOJhUtAaMjYTg7dq8N1cXhs0Pkz3kdQCa7ura+up41Hyij4CbtsYx64p35sm93CJWm6 uYfA== X-Gm-Message-State: AOAM531njJEbtOoAILjI4e9Fm1pDga83s2elGbFzXV1IvJHtms/Evyhk nG4+XSf5uiXsuFTh9xljY/Q= X-Google-Smtp-Source: ABdhPJyY3jP8wwnvbbC2f3DZogC4FFNMafDx+skaxoihDtfdOyfN1UHIM2NQwaSkDGTTReWxRWE9gQ== X-Received: by 2002:a05:6402:1547:: with SMTP id p7mr41505344edx.319.1620801519786; Tue, 11 May 2021 23:38:39 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:39 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 15/25] trace-cmd record: Add new parameter --compression Date: Wed, 12 May 2021 09:38:13 +0300 Message-Id: <20210512063823.324610-16-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added a new parameter "trace-cmd record --compression", can be used to select the desired compression algorithm for the trace output file. One of these strings can be passed: "any" - auto select the best available compression algorithm "none" - do not compress the trace file name - the name of the desired compression algorithms, available algorithms can be listed with "trace-cmd list -c" If a compression is used, trace file version 7 is selected, otherwise trace file version 6 is used. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-record.c | 24 +++++++++++++++++++----- tracecmd/trace-usage.c | 5 +++++ 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index e7a360a4..5e969dc7 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -199,6 +199,7 @@ struct common_record_context { char *date2ts; char *user; const char *clock; + const char *compression; struct tsc_nsec tsc2nsec; int data_flags; int tsync_loop_interval; @@ -3642,7 +3643,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { - network_handle = tracecmd_create_init_fd_msg(msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(msg_handle, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3660,7 +3662,8 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; } else { - network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, listed_events, NULL); + network_handle = tracecmd_create_init_fd_glob(msg_handle->fd, + listed_events, ctx->compression); if (!network_handle) goto error; tracecmd_set_quiet(network_handle, quiet); @@ -3848,7 +3851,8 @@ static void setup_agent(struct buffer_instance *instance, { struct tracecmd_output *network_handle; - network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, listed_events, NULL); + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, + listed_events, ctx->compression); add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); @@ -4240,7 +4244,8 @@ static void record_data(struct common_record_context *ctx) return; if (latency) { - handle = tracecmd_create_file_latency(ctx->output, local_cpu_count, NULL); + handle = tracecmd_create_file_latency(ctx->output, + local_cpu_count, ctx->compression); tracecmd_set_quiet(handle, quiet); } else { if (!local_cpu_count) @@ -4271,7 +4276,8 @@ static void record_data(struct common_record_context *ctx) touch_file(temp_files[i]); } - handle = tracecmd_create_init_file_glob(ctx->output, listed_events, NULL); + handle = tracecmd_create_init_file_glob(ctx->output, + listed_events, ctx->compression); if (!handle) die("Error creating output file"); tracecmd_set_quiet(handle, quiet); @@ -5531,6 +5537,7 @@ void init_top_instance(void) } enum { + OPT_comporession = 239, OPT_tsc2nsec = 240, OPT_fork = 241, OPT_tsyncinterval = 242, @@ -5965,6 +5972,7 @@ static void parse_record_options(int argc, {"tsync-interval", required_argument, NULL, OPT_tsyncinterval}, {"fork", no_argument, NULL, OPT_fork}, {"tsc2nsec", no_argument, NULL, OPT_tsc2nsec}, + {"compression", required_argument, NULL, OPT_comporession}, {NULL, 0, NULL, 0} }; @@ -6386,6 +6394,12 @@ static void parse_record_options(int argc, die("TSC to nanosecond is not supported"); ctx->instance->flags |= BUFFER_FL_TSC2NSEC; break; + case OPT_comporession: + if (strcmp(optarg, "any") && strcmp(optarg, "none") && + !tracecmd_compress_is_supported(optarg, NULL)) + die("Compression algorithm %s is not supported", optarg); + ctx->compression = strdup(optarg); + break; case OPT_quiet: case 'q': quiet = true; diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index c388eccd..a1d7799a 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -68,6 +68,11 @@ static struct usage_help usage_help[] = { " If a negative number is specified, timestamps synchronization is disabled" " If 0 is specified, no loop is performed - timestamps offset is calculated only twice," " at the beginnig and at the end of the trace\n" + " --compression compress the trace output file, one of these strings can be passed:\n" + " any - auto select the best available compression algorithm\n" + " none - do not compress the trace file\n" + " name - the name of the desired compression algorithms\n" + " available algorithms can be listed with trace-cmd list -c\n" }, { "set", From patchwork Wed May 12 06:38:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFC5CC43461 for ; Wed, 12 May 2021 06:38:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F5B5613B4 for ; Wed, 12 May 2021 06:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230114AbhELGju (ORCPT ); Wed, 12 May 2021 02:39:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230109AbhELGjt (ORCPT ); Wed, 12 May 2021 02:39:49 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F115EC06175F for ; Tue, 11 May 2021 23:38:41 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id b17so25720331ede.0 for ; Tue, 11 May 2021 23:38:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=PdXUYxxbp6Uzd2DrV4SFI2//KO3rG1Oormqx1CfetgyBcIoZ81y5Q33M85+tqDwIww 6zgIC/TZBtTEyBg3uGy5I5dhrE33ZdcBXOB7Cd202h1n11N0ZQlS0QTxEfRFwJNxmmXs IIFAN1Z+JEKt4n96O2YNPi2u7780j9HWu/uCG48e93juLlmEIuNZyNDtVzg9+BJm2fKD jNDoSiQcpG+Q7DXT8/k9JGBkkM2J5bYVIvfAgzyAwfTxRDugghHL3ungiuMkJ5iZV7ST 7k4YKl4kLbBoQl1lnLo5qOCn/vkKs7hdzdTjDvfI11/8qRDHFvnQWWUOmdR8Y4SEnjwn a+fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KcqaeC0E8JB2QkU0ajzNoubjwOnME2GGI/iefg1BUC4=; b=Y5L2VyCINSVQWSbIgJIHXne+TKv0yWOlSAEty4d5J9Fcsn7nggYQZWmtEpdtT1Kqt5 c2XJN+B72NwKHSD8fL1oDpPGhp2Ra0T/Q5CKtdK7GbB9XBxaI0x9MILj1qewvHH0jJ/l vWm8yTTtvqTF2NxonUxRLVFgq49b0QijRiHhu91ZbzOGsMdl9XGhNjhDHIDUfLL2wwrO DbEH1cVY5e5uFG9oA++2a3Q78E09DwS2tKaweWIwDkWiGFum0UbqEFaDd31u7EVSbJyF gFyYJkTKLMr/S7iDHjbGyoIiSzQk/0Aog62XuFHOTanWlob5z9t8tTRAdjcFF47six49 rnYw== X-Gm-Message-State: AOAM533lBwOTlenAD4G4nDUvWAP5JmuJTPe5pKAY7W5NGjXOr8BPkdJX 3AW5OaEYaBcPPHu5ZUB9h7i4sv3MXYxgyQ== X-Google-Smtp-Source: ABdhPJzVCj7zKfpgm82x7f5HPhDLXjpqoeXJAVpPzcQRGkFAH+fthJ9Zp7HSYIOCuJzTGf4GOsfdyw== X-Received: by 2002:a50:8fe6:: with SMTP id y93mr41004890edy.224.1620801520691; Tue, 11 May 2021 23:38:40 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:40 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 16/25] trace-cmd dump: Add support for trace files version 7 Date: Wed, 12 May 2021 09:38:14 +0300 Message-Id: <20210512063823.324610-17-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Added support to read and dump information of trace files version 7. Signed-off-by: Tzvetomir Stoyanov (VMware) --- tracecmd/trace-dump.c | 125 +++++++++++++++++++++++++++++++++++------- 1 file changed, 105 insertions(+), 20 deletions(-) diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 03cc82b4..ed82104b 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -26,6 +26,9 @@ static struct tep_handle *tep; static unsigned int trace_cpus; static int has_clock; +static unsigned long file_version; +static bool read_compress; +static struct tracecmd_compression *compress; enum dump_items { SUMMARY = (1 << 0), @@ -52,46 +55,62 @@ enum dump_items verbosity; tracecmd_plog(fmt, ##__VA_ARGS__); \ } while (0) -static int read_file_string(int fd, char *dst, int len) +static int read_fd(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, 1); + r = read(fd, dst+size, len); if (r > 0) { - size++; - len--; + size += r; + len -= r; } else break; - if (!dst[size - 1]) - break; - } while (r > 0 && len); + } while (r > 0); - if (!size || dst[size - 1]) + if (len) return -1; - return 0; + return size; } -static int read_file_bytes(int fd, char *dst, int len) +static int read_compressed(int fd, char *dst, int len) +{ + + if (read_compress) + return tracecmd_compress_read(compress, dst, len); + return read_fd(fd, dst, len); +} + +static int read_file_string(int fd, char *dst, int len) { size_t size = 0; int r; do { - r = read(fd, dst+size, len); + r = read_compressed(fd, dst+size, 1); if (r > 0) { - size += r; - len -= r; + size++; + len--; } else break; - } while (r > 0); + if (!dst[size - 1]) + break; + } while (r > 0 && len); - if (len) + if (!size || dst[size - 1]) return -1; return 0; } +static int read_file_bytes(int fd, char *dst, int len) +{ + int ret; + + ret = read_compressed(fd, dst, len); + return ret < 0 ? ret : 0; +} + static void read_dump_string(int fd, int size, enum dump_items id) { char buf[DUMP_SIZE]; @@ -146,7 +165,6 @@ static void dump_initial_format(int fd) char magic[] = TRACECMD_MAGIC; char buf[DUMP_SIZE]; int val4; - unsigned long ver; do_print(SUMMARY, "\t[Initial format]\n"); @@ -168,11 +186,11 @@ static void dump_initial_format(int fd) die("no version string"); do_print(SUMMARY, "\t\t%s\t[Version]\n", buf); - ver = strtol(buf, NULL, 10); - if (!ver && errno) + file_version = strtol(buf, NULL, 10); + if (!file_version && errno) die("Invalid file version string %s", buf); - if (!tracecmd_is_version_supported(ver)) - die("Unsupported file version %lu", ver); + if (!tracecmd_is_version_supported(file_version)) + die("Unsupported file version %lu", file_version); /* get file endianness*/ if (read_file_bytes(fd, buf, 1)) @@ -192,6 +210,29 @@ static void dump_initial_format(int fd) do_print(SUMMARY, "\t\t%d\t[Page size, bytes]\n", val4); } +static void dump_compress(int fd) +{ + char compr[DUMP_SIZE]; + char *ver = NULL; + + if (file_version < 7) + return; + + /* get compression header */ + if (read_file_string(fd, compr, DUMP_SIZE)) + die("no compression header"); + ver = strchr(compr, ' '); + if (!ver) + die("no compression version"); + *ver = '\0'; + do_print((SUMMARY), "\t\t%s\t[Compression algorithm]\n", compr); + do_print((SUMMARY), "\t\t%s\t[Compression version]\n", ver + 1); + + compress = tracecmd_compress_alloc(compr, ver + 1, fd, tep, NULL); + if (!compress) + die("cannot uncomress the file"); +} + static void dump_header_page(int fd) { unsigned long long size; @@ -234,11 +275,34 @@ static void dump_header_event(int fd) read_dump_string(fd, size, HEAD_EVENT); } +static void uncompress_reset(void) +{ + if (compress && file_version >= 7) { + read_compress = false; + tracecmd_compress_reset(compress); + } +} + +static int uncompress_block(void) +{ + int ret = 0; + + if (compress && file_version >= 7) { + ret = tracecmd_uncompress_block(compress); + if (!ret) + read_compress = true; + + } + return ret; +} + static void dump_ftrace_events_format(int fd) { unsigned long long size; unsigned int count; + if (uncompress_block()) + die("cannot uncompress file block"); do_print((SUMMARY | FTRACE_FORMAT), "\t[Ftrace format, "); if (read_file_number(fd, &count, 4)) die("cannot read the count of the ftrace events"); @@ -251,6 +315,7 @@ static void dump_ftrace_events_format(int fd) read_dump_string(fd, size, FTRACE_FORMAT); count--; } + uncompress_reset(); } static void dump_events_format(int fd) @@ -262,6 +327,9 @@ static void dump_events_format(int fd) do_print((SUMMARY | EVENT_FORMAT | EVENT_SYSTEMS), "\t[Events format, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &systems, 4)) die("cannot read the count of the event systems"); @@ -284,6 +352,7 @@ static void dump_events_format(int fd) } systems--; } + uncompress_reset(); } static void dump_kallsyms(int fd) @@ -292,12 +361,17 @@ static void dump_kallsyms(int fd) do_print((SUMMARY | KALLSYMS), "\t[Kallsyms, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the kallsyms"); do_print((SUMMARY | KALLSYMS), "%d bytes]\n", size); read_dump_string(fd, size, KALLSYMS); + + uncompress_reset(); } static void dump_printk(int fd) @@ -306,12 +380,17 @@ static void dump_printk(int fd) do_print((SUMMARY | TRACE_PRINTK), "\t[Trace printk, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 4)) die("cannot read the size of the trace printk"); do_print((SUMMARY | TRACE_PRINTK), "%d bytes]\n", size); read_dump_string(fd, size, TRACE_PRINTK); + + uncompress_reset(); } static void dump_cmdlines(int fd) @@ -320,12 +399,17 @@ static void dump_cmdlines(int fd) do_print((SUMMARY | CMDLINES), "\t[Saved command lines, "); + if (uncompress_block()) + die("cannot uncompress file block"); + if (read_file_number(fd, &size, 8)) die("cannot read the size of the saved command lines"); do_print((SUMMARY | CMDLINES), "%d bytes]\n", size); read_dump_string(fd, size, CMDLINES); + + uncompress_reset(); } static void dump_cpus_count(int fd) @@ -669,6 +753,7 @@ static void dump_file(const char *file) do_print(SUMMARY, "\n Tracing meta data in file %s:\n", file); dump_initial_format(fd); + dump_compress(fd); dump_header_page(fd); dump_header_event(fd); dump_ftrace_events_format(fd); From patchwork Wed May 12 06:38:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLACK,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B08AC43460 for ; Wed, 12 May 2021 06:38:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5C3EC613B4 for ; Wed, 12 May 2021 06:38:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230096AbhELGjw (ORCPT ); Wed, 12 May 2021 02:39:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230129AbhELGjv (ORCPT ); Wed, 12 May 2021 02:39:51 -0400 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F5B0C061574 for ; Tue, 11 May 2021 23:38:43 -0700 (PDT) Received: by mail-ed1-x52d.google.com with SMTP id l7so25742831edb.1 for ; Tue, 11 May 2021 23:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rgszc3tN2S8QInYyAADhZ32YXe4kOwxRzeHif1prIi8=; b=TB4jHtVu6aZePTf4k3EyeAKafkeeGJmzGuatA9tujs5/AQnBDnb+0f+Zau9nbXT7WM 3AHoPOrxoEuOssFlACL26eOA3dVFqprSKuJs5k5Nhc16lOZ5VZsuW3wWFX2QKy4jaSZH JUIehz9eWVkNu6aZ10uDZ28seQQJTADcs1nwQoaBS5ZRy3EV05byHVCoWe8Bnh7+3eVV WRZ7dEe3lfq5o5+RIJib9YRMmzf+AnqLF6TXpFNR9vdNNKg1vYDnICbJdckHlNeFyS+D MGdSQTPV+/gsslXVuJAN+V1+JKlpf0MgUHJZI4k3cYXcmfJI+mv1Gzh+i1HcZXFfokLe mJ2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rgszc3tN2S8QInYyAADhZ32YXe4kOwxRzeHif1prIi8=; b=pZoHa+oCwLvmmD1E263r5MRw09mVOcAswm6ioaQFjgUR2gXY7gF/+L+NT0wDrRknMs QottDKggWmtL0n3nHnk0DqIWUqCpTja/eBray1t5JeOoxGdKJe30LVToCf8r8BYkC8t3 QyM/5I13fodjJuuu+M2VYh+xnl+x93u4pJSb43HKElMdfOPe3ZBaa1oK4m5j+YP0Qzif x1jJ4TlhP5lYP5JjGLubmbfYrYcP0KAi+Y17fixhwsM1PV8u7E+UeBDE51+4YSRU323b +QTm/TsVsHdRvuPyRV0JGqut11xUjZHCBU444qlUIk2eyOsIcewlvyE81PEVneNrWoiZ 3xVA== X-Gm-Message-State: AOAM531K7/0ADWi620whhV/1NoprhID5hpt0mxtK93Vo7vDtgQ7Od++P IA/e5PC8zVoWIbWC+doN9w8= X-Google-Smtp-Source: ABdhPJyQk+ARaQJJmJDxtCI9gv6tGJDkkYr/mPd7kBTkYBFlHp7GndYdeGkopCPy+4kbK5WrvZpBWw== X-Received: by 2002:a05:6402:10c9:: with SMTP id p9mr41724855edu.268.1620801521946; Tue, 11 May 2021 23:38:41 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:41 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 17/25] trace-cmd library: Add support for zlib compression library Date: Wed, 12 May 2021 09:38:15 +0300 Message-Id: <20210512063823.324610-18-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org If libz is available, use that library to provide trace file compression support. The library is detected runtime. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/Makefile | 10 ++ lib/trace-cmd/include/trace-cmd-local.h | 5 + lib/trace-cmd/trace-compress-zlib.c | 172 ++++++++++++++++++++++++ lib/trace-cmd/trace-compress.c | 8 ++ 4 files changed, 195 insertions(+) create mode 100644 lib/trace-cmd/trace-compress-zlib.c diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile index bab4322d..83ba7016 100644 --- a/lib/trace-cmd/Makefile +++ b/lib/trace-cmd/Makefile @@ -7,6 +7,13 @@ ldir:=$(src)/lib/trace-cmd DEFAULT_TARGET = $(LIBTRACECMD_STATIC) +pound := \# +ZLIB_INSTALLED := $(shell if (printf "$(pound)include \n void main(){deflateInit(NULL, Z_BEST_COMPRESSION);}" | $(CC) -o /dev/null -x c - -lz >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) +ifeq ($(ZLIB_INSTALLED), 1) +CFLAGS += -DHAVE_ZLIB +$(info Have zlib compression support) +endif + OBJS = OBJS += trace-hash.o OBJS += trace-hooks.o @@ -26,6 +33,9 @@ OBJS += trace-timesync-ptp.o OBJS += trace-timesync-kvm.o endif OBJS += trace-compress.o +ifeq ($(ZLIB_INSTALLED), 1) +OBJS += trace-compress-zlib.o +endif # Additional util objects OBJS += trace-blk-hack.o diff --git a/lib/trace-cmd/include/trace-cmd-local.h b/lib/trace-cmd/include/trace-cmd-local.h index c8c51ff0..86d91caf 100644 --- a/lib/trace-cmd/include/trace-cmd-local.h +++ b/lib/trace-cmd/include/trace-cmd-local.h @@ -30,6 +30,11 @@ void tracecmd_fatal(const char *fmt, ...); #endif #endif +#ifdef HAVE_ZLIB +int tracecmd_zlib_init(void); +void tracecmd_zlib_free(void); +#endif + int tracecmd_compress_init(void); void tracecmd_compress_free(void); diff --git a/lib/trace-cmd/trace-compress-zlib.c b/lib/trace-cmd/trace-compress-zlib.c new file mode 100644 index 00000000..3208d57b --- /dev/null +++ b/lib/trace-cmd/trace-compress-zlib.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: LGPL-2.1 +/* + * Copyright (C) 2021, VMware, Tzvetomir Stoyanov tz.stoyanov@gmail.com> + * + */ +#include +#include +#include +#include + +#include "trace-cmd-private.h" + +#define __ZLIB_NAME "zlib" +#define __ZLIB_WEIGTH 10 +#define __ZLIB_FILE "libz.so" +#define ZLIB_FUNC_COMPRESS "compress2" +#define ZLIB_FUNC_DECOMOPRESS "uncompress" +#define ZLIB_FUNC_SIZE "compressBound" +#define ZLIB_FUNC_VERSION "zlibVersion" + +static void *zlib_handle; +static int (*_lib_compress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes, int level); +static int (*_libz_decompress)(unsigned char *out, unsigned long *out_bytes, + unsigned char *in, unsigned long in_bytes); +static unsigned long (*_libz_compress_bound)(unsigned long in_bytes); +static const char *(*_libz_version)(void); + +static int zlib_compress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_lib_compress) + return -1; + + ret = _lib_compress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes, Z_BEST_COMPRESSION); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_STREAM_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static int zlib_decompress(char *in, unsigned int in_bytes, + char *out, unsigned int *out_bytes) +{ + unsigned long out_size = *out_bytes; + int ret; + + if (!_libz_decompress) + return -1; + + ret = _libz_decompress((unsigned char *)out, &out_size, + (unsigned char *)in, (unsigned long)in_bytes); + *out_bytes = out_size; + errno = 0; + switch (ret) { + case Z_OK: + return 0; + case Z_BUF_ERROR: + errno = -ENOBUFS; + break; + case Z_MEM_ERROR: + errno = -ENOMEM; + break; + case Z_DATA_ERROR: + errno = -EINVAL; + break; + default: + errno = -EFAULT; + break; + } + + return -1; +} + +static unsigned int zlib_compress_bound(unsigned int in_bytes) +{ + if (!_libz_compress_bound) + return 0; + return _libz_compress_bound(in_bytes); +} + +static bool zlib_is_supported(const char *name, const char *version) +{ + const char *zver; + + if (!name) + return false; + if (strlen(name) != strlen(__ZLIB_NAME) || strcmp(name, __ZLIB_NAME)) + return false; + + if (!version) + return true; + + if (!_libz_version) + return false; + zver = _libz_version(); + if (!zver) + return false; + + /* Compare the major version number */ + if (atoi(version) <= atoi(zver)) + return true; + + return false; +} + +int tracecmd_zlib_init(void) +{ + if (zlib_handle) + return 0; + + zlib_handle = dlopen(__ZLIB_FILE, RTLD_NOW | RTLD_GLOBAL); + if (!zlib_handle) + return -1; + _lib_compress = dlsym(zlib_handle, ZLIB_FUNC_COMPRESS); + if (!_lib_compress) + goto error; + _libz_decompress = dlsym(zlib_handle, ZLIB_FUNC_DECOMOPRESS); + if (!_libz_decompress) + goto error; + _libz_compress_bound = dlsym(zlib_handle, ZLIB_FUNC_SIZE); + if (!_libz_compress_bound) + goto error; + _libz_version = dlsym(zlib_handle, ZLIB_FUNC_VERSION); + if (!_libz_version) + goto error; + + return tracecmd_compress_proto_register(__ZLIB_NAME, _libz_version(), __ZLIB_WEIGTH, + zlib_compress, zlib_decompress, + zlib_compress_bound, zlib_is_supported); + +error: + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + dlclose(zlib_handle); + zlib_handle = NULL; + return -1; +} + +void tracecmd_zlib_free(void) +{ + _lib_compress = NULL; + _libz_decompress = NULL; + _libz_version = NULL; + + if (zlib_handle) { + dlclose(zlib_handle); + zlib_handle = NULL; + } + +} diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index ff742226..2b329d00 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -353,6 +353,10 @@ int tracecmd_compress_init(void) gettimeofday(&time, NULL); srand((time.tv_sec * 1000) + (time.tv_usec / 1000)); +#ifdef HAVE_ZLIB + tracecmd_zlib_init(); +#endif + return 0; } @@ -530,6 +534,10 @@ void tracecmd_compress_free(void) struct compress_proto *proto = proto_list; struct compress_proto *del; +#ifdef HAVE_ZLIB + tracecmd_zlib_free(); +#endif + while (proto) { del = proto; proto = proto->next; From patchwork Wed May 12 06:38:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6B40C433B4 for ; Wed, 12 May 2021 06:38:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B718D613B4 for ; Wed, 12 May 2021 06:38:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230134AbhELGjw (ORCPT ); Wed, 12 May 2021 02:39:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230129AbhELGjw (ORCPT ); Wed, 12 May 2021 02:39:52 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C6CFC06174A for ; Tue, 11 May 2021 23:38:44 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id b17so25720412ede.0 for ; Tue, 11 May 2021 23:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2IqNx8fXpgOaOemYCe6aYMTLlINXqpGBKKydV7rHt1I=; b=TUpmspB0yZGros/MCiAACjpq5Brm9YW9SoGtuq2CjHCnwbdnuk98ReN9YOjHO2XKuQ 9IYV/FJLZuV8NfL5T4g37RItjx9xjafNwIl3NjNtqaNS9zBFe/bK39OTymc5NQwKyS4e 4zDUYMVmsHqiIkffc5Pwt/nIKXJQp9TAmFw/NJ/5BiXF5aJYR28p7Gqqiptvx1Z91Zib fWkLc8t/FISLSlCH+PDbuw/Vew3w7iLWT/6GRPw3auvBixp+V/vNn2eplFmu6OZs9y5q aoTNeMujjzku0xFAfxl1IwsJU4q3AIaad86Dxpk3dKaUMedITY3EQ6RA+fBhE6khZWJ9 4Yxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2IqNx8fXpgOaOemYCe6aYMTLlINXqpGBKKydV7rHt1I=; b=AOIakQ8+eeTI7MdcUlKhXYgD+uYfH4kZlDks8GebTiqYM/4yOOBDgQn0+4IlSbnT8h gCoz3gbKaesGWdqJaE1xGt7IpR6u+PGj7NEBRWW50QFgzJFYMbICjr0YZ8q0cnymuDx5 fTlg48dxVnd/a3G3E85hqw82hMBhVuIZdDIvjs5+zJ4M04JDUTgK7MvMSBSz1h09VXWW /0zwdBVTRYzlilSfUv/ws0qiHIW+/RPI7pcjsX6VA5+gV3Z8748o/SI0nA/MCG70Qm2f CFisZAft4KJNgoZuAZxsOHrK7b/sQniIS1xvc4nY/jeYBMB1ZqJVqLktW9TR6YqmytB4 +8yg== X-Gm-Message-State: AOAM532oBPPbUFXD3A5QHli5jjDLXRBFiUNoonv9CH1DXeYCn+pBPGlr GyZMLVPQ0BxJSPCBx/a4AphEquQCV8O/xA== X-Google-Smtp-Source: ABdhPJxOlTqYGRT9ubGRHyyCWWFMNQh3mIaSR4TRMUBilxt24qZ5pSDtHEjBLyAs3p/PwF7ky8KUxA== X-Received: by 2002:a05:6402:154d:: with SMTP id p13mr42009491edx.371.1620801522770; Tue, 11 May 2021 23:38:42 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:42 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 18/25] trace-cmd library: Hide the logic for updating buffer offset Date: Wed, 12 May 2021 09:38:16 +0300 Message-Id: <20210512063823.324610-19-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a trace buffer data are written in the trace file, the buffer option in the file metadata is updated with the file offset of the tracing data. Hide this logic into the trace-cmd library. Added new APIs: tracecmd_add_buffer_description() tracecmd_write_buffers_description() Changed APIs: tracecmd_append_buffer_cpu_data() Removed APIs: tracecmd_add_buffer_option() This internal refactoring is needed for changes, related to compression of the options sections of the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 8 +- lib/trace-cmd/trace-output.c | 83 +++++++++++++++++-- tracecmd/trace-record.c | 16 ++-- 3 files changed, 84 insertions(+), 23 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 7c5e3fe1..7d34f27b 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -295,8 +295,8 @@ struct tracecmd_option * tracecmd_add_option_v(struct tracecmd_output *handle, unsigned short id, const struct iovec *vector, int count); -struct tracecmd_option *tracecmd_add_buffer_option(struct tracecmd_output *handle, - const char *name, int cpus); +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus); +int tracecmd_write_buffers_description(struct tracecmd_output *handle); int tracecmd_write_cpus(struct tracecmd_output *handle, int cpus); int tracecmd_write_cmdlines(struct tracecmd_output *handle); @@ -312,9 +312,7 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, int tracecmd_append_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files); int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files); - + const char *name, int cpus, char * const *cpu_data_files); struct tracecmd_output *tracecmd_get_output_handle_fd(int fd); /* --- Reading the Fly Recorder Trace --- */ diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 56b68edd..aa55a089 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -44,6 +44,14 @@ struct tracecmd_option { struct list_head list; }; +struct tracecmd_buffer { + int cpus; + void *name; + tsize_t offset; + struct tracecmd_option *option; + struct list_head list; +}; + enum { OUTPUT_FL_SEND_META = (1 << 0), }; @@ -63,6 +71,7 @@ struct tracecmd_output { struct tracecmd_compression *compress; struct list_head options; + struct list_head buffers; struct tracecmd_msg_handle *msg_handle; char *trace_clock; }; @@ -189,6 +198,7 @@ bool tracecmd_get_quiet(struct tracecmd_output *handle) void tracecmd_output_free(struct tracecmd_output *handle) { struct tracecmd_option *option; + struct tracecmd_buffer *buffer; if (!handle) return; @@ -199,6 +209,13 @@ void tracecmd_output_free(struct tracecmd_output *handle) if (handle->pevent) tep_unref(handle->pevent); + while (!list_empty(&handle->buffers)) { + buffer = container_of(handle->buffers.next, + struct tracecmd_buffer, list); + list_del(&buffer->list); + free(buffer->name); + free(buffer); + } while (!list_empty(&handle->options)) { option = container_of(handle->options.next, struct tracecmd_option, list); @@ -1075,6 +1092,7 @@ create_file_fd(int fd, struct tracecmd_input *ihandle, goto out_free; list_head_init(&handle->options); + list_head_init(&handle->buffers); buf[0] = 23; buf[1] = 8; @@ -1373,9 +1391,8 @@ int tracecmd_append_options(struct tracecmd_output *handle) return 0; } -struct tracecmd_option * -tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, - int cpus) +static struct tracecmd_option * +add_buffer_option(struct tracecmd_output *handle, const char *name, int cpus) { struct tracecmd_option *option; char *buf; @@ -1403,6 +1420,52 @@ tracecmd_add_buffer_option(struct tracecmd_output *handle, const char *name, return option; } +int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char *name, int cpus) +{ + struct tracecmd_buffer *buf; + + buf = calloc(1, sizeof(struct tracecmd_buffer)); + if (!buf) + return -1; + buf->name = strdup(name); + if (!buf->name) { + free(buf); + return -1; + } + list_add_tail(&buf->list, &handle->buffers); + return 0; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + struct tracecmd_option *option; + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + option = add_buffer_option(handle, buf->name, buf->cpus); + if (!option) + return -1; + buf->option = option; + } + return 0; +} + +static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) +{ + struct tracecmd_buffer *buf; + + list_for_each_entry(buf, &handle->buffers, list) { + if (strlen(name) == strlen(buf->name) && !strcmp(name, buf->name)) { + if (handle->file_version >= 7) + return buf->offset; + if (!buf->option) + break; + return buf->option->offset; + } + } + return 0; +} + int tracecmd_write_cmdlines(struct tracecmd_output *handle) { int ret; @@ -1647,18 +1710,23 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, } int tracecmd_append_buffer_cpu_data(struct tracecmd_output *handle, - struct tracecmd_option *option, - int cpus, char * const *cpu_data_files) + const char *name, int cpus, char * const *cpu_data_files) { + tsize_t b_offset; tsize_t offset; stsize_t ret; + b_offset = get_buffer_file_offset(handle, name); + if (!b_offset) { + tracecmd_warning("Cannot find description for buffer %s\n", name); + return -1; + } offset = lseek64(handle->fd, 0, SEEK_CUR); /* Go to the option data, where will write the offest */ - ret = lseek64(handle->fd, option->offset, SEEK_SET); + ret = lseek64(handle->fd, b_offset, SEEK_SET); if (ret == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", option->offset); + tracecmd_warning("could not seek to %lld\n", b_offset); return -1; } @@ -1717,6 +1785,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_file_version(ihandle); list_head_init(&handle->options); + list_head_init(&handle->buffers); if (!tracecmd_get_file_compress_proto(ihandle, &cname, &cver)) { handle->compress = tracecmd_compress_alloc(cname, cver, handle->fd, diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 5e969dc7..e12d77cc 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3934,7 +3934,6 @@ static void touch_file(const char *file) } static void append_buffer(struct tracecmd_output *handle, - struct tracecmd_option *buffer_option, struct buffer_instance *instance, char **temp_files) { @@ -3962,7 +3961,7 @@ static void append_buffer(struct tracecmd_output *handle, touch_file(temp_files[i]); } - tracecmd_append_buffer_cpu_data(handle, buffer_option, + tracecmd_append_buffer_cpu_data(handle, tracefs_instance_get_name(instance->tracefs), cpu_count, temp_files); for (i = 0; i < instance->cpu_count; i++) { @@ -4223,7 +4222,6 @@ static void write_guest_file(struct buffer_instance *instance) static void record_data(struct common_record_context *ctx) { - struct tracecmd_option **buffer_options; struct tracecmd_output *handle; struct buffer_instance *instance; bool local = false; @@ -4294,9 +4292,6 @@ static void record_data(struct common_record_context *ctx) } if (buffers) { - buffer_options = malloc(sizeof(*buffer_options) * buffers); - if (!buffer_options) - die("Failed to allocate buffer options"); i = 0; for_each_instance(instance) { int cpus = instance->cpu_count != local_cpu_count ? @@ -4304,10 +4299,9 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; - - buffer_options[i++] = tracecmd_add_buffer_option(handle, - tracefs_instance_get_name(instance->tracefs), - cpus); + tracecmd_add_buffer_description(handle, + tracefs_instance_get_name(instance->tracefs), + cpus); add_buffer_stat(handle, instance); } } @@ -4342,7 +4336,7 @@ static void record_data(struct common_record_context *ctx) if (instance->msg_handle) continue; print_stat(instance); - append_buffer(handle, buffer_options[i++], instance, temp_files); + append_buffer(handle, instance, temp_files); } } From patchwork Wed May 12 06:38:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F362C43461 for ; Wed, 12 May 2021 06:38:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 13696613B4 for ; Wed, 12 May 2021 06:38:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230129AbhELGjx (ORCPT ); Wed, 12 May 2021 02:39:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230135AbhELGjx (ORCPT ); Wed, 12 May 2021 02:39:53 -0400 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E973DC061574 for ; Tue, 11 May 2021 23:38:44 -0700 (PDT) Received: by mail-ed1-x52d.google.com with SMTP id f1so3472170edt.4 for ; Tue, 11 May 2021 23:38:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Nl8zn9xpnr7w2/VoLzRibtEBiv7NU2J3cESIglbFL/0=; b=Xebx9GhaOAfu5KimjMWcXyfS1VdMp3Wiy/oiLhVPjOc4XTJH2Nq9BSA+Vs7Z6TVB5f DmA5HuN1Hf2VBnpjsiWqiY/NGRQtzruuai/tfqTm178w6DqrbV69TWTRL8havYRTRKN7 CQYs8OEm8ODPig4Dp6l3TNzkUw0Ios+vtVJ/RcbqBKQfjdTp1WWLMDL3qZhkKFbMbBVL qi36BkHfvBjC3gJbXzEm33NUQtwtDV8u4nxy4uRhfPsR/Xv2QSumGK3LnC843oyUbPrn T8Jw9Et9aNSekAmf/aH3fDz5oMHkUNbGItlba9EWzMevDf15qjiBTO/3zfCBfrfMv2ad WkOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Nl8zn9xpnr7w2/VoLzRibtEBiv7NU2J3cESIglbFL/0=; b=UHBgkb2Nlsy8MTRLyQxHCOV6bsObQjXUcJ7WNYFQhpXnrRx2OpQ1UAg4cnCoikguWm g0Xsvnnl0lQGEJC9YQ/wDq/Oh/WYdnubmo1bWr9Go3Q1ltbL4lQLJwbaq41H7TpZ9Nx9 kJe5FqNjwfsf8HU4uhNTOi6rfAOLjsZqzz9l5maUCO2i/xo2UAkpb9X9bAnarg1Podvw 2nO+vNs3hSAlI0FPwX8UHY7O0yJoVxt8Po+dglBsNuR8wj2IliKi4V8VnDD9EwvOzdx6 mWOl/p/nwd4o/m0mqanYmw/eH03Y7N/NHJViOijFNvw9VygbJt+1vW+94gmRAw6nYYeV E4Gw== X-Gm-Message-State: AOAM532/pVw47vvABKOpKgYT9hx8OiQ7VyVB8z+6lib6o85FCNyNUlDp x0l5OWYCBr8S4FFe18Rv38U= X-Google-Smtp-Source: ABdhPJwF9MQ4JiEoZaxnMBMcPOoXQZ/ZiN9VW7tglGT1pK+zwKZjVyohJ2CFbvXy1gwNBmSzHeA9rg== X-Received: by 2002:aa7:cb43:: with SMTP id w3mr41143456edt.287.1620801523657; Tue, 11 May 2021 23:38:43 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:43 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 19/25] trace-cmd: Move buffers description outside of options Date: Wed, 12 May 2021 09:38:17 +0300 Message-Id: <20210512063823.324610-20-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When a new buffer data are written in the trace file, the file offset is stored in the buffer description. As the buffer description is part of the trace options, which are already written in the file, the offset field in this option is updated with the new value. This causes a serious problem when option section of the file is compressed. Updating a value in already compressed section may change the size of this section, which can make the calculated offset of the buffer trace data invalid. To solve that problem and to enable the compression of the option section of the file, the structure of the trace file is changed: - In trace file v7, the description of the buffers is moved to a separate section in the trace file medatata. It is stored after the CPU count section and before the option section. The format of this new buffers description is: 4 bytes - count of the buffers, stored in the file. The top buffer is not counted. For each buffer: 8 bytes, offset of the trace data for this buffer in the file. null-terminated string, name of the buffer. The new section is mandatory, in case of no buffers - the count 0 is stored in the first 4 bytes. - In trace file v6, the description of the buffers is not changed. These changes are needed for compression of the options sections from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 40 ++++++++++ lib/trace-cmd/trace-output.c | 78 ++++++++++++++++++- tracecmd/trace-dump.c | 24 ++++++ tracecmd/trace-listen.c | 3 + tracecmd/trace-record.c | 4 + 6 files changed, 146 insertions(+), 4 deletions(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index 7d34f27b..d7488346 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -112,6 +112,7 @@ enum tracecmd_file_states { TRACECMD_FILE_PRINTK, TRACECMD_FILE_CMD_LINES, TRACECMD_FILE_CPU_COUNT, + TRACECMD_FILE_BUFERS, TRACECMD_FILE_OPTIONS, TRACECMD_FILE_CPU_LATENCY, TRACECMD_FILE_CPU_FLYRECORD, diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 9dff2188..5088a059 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -881,6 +881,39 @@ static int read_cpus(struct tracecmd_input *handle) return 0; } +static int read_buffers(struct tracecmd_input *handle) +{ + struct input_buffer_instance *buffer; + unsigned long long offset; + unsigned int count; + int i; + + if (handle->file_version < 7 || handle->file_state >= TRACECMD_FILE_BUFERS) + return 0; + + if (read4(handle, &count) < 0) + return -1; + i = handle->nr_buffers; + handle->nr_buffers += count; + handle->buffers = realloc(handle->buffers, + sizeof(*handle->buffers) * handle->nr_buffers); + if (!handle->buffers) + return -1; + + for (; i < handle->nr_buffers; i++) { + buffer = &handle->buffers[i]; + if (read8(handle, &offset) < 0) + return -1; + buffer->offset = offset; + buffer->name = read_string(handle); + if (!buffer->name) + return -1; + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +} + /** * tracecmd_read_headers - read the header information from trace.dat * @handle: input handle for the trace.dat file @@ -955,6 +988,13 @@ int tracecmd_read_headers(struct tracecmd_input *handle, if (state <= handle->file_state) return 0; + if (read_buffers(handle) < 0) + return -1; + + if (state <= handle->file_state) + return 0; + + if (read_options_type(handle) < 0) return -1; diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index aa55a089..57b11493 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -376,6 +376,17 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) if (!handle) return -1; + if (handle->file_version < 7) { + switch (new_state) { + case TRACECMD_FILE_BUFERS: + tracecmd_warning("State %d is not supported in trace file version %d", + new_state, handle->file_version); + return -1; + default: + break; + } + } + switch (new_state) { case TRACECMD_FILE_HEADERS: case TRACECMD_FILE_FTRACE_EVENTS: @@ -384,10 +395,16 @@ static int check_out_state(struct tracecmd_output *handle, int new_state) case TRACECMD_FILE_PRINTK: case TRACECMD_FILE_CMD_LINES: case TRACECMD_FILE_CPU_COUNT: - case TRACECMD_FILE_OPTIONS: + case TRACECMD_FILE_BUFERS: if (handle->file_state == (new_state - 1)) return 0; break; + case TRACECMD_FILE_OPTIONS: + if (handle->file_version < 7 && handle->file_state == TRACECMD_FILE_CPU_COUNT) + return 0; + if (handle->file_version >= 7 && handle->file_state == TRACECMD_FILE_BUFERS) + return 0; + break; case TRACECMD_FILE_CPU_LATENCY: case TRACECMD_FILE_CPU_FLYRECORD: if (handle->file_state == TRACECMD_FILE_OPTIONS) @@ -1436,7 +1453,7 @@ int tracecmd_add_buffer_description(struct tracecmd_output *handle, const char * return 0; } -int tracecmd_write_buffers_description(struct tracecmd_output *handle) +static int write_buffers_description_v6(struct tracecmd_output *handle) { struct tracecmd_option *option; struct tracecmd_buffer *buf; @@ -1450,6 +1467,56 @@ int tracecmd_write_buffers_description(struct tracecmd_output *handle) return 0; } +static int write_buffers_description_v7(struct tracecmd_output *handle) +{ + struct tracecmd_buffer *buf; + char *data = NULL; + int count = 0; + int size; + int ret; + + ret = check_out_state(handle, TRACECMD_FILE_BUFERS); + if (ret < 0) { + tracecmd_warning("Cannot write buffers descriptions"); + return ret; + } + + list_for_each_entry(buf, &handle->buffers, list) { + count++; + } + + count = convert_endian_4(handle, count); + ret = do_write_check(handle, &count, 4); + if (ret < 0) + return ret; + + list_for_each_entry(buf, &handle->buffers, list) { + size = 8 + strlen(buf->name) + 1; + data = malloc(size); + if (!data) + goto error; + *(tsize_t *)data = 0; + strcpy(data + 8, buf->name); + /* Save the data location in case it needs to be updated */ + buf->offset = do_lseek(handle, 0, SEEK_CUR); + if (do_write_check(handle, data, size)) + goto error; + } + + handle->file_state = TRACECMD_FILE_BUFERS; + return 0; +error: + return -1; +} + +int tracecmd_write_buffers_description(struct tracecmd_output *handle) +{ + if (handle->file_version >= 7) + return write_buffers_description_v7(handle); + + return write_buffers_description_v6(handle); +} + static tsize_t get_buffer_file_offset(struct tracecmd_output *handle, const char *name) { struct tracecmd_buffer *buf; @@ -1508,7 +1575,8 @@ struct tracecmd_output *tracecmd_create_file_latency(const char *output_file, in if (tracecmd_write_cpus(handle, cpus) < 0) goto out_free; - + if (tracecmd_write_buffers_description(handle) < 0) + goto out_free; if (tracecmd_write_options(handle) < 0) goto out_free; @@ -1701,7 +1769,9 @@ int tracecmd_append_cpu_data(struct tracecmd_output *handle, ret = tracecmd_write_cpus(handle, cpus); if (ret) return ret; - + ret = tracecmd_write_buffers_description(handle); + if (ret) + return ret; ret = tracecmd_write_options(handle); if (ret) return ret; diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index ed82104b..56acb01d 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -420,6 +420,29 @@ static void dump_cpus_count(int fd) do_print(SUMMARY, "\t%d [CPUs with tracing data]\n", trace_cpus); } +static void dump_buffers(int fd) +{ + unsigned long long offset; + char name[DUMP_SIZE]; + int count; + + if (file_version < 7) + return; + + if (read_file_number(fd, &count, 4)) + die("cannot read the buffers count"); + do_print(SUMMARY, "\t%d [buffers with tracing data]\n", count); + + while (count) { + if (read_file_number(fd, &offset, 8)) + die("cannot read the buffer offset"); + if (read_file_string(fd, name, DUMP_SIZE)) + die("cannot read the buffer name"); + do_print(SUMMARY, "\t\t 0x%llx %s[data offset, buffer name]\n", offset, name); + count--; + } +} + static void dump_option_string(int fd, int size, char *desc) { do_print(OPTIONS, "\t\t[Option %s, %d bytes]\n", desc, size); @@ -762,6 +785,7 @@ static void dump_file(const char *file) dump_printk(fd); dump_cmdlines(fd); dump_cpus_count(fd); + dump_buffers(fd); dump_therest(fd); tep_free(tep); diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c index 0ae1c948..62debdb6 100644 --- a/tracecmd/trace-listen.c +++ b/tracecmd/trace-listen.c @@ -604,6 +604,9 @@ static int put_together_file(int cpus, int ofd, const char *node, if (write_options) { ret = tracecmd_write_cpus(handle, cpus); + if (ret) + goto out; + ret = tracecmd_write_buffers_description(handle); if (ret) goto out; ret = tracecmd_write_options(handle); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index e12d77cc..5516fe3a 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -3653,6 +3653,9 @@ setup_connection(struct buffer_instance *instance, struct common_record_context if (ret) goto error; ret = tracecmd_write_cpus(network_handle, instance->cpu_count); + if (ret) + goto error; + ret = tracecmd_write_buffers_description(network_handle); if (ret) goto error; ret = tracecmd_write_options(network_handle); @@ -3856,6 +3859,7 @@ static void setup_agent(struct buffer_instance *instance, add_options(network_handle, ctx); tracecmd_write_cmdlines(network_handle); tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_buffers_description(network_handle); tracecmd_write_options(network_handle); tracecmd_msg_finish_sending_data(instance->msg_handle); instance->network_handle = network_handle; From patchwork Wed May 12 06:38:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A310CC433ED for ; Wed, 12 May 2021 06:38:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 84EF8613BA for ; Wed, 12 May 2021 06:38:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230160AbhELGjy (ORCPT ); Wed, 12 May 2021 02:39:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230135AbhELGjy (ORCPT ); Wed, 12 May 2021 02:39:54 -0400 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9A63C06174A for ; Tue, 11 May 2021 23:38:45 -0700 (PDT) Received: by mail-ed1-x52e.google.com with SMTP id a25so1571583edr.12 for ; Tue, 11 May 2021 23:38:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lwqPloBHhTOpLctpjqXbzcxqgIoZCw15PdcDzojOOdA=; b=qatz6s+1e8k6jSwSX+dzson52c6/BaBroxkDE3q6frJ9QWEi6DIJdEKlOPwRNn/Fqz YIygJL1/gld+NMAVrwZJbmb07z4u68HXaUTnV8pDfA4+zhRuxN1eDJBDfEQNj+v6s0aO QoeCoZs9c3WgRER/HrOSO+yYrddLcegtFipXUmekQUl20GRJHNO7cuBihziGfj0qA6rR r1rCdTHLRhDy2kxsnjeqWaueh+wY4rAiBR8Gx1NMBF5OoUZvhzo7CwfTST0LtIYVjJw1 I7PzFRsNerjHGrs9102MPtHkRRwPK7bMu5NvXMoH4fhVNUBq7fyxzv1ToIQqJCoomhey vPyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lwqPloBHhTOpLctpjqXbzcxqgIoZCw15PdcDzojOOdA=; b=qApNLSM9ohM1tIHQAMCUYdARuJZ86TrFWuUl2JzNoGLvsqBFMKOQGK8dlgqDbvi+Af VZyVJ5FFfchB5YAjZRfc0fG6T4Cah0e2WksKLRq8KdLrnh0ugipTfnWX4kx0f8mWyari 2r5FGpdZC9IFyUZCMc0RWkg3n2QUgrLpocxOrvxaN30VjAW4D0qPWjGQXzTwN4HgDYHu 1fGuiCbqxye7CQ9qdz05a9oXW7USZZH/elgwXe+iDm5iFJUZCe+RVoZLmTmaASSWs4K4 4ZZsL0cF78eqsZAgoLwybHbu6ODlWGmDobCRxsDy3EZVkNkYbs+BsOVUAiOGcRTEX0Rf nBAQ== X-Gm-Message-State: AOAM533HeUCpTjmz9Q5gG1cuMWPGt3uOjTMq3em5o4+TbEl1jzuOgcpr DDeM45tmPt6+qg5Q6aAFipALn4GXTFcitg== X-Google-Smtp-Source: ABdhPJzV6iPhGP8b2K42osj5k3YempeJuJgC0VepNsb0ML61bx+Z/C5TfmdBE//sDVDxXmWLKgDO6A== X-Received: by 2002:a05:6402:849:: with SMTP id b9mr27067212edz.126.1620801524562; Tue, 11 May 2021 23:38:44 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:44 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 20/25] trace-cmd library: Track the offset in the option section in the trace file Date: Wed, 12 May 2021 09:38:18 +0300 Message-Id: <20210512063823.324610-21-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading / writing trace file, store the offset of the option section in the file. A new internal API is added, to get the offset of the option section from an input file handler: tracecmd_get_options_offset() These changes are needed for compression of the option section from the trace file. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/include/private/trace-cmd-private.h | 1 + lib/trace-cmd/trace-input.c | 11 +++++++++++ lib/trace-cmd/trace-output.c | 2 ++ 3 files changed, 14 insertions(+) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index d7488346..a9b66736 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -239,6 +239,7 @@ tracecmd_get_cursor(struct tracecmd_input *handle, int cpu); unsigned long tracecmd_get_file_version(struct tracecmd_input *handle); int tracecmd_get_file_compress_proto(struct tracecmd_input *handle, const char **name, const char **version); +size_t tracecmd_get_options_offset(struct tracecmd_input *handle); int tracecmd_ftrace_overrides(struct tracecmd_input *handle, struct tracecmd_ftrace *finfo); bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle); diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 5088a059..78581789 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -157,6 +157,7 @@ struct tracecmd_input { size_t header_files_start; size_t ftrace_files_start; size_t event_files_start; + size_t options_start; size_t total_file_size; /* For custom profilers. */ @@ -2759,6 +2760,7 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); for (;;) { if (do_read_check(handle, &option, 2)) @@ -4215,6 +4217,15 @@ bool tracecmd_get_use_trace_clock(struct tracecmd_input *handle) return handle->use_trace_clock; } +/** + * tracecmd_get_options_offset - get offset of the options sections in the file + * @handle: input handle for the trace.dat file + */ +size_t tracecmd_get_options_offset(struct tracecmd_input *handle) +{ + return handle->options_start; +} + /** * tracecmd_get_show_data_func - return the show data func * @handle: input handle for the trace.dat file diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 57b11493..1bd2351f 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -66,6 +66,7 @@ struct tracecmd_output { bool quiet; unsigned long file_state; unsigned long file_version; + size_t options_start; bool do_compress; struct tracecmd_compression *compress; @@ -1854,6 +1855,7 @@ struct tracecmd_output *tracecmd_get_output_handle_fd(int fd) tep_ref(handle->pevent); handle->page_size = tracecmd_page_size(ihandle); handle->file_version = tracecmd_get_file_version(ihandle); + handle->options_start = tracecmd_get_options_offset(ihandle); list_head_init(&handle->options); list_head_init(&handle->buffers); From patchwork Wed May 12 06:38:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5CB2C43460 for ; Wed, 12 May 2021 06:38:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B4CEC617C9 for ; Wed, 12 May 2021 06:38:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230135AbhELGjy (ORCPT ); Wed, 12 May 2021 02:39:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230157AbhELGjy (ORCPT ); Wed, 12 May 2021 02:39:54 -0400 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC9D9C061574 for ; Tue, 11 May 2021 23:38:46 -0700 (PDT) Received: by mail-ed1-x52b.google.com with SMTP id di13so25713929edb.2 for ; Tue, 11 May 2021 23:38:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tLnpfeNVWbyIOVfArLlXsardczbIEiDd7wB8LQwgGYw=; b=B4bUZVF7Szi0Q4JvzqfAKLmKzqtnB9yyhp2mtDqun5GI6WdOUUxQeSR6M4L5q0ihHo L4fetodIsedYLWHeEFVGiILdMKucafiWqmEoczWlAZ7GhyuTW2hdB1lwfHEJ+EwFPkGd nTOeFgt7+sOGeqKtroeAE7ml6AUSyWFt8dG1JkDalHCMv2OzI9lNt+9nugGV/Yw0iw3z HcQ2fJLER0ybmWmBfAJGCL1WKk2GA4izKNG+/tDJlFhhH7LHqr0JphS2VGhxzEXQy9+O nBCpcyzroXJ70vGMXhiN2OF6VLOvk3eFOI2waOTlS/MsxkBpJmUhPDpTy3y3INqa8qqN JHZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tLnpfeNVWbyIOVfArLlXsardczbIEiDd7wB8LQwgGYw=; b=A6h2tFeegeqGWCHzdE2Nkb9tm0UrvGMfihpnQE36uQ/0309CbIxuoNOWDBjcaXOO2V /0luCtdLM8Kdnv4Sr+h80jH2NT/hug8HjoPNJGMTK8AQB/XVm0Ipu1victtdMhc6tWi/ Er6QHXo7PcftSQ38KOtmlcHvG8VFNwHATEVrSV+6o3aGhSLdJlDXlJU0jQDacsp71mfH lM8g8U6oob1cly7qGCxSgZFwyxGJX5oO09dJ4cqjbZnYi/t6YGn3iD2VpJwmO7n0ufEd kZld5bOBTBZvoAlEoEHAfLZfCLhvalZiiiBaOmxG+kw717ofTunVx/sFxR08RfJLhCug KgXw== X-Gm-Message-State: AOAM533PA6gEmMwRoy8UUVr6iCUJjXERiUOpLrGcMdWSH4MPuCMZBv2U OOB9IbrmJDhcZzaPL9XdSw5Vshv6LpEsMg== X-Google-Smtp-Source: ABdhPJyD4lAKtffH8+UM/meUDEGFRicYWSr1T5A7OjOVhKq9zoA7KgPSlAjRO/FitdvjyVwlWxR1xw== X-Received: by 2002:a05:6402:da:: with SMTP id i26mr4833888edu.379.1620801525477; Tue, 11 May 2021 23:38:45 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:45 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 21/25] trace-cmd library: Add compression of the option section of the trace file Date: Wed, 12 May 2021 09:38:19 +0300 Message-Id: <20210512063823.324610-22-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Comperss the option section of the trace file. This section is not big currently and compressing it does not reduce significantly the size of the file. This could be useful in the future as new options can be added, storing a potentially huge amount of data. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 46 +++++++++++++++--------- lib/trace-cmd/trace-output.c | 68 ++++++++++++++++++++++++------------ tracecmd/trace-dump.c | 16 +++++++-- 3 files changed, 89 insertions(+), 41 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 78581789..1cec6917 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -2761,23 +2761,29 @@ static int handle_options(struct tracecmd_input *handle) /* By default, use usecs, unless told otherwise */ handle->flags |= TRACECMD_FL_IN_USECS; handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); - + if (uncompress_block(handle)) + return -1; for (;;) { - if (do_read_check(handle, &option, 2)) - return -1; + ret = do_read_check(handle, &option, 2); + if (ret) + goto out; if (option == TRACECMD_OPTION_DONE) break; /* next 4 bytes is the size of the option */ - if (do_read_check(handle, &size, 4)) - return -1; + ret = do_read_check(handle, &size, 4); + if (ret) + goto out; size = tep_read_number(handle->pevent, &size, 4); buf = malloc(size); - if (!buf) - return -ENOMEM; - if (do_read_check(handle, buf, size)) - return -1; + if (!buf) { + ret = -ENOMEM; + goto out; + } + ret = do_read_check(handle, buf, size); + if (ret) + goto out; switch (option) { case TRACECMD_OPTION_DATE: @@ -2827,14 +2833,16 @@ static int handle_options(struct tracecmd_input *handle) buf + 8, 4); ret = tsync_cpu_offsets_load(handle, buf + 12, size - 12); if (ret < 0) - return ret; + goto out; tracecmd_enable_tsync(handle, true); break; case TRACECMD_OPTION_CPUSTAT: buf[size-1] = '\n'; cpustats = realloc(cpustats, cpustats_size + size + 1); - if (!cpustats) - return -ENOMEM; + if (!cpustats) { + ret = -ENOMEM; + goto out; + } memcpy(cpustats + cpustats_size, buf, size); cpustats_size += size; cpustats[cpustats_size] = 0; @@ -2844,14 +2852,17 @@ static int handle_options(struct tracecmd_input *handle) handle->nr_buffers++; handle->buffers = realloc(handle->buffers, sizeof(*handle->buffers) * handle->nr_buffers); - if (!handle->buffers) - return -ENOMEM; + if (!handle->buffers) { + ret = -ENOMEM; + goto out; + } buffer = &handle->buffers[handle->nr_buffers - 1]; buffer->name = strdup(buf + 8); if (!buffer->name) { free(handle->buffers); handle->buffers = NULL; - return -ENOMEM; + ret = -ENOMEM; + goto out; } offset = *(unsigned long long *)buf; buffer->offset = tep_read_number(handle->pevent, &offset, 8); @@ -2908,8 +2919,11 @@ static int handle_options(struct tracecmd_input *handle) } handle->cpustats = cpustats; + ret = 0; - return 0; +out: + uncompress_reset(handle); + return ret; } static int read_options_type(struct tracecmd_input *handle) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 1bd2351f..ca115872 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -150,6 +150,18 @@ static inline void compression_reset(struct tracecmd_output *handle) handle->do_compress = false; } +static int uncompress_block(struct tracecmd_output *handle) +{ + int ret = 0; + + if (handle->file_version < 7 || !handle->compress) + return 0; + ret = tracecmd_uncompress_block(handle->compress); + if (!ret) + handle->do_compress = true; + return ret; +} + static inline int compression_start(struct tracecmd_output *handle) { if (handle->file_version < 7 || !handle->compress) @@ -1330,32 +1342,38 @@ int tracecmd_write_options(struct tracecmd_output *handle) if (do_write_check(handle, "options ", 10)) return -1; - + handle->options_start = lseek64(handle->fd, 0, SEEK_CUR); + compression_start(handle); list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; + goto error; + if (compression_end(handle)) + goto error; handle->file_state = TRACECMD_FILE_OPTIONS; return 0; +error: + compression_reset(handle); + return -1; } int tracecmd_append_options(struct tracecmd_output *handle) @@ -1371,42 +1389,48 @@ int tracecmd_append_options(struct tracecmd_output *handle) * We can append only if options are already written and tracing data * is not yet written */ - if (handle->file_state != TRACECMD_FILE_OPTIONS) + if (handle->file_state != TRACECMD_FILE_OPTIONS || !handle->options_start) return -1; - - if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) return -1; - offset = lseek64(handle->fd, -2, SEEK_CUR); - if (offset == (off_t)-1) + if (uncompress_block(handle)) return -1; - - r = pread(handle->fd, &option, 2, offset); + if (do_lseek(handle, 0, SEEK_END) == -1) + goto error; + offset = do_lseek(handle, -2, SEEK_CUR); + if (offset == (off_t)-1) + goto error; + r = do_preed(handle, &option, 2, offset); if (r != 2 || option != TRACECMD_OPTION_DONE) - return -1; - + goto error; list_for_each_entry(options, &handle->options, list) { endian2 = convert_endian_2(handle, options->id); if (do_write_check(handle, &endian2, 2)) - return -1; + goto error; endian4 = convert_endian_4(handle, options->size); if (do_write_check(handle, &endian4, 4)) - return -1; + goto error; /* Save the data location in case it needs to be updated */ - options->offset = lseek64(handle->fd, 0, SEEK_CUR); + options->offset = do_lseek(handle, 0, SEEK_CUR); if (do_write_check(handle, options->data, options->size)) - return -1; + goto error; } - option = TRACECMD_OPTION_DONE; if (do_write_check(handle, &option, 2)) - return -1; - + goto error; + if (lseek64(handle->fd, handle->options_start, SEEK_SET) == (off_t)-1) + goto error; + if (compression_end(handle)) + goto error; return 0; +error: + compression_reset(handle); + return -1; } static struct tracecmd_option * diff --git a/tracecmd/trace-dump.c b/tracecmd/trace-dump.c index 56acb01d..8d0f2251 100644 --- a/tracecmd/trace-dump.c +++ b/tracecmd/trace-dump.c @@ -82,6 +82,13 @@ static int read_compressed(int fd, char *dst, int len) return read_fd(fd, dst, len); } +static int do_lseek(int fd, int offset, int whence) +{ + if (read_compress) + return tracecmd_compress_lseek(compress, offset, whence); + return lseek64(fd, offset, whence); +} + static int read_file_string(int fd, char *dst, int len) { size_t size = 0; @@ -625,6 +632,9 @@ static void dump_options(int fd) unsigned int size; int count = 0; + if (uncompress_block()) + die("cannot uncompress file block"); + for (;;) { if (read_file_number(fd, &option, 2)) die("cannot read the option id"); @@ -635,7 +645,7 @@ static void dump_options(int fd) count++; if (!DUMP_CHECK(OPTIONS) && !DUMP_CHECK(CLOCK) && !DUMP_CHECK(SUMMARY)) { - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); continue; } switch (option) { @@ -685,12 +695,12 @@ static void dump_options(int fd) default: do_print(OPTIONS, " %d %d\t[Unknown option, size - skipping]\n", option, size); - lseek64(fd, size, SEEK_CUR); + do_lseek(fd, size, SEEK_CUR); break; } } do_print(SUMMARY, "\t[%d options]\n", count); - + uncompress_reset(); } static void dump_latency(int fd) From patchwork Wed May 12 06:38:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9BEEC433B4 for ; Wed, 12 May 2021 06:38:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC2B8613B4 for ; Wed, 12 May 2021 06:38:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230157AbhELGj4 (ORCPT ); Wed, 12 May 2021 02:39:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230154AbhELGj4 (ORCPT ); Wed, 12 May 2021 02:39:56 -0400 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FC6EC061574 for ; Tue, 11 May 2021 23:38:47 -0700 (PDT) Received: by mail-ed1-x52b.google.com with SMTP id u13so25710546edd.3 for ; Tue, 11 May 2021 23:38:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VBejlyIMKlFLT2t7QvD/+a7jOXPgFEcU9SQMoSVZ4vs=; b=PeACu3we6HkiNCHKj+7Yr+zlOxv2TRXbL4tpJoA6uNQbrUmKCNCAcVNadtEWUvm9LR jWaxRiJasyeAPqmeHpIRDDejtLHvCh37pfL0c3R5VlIZA7J246lA3zfUbXF3oTm6GcmC 9qwi0KrNF6CDFb+f+GWdk5m2mjSwY0YV4mb8nl+ynp1c066kalNQlJEYkgWoRGG38x2e TAVktuplpTpER3m4EP2MjNKPhSMO0UDJnAxtNI5KOPA/gothTk/iWLaOMj0DNOmUsTBc aObqX7I418f9WR4N3SMq2Iu5kERMO41q3h9uooFSMyHBBUDnWezA9CjRkohFUjktrNRQ RlCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VBejlyIMKlFLT2t7QvD/+a7jOXPgFEcU9SQMoSVZ4vs=; b=drv6WTIg7wk3ciA1+fuyT4kBIaTBpbxUNcSLUBS2FxH/+nWyTMHaKZG4hTg/afE4o1 RnraYqj1tSxj1IbuC3BpY6qukwguxnSxMg+MMtQFcgW8F0n3lc2SHEZ5sM0oITfrCqZp SDooyUNZmqpainHBuR18jHRfSut/vEDpvU2XROM0+lTPAtFx9fhViUIwdH2BCmdR5hmA G3ncbG8IpEshNA5o2Mw5Yh6/AzcY4BRPKPp6hJWvNJ9ZFSQF69B2JsEGsXEl8VHSv9eQ 3U4oqYly7XzZWdd8YyTaN6tLSpJ79q40FixQqQTNMbZVQNh5rISm9Ue+RHHoai4yiz4w 3qtw== X-Gm-Message-State: AOAM531dG3LbYNxD4u7oFchOPF1z6WJ1Z809zhZFAWZbNyNO5eJXrKDT NGY1IfAYs0diJhyRErlNFYrr4pUh9FDAiw== X-Google-Smtp-Source: ABdhPJzyceX3tr/uLhZJCpsDcnI5rYbjUcLtPLrp7YCwlJu1hfMrEE0bexQvzVXAOxnDaejjY+O4UA== X-Received: by 2002:aa7:c9cb:: with SMTP id i11mr41728473edt.331.1620801526329; Tue, 11 May 2021 23:38:46 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:45 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 22/25] trace-cmd library: Refactor the logic for writing trace data in the file Date: Wed, 12 May 2021 09:38:20 +0300 Message-Id: <20210512063823.324610-23-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Refactored the internal logic of tracecmd_write_cpu_data() API to be suitable for adding trace data compression. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 97 +++++++++++++++++------------------- 1 file changed, 46 insertions(+), 51 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index ca115872..08d05744 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -155,8 +155,7 @@ static int uncompress_block(struct tracecmd_output *handle) int ret = 0; if (handle->file_version < 7 || !handle->compress) - return 0; - ret = tracecmd_uncompress_block(handle->compress); + return 0; ret = tracecmd_uncompress_block(handle->compress); if (!ret) handle->do_compress = true; return ret; @@ -1672,11 +1671,17 @@ static char *get_clock(struct tracecmd_output *handle) return handle->trace_clock; } +struct data_file_write { + int file_size; + off64_t soffset; + off64_t data_offset; + off64_t doffset; +}; + int tracecmd_write_cpu_data(struct tracecmd_output *handle, int cpus, char * const *cpu_data_files) { - off64_t *offsets = NULL; - unsigned long long *sizes = NULL; + struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; char *clock = NULL; @@ -1698,37 +1703,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, if (do_write_check(handle, "flyrecord", 10)) goto out_free; - offsets = malloc(sizeof(*offsets) * cpus); - if (!offsets) - goto out_free; - sizes = malloc(sizeof(*sizes) * cpus); - if (!sizes) - goto out_free; - - offset = lseek64(handle->fd, 0, SEEK_CUR); - - /* hold any extra data for data */ - offset += cpus * (16); - - /* - * Unfortunately, the trace_clock data was placed after the - * cpu data, and wasn't accounted for with the offsets. - * We need to save room for the trace_clock file. This means - * we need to find the size of it before we define the final - * offsets. - */ - clock = get_clock(handle); - if (!clock) + data_files = calloc(cpus, sizeof(struct data_file_write)); + if (!data_files) goto out_free; - /* Save room for storing the size */ - offset += 8; - offset += strlen(clock); - /* 2 bytes for [] around the clock */ - offset += 2; - - /* Page align offset */ - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + /* Write 0 for trace data offset and size and store offsets of these fields */ for (i = 0; i < cpus; i++) { file = cpu_data_files[i]; ret = stat(file, &st); @@ -1736,53 +1715,69 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, tracecmd_warning("can not stat '%s'", file); goto out_free; } - offsets[i] = offset; - sizes[i] = st.st_size; - offset += st.st_size; - offset = (offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].file_size = st.st_size; - endian8 = convert_endian_8(handle, offsets[i]); + endian8 = 0; + data_files[i].doffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; - endian8 = convert_endian_8(handle, sizes[i]); + data_files[i].soffset = lseek64(handle->fd, 0, SEEK_CUR); if (do_write_check(handle, &endian8, 8)) goto out_free; } + clock = get_clock(handle); + if (!clock) + goto out_free; if (save_clock(handle, clock)) goto out_free; for (i = 0; i < cpus; i++) { + data_files[i].data_offset = lseek64(handle->fd, 0, SEEK_CUR); + /* Page align offset */ + data_files[i].data_offset = (data_files[i].data_offset + (handle->page_size - 1)) & ~(handle->page_size - 1); + data_files[i].data_offset = lseek64(handle->fd, data_files[i].data_offset, SEEK_SET); + if (data_files[i].data_offset == (off64_t)-1) + goto out_free; if (!tracecmd_get_quiet(handle)) fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", - i, (unsigned long long) offsets[i]); - offset = lseek64(handle->fd, offsets[i], SEEK_SET); - if (offset == (off64_t)-1) { - tracecmd_warning("could not seek to %lld\n", offsets[i]); - goto out_free; - } + i, (unsigned long long) data_files[i].data_offset); + check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != sizes[i]) { + if (check_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, sizes[i]); + check_size, data_files[i].file_size); goto out_free; } + /* Write the real CPU data offset inthe file */ + offset = lseek64(handle->fd, data_files[i].doffset, SEEK_SET); + endian8 = convert_endian_8(handle, data_files[i].data_offset); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + /* Write the real CPU data size in the file */ + offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); + endian8 = convert_endian_8(handle, check_size); + if (do_write_check(handle, &endian8, 8)) + goto out_free; + offset = data_files[i].data_offset + check_size; + offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", (unsigned long long)check_size); } - free(offsets); - free(sizes); + if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) + goto out_free; + free(data_files); handle->file_state = TRACECMD_FILE_CPU_FLYRECORD; return 0; out_free: - free(offsets); - free(sizes); + lseek64(handle->fd, 0, SEEK_END); + free(data_files); return -1; } From patchwork Wed May 12 06:38:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FD01C433ED for ; Wed, 12 May 2021 06:38:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0335D613BA for ; Wed, 12 May 2021 06:38:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230168AbhELGkE (ORCPT ); Wed, 12 May 2021 02:40:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230154AbhELGj4 (ORCPT ); Wed, 12 May 2021 02:39:56 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65135C06174A for ; Tue, 11 May 2021 23:38:48 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id v5so14632326edc.8 for ; Tue, 11 May 2021 23:38:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BbgUX1dAmAvyY+7LZeJ/4MbTgGA6KeIVuGIq+vN1d5I=; b=sNj2aCIPvKXMSeWSiwAWuhZo17reqY74KB1iO51kiCiXojKs4rzd6vwgUIF+AERI5l bn8NSzEywp6rrG0091ZSJT45dHvdMtP6iUTE8y4pXP7EkuypbL5CLO7dWssouzFMPE+3 zSV1PMgng1Kvj7Np1WOX8jMiGdupg6+aXCh3x3l5d8AeKYGlOLjoiE1ZUtEY7Y5pJrDg ZNxob4I3MB/fdC8BtkDz0eJrs/g8Rs4aS5mNdKW6Eub+n0I97jNOoCkwu2rezrn7qP1a XZPio8FY+X2CMJsdfrsOy5Pxz4NmPf7qFetlYP9GhZOVW0PW0aLgkdAIZnGPOWK5ad7b cu4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BbgUX1dAmAvyY+7LZeJ/4MbTgGA6KeIVuGIq+vN1d5I=; b=MvlAEZyfBXuaWdKbIGRAKwto775AyjP1p827wo4uz0NvHtUCqCWpAqxNCinuT4nhva hwwDB8HFVN3yhBmE/ibRQ+EpJxXQIQEqB+fndklsrTijBIi1h7XKQSRQEA6H3pAYLnLl f8dydEV90ZlaXk+n9GLzwNgJbFwrNaUH5o4yF/Q1flnTTRL+M7uEeA1m9jlJEk7NbVpQ PYXqNyxoLRWeOvw90eUQ0D4qrVM7+sxHHP8cZ8L1Kie4xl08B+xZn+fi2fVwbGUWrV+8 6OjBTMIeljaf40m+Yll/n8GVPjBMVGM/QzJZHBlhy6szGwRzCnNzmlDLOCoAI3Enz+aB 9ryQ== X-Gm-Message-State: AOAM533siaiVp8INkJSp7xxvPChvp1QOp7GPitl7rQaLxTsMDfwCwBBd F1jB2D3pLhvjxJARCDGIftgTrOonMRdXBA== X-Google-Smtp-Source: ABdhPJzRsJah2MoijZvsMQZxiww5Pcg/LVzUTPTbNXZ9DPFKO0SfDzHDcrPLQQ4DZq+otaEJ1zFXBA== X-Received: by 2002:a50:8e44:: with SMTP id 4mr41322838edx.244.1620801527184; Tue, 11 May 2021 23:38:47 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:46 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 23/25] trace-cmd library: Add APIs for read and write compressed data in chunks Date: Wed, 12 May 2021 09:38:21 +0300 Message-Id: <20210512063823.324610-24-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org New compression APIs are added for writing and reading compressed data from / to files in chunks: tracecmd_compress_copy_from() tracecmd_uncompress_copy_to() Format of the compressed data, used by these APIs, is: - 4 bytes, chunks count - for each chunk: - 4 bytes, size of compressed data in this chunk - 4 bytes, uncompressed size of the data in this chunk - data, bytes of Signed-off-by: Tzvetomir Stoyanov (VMware) --- .../include/private/trace-cmd-private.h | 5 +- lib/trace-cmd/trace-compress.c | 191 ++++++++++++++++++ 2 files changed, 195 insertions(+), 1 deletion(-) diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h index a9b66736..950e6b51 100644 --- a/lib/trace-cmd/include/private/trace-cmd-private.h +++ b/lib/trace-cmd/include/private/trace-cmd-private.h @@ -493,7 +493,10 @@ int tracecmd_compress_proto_register(const char *name, const char *version, int char *out, unsigned int *out_bytes), unsigned int (*comress_size)(unsigned int bytes), bool (*is_supported)(const char *name, const char *version)); - +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size); +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size); /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/lib/trace-cmd/trace-compress.c b/lib/trace-cmd/trace-compress.c index 2b329d00..3a7accc2 100644 --- a/lib/trace-cmd/trace-compress.c +++ b/lib/trace-cmd/trace-compress.c @@ -594,3 +594,194 @@ error: free(v); return -1; } + +/** + * tracecmd_compress_copy_from - Copy and compress data from a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data + * @chunk_size: size of one compression chunk + * @read_size: return, size of the uncompressed data read from @fd + * @write_size: return, size of the compressed data written into @handle + * + * This function reads uncompressed data from given @fd until EOF is reached, compresses + * the data using the @handle compression context and writes the compressed data into the fd + * associated with the @handle. The data is compressed on chunks with given @chunk_size size. + * The compressed data is written in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_compress_copy_from(struct tracecmd_compression *handle, int fd, int chunk_size, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int chunks = 0; + unsigned int rsize = 0; + unsigned int wsize = 0; + unsigned int csize; + unsigned int size; + unsigned int r; + off64_t offset; + char *buf_from; + char *buf_to; + int endian4; + int ret; + + if (!handle || !handle->proto || + !handle->proto->comress_block || !handle->proto->comress_size) + return 0; + + csize = handle->proto->comress_size(chunk_size); + buf_from = malloc(chunk_size); + if (!buf_from) + return -1; + buf_to = malloc(csize); + if (!buf_to) + return -1; + /* save the initial offset and write 0 chunks */ + offset = lseek64(handle->fd, 0, SEEK_CUR); + write_fd(handle->fd, &chunks, 4); + + do { + r = read(fd, buf_from, chunk_size); + if (r < 0) + break; + rsize += r; + size = csize; + if (r > 0) { + ret = handle->proto->comress_block(buf_from, r, buf_to, &size); + if (ret < 0) { + if (errno == EINTR) + continue; + break; + } + /* Write compressed data size */ + endian4 = tep_read_number(handle->tep, &size, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write uncompressed data size */ + endian4 = tep_read_number(handle->tep, &r, 4); + ret = write_fd(handle->fd, &endian4, 4); + if (ret != 4) + break; + /* Write the compressed data */ + ret = write_fd(handle->fd, buf_to, size); + if (ret != size) + break; + /* data + compress header */ + wsize += (size + 8); + chunks++; + } + } while (r > 0); + free(buf_from); + free(buf_to); + if (r) + return -1; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + endian4 = tep_read_number(handle->tep, &chunks, 4); + /* write chunks count*/ + write_fd(handle->fd, &chunks, 4); + lseek64(handle->fd, offset, SEEK_SET); + if (lseek64(handle->fd, 0, SEEK_END) == (off_t)-1) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} + +/** + * tracecmd_uncompress_copy_to - Uncompress data and copy to a file + * @handle: compression handler + * @fd: file descriptor to uncompressed data + * @read_size: return, size of the compressed data read from @handle + * @write_size: return, size of the uncompressed data written into @fd + * + * This function reads compressed data from the fd, associated with @handle, uncompresses it + * using the @handle compression context and writes the uncompressed data into the fd. + * The compressed data must be in the format: + * - 4 bytes, chunks count + * - for each chunk: + * - 4 bytes, size of compressed data in this chunk + * - 4 bytes, uncompressed size of the data in this chunk + * - data, bytes of + * + * On success 0 is returned, @read_size and @write_size are updated with the size of + * read and written data. + */ +int tracecmd_uncompress_copy_to(struct tracecmd_compression *handle, int fd, + unsigned long long *read_size, unsigned long long *write_size) +{ + unsigned int s_uncompressed; + unsigned int s_compressed; + unsigned int rsize = 0; + unsigned int wsize = 0; + char *bytes_out = NULL; + char *bytes_in = NULL; + int size_out; + int size_in; + int chunks; + char buf[4]; + char *tmp; + int ret; + + if (!handle || !handle->proto || !handle->proto->uncompress_block) + return -1; + + if (read(handle->fd, buf, 4) != 4) + return -1; + chunks = tep_read_number(handle->tep, buf, 4); + rsize += 4; + while (chunks) { + if (read(handle->fd, buf, 4) != 4) + break; + s_compressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (read(handle->fd, buf, 4) != 4) + break; + s_uncompressed = tep_read_number(handle->tep, buf, 4); + rsize += 4; + if (!bytes_in || size_in < s_compressed) { + tmp = realloc(bytes_in, s_compressed); + if (!tmp) + break; + bytes_in = tmp; + size_in = s_compressed; + } + + if (!bytes_out || size_out < s_uncompressed) { + tmp = realloc(bytes_out, s_uncompressed); + if (!tmp) + break; + bytes_out = tmp; + size_out = s_uncompressed; + } + + if (read_fd(handle->fd, bytes_in, s_compressed) < 0) + break; + rsize += s_compressed; + ret = handle->proto->uncompress_block(bytes_in, s_compressed, + bytes_out, &s_uncompressed); + if (ret) + break; + write_fd(fd, bytes_out, s_uncompressed); + wsize += s_uncompressed; + chunks--; + } + free(bytes_in); + free(bytes_out); + if (chunks) + return -1; + if (read_size) + *read_size = rsize; + if (write_size) + *write_size = wsize; + return 0; +} From patchwork Wed May 12 06:38:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9523C433B4 for ; Wed, 12 May 2021 06:38:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8C2C3613BA for ; Wed, 12 May 2021 06:38:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230154AbhELGkE (ORCPT ); Wed, 12 May 2021 02:40:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230166AbhELGj5 (ORCPT ); Wed, 12 May 2021 02:39:57 -0400 Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BBB9C06175F for ; Tue, 11 May 2021 23:38:49 -0700 (PDT) Received: by mail-ed1-x533.google.com with SMTP id s6so25663169edu.10 for ; Tue, 11 May 2021 23:38:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g1oiBlQjU7BXkc4N3aSbi+iGHH5sqHO8pTrGqpp+wn4=; b=AJNHMYNj6SjaLClRrFMdzZAMHIMIJUySmbAi5teAQ2R1HqZyIrl5HHHE1YYKMA1Bhc KaBcRqsgOYKdKjrN0Ex/9QEvYqHpgRu4436d5FWfMX3soYeO6cv2+PGOITNjedGbaLnW SsihucLzOa3fLZYxENpvHg0AUHIqRHVnfNr5qLV1YeQ8r7efooZNYD2qys/24uNyyRvE Jg5kh8juSSzHizvjSxe7KsKd7V8pCh+xpC61krdp9wDczyIqF+MDJk6nVPostX5F37zT DR8BB18tMt6g5vBVdnsRyVuIYRtpLwfdXXot5wi+IUc2qZXmFsxPoUGWh4C6TaNa9gIk ycMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g1oiBlQjU7BXkc4N3aSbi+iGHH5sqHO8pTrGqpp+wn4=; b=FV7zT/PLlXAWIqdSsosWyZGfn4ybyZewrFr+T9hzVXkVhRm0qzAqEnYu52abwoaGBE YKe7JbAMfZ9x9dNL9TIiSKsxraPh1BR14zifN87UYRQSGVKH+a6/LDsbiHlLMVseNyoi RpPX+JCIu/3onCNp98vNffHs6+Ik4wIRp9uUc1OEdiIkCTHLHZICOW+jbTcR9/t1evzL F+Gn2tBaWo20SH4nWk20WKKicbtcibgAE49P/VSK9/AMR/Fvsj1RNqMHo1/f+VKakWcu SyPOABUmEmVfThTbJPRZ28Okg8MfF+mwa0hwI5NPVSK6fQ8IIEBIf8pOBSOk5JMnBc7R WmvA== X-Gm-Message-State: AOAM532fEdlfZ9z+iilenyUuYDcfyYL/vKr8nd/FZRms9+tJU4S4tAI3 1TGebDk1J7sway95kSNBoQZ+siq/uOUImQ== X-Google-Smtp-Source: ABdhPJyelqriB4FTDthOVM5+ZgM5mKZPva1Iky8GK3Gk3FcBfy5kN4OJxoV8f9bf6BgIlkZcIxon8w== X-Received: by 2002:a05:6402:750:: with SMTP id p16mr41157019edy.156.1620801527988; Tue, 11 May 2021 23:38:47 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:47 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 24/25] trace-cmd: Compress trace data Date: Wed, 12 May 2021 09:38:22 +0300 Message-Id: <20210512063823.324610-25-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When creating a trace.dat file of version 7, compress the trace data. The data is compressed in chunks. The chunk size is a compile time parameter, set by default to 1MB. Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-output.c | 50 +++++++++++++++++++++++++++++++----- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/lib/trace-cmd/trace-output.c b/lib/trace-cmd/trace-output.c index 08d05744..fab01f56 100644 --- a/lib/trace-cmd/trace-output.c +++ b/lib/trace-cmd/trace-output.c @@ -320,6 +320,41 @@ static tsize_t copy_file(struct tracecmd_output *handle, return size; } +#define COMPRESS_CHUNK_SIZE (1*1024*1024) +static tsize_t copy_file_compress(struct tracecmd_output *handle, + const char *file, unsigned long long *write_size) +{ + unsigned long long rsize = 0; + unsigned long long wsize = 0; + tsize_t size; + int ret; + int fd; + + fd = open(file, O_RDONLY); + if (fd < 0) { + tracecmd_warning("Can't read '%s'", file); + return 0; + } + + if (handle->file_version >= 7) { + ret = tracecmd_compress_copy_from(handle->compress, fd, + COMPRESS_CHUNK_SIZE, &rsize, &wsize); + if (ret < 0) { + tracecmd_warning("Can't compress '%s'", file); + close(fd); + return 0; + } + size = rsize; + *write_size = wsize; + } else { + size = copy_file_fd(handle, fd); + *write_size = size; + } + + close(fd); + return size; +} + /* * Finds the path to the debugfs/tracing * Allocates the string and stores it. @@ -1684,8 +1719,9 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, struct data_file_write *data_files = NULL; off64_t offset; unsigned long long endian8; + unsigned long long read_size; + unsigned long long write_size; char *clock = NULL; - off64_t check_size; char *file; struct stat st; int ret; @@ -1743,11 +1779,11 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, fprintf(stderr, "CPU%d data recorded at offset=0x%llx\n", i, (unsigned long long) data_files[i].data_offset); - check_size = copy_file(handle, cpu_data_files[i]); - if (check_size != data_files[i].file_size) { + read_size = copy_file_compress(handle, cpu_data_files[i], &write_size); + if (read_size != data_files[i].file_size) { errno = EINVAL; tracecmd_warning("did not match size of %lld to %lld", - check_size, data_files[i].file_size); + read_size, data_files[i].file_size); goto out_free; } /* Write the real CPU data offset inthe file */ @@ -1757,14 +1793,14 @@ int tracecmd_write_cpu_data(struct tracecmd_output *handle, goto out_free; /* Write the real CPU data size in the file */ offset = lseek64(handle->fd, data_files[i].soffset, SEEK_SET); - endian8 = convert_endian_8(handle, check_size); + endian8 = convert_endian_8(handle, write_size); if (do_write_check(handle, &endian8, 8)) goto out_free; - offset = data_files[i].data_offset + check_size; + offset = data_files[i].data_offset + write_size; offset = lseek64(handle->fd, offset, SEEK_SET); if (!tracecmd_get_quiet(handle)) fprintf(stderr, " %llu bytes in size\n", - (unsigned long long)check_size); + (unsigned long long)write_size); } if (lseek64(handle->fd, 0, SEEK_END) == (off64_t)-1) From patchwork Wed May 12 06:38:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12252893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D018C43460 for ; Wed, 12 May 2021 06:38:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3754561628 for ; Wed, 12 May 2021 06:38:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230172AbhELGkF (ORCPT ); Wed, 12 May 2021 02:40:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229580AbhELGkE (ORCPT ); Wed, 12 May 2021 02:40:04 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18CB5C061574 for ; Tue, 11 May 2021 23:38:50 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id di13so25714068edb.2 for ; Tue, 11 May 2021 23:38:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9p3HZ+fV114HUtuCsEoFxcTzF+q4DNlNLh6cTUwrBE0=; b=tF0aI57xN7PJ0PBmguLDhVvbJwSBq9ezYmLRLLl3j0t7K5thEuQegRTmaTTkI+GPhM l0RfPGv6J+zyBKlkrTRQqZy0U5wM5bgr5P0xIFTMnCfQycyCepXIoQ4DNtJbpw+na9QT Jfbf9Peh4QsuiVy02SrCxKI4LZS5bFFCOD3KwZRdOVKgFYplqA3k8/5dfUvEMdj1HOMD ByDV7GIx6FuSf52I/hQTESxq9KwsclPqMeR0gOnf2Iv2g0k43NETQdTmmyzXPCBVdIbh lvk0is6oeUVDUd0ALU+niR82MIulWimOdXDavZTIrxeO5CBNMcBdMsY3ErXlLDvHBwFp MpAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9p3HZ+fV114HUtuCsEoFxcTzF+q4DNlNLh6cTUwrBE0=; b=Aiv+bZQst/NIr5U7/lVsuYspGBCIcRKXhxHMGzDV8SupMuKwOVHZZPP9CVN6CXqGZt BWQfDyKpg8f9Ou4k0XL7JaebJqllAN1J4N/7BEMXjTdNVv6sMbOQ2WhPqlGrybR9+9yZ q0BtjNEtU4DHTa01Bq+sQzqGOsMj6NEIfOa36qIPbMffm+c7opxiRNNUmwICLKZqr7Lw 3lI3FCsIoT24DSVSoGiITqLdXV1CFZjODjF4+KBnvqNnYb7+XvuUU2wqJiQYqoa2dQiH 0NP68qGD5ZfAUGTQTVTt7yoTyLi2ijQa6ZPBfKUtw4kxmmo5rsDFXLtVk54HNE6Lkw/x Mivw== X-Gm-Message-State: AOAM531oR9nlBSaZS3I448QlVur5/qCyml+NC7QGwrQc7LZqyGv4ZHF1 2CAYoAz/8XKaSuR2VYcrl1lkXBCYfxzfbg== X-Google-Smtp-Source: ABdhPJwVM8CzIOwtqftv/fiMUNMzAXks644NWHTqVdAiaO9UjplSGa33+7uUX/2vt3yp86O0dY0sHg== X-Received: by 2002:aa7:cb48:: with SMTP id w8mr41165362edt.55.1620801528888; Tue, 11 May 2021 23:38:48 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id h23sm13387723ejx.90.2021.05.11.23.38.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 May 2021 23:38:48 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v3 25/25] trace-cmd: Read compressed trace data Date: Wed, 12 May 2021 09:38:23 +0300 Message-Id: <20210512063823.324610-26-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210512063823.324610-1-tz.stoyanov@gmail.com> References: <20210512063823.324610-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org When reading a trace.dat file of version 7, uncompress the trace data. The trace data for each CPU is uncompressed in a temporary file, located in /tmp directory with prefix "trace_cpu_data". Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 74 +++++++++++++++++++++++++++++-------- tracecmd/trace-read.c | 8 ++++ 2 files changed, 67 insertions(+), 15 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 1cec6917..e7fd0d30 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,7 @@ struct page { #endif }; +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +73,10 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + + /* temporary file for uncompressed cpu data */ + int cfd; + char cfile[26]; /* strlen(COMPR_TEMP_FILE) */ }; struct input_buffer_instance { @@ -1080,6 +1085,7 @@ static void *allocate_page_map(struct tracecmd_input *handle, off64_t map_offset; void *map; int ret; + int fd; if (handle->read_page) { map = malloc(handle->page_size); @@ -1119,12 +1125,15 @@ static void *allocate_page_map(struct tracecmd_input *handle, map_size -= map_offset + map_size - (cpu_data->file_offset + cpu_data->file_size); + if (cpu_data->cfd >= 0) + fd = cpu_data->cfd; + else + fd = handle->fd; again: page_map->size = map_size; page_map->offset = map_offset; - page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, - handle->fd, map_offset); + page_map->map = mmap(NULL, map_size, PROT_READ, MAP_PRIVATE, fd, map_offset); if (page_map->map == MAP_FAILED) { /* Try a smaller map */ @@ -2316,13 +2325,41 @@ tracecmd_read_prev(struct tracecmd_input *handle, struct tep_record *record) /* Not reached */ } +static int cpu_data_uncompress(struct tracecmd_input *handle, int cpu, unsigned long long *size) +{ + struct cpu_data *cpu_data; + + cpu_data = &handle->cpu_data[cpu]; + strcpy(cpu_data->cfile, COMPR_TEMP_FILE); + cpu_data->cfd = mkstemp(cpu_data->cfile); + if (cpu_data->cfd < 0) + return -1; + return tracecmd_uncompress_copy_to(handle->compress, cpu_data->cfd, NULL, size); +} + static int init_cpu(struct tracecmd_input *handle, int cpu) { struct cpu_data *cpu_data = &handle->cpu_data[cpu]; + unsigned long long size; + off64_t offset; int i; - cpu_data->offset = cpu_data->file_offset; - cpu_data->size = cpu_data->file_size; + if (handle->file_version >= 7) { + offset = lseek64(handle->fd, 0, SEEK_CUR); + if (lseek64(handle->fd, cpu_data->file_offset, SEEK_SET) == (off_t)-1) + return -1; + if (cpu_data_uncompress(handle, cpu, &size) < 0) + return -1; + cpu_data->offset = 0; + cpu_data->file_offset = 0; + cpu_data->file_size = size; + cpu_data->size = size; + if (lseek64(handle->fd, offset, SEEK_SET) == (off_t)-1) + return -1; + } else { + cpu_data->offset = cpu_data->file_offset; + cpu_data->size = cpu_data->file_size; + } cpu_data->timestamp = 0; list_head_init(&cpu_data->page_maps); @@ -3015,6 +3052,7 @@ static int read_cpu_data(struct tracecmd_input *handle) handle->cpu_data[cpu].file_offset = offset; handle->cpu_data[cpu].file_size = size; + handle->cpu_data[cpu].cfd = -1; if (size > max_size) max_size = size; @@ -3637,17 +3675,23 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].cfd >= 0) { + close(handle->cpu_data[cpu].cfd); + unlink(handle->cpu_data[cpu].cfile); + } } } diff --git a/tracecmd/trace-read.c b/tracecmd/trace-read.c index b315283e..e7a4f196 100644 --- a/tracecmd/trace-read.c +++ b/tracecmd/trace-read.c @@ -1363,7 +1363,14 @@ struct tracecmd_input *read_trace_header(const char *file, int flags) static void sig_end(int sig) { + struct handle_list *handles; + fprintf(stderr, "trace-cmd: Received SIGINT\n"); + + list_for_each_entry(handles, &handle_list, list) { + tracecmd_close(handles->handle); + } + exit(0); } @@ -1922,6 +1929,7 @@ void trace_report (int argc, char **argv) /* and version overrides uname! */ if (show_version) otype = OUTPUT_VERSION_ONLY; + read_data_info(&handle_list, otype, global, align_ts); list_for_each_entry(handles, &handle_list, list) {