From patchwork Mon Sep 13 12:41:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12488829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C090FC433F5 for ; Mon, 13 Sep 2021 12:42:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AC61960FF2 for ; Mon, 13 Sep 2021 12:42:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239116AbhIMMnd (ORCPT ); Mon, 13 Sep 2021 08:43:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239963AbhIMMnd (ORCPT ); Mon, 13 Sep 2021 08:43:33 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C580C061762 for ; Mon, 13 Sep 2021 05:42:17 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id d207-20020a1c1dd8000000b00307e2d1ec1aso725692wmd.5 for ; Mon, 13 Sep 2021 05:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hFneIwwflyvYS50+7Kf5cz199vOtKGcFApls84TqVf4=; b=cyJXTo2w/paUDVsrmRRUG36e80vtRxv/pxKE8C/zJ1wOuX2w3OWMt9lUKpObTkU2+r wkTmclkKPi4ouGAt/dlam18lMeB/lxe5ZanbChziiOUHb58fJvrKVfPYg4n+pDpCMZhT vKPrwsciYz76+ylhalcKZPUZrbAE+3LCl3Wji1Pa5o2dJZos7flPbcwHIZJ3aH6lKnYV bYODKyU8sLQskaK7EkYbL9KMl3XU0k9fXtosXpERiYzn+kTh6PfE0tzutAoC2e6EcmHy WHkOiT9ldQLIYaDmxdmQkVzXASzHN5Dq1Fgn5jAcuMdiwRAfRNc09RzV+3tCvugv3Dtk qLAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hFneIwwflyvYS50+7Kf5cz199vOtKGcFApls84TqVf4=; b=A8ZM70WNhXl9EN8PnTKy+ilwh3hSRvOqUN4D+qXJH8jQYif0q+97nSdpvB0WWyzK+6 xQBztCOHGnfqMxh53bBkPMuf9ZpmDi8V8JNYt/yQ+p+w09qX4Zn3ptc1+NYPz52C3F4L nzl/NyNDHbu7CD67UmBbTgzGt0kbJOlotf2EpScjpfou1hc6McMTz/rgUiifikceIlR9 /dTj2DIo/F8ouL6E2huH9oZdT/PuQhosnkhx7DfYlIgs5UROjHNspq7ZA6zyh8xOQcC5 5B/5Wpk4oUgXnmQSQInxhxDvHgf5jYM04qsCAwQ415yQUYvlAr/uqzKYq/lu+yJC0oaF e2vg== X-Gm-Message-State: AOAM530kPlxW6ab4thF2lU+AFoDY6+mukwmfjor9hd9t9lskPD/9Pa0Q 7OPiW+UNhgTdC22XKejb4aFOaJeTXWA= X-Google-Smtp-Source: ABdhPJx3coxDHRXP6FpIkfEjjnON+sn5A3dCnYayZKVrKloiy4Ll0vIxwkhy86FizVTkUb1PGNlsRQ== X-Received: by 2002:a05:600c:3641:: with SMTP id y1mr10733425wmq.43.1631536935804; Mon, 13 Sep 2021 05:42:15 -0700 (PDT) Received: from oberon.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id u13sm7244290wrt.41.2021.09.13.05.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Sep 2021 05:42:15 -0700 (PDT) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH 12/20] trace-cmd library: Extend the input handler with trace data decompression context Date: Mon, 13 Sep 2021 15:41:55 +0300 Message-Id: <20210913124203.3677760-13-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210913124203.3677760-1-tz.stoyanov@gmail.com> References: <20210913124203.3677760-1-tz.stoyanov@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org The CPU trace data is compressed in chunks, as chunk's size is multiple trace pages. The input handler is extended with the necessary structures, to control the data decompression. There are two approaches for data decompression, both are supported and can be used in different use cases: - in-memory decompression, page by page. - using a temporary file Signed-off-by: Tzvetomir Stoyanov (VMware) --- lib/trace-cmd/trace-input.c | 59 ++++++++++++++++++++++++++++++------- 1 file changed, 48 insertions(+), 11 deletions(-) diff --git a/lib/trace-cmd/trace-input.c b/lib/trace-cmd/trace-input.c index 19a86269..ce74252e 100644 --- a/lib/trace-cmd/trace-input.c +++ b/lib/trace-cmd/trace-input.c @@ -54,6 +54,24 @@ struct page { #endif }; +struct zchunk_cache { + struct list_head list; + struct tracecmd_compress_chunk *chunk; + void *map; + int ref; +}; + +struct cpu_zdata { + /* uncompressed cpu data */ + int fd; + char file[26]; /* strlen(COMPR_TEMP_FILE) */ + unsigned int count; + unsigned int last_chunk; + struct list_head cache; + struct tracecmd_compress_chunk *chunks; +}; + +#define COMPR_TEMP_FILE "/tmp/trace_cpu_dataXXXXXX" struct cpu_data { /* the first two never change */ unsigned long long file_offset; @@ -72,6 +90,7 @@ struct cpu_data { int page_cnt; int cpu; int pipe_fd; + struct cpu_zdata compress; }; struct cpu_file_data { @@ -150,6 +169,8 @@ struct tracecmd_input { bool use_trace_clock; bool read_page; bool use_pipe; + bool read_zpage; /* uncompress pages in memory, do not use tmp files */ + bool cpu_compressed; int file_version; struct cpu_data *cpu_data; long long ts_offset; @@ -3300,6 +3321,7 @@ static int init_cpu_data(struct tracecmd_input *handle) endian = KBUFFER_ENDIAN_LITTLE; for (cpu = 0; cpu < handle->cpus; cpu++) { + handle->cpu_data[cpu].compress.fd = -1; handle->cpu_data[cpu].kbuf = kbuffer_alloc(long_size, endian); if (!handle->cpu_data[cpu].kbuf) goto out_free; @@ -4041,6 +4063,7 @@ static inline void free_buffer(struct input_buffer_instance *buf) */ void tracecmd_close(struct tracecmd_input *handle) { + struct zchunk_cache *cache; struct file_section *del_sec; int cpu; int i; @@ -4060,17 +4083,31 @@ void tracecmd_close(struct tracecmd_input *handle) /* The tracecmd_peek_data may have cached a record */ free_next(handle, cpu); free_page(handle, cpu); - if (handle->cpu_data && handle->cpu_data[cpu].kbuf) { - kbuffer_free(handle->cpu_data[cpu].kbuf); - if (handle->cpu_data[cpu].page_map) - free_page_map(handle->cpu_data[cpu].page_map); - - if (handle->cpu_data[cpu].page_cnt) - tracecmd_warning("%d pages still allocated on cpu %d%s", - handle->cpu_data[cpu].page_cnt, cpu, - show_records(handle->cpu_data[cpu].pages, - handle->cpu_data[cpu].nr_pages)); - free(handle->cpu_data[cpu].pages); + if (handle->cpu_data) { + if (handle->cpu_data[cpu].kbuf) { + kbuffer_free(handle->cpu_data[cpu].kbuf); + if (handle->cpu_data[cpu].page_map) + free_page_map(handle->cpu_data[cpu].page_map); + + if (handle->cpu_data[cpu].page_cnt) + tracecmd_warning("%d pages still allocated on cpu %d%s", + handle->cpu_data[cpu].page_cnt, cpu, + show_records(handle->cpu_data[cpu].pages, + handle->cpu_data[cpu].nr_pages)); + free(handle->cpu_data[cpu].pages); + } + if (handle->cpu_data[cpu].compress.fd >= 0) { + close(handle->cpu_data[cpu].compress.fd); + unlink(handle->cpu_data[cpu].compress.file); + } + while (!list_empty(&handle->cpu_data[cpu].compress.cache)) { + cache = container_of(handle->cpu_data[cpu].compress.cache.next, + struct zchunk_cache, list); + list_del(&cache->list); + free(cache->map); + free(cache); + } + free(handle->cpu_data[cpu].compress.chunks); } }