From patchwork Thu Nov 11 15:11:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Tzvetomir Stoyanov (VMware)" X-Patchwork-Id: 12615081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EF74C433F5 for ; Thu, 11 Nov 2021 15:11:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F281D61252 for ; Thu, 11 Nov 2021 15:11:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233966AbhKKPOq (ORCPT ); Thu, 11 Nov 2021 10:14:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233715AbhKKPOq (ORCPT ); Thu, 11 Nov 2021 10:14:46 -0500 Received: from mail-ed1-x52f.google.com (mail-ed1-x52f.google.com [IPv6:2a00:1450:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE38DC061766 for ; Thu, 11 Nov 2021 07:11:56 -0800 (PST) Received: by mail-ed1-x52f.google.com with SMTP id z21so25412063edb.5 for ; Thu, 11 Nov 2021 07:11:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=yMQOLtfOz5VN82+bRaY8epeT4xaUHOYs9zrre+U3dMQ=; b=aPg3t1PIziViouPpM7mOEiBfANjje6TDZcaV9Q+7tCIzGqAvnYiuaRcYT/+mIihy6c uOK1HAIgYuJzGI/S+XK81xzA1Wn4q0t0GB3SnfOh3iO2CIXZsbnX7ZOSxqY9Dfiedj2i IFlXalxN/aTzt0cUkWR7euLIpkftntARpb2COhhMQggGSHKUpnIx2U04bOOLj3/EbFyR 5rHxdrUPCRdX4WmeBt6PCXHMBQeOw92IDvqjZnhJNc48OTGyB7BJL8VRoAC65UYqiZQg PI3vSp95kNqvXap1zxDdC6zK+4M3J6QfUTKHgP+hUyTPYF72fIPphJJRihDdETNMzZf0 QM0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=yMQOLtfOz5VN82+bRaY8epeT4xaUHOYs9zrre+U3dMQ=; b=LwOPWR0QTUFxiYkeWL+ghF65pn6ENX8xEC/QCS/r0X5ttnHzIT64FGzUvbPsRhT08h vASpAHcvYvfBZpp4ZYw1/msLoCT8zgJugQJoFeBOOR/4Se0wMNfY0JQXwykKpgz4tpgN meO4tyg2Ia8xP6D353oglVvrc9ZngTnsRjL72iK4lU/UKtzgr7BAbe/RhxPYPeYQpzIi aaqRSBbuTKOoE0dVbFxEaaG0eMYjL0nsM7hVyB3EtH/lpGZ7msr1k9+/C5WFn4mqTorK ucX2m5GzAHLQtm3iZObCZ2aFnr1gVFskJRV568brAFGixZUX8TPNrQ8wQk1F2DC6xJwC scZQ== X-Gm-Message-State: AOAM532czjTufoBWM8RrFmb3IB2BAGsmNVkxltYWWH8a5t0M7DOC3bjM QaKDsTYKiuWLDkdglc3Bc1vLhBpGTQbJlQ== X-Google-Smtp-Source: ABdhPJz+qS+ISUzIa8EreZzDT4ExzcnfLSMH/BW0JeT73OPk4gvhKpbZW68Mj2fUAcq9qfsqk8DPVg== X-Received: by 2002:aa7:c44b:: with SMTP id n11mr10588577edr.238.1636643515352; Thu, 11 Nov 2021 07:11:55 -0800 (PST) Received: from oberon.zico.biz.zico.biz ([83.222.187.186]) by smtp.gmail.com with ESMTPSA id qf9sm1479509ejc.18.2021.11.11.07.11.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Nov 2021 07:11:54 -0800 (PST) From: "Tzvetomir Stoyanov (VMware)" To: rostedt@goodmis.org Cc: linux-trace-devel@vger.kernel.org Subject: [PATCH v4 00/20] Trace file version 7 - compression Date: Thu, 11 Nov 2021 17:11:33 +0200 Message-Id: <20211111151153.86855-1-tz.stoyanov@gmail.com> X-Mailer: git-send-email 2.33.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Introduced compression of the metadata and trace data in trace files version 7. The compression is optional and disabled by default. A new parameter of "trace-cmd record" is introduced, which can be used to configure the trace file compression: --compression < none / any / name of the desired algorithm > This patch-set depends on: "trace-cmd fixes and clean-ups" patch set. "Refactor APIs for creating output handler" patch set. "Refactor the logic for writing trace data in the file" patch set. "Trace file version 7 - sections" patch set. v4 changes: - Rebased on top of the latest master. v3 changes: - Rebased on top of the latest master. v2 changes: - fixed issues of split and convert commands with some corner cases Tzvetomir Stoyanov (VMware) (20): trace-cmd library: Add support for compression algorithms trace-cmd library: Internal helpers for compressing data trace-cmd library: Internal helpers for uncompressing data trace-cmd library: Inherit compression algorithm from input file trace-cmd library: New API to configure compression on an output handler trace-cmd library: Write compression header in the trace file trace-cmd library: Compress part of the trace file trace-cmd library: Add local helper function for data compression trace-cmd library: Compress the trace data trace-cmd library: Decompress the options section, if it is compressed trace-cmd library: Read compression header trace-cmd library: Extend the input handler with trace data decompression context trace-cmd library: Initialize CPU data decompression logic trace-cmd library: Add logic for in-memory decompression trace-cmd library: Read compressed latency data trace-cmd library: Decompress file sections on reading trace-cmd library: Add zlib compression algorithm trace-cmd list: Show supported compression algorithms trace-cmd record: Add compression to the trace context trace-cmd report: Add new parameter for trace file compression Documentation/trace-cmd/trace-cmd-list.1.txt | 3 + Makefile | 7 + lib/trace-cmd/Makefile | 8 + .../include/private/trace-cmd-private.h | 46 +- lib/trace-cmd/include/trace-cmd-local.h | 17 + lib/trace-cmd/trace-compress-zlib.c | 109 +++ lib/trace-cmd/trace-compress.c | 910 ++++++++++++++++++ lib/trace-cmd/trace-input.c | 469 ++++++++- lib/trace-cmd/trace-output.c | 316 +++++- lib/trace-cmd/trace-util.c | 10 + tracecmd/Makefile | 4 + tracecmd/trace-list.c | 26 + tracecmd/trace-record.c | 34 +- tracecmd/trace-usage.c | 6 + 14 files changed, 1892 insertions(+), 73 deletions(-) create mode 100644 lib/trace-cmd/trace-compress-zlib.c create mode 100644 lib/trace-cmd/trace-compress.c