From patchwork Mon Feb 4 06:58:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10794991 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A96A159A for ; Mon, 4 Feb 2019 06:59:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47C2E2B03E for ; Mon, 4 Feb 2019 06:59:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A8132B042; Mon, 4 Feb 2019 06:59:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A40AD2B03E for ; Mon, 4 Feb 2019 06:58:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725902AbfBDG67 (ORCPT ); Mon, 4 Feb 2019 01:58:59 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:54223 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG67 (ORCPT ); Mon, 4 Feb 2019 01:58:59 -0500 Received: by mail-wm1-f66.google.com with SMTP id d15so11932966wmb.3 for ; Sun, 03 Feb 2019 22:58:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ajXfUh/mnU+wtU0V6S7mrCQeLCbVutVc/GYt5XQFIvY=; b=oy23V0Qt4KBcAlNmg6nrCYOZL8jvX0bY5hISkXGWjbEy3QMRDU3fOl7EhYefQQ039R or/FVOamoltsB4b7U85c0iIyLjuEyPwr55eZzVbtODyWoTmxXSPq4lezYsj7qT3v1M9O BmofzzZWi5RXJX8E/DX6js1me6skMFIiwJfQ3vuJwmlbP1MiPW6A8ib9wh6zjcdyaCwG Mb/DKk+hbuZqKewz8f/RAG4UXFKWkF2qDVU/yQTgLweIeWqrwfzEnP6XLDWCRpoH3vg1 E5AN5gB1lHKuDhaaFT/qZ+i//7ZqGVJ8T4UZ5dozoHBtaUplls2qQo3dr1tuQnbxVopW wMaQ== X-Gm-Message-State: AHQUAuatHC74qDplw51v3oKRKjf4g+fYGZfQpn4KAZLSHCk8EiFU+0YZ P667EHvumcYEXKcRAERFobDYJ7Wo2g== X-Google-Smtp-Source: AHgI3IYPOm8vecZl19zjLGH8IHngO82TzpJ9bAmTw5AjYquc34tqbg1Lj9gq6RzAlH+Hz5sXYZeSmw== X-Received: by 2002:a1c:a70a:: with SMTP id q10mr11138259wme.58.1549263537134; Sun, 03 Feb 2019 22:58:57 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.58.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:58:56 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 01/11] trace-cmd: Detect if vsockets are available Date: Mon, 4 Feb 2019 08:58:38 +0200 Message-Id: <20190204065848.8248-2-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Steven Rostedt (VMware)" Detect and define VSOCK if vsockets are available on the system. This macro is used to disable VM remote tracing features on older kernels. Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Slavomir Kaslev --- Makefile | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Makefile b/Makefile index f0f7e92..ce16b83 100644 --- a/Makefile +++ b/Makefile @@ -204,6 +204,13 @@ CFLAGS ?= -g -Wall CPPFLAGS ?= LDFLAGS ?= +VSOCK_DEFINED := $(shell if (echo "\#include " | $(CC) -E - >/dev/null 2>&1) ; then echo 1; else echo 0 ; fi) + +export VSOCK_DEFINED +ifeq ($(VSOCK_DEFINED), 1) +CFLAGS += -DVSOCK +endif + export CFLAGS export INCLUDES From patchwork Mon Feb 4 06:58:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10794993 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AACC06C2 for ; Mon, 4 Feb 2019 06:59:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 981892B03E for ; Mon, 4 Feb 2019 06:59:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C48E2B042; Mon, 4 Feb 2019 06:59:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1F2AD2B03E for ; Mon, 4 Feb 2019 06:59:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726006AbfBDG7B (ORCPT ); Mon, 4 Feb 2019 01:59:01 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:42458 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG7A (ORCPT ); Mon, 4 Feb 2019 01:59:00 -0500 Received: by mail-wr1-f68.google.com with SMTP id q18so13121828wrx.9 for ; Sun, 03 Feb 2019 22:58:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=S8/m/OZ2EmADcsb4CCYO4FGTMLTjcD4Du9Mh697xVdY=; b=EiQjHWDER6i2WSXsxwQprcxvr78daAp0trk6Adw4ymhJjiObhE+1rFvbvjda21Ojzq OnG2Ud4Z23fkKcywGGygpNGuvgVpy5tvl/Hrc7G/qCrUw5LvxgrIwt6SbnUvLX9tJ87V HdFOvccD5JjX1+RIv7IsZ1uw9b9mwikHvV9viaxRsQaEKzRhDTZFvcSrZfFn7l3KENa3 Re9k7+dQ8Uv5gVL3N07C/xYdAOKBDJq3F/+GfTv3/YCwQVAjnOr2mjOuBAqwaB7K42uH V/I7rENi2FgHmWCT1E4GTHeWX3a8D8MxuMvYPmA764ulcZurTBIheqYDaLYeSQDQKdU7 cxvA== X-Gm-Message-State: AHQUAuZtSRkdrKR1J/CU6kowDcKdtq0GABqWlb42qFapAsxOYJ80x1aD EwwJR4kK8LoW59+AtLau19v2ylW3bw== X-Google-Smtp-Source: AHgI3IZlloizZ+5wB0GBKgW4ryRj70bCYF0T6RXpRsBC1ZrvPChLh0lXHQyED7IZkPmvZTWWn0ZZ0A== X-Received: by 2002:a5d:5285:: with SMTP id c5mr5756354wrv.167.1549263538693; Sun, 03 Feb 2019 22:58:58 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.58.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:58:57 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 02/11] trace-cmd: Add tracecmd_create_recorder_virt function Date: Mon, 4 Feb 2019 08:58:39 +0200 Message-Id: <20190204065848.8248-3-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add tracecmd_create_recorder_virt function that will be used for tracing VM guests. Signed-off-by: Slavomir Kaslev --- include/trace-cmd/trace-cmd.h | 1 + lib/trace-cmd/trace-recorder.c | 53 ++++++++++++++++++++++++---------- 2 files changed, 39 insertions(+), 15 deletions(-) diff --git a/include/trace-cmd/trace-cmd.h b/include/trace-cmd/trace-cmd.h index 86b9b53..8b43462 100644 --- a/include/trace-cmd/trace-cmd.h +++ b/include/trace-cmd/trace-cmd.h @@ -277,6 +277,7 @@ enum { void tracecmd_free_recorder(struct tracecmd_recorder *recorder); struct tracecmd_recorder *tracecmd_create_recorder(const char *file, int cpu, unsigned flags); struct tracecmd_recorder *tracecmd_create_recorder_fd(int fd, int cpu, unsigned flags); +struct tracecmd_recorder *tracecmd_create_recorder_virt(const char *file, int cpu, unsigned flags, int trace_fd); struct tracecmd_recorder *tracecmd_create_recorder_maxkb(const char *file, int cpu, unsigned flags, int maxkb); struct tracecmd_recorder *tracecmd_create_buffer_recorder_fd(int fd, int cpu, unsigned flags, const char *buffer); struct tracecmd_recorder *tracecmd_create_buffer_recorder(const char *file, int cpu, unsigned flags, const char *buffer); diff --git a/lib/trace-cmd/trace-recorder.c b/lib/trace-cmd/trace-recorder.c index 5331925..83a12f4 100644 --- a/lib/trace-cmd/trace-recorder.c +++ b/lib/trace-cmd/trace-recorder.c @@ -148,16 +148,22 @@ tracecmd_create_buffer_recorder_fd2(int fd, int fd2, int cpu, unsigned flags, recorder->fd1 = fd; recorder->fd2 = fd2; - if (flags & TRACECMD_RECORD_SNAPSHOT) - ret = asprintf(&path, "%s/per_cpu/cpu%d/snapshot_raw", buffer, cpu); - else - ret = asprintf(&path, "%s/per_cpu/cpu%d/trace_pipe_raw", buffer, cpu); - if (ret < 0) - goto out_free; + if (buffer) { + if (flags & TRACECMD_RECORD_SNAPSHOT) + ret = asprintf(&path, "%s/per_cpu/cpu%d/snapshot_raw", + buffer, cpu); + else + ret = asprintf(&path, "%s/per_cpu/cpu%d/trace_pipe_raw", + buffer, cpu); + if (ret < 0) + goto out_free; + + recorder->trace_fd = open(path, O_RDONLY); + free(path); - recorder->trace_fd = open(path, O_RDONLY); - if (recorder->trace_fd < 0) - goto out_free; + if (recorder->trace_fd < 0) + goto out_free; + } if ((recorder->flags & TRACECMD_RECORD_NOSPLICE) == 0) { ret = pipe(recorder->brass); @@ -177,13 +183,9 @@ tracecmd_create_buffer_recorder_fd2(int fd, int fd2, int cpu, unsigned flags, recorder->pipe_size = pipe_size; } - free(path); - return recorder; out_free: - free(path); - tracecmd_free_recorder(recorder); return NULL; } @@ -194,8 +196,9 @@ tracecmd_create_buffer_recorder_fd(int fd, int cpu, unsigned flags, const char * return tracecmd_create_buffer_recorder_fd2(fd, -1, cpu, flags, buffer, 0); } -struct tracecmd_recorder * -tracecmd_create_buffer_recorder(const char *file, int cpu, unsigned flags, const char *buffer) +static struct tracecmd_recorder * +__tracecmd_create_buffer_recorder(const char *file, int cpu, unsigned flags, + const char *buffer) { struct tracecmd_recorder *recorder; int fd; @@ -258,6 +261,26 @@ tracecmd_create_buffer_recorder_maxkb(const char *file, int cpu, unsigned flags, goto out; } +struct tracecmd_recorder * +tracecmd_create_buffer_recorder(const char *file, int cpu, unsigned flags, + const char *buffer) +{ + return __tracecmd_create_buffer_recorder(file, cpu, flags, buffer); +} + +struct tracecmd_recorder * +tracecmd_create_recorder_virt(const char *file, int cpu, unsigned flags, + int trace_fd) +{ + struct tracecmd_recorder *recorder; + + recorder = __tracecmd_create_buffer_recorder(file, cpu, flags, NULL); + if (recorder) + recorder->trace_fd = trace_fd; + + return recorder; +} + struct tracecmd_recorder *tracecmd_create_recorder_fd(int fd, int cpu, unsigned flags) { const char *tracing; From patchwork Mon Feb 4 06:58:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10794995 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60A53159A for ; Mon, 4 Feb 2019 06:59:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E5A82B03C for ; Mon, 4 Feb 2019 06:59:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3EA9E2B03F; Mon, 4 Feb 2019 06:59:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A3612B03C for ; Mon, 4 Feb 2019 06:59:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726159AbfBDG7D (ORCPT ); Mon, 4 Feb 2019 01:59:03 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:38906 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG7D (ORCPT ); Mon, 4 Feb 2019 01:59:03 -0500 Received: by mail-wr1-f67.google.com with SMTP id v13so13139515wrw.5 for ; Sun, 03 Feb 2019 22:59:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nchF6aP4wBEXwHmlyn2j5ppBbARyelas+/PlNoYY8xo=; b=NQ8wAsy2OZTawj/kGKZfmSkvfb59SY0IqublCRC15COmu3I3iSmWloyEwVaPNeisz2 rCT3IJSeenJ/wFzy0Z1kJ2lPpyFBFhOHrLxO+RKaPoae8qLGhnGrC+7DQ6+lWfjdtHf3 TdpxZLs8J/hyAVbE6cFGwFfZMriaJ2aAutsgiPt9aVSYcWNHoZ1eRFNKu0MBt+W+ZD6v qWo1QeUTYns0XHATUxyVQ7vbcpZI70J/6zIMeNE0xJ4kO8lQTC/j9rocw5t86ZtgwV6D av2k7upino+SGMWd+u5/qsZo3uqUcLuTRNFHs6fT4MEl2bM7Z73uMmp2HWHnRpYGLMlH eHDQ== X-Gm-Message-State: AJcUukfkdOHHS58aO2fbpG0sENO058OHh7fuy2ZVvdWir+0JE+JAvPG/ 2vvvW53NRr84Xwpr+IcuHO4Hhqk7vw== X-Google-Smtp-Source: ALg8bN4noUGvRpyd2H3HWY7fCSoq6KqNpG5kcC8ycgBhl3CKdqHXXFzkIouJRgFY/PhivawiSzmY8w== X-Received: by 2002:adf:8b4d:: with SMTP id v13mr46083721wra.282.1549263540173; Sun, 03 Feb 2019 22:59:00 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.58.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:58:59 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 03/11] trace-cmd: Add TRACE_REQ and TRACE_RESP messages Date: Mon, 4 Feb 2019 08:58:40 +0200 Message-Id: <20190204065848.8248-4-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add TRACE_REQ and TRACE_RESP messages which are used for initiating guest VM tracing. Signed-off-by: Slavomir Kaslev --- include/trace-cmd/trace-cmd.h | 16 ++- tracecmd/trace-msg.c | 213 +++++++++++++++++++++++++++++++++- 2 files changed, 224 insertions(+), 5 deletions(-) diff --git a/include/trace-cmd/trace-cmd.h b/include/trace-cmd/trace-cmd.h index 8b43462..7664074 100644 --- a/include/trace-cmd/trace-cmd.h +++ b/include/trace-cmd/trace-cmd.h @@ -326,7 +326,7 @@ int tracecmd_msg_send_init_data(struct tracecmd_msg_handle *msg_handle, int tracecmd_msg_data_send(struct tracecmd_msg_handle *msg_handle, const char *buf, int size); int tracecmd_msg_finish_sending_data(struct tracecmd_msg_handle *msg_handle); -void tracecmd_msg_send_close_msg(struct tracecmd_msg_handle *msg_handle); +int tracecmd_msg_send_close_msg(struct tracecmd_msg_handle *msg_handle); /* for server */ int tracecmd_msg_initial_setting(struct tracecmd_msg_handle *msg_handle); @@ -337,6 +337,20 @@ int tracecmd_msg_collect_data(struct tracecmd_msg_handle *msg_handle, int ofd); bool tracecmd_msg_done(struct tracecmd_msg_handle *msg_handle); void tracecmd_msg_set_done(struct tracecmd_msg_handle *msg_handle); +int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, + int argc, char **argv); +int tracecmd_msg_recv_trace_req(struct tracecmd_msg_handle *msg_handle, + int *argc, char ***argv); + +int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, + int nr_cpus, int page_size, + unsigned int *ports); +int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, + int *nr_cpus, int *page_size, + unsigned int **ports); + +int tracecmd_msg_wait_close(struct tracecmd_msg_handle *msg_handle); + /* --- Plugin handling --- */ extern struct tep_plugin_option trace_ftrace_options[]; diff --git a/tracecmd/trace-msg.c b/tracecmd/trace-msg.c index 529ae2a..8b6fb03 100644 --- a/tracecmd/trace-msg.c +++ b/tracecmd/trace-msg.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -79,6 +80,16 @@ struct tracecmd_msg_rinit { be32 cpus; } __attribute__((packed)); +struct tracecmd_msg_trace_req { + be32 flags; + be32 argc; +} __attribute__((packed)); + +struct tracecmd_msg_trace_resp { + be32 cpus; + be32 page_size; +} __attribute__((packed)); + struct tracecmd_msg_header { be32 size; be32 cmd; @@ -90,7 +101,9 @@ struct tracecmd_msg_header { C(TINIT, 1, sizeof(struct tracecmd_msg_tinit)), \ C(RINIT, 2, sizeof(struct tracecmd_msg_rinit)), \ C(SEND_DATA, 3, 0), \ - C(FIN_DATA, 4, 0), + C(FIN_DATA, 4, 0), \ + C(TRACE_REQ, 5, sizeof(struct tracecmd_msg_trace_req)), \ + C(TRACE_RESP, 6, sizeof(struct tracecmd_msg_trace_resp)), #undef C #define C(a,b,c) MSG_##a = b @@ -122,6 +135,8 @@ struct tracecmd_msg { union { struct tracecmd_msg_tinit tinit; struct tracecmd_msg_rinit rinit; + struct tracecmd_msg_trace_req trace_req; + struct tracecmd_msg_trace_resp trace_resp; }; union { struct tracecmd_msg_opt *opt; @@ -569,12 +584,12 @@ int tracecmd_msg_send_port_array(struct tracecmd_msg_handle *msg_handle, return 0; } -void tracecmd_msg_send_close_msg(struct tracecmd_msg_handle *msg_handle) +int tracecmd_msg_send_close_msg(struct tracecmd_msg_handle *msg_handle) { struct tracecmd_msg msg; tracecmd_msg_init(MSG_CLOSE, &msg); - tracecmd_msg_send(msg_handle->fd, &msg); + return tracecmd_msg_send(msg_handle->fd, &msg); } int tracecmd_msg_data_send(struct tracecmd_msg_handle *msg_handle, @@ -684,7 +699,7 @@ int tracecmd_msg_collect_data(struct tracecmd_msg_handle *msg_handle, int ofd) int ret; ret = tracecmd_msg_read_data(msg_handle, ofd); - if (ret) + if (ret < 0) goto error; /* check the finish message of the client */ @@ -715,3 +730,193 @@ error: msg_free(&msg); return ret; } + +static int make_trace_req(struct tracecmd_msg *msg, int argc, char **argv) +{ + size_t args_size = 0; + char *p; + int i; + + for (i = 0; i < argc; i++) + args_size += strlen(argv[i]) + 1; + + msg->hdr.size = htonl(ntohl(msg->hdr.size) + args_size); + msg->trace_req.argc = htonl(argc); + msg->buf = calloc(args_size, 1); + if (!msg->buf) + return -ENOMEM; + + p = msg->buf; + for (i = 0; i < argc; i++) + p = stpcpy(p, argv[i]) + 1; + + return 0; +} + +int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, + int argc, char **argv) +{ + struct tracecmd_msg msg; + int ret; + + tracecmd_msg_init(MSG_TRACE_REQ, &msg); + ret = make_trace_req(&msg, argc, argv); + if (ret < 0) + return ret; + + return tracecmd_msg_send(msg_handle->fd, &msg); +} + + /* + * NOTE: On success, the returned `argv` should be freed with: + * free(argv[0]); + * free(argv); + */ +int tracecmd_msg_recv_trace_req(struct tracecmd_msg_handle *msg_handle, + int *argc, char ***argv) +{ + struct tracecmd_msg msg; + char *p, *buf_end, **args; + int i, ret, nr_args; + size_t buf_len; + + ret = tracecmd_msg_recv(msg_handle->fd, &msg); + if (ret < 0) + return ret; + + if (ntohl(msg.hdr.cmd) != MSG_TRACE_REQ) + goto out; + + if (ntohl(msg.trace_req.argc) < 0) + goto out; + + buf_len = ntohl(msg.hdr.size) - MSG_HDR_LEN - ntohl(msg.hdr.cmd_size); + buf_end = (char *)msg.buf + buf_len; + p = msg.buf; + nr_args = ntohl(msg.trace_req.argc); + args = calloc(nr_args, sizeof(*args)); + if (!args) { + ret = -ENOMEM; + goto out; + } + for (i = 0; i < nr_args; i++) { + if (p >= buf_end) { + ret = -1; + free(args); + goto out; + } + + args[i] = p; + p = strchr(p, '\0'); + p++; + } + + /* + * On success we're passing msg.buf to the caller through argv[0] so we + * reset it here to avoid getting it freed below. + */ + msg.buf = NULL; + *argc = nr_args; + *argv = args; + ret = 0; + +out: + msg_free(&msg); + return ret; +} + +static int make_trace_resp(struct tracecmd_msg *msg, + int page_size, int nr_cpus, unsigned int *ports) +{ + int ports_size = nr_cpus * sizeof(*msg->port_array); + int i; + + msg->hdr.size = htonl(ntohl(msg->hdr.size) + ports_size); + msg->trace_resp.cpus = htonl(nr_cpus); + msg->trace_resp.page_size = htonl(page_size); + + msg->port_array = malloc(ports_size); + if (!msg->port_array) + return -ENOMEM; + + for (i = 0; i < nr_cpus; i++) + msg->port_array[i] = htonl(ports[i]); + + return 0; +} + +int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, + int nr_cpus, int page_size, + unsigned int *ports) +{ + struct tracecmd_msg msg; + int ret; + + tracecmd_msg_init(MSG_TRACE_RESP, &msg); + ret = make_trace_resp(&msg, page_size, nr_cpus, ports); + if (ret < 0) + return ret; + + return tracecmd_msg_send(msg_handle->fd, &msg); +} + +int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, + int *nr_cpus, int *page_size, + unsigned int **ports) +{ + struct tracecmd_msg msg; + size_t buf_len; + int i, ret; + + ret = tracecmd_msg_recv(msg_handle->fd, &msg); + if (ret < 0) + return ret; + + if (ntohl(msg.hdr.cmd) != MSG_TRACE_RESP) { + ret = -1; + goto out; + } + + buf_len = ntohl(msg.hdr.size) - MSG_HDR_LEN - ntohl(msg.hdr.cmd_size); + if (buf_len <= 0 || + buf_len != sizeof(*msg.port_array) * ntohl(msg.trace_resp.cpus)) { + ret = -1; + goto out; + } + + *nr_cpus = ntohl(msg.trace_resp.cpus); + *page_size = ntohl(msg.trace_resp.page_size); + *ports = calloc(*nr_cpus, sizeof(**ports)); + if (!*ports) { + ret = -ENOMEM; + goto out; + } + for (i = 0; i < *nr_cpus; i++) + (*ports)[i] = ntohl(msg.port_array[i]); + + ret = 0; + +out: + msg_free(&msg); + return ret; +} + +int tracecmd_msg_wait_close(struct tracecmd_msg_handle *msg_handle) +{ + struct tracecmd_msg msg; + int ret; + + memset(&msg, 0, sizeof(msg)); + for (;;) { + ret = tracecmd_msg_recv(msg_handle->fd, &msg); + if (ret < 0) + return ret; + + if (ntohl(msg.hdr.cmd) == MSG_CLOSE) + return 0; + + msg_free(&msg); + } + + return -1; +} From patchwork Mon Feb 4 06:58:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10794997 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EA241805 for ; Mon, 4 Feb 2019 06:59:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFFD22B03C for ; Mon, 4 Feb 2019 06:59:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E25692B03F; Mon, 4 Feb 2019 06:59:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 600842B03E for ; Mon, 4 Feb 2019 06:59:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726185AbfBDG7E (ORCPT ); Mon, 4 Feb 2019 01:59:04 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:34616 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726066AbfBDG7E (ORCPT ); Mon, 4 Feb 2019 01:59:04 -0500 Received: by mail-wm1-f68.google.com with SMTP id y185so9296551wmd.1 for ; Sun, 03 Feb 2019 22:59:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cILFA6YqC26BwdhG6xlYaNZ/d3tVOXiWsCpNOhOBXGY=; b=QZFXML4oD0V0BxoiE6YR1x4fCohpwOyTVdqCdQwnyihGDPAqx/1/0DGhcH7oOWmQZc fAdxY+HFhj1VoFU88B8qDima1+9613oK9phYXTyaOWPR3Vp/HFapouRe48TWnLxzFpeM OiCZY+0r2PR6/RQewkAUo+P4rCImiWKuU/CW2dClvceLJn3nnFDCIilZGGOYH6rzLdu5 oUJdRUUbABTDfjhCUDzj+kDTsHoNJvAr7X42Yiqtq5vitFubSB+N9pxlOMiecddPz4Nw d+/+zU8s58sHCfA34xaNH9ZNt2yULy9IZDQnpkObTnxJdyC3GTR8nSQvbbbCdUZvzKug p83g== X-Gm-Message-State: AHQUAubu4lXWc7zr1cClgfpBV3+hgGhV22tTDvmwc/D1tX0XkrAaT1LY +KbbJts8wJowzk3CfY2X6fj1I3kCrw== X-Google-Smtp-Source: AHgI3Ibu+aYbEqIyKDxQ3+wAULx2ojvq8+6fuLeTETePVQ9c3slx/3MZgtcDoGhmj52kDBEEBQfhnw== X-Received: by 2002:a7b:c76b:: with SMTP id x11mr11602118wmk.37.1549263541352; Sun, 03 Feb 2019 22:59:01 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:00 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 04/11] trace-cmd: Add buffer instance flags for tracing in guest and agent context Date: Mon, 4 Feb 2019 08:58:41 +0200 Message-Id: <20190204065848.8248-5-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add BUFFER_FL_GUEST and BUFFER_FL_AGENT flags to differentiate when trace-record.c is being called to trace guest or the VM tracing agent. Also disable functions talking to the local tracefs when called in recording guest instances context. Signed-off-by: Slavomir Kaslev --- tracecmd/include/trace-local.h | 2 ++ tracecmd/trace-record.c | 55 ++++++++++++++++++++++++++++++++-- 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index a1a06e9..f19c8bb 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -149,6 +149,8 @@ char *strstrip(char *str); enum buffer_instance_flags { BUFFER_FL_KEEP = 1 << 0, BUFFER_FL_PROFILE = 1 << 1, + BUFFER_FL_GUEST = 1 << 2, + BUFFER_FL_AGENT = 1 << 3, }; struct func_list { diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 3034a4b..b911c34 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -781,6 +781,9 @@ static void __clear_trace(struct buffer_instance *instance) FILE *fp; char *path; + if (instance->flags & BUFFER_FL_GUEST) + return; + /* reset the trace */ path = get_instance_file(instance, "trace"); fp = fopen(path, "w"); @@ -1264,6 +1267,9 @@ set_plugin_instance(struct buffer_instance *instance, const char *name) char *path; char zero = '0'; + if (instance->flags & BUFFER_FL_GUEST) + return; + path = get_instance_file(instance, "current_tracer"); fp = fopen(path, "w"); if (!fp) { @@ -1360,6 +1366,9 @@ static void disable_func_stack_trace_instance(struct buffer_instance *instance) int size; int ret; + if (instance->flags & BUFFER_FL_GUEST) + return; + path = get_instance_file(instance, "current_tracer"); ret = stat(path, &st); tracecmd_put_tracing_file(path); @@ -1553,6 +1562,9 @@ reset_events_instance(struct buffer_instance *instance) int i; int ret; + if (instance->flags & BUFFER_FL_GUEST) + return; + if (use_old_event_method()) { /* old way only had top instance */ if (!is_top_instance(instance)) @@ -1904,6 +1916,9 @@ static void write_tracing_on(struct buffer_instance *instance, int on) int ret; int fd; + if (instance->flags & BUFFER_FL_GUEST) + return; + fd = open_tracing_on(instance); if (fd < 0) return; @@ -1923,6 +1938,9 @@ static int read_tracing_on(struct buffer_instance *instance) char buf[10]; int ret; + if (instance->flags & BUFFER_FL_GUEST) + return -1; + fd = open_tracing_on(instance); if (fd < 0) return fd; @@ -2156,6 +2174,9 @@ static void set_mask(struct buffer_instance *instance) int fd; int ret; + if (instance->flags & BUFFER_FL_GUEST) + return; + if (!instance->cpumask) return; @@ -2186,6 +2207,9 @@ static void enable_events(struct buffer_instance *instance) { struct event_list *event; + if (instance->flags & BUFFER_FL_GUEST) + return; + for (event = instance->events; event; event = event->next) { if (!event->neg) update_event(event, event->filter, 0, '1'); @@ -2209,6 +2233,9 @@ static void set_clock(struct buffer_instance *instance) char *content; char *str; + if (instance->flags & BUFFER_FL_GUEST) + return; + if (!instance->clock) return; @@ -2238,6 +2265,9 @@ static void set_max_graph_depth(struct buffer_instance *instance, char *max_grap char *path; int ret; + if (instance->flags & BUFFER_FL_GUEST) + return; + path = get_instance_file(instance, "max_graph_depth"); reset_save_file(path, RESET_DEFAULT_PRIO); tracecmd_put_tracing_file(path); @@ -2463,6 +2493,9 @@ static void expand_event_instance(struct buffer_instance *instance) struct event_list *compressed_list = instance->events; struct event_list *event; + if (instance->flags & BUFFER_FL_GUEST) + return; + reset_event_list(instance); while (compressed_list) { @@ -3336,6 +3369,9 @@ static void set_funcs(struct buffer_instance *instance) int set_notrace = 0; int ret; + if (instance->flags & BUFFER_FL_GUEST) + return; + ret = write_func_file(instance, "set_ftrace_filter", &instance->filter_funcs); if (ret < 0) die("set_ftrace_filter does not exist. Can not filter functions"); @@ -3632,6 +3668,9 @@ static void set_buffer_size_instance(struct buffer_instance *instance) int ret; int fd; + if (instance->flags & BUFFER_FL_GUEST) + return; + if (!buffer_size) return; @@ -3829,6 +3868,9 @@ static void make_instances(void) int ret; for_each_instance(instance) { + if (instance->flags & BUFFER_FL_GUEST) + continue; + path = get_instance_dir(instance); ret = stat(path, &st); if (ret < 0) { @@ -3850,7 +3892,7 @@ void tracecmd_remove_instances(void) for_each_instance(instance) { /* Only delete what we created */ - if (instance->flags & BUFFER_FL_KEEP) + if (instance->flags & (BUFFER_FL_KEEP | BUFFER_FL_GUEST)) continue; if (instance->tracing_on_fd > 0) { close(instance->tracing_on_fd); @@ -3932,7 +3974,7 @@ static void check_function_plugin(void) static int __check_doing_something(struct buffer_instance *instance) { - return (instance->flags & BUFFER_FL_PROFILE) || + return (instance->flags & (BUFFER_FL_PROFILE | BUFFER_FL_GUEST)) || instance->plugin || instance->events; } @@ -3954,6 +3996,9 @@ update_plugin_instance(struct buffer_instance *instance, { const char *plugin = instance->plugin; + if (instance->flags & BUFFER_FL_GUEST) + return; + if (!plugin) return; @@ -4053,6 +4098,9 @@ static void record_stats(void) int cpu; for_all_instances(instance) { + if (instance->flags & BUFFER_FL_GUEST) + continue; + s_save = instance->s_save; s_print = instance->s_print; for (cpu = 0; cpu < instance->cpu_count; cpu++) { @@ -4079,6 +4127,9 @@ static void destroy_stats(void) int cpu; for_all_instances(instance) { + if (instance->flags & BUFFER_FL_GUEST) + continue; + for (cpu = 0; cpu < instance->cpu_count; cpu++) { trace_seq_destroy(&instance->s_save[cpu]); trace_seq_destroy(&instance->s_print[cpu]); From patchwork Mon Feb 4 06:58:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10795005 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D84B159A for ; Mon, 4 Feb 2019 06:59:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 474FD2B03C for ; Mon, 4 Feb 2019 06:59:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3B40C2B03F; Mon, 4 Feb 2019 06:59:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DC602B03E for ; Mon, 4 Feb 2019 06:59:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726066AbfBDG7H (ORCPT ); Mon, 4 Feb 2019 01:59:07 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:45803 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG7G (ORCPT ); Mon, 4 Feb 2019 01:59:06 -0500 Received: by mail-wr1-f66.google.com with SMTP id q15so2204599wro.12 for ; Sun, 03 Feb 2019 22:59:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WqOIZneW6KsfZInxi4hVISMi7JB4CsuLKo+xK2MW9xo=; b=oppALmzEZrkBWejR7Ct1pufgw0ZuSug0e0FVZAdr7xP19KguncVj9DuYZFcLtkeI86 sOs8Vfc+yEdTF9wb/CgrsPxaeikYiSPX874cqGXF+RVpGxWxtBYSmEz7qN4/9BbaUrVI lUV3326INNFqV7FN7QmF22NussFfdiFuDG3cwmcOHnhJkkgfb2Itwm94gFHJ0irvsVrB AsBWw/aQyIDQgHpyLObdxM1TXtbkk3QA9YT5XOJHkxPGbbJQWO7VB6oSGn6dyGPw9NoL Mz7VUPcgyy3epHoomkCeuAjcYX+sAFE7VquadeQ2FQHYazwjOzgXEhk3Z1owgEH3jao9 CoOw== X-Gm-Message-State: AJcUukduyzWiY1cd9EqXT3CatGjOCp+q3fKDfaCxZoVfagNqYQM2PwaU PCvXy5HPZmIYVJ3nzMqbCED26yhe/w== X-Google-Smtp-Source: ALg8bN6ASgNE7h5y/EfJVDvK+Op1NvAsfUBrSvfCdtsIHUf9/edYVOxcrShLtBn0PctAkElow+EqzQ== X-Received: by 2002:a5d:4250:: with SMTP id s16mr48239966wrr.253.1549263542560; Sun, 03 Feb 2019 22:59:02 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:01 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 05/11] trace-cmd: Add VM kernel tracing over vsockets transport Date: Mon, 4 Feb 2019 08:58:42 +0200 Message-Id: <20190204065848.8248-6-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds VM tracing over vsockets. The new `trace-cmd agent` command needs run on each guest we want to trace: you@guest2 # trace-cmd agent Then `trace-cmd record` on the host can collect data from both the host and several guests simultaneously: you@host $ trace-cmd record -A guest1 -e irq -e sched \ -A guest2 -p function -e all Signed-off-by: Slavomir Kaslev --- tracecmd/Makefile | 6 +- tracecmd/include/trace-local.h | 16 + tracecmd/trace-agent.c | 225 +++++++++++++ tracecmd/trace-cmd.c | 3 + tracecmd/trace-msg.c | 1 + tracecmd/trace-record.c | 594 ++++++++++++++++++++++++++++++--- tracecmd/trace-usage.c | 13 +- 7 files changed, 811 insertions(+), 47 deletions(-) create mode 100644 tracecmd/trace-agent.c diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 3a11024..865b1c6 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -33,13 +33,17 @@ TRACE_CMD_OBJS += trace-output.o TRACE_CMD_OBJS += trace-usage.o TRACE_CMD_OBJS += trace-msg.o +ifeq ($(VSOCK_DEFINED), 1) +TRACE_CMD_OBJS += trace-agent.o +endif + ALL_OBJS := $(TRACE_CMD_OBJS:%.o=$(bdir)/%.o) all_objs := $(sort $(ALL_OBJS)) all_deps := $(all_objs:$(bdir)/%.o=$(bdir)/.%.d) CONFIG_INCLUDES = -CONFIG_LIBS = +CONFIG_LIBS = -lrt CONFIG_FLAGS = all: $(TARGETS) diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index f19c8bb..823d323 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -12,6 +12,8 @@ #include "trace-cmd.h" #include "event-utils.h" +#define TRACE_AGENT_DEFAULT_PORT 823 + extern int debug; extern int quiet; @@ -64,6 +66,8 @@ void trace_split(int argc, char **argv); void trace_listen(int argc, char **argv); +void trace_agent(int argc, char **argv); + void trace_restore(int argc, char **argv); void trace_clear(int argc, char **argv); @@ -88,6 +92,10 @@ void trace_list(int argc, char **argv); void trace_usage(int argc, char **argv); +int trace_record_agent(struct tracecmd_msg_handle *msg_handle, + int cpus, int *fds, + int argc, char **argv); + struct hook_list; void trace_init_profile(struct tracecmd_input *handle, struct hook_list *hooks, @@ -176,6 +184,7 @@ struct buffer_instance { struct func_list *notrace_funcs; const char *clock; + unsigned int *client_ports; struct trace_seq *s_save; struct trace_seq *s_print; @@ -190,6 +199,13 @@ struct buffer_instance { int tracing_on_fd; int buffer_size; int cpu_count; + + int argc; + char **argv; + + int cid; + int port; + int *fds; }; extern struct buffer_instance top_instance; diff --git a/tracecmd/trace-agent.c b/tracecmd/trace-agent.c new file mode 100644 index 0000000..b4b3bbb --- /dev/null +++ b/tracecmd/trace-agent.c @@ -0,0 +1,225 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2018 VMware Inc, Slavomir Kaslev + * + * based on prior implementation by Yoshihiro Yunomae + * Copyright (C) 2013 Hitachi, Ltd. + * Yoshihiro YUNOMAE + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "trace-local.h" +#include "trace-msg.h" + +static int make_vsock(unsigned int port) +{ + struct sockaddr_vm addr = { + .svm_family = AF_VSOCK, + .svm_cid = VMADDR_CID_ANY, + .svm_port = port, + }; + int sd; + + sd = socket(AF_VSOCK, SOCK_STREAM, 0); + if (sd < 0) + return -1; + + setsockopt(sd, SOL_SOCKET, SO_REUSEADDR, &(int){1}, sizeof(int)); + + if (bind(sd, (struct sockaddr *)&addr, sizeof(addr))) + return -1; + + if (listen(sd, SOMAXCONN)) + return -1; + + return sd; +} + +static int get_vsock_port(int sd, unsigned int *port) +{ + struct sockaddr_vm addr; + socklen_t addr_len = sizeof(addr); + + if (getsockname(sd, (struct sockaddr *)&addr, &addr_len)) + return -1; + + if (addr.svm_family != AF_VSOCK) + return -1; + + if (port) + *port = addr.svm_port; + + return 0; +} + +static void make_vsocks(int nr, int *fds, unsigned int *ports) +{ + unsigned int port; + int i, fd, ret; + + for (i = 0; i < nr; i++) { + fd = make_vsock(VMADDR_PORT_ANY); + if (fd < 0) + die("Failed to open vsocket"); + + ret = get_vsock_port(fd, &port); + if (ret < 0) + die("Failed to get vsocket address"); + + fds[i] = fd; + ports[i] = port; + } +} + +static void agent_handle(int sd, int nr_cpus, int page_size) +{ + struct tracecmd_msg_handle *msg_handle; + unsigned int *ports; + char **argv = NULL; + int argc = 0; + int *fds; + int ret; + + fds = calloc(nr_cpus, sizeof(*fds)); + ports = calloc(nr_cpus, sizeof(*ports)); + if (!fds || !ports) + die("Failed to allocate memory"); + + msg_handle = tracecmd_msg_handle_alloc(sd, TRACECMD_MSG_FL_CLIENT); + if (!msg_handle) + die("Failed to allocate message handle"); + + ret = tracecmd_msg_recv_trace_req(msg_handle, &argc, &argv); + if (ret < 0) + die("Failed to receive trace request"); + + make_vsocks(nr_cpus, fds, ports); + + ret = tracecmd_msg_send_trace_resp(msg_handle, nr_cpus, page_size, ports); + if (ret < 0) + die("Failed to send trace response"); + + trace_record_agent(msg_handle, nr_cpus, fds, argc, argv); + + free(argv[0]); + free(argv); + free(ports); + free(fds); + tracecmd_msg_handle_close(msg_handle); + exit(0); +} + +static volatile pid_t handler_pid; + +static void handle_sigchld(int sig) +{ + int wstatus; + pid_t pid; + + for (;;) { + pid = waitpid(-1, &wstatus, WNOHANG); + if (pid <= 0) + break; + + if (pid == handler_pid) + handler_pid = 0; + } +} + +static void agent_serve(unsigned int port) +{ + int sd, cd, nr_cpus; + pid_t pid; + + signal(SIGCHLD, handle_sigchld); + + nr_cpus = count_cpus(); + page_size = getpagesize(); + + sd = make_vsock(port); + if (sd < 0) + die("Failed to open vsocket"); + + for (;;) { + cd = accept(sd, NULL, NULL); + if (cd < 0) { + if (errno == EINTR) + continue; + die("accept"); + } + + if (handler_pid) + goto busy; + + pid = fork(); + if (pid == 0) { + close(sd); + signal(SIGCHLD, SIG_DFL); + agent_handle(cd, nr_cpus, page_size); + } + if (pid > 0) + handler_pid = pid; + +busy: + close(cd); + } +} + +void trace_agent(int argc, char **argv) +{ + bool do_daemon = false; + unsigned int port = TRACE_AGENT_DEFAULT_PORT; + + if (argc < 2) + usage(argv); + + if (strcmp(argv[1], "agent") != 0) + usage(argv); + + for (;;) { + int c, option_index = 0; + static struct option long_options[] = { + {"port", required_argument, NULL, 'p'}, + {"help", no_argument, NULL, '?'}, + {NULL, 0, NULL, 0} + }; + + c = getopt_long(argc-1, argv+1, "+hp:D", + long_options, &option_index); + if (c == -1) + break; + switch (c) { + case 'h': + usage(argv); + break; + case 'p': + port = atoi(optarg); + break; + case 'D': + do_daemon = true; + break; + default: + usage(argv); + } + } + + if (optind < argc-1) + usage(argv); + + if (do_daemon && daemon(1, 0)) + die("daemon"); + + agent_serve(port); +} diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 797b303..3ae5e2e 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -83,6 +83,9 @@ struct command commands[] = { {"hist", trace_hist}, {"mem", trace_mem}, {"listen", trace_listen}, +#ifdef VSOCK + {"agent", trace_agent}, +#endif {"split", trace_split}, {"restore", trace_restore}, {"stack", trace_stack}, diff --git a/tracecmd/trace-msg.c b/tracecmd/trace-msg.c index 8b6fb03..7ffb8a7 100644 --- a/tracecmd/trace-msg.c +++ b/tracecmd/trace-msg.c @@ -86,6 +86,7 @@ struct tracecmd_msg_trace_req { } __attribute__((packed)); struct tracecmd_msg_trace_resp { + be32 flags; be32 cpus; be32 page_size; } __attribute__((packed)); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index b911c34..3c28314 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -33,6 +33,9 @@ #include #include #include +#ifdef VSOCK +#include +#endif #include "trace-local.h" #include "trace-msg.h" @@ -74,8 +77,6 @@ static int buffers; static int clear_function_filters; static char *host; -static unsigned int *client_ports; -static int sfd; /* Max size to let a per cpu file get */ static int max_kb; @@ -518,6 +519,22 @@ static char *get_temp_file(struct buffer_instance *instance, int cpu) return file; } +static char *get_guest_file(const char *file, const char *guest) +{ + const char *p; + char *out = NULL; + int base_len; + + p = strrchr(file, '.'); + if (p && p != file) + base_len = p - file; + else + base_len = strlen(file); + + asprintf(&out, "%.*s-%s%s", base_len, file, guest, file + base_len); + return out; +} + static void put_temp_file(char *file) { free(file); @@ -623,6 +640,18 @@ static void delete_thread_data(void) } } +static void tell_guests_to_stop(void) +{ + struct buffer_instance *instance; + + for_all_instances(instance) { + if (instance->flags & BUFFER_FL_GUEST) { + tracecmd_msg_send_close_msg(instance->msg_handle); + tracecmd_msg_handle_close(instance->msg_handle); + } + } +} + static void stop_threads(enum trace_type type) { struct timeval tv = { 0, 0 }; @@ -2571,14 +2600,14 @@ static void flush(int sig) tracecmd_stop_recording(recorder); } -static void connect_port(int cpu) +static int connect_port(const char *host, unsigned int port) { struct addrinfo hints; struct addrinfo *results, *rp; - int s; + int s, sfd; char buf[BUFSIZ]; - snprintf(buf, BUFSIZ, "%u", client_ports[cpu]); + snprintf(buf, BUFSIZ, "%u", port); memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_UNSPEC; @@ -2605,7 +2634,190 @@ static void connect_port(int cpu) freeaddrinfo(results); - client_ports[cpu] = sfd; + return sfd; +} + +#ifdef VSOCK +static int open_vsock(unsigned int cid, unsigned int port) +{ + struct sockaddr_vm addr = { + .svm_family = AF_VSOCK, + .svm_cid = cid, + .svm_port = port, + }; + int sd; + + sd = socket(AF_VSOCK, SOCK_STREAM, 0); + if (sd < 0) + return -1; + + if (connect(sd, (struct sockaddr *)&addr, sizeof(addr))) + return -1; + + return sd; +} +#else +static inline int open_vsock(unsigned int cid, unsigned int port) +{ + die("vsock is not supported"); + return -1; +} +#endif + +static int do_accept(int sd) +{ + int cd; + + for (;;) { + cd = accept(sd, NULL, NULL); + if (cd < 0) { + if (errno == EINTR) + continue; + die("accept"); + } + + return cd; + } + + return -1; +} + +static bool is_digits(const char *s) +{ + const char *p; + + for (p = s; *p; p++) + if (!isdigit(*p)) + return false; + + return true; +} + +struct guest { + char *name; + int cid; + int pid; +}; + +static struct guest *guests; +static size_t guests_len; + +static char *get_qemu_guest_name(char *arg) +{ + char *tok, *end = arg; + + while ((tok = strsep(&end, ","))) { + if (strncmp(tok, "guest=", 6) == 0) + return tok + 6; + } + + return arg; +} + +static void read_qemu_guests(void) +{ + static bool initialized; + struct dirent *entry; + char path[PATH_MAX]; + DIR *dir; + + if (initialized) + return; + + initialized = true; + dir = opendir("/proc"); + if (!dir) + die("opendir"); + + for (entry = readdir(dir); entry; entry = readdir(dir)) { + bool is_qemu = false, last_was_name = false; + struct guest guest = {}; + char *p, *arg = NULL; + size_t arg_size = 0; + FILE *f; + + if (!(entry->d_type == DT_DIR && is_digits(entry->d_name))) + continue; + + guest.pid = atoi(entry->d_name); + snprintf(path, sizeof(path), "/proc/%s/cmdline", entry->d_name); + f = fopen(path, "r"); + if (!f) + continue; + + while (getdelim(&arg, &arg_size, 0, f) != -1) { + if (!is_qemu && strstr(arg, "qemu-system-")) { + is_qemu = true; + continue; + } + + if (!is_qemu) + continue; + + if (strcmp(arg, "-name") == 0) { + last_was_name = true; + continue; + } + + if (last_was_name) { + guest.name = strdup(get_qemu_guest_name(arg)); + last_was_name = false; + continue; + } + + p = strstr(arg, "guest-cid="); + if (p) { + guest.cid = atoi(p + 10); + continue; + } + } + + if (!is_qemu) + goto next; + + guests = realloc(guests, (guests_len + 1) * sizeof(*guests)); + if (!guests) + die("Can not allocate guest buffer"); + guests[guests_len++] = guest; + +next: + free(arg); + fclose(f); + } + + closedir(dir); +} + +static char *parse_guest_name(char *guest, int *cid, int *port) +{ + size_t i; + char *p; + + *port = -1; + p = strrchr(guest, ':'); + if (p) { + *p = '\0'; + *port = atoi(p + 1); + } + + *cid = -1; + p = strrchr(guest, '@'); + if (p) { + *p = '\0'; + *cid = atoi(p + 1); + } else if (is_digits(guest)) + *cid = atoi(guest); + + read_qemu_guests(); + for (i = 0; i < guests_len; i++) { + if ((*cid > 0 && *cid == guests[i].cid) || + strcmp(guest, guests[i].name) == 0) { + *cid = guests[i].cid; + return guests[i].name; + } + } + + return guest; } static void set_prio(int prio) @@ -2652,6 +2864,17 @@ create_recorder_instance(struct buffer_instance *instance, const char *file, int struct tracecmd_recorder *record; char *path; + if (instance->flags & BUFFER_FL_GUEST) { + int sd; + + sd = open_vsock(instance->cid, instance->client_ports[cpu]); + if (sd < 0) + die("Failed to connect to agent"); + + return tracecmd_create_recorder_virt( + file, cpu, recorder_flags | TRACECMD_RECORD_NOSPLICE, sd); + } + if (brass) return create_recorder_instance_pipe(instance, cpu, brass); @@ -2676,7 +2899,7 @@ static int create_recorder(struct buffer_instance *instance, int cpu, { long ret; char *file; - int pid; + pid_t pid; if (type != TRACE_TYPE_EXTRACT) { signal(SIGUSR1, flush); @@ -2695,19 +2918,24 @@ static int create_recorder(struct buffer_instance *instance, int cpu, instance->cpu_count = 0; } - if (client_ports) { - char *path; + if ((instance->client_ports && !(instance->flags & BUFFER_FL_GUEST)) || + (instance->flags & BUFFER_FL_AGENT)) { + unsigned int flags = recorder_flags; + char *path = NULL; + int fd; - connect_port(cpu); - if (instance->name) + if (instance->flags & BUFFER_FL_AGENT) + fd = do_accept(instance->fds[cpu]); + else + fd = connect_port(host, instance->client_ports[cpu]); + if (fd < 0) + die("Failed connecting to client"); + if (instance->name && !(instance->flags & BUFFER_FL_AGENT)) path = get_instance_dir(instance); else path = tracecmd_find_tracing_dir(); - recorder = tracecmd_create_buffer_recorder_fd(client_ports[cpu], - cpu, recorder_flags, - path); - if (instance->name) - tracecmd_put_tracing_file(path); + recorder = tracecmd_create_buffer_recorder_fd(fd, cpu, flags, path); + tracecmd_put_tracing_file(path); } else { file = get_temp_file(instance, cpu); recorder = create_recorder_instance(instance, file, cpu, brass); @@ -2745,7 +2973,8 @@ static void check_first_msg_from_server(struct tracecmd_msg_handle *msg_handle) die("server not tracecmd server"); } -static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle) +static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle, + unsigned int **client_ports) { char buf[BUFSIZ]; ssize_t n; @@ -2788,8 +3017,8 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle) /* No options */ write(msg_handle->fd, "0", 2); - client_ports = malloc(local_cpu_count * sizeof(*client_ports)); - if (!client_ports) + *client_ports = malloc(local_cpu_count * sizeof(*client_ports)); + if (!*client_ports) die("Failed to allocate client ports for %d cpus", local_cpu_count); /* @@ -2807,13 +3036,14 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle) if (i == BUFSIZ) die("read bad port number"); buf[i] = 0; - client_ports[cpu] = atoi(buf); + (*client_ports)[cpu] = atoi(buf); } } -static void communicate_with_listener_v3(struct tracecmd_msg_handle *msg_handle) +static void communicate_with_listener_v3(struct tracecmd_msg_handle *msg_handle, + unsigned int **client_ports) { - if (tracecmd_msg_send_init_data(msg_handle, &client_ports) < 0) + if (tracecmd_msg_send_init_data(msg_handle, client_ports) < 0) die("Cannot communicate with server"); } @@ -2864,7 +3094,7 @@ static void check_protocol_version(struct tracecmd_msg_handle *msg_handle) } } -static struct tracecmd_msg_handle *setup_network(void) +static struct tracecmd_msg_handle *setup_network(struct buffer_instance *instance) { struct tracecmd_msg_handle *msg_handle = NULL; struct addrinfo hints; @@ -2934,11 +3164,11 @@ again: close(sfd); goto again; } - communicate_with_listener_v3(msg_handle); + communicate_with_listener_v3(msg_handle, &instance->client_ports); } if (msg_handle->version == V1_PROTOCOL) - communicate_with_listener_v1(msg_handle); + communicate_with_listener_v1(msg_handle, &instance->client_ports); return msg_handle; } @@ -2951,7 +3181,7 @@ setup_connection(struct buffer_instance *instance, struct common_record_context struct tracecmd_msg_handle *msg_handle; struct tracecmd_output *network_handle; - msg_handle = setup_network(); + msg_handle = setup_network(instance); /* Now create the handle through this socket */ if (msg_handle->version == V3_PROTOCOL) { @@ -2978,28 +3208,99 @@ static void finish_network(struct tracecmd_msg_handle *msg_handle) free(host); } +static void connect_to_agent(struct buffer_instance *instance) +{ + struct tracecmd_msg_handle *msg_handle; + int sd, ret, nr_cpus, page_size; + unsigned int *ports; + + sd = open_vsock(instance->cid, instance->port); + if (sd < 0) + die("Failed to connect to vsocket @%d:%d", + instance->cid, instance->port); + + msg_handle = tracecmd_msg_handle_alloc(sd, TRACECMD_MSG_FL_CLIENT); + if (!msg_handle) + die("Failed to allocate message handle"); + + ret = tracecmd_msg_send_trace_req(msg_handle, instance->argc, instance->argv); + if (ret < 0) + die("Failed to send trace request"); + + ret = tracecmd_msg_recv_trace_resp(msg_handle, &nr_cpus, &page_size, &ports); + if (ret < 0) + die("Failed to receive trace response"); + + instance->client_ports = ports; + instance->cpu_count = nr_cpus; + + /* the msg_handle now points to the guest fd */ + instance->msg_handle = msg_handle; +} + +static void setup_guest(struct buffer_instance *instance) +{ + struct tracecmd_msg_handle *msg_handle = instance->msg_handle; + char *file; + int fd; + + /* Create a place to store the guest meta data */ + file = get_guest_file(output_file, instance->name); + if (!file) + die("Failed to allocate memory"); + + fd = open(file, O_CREAT|O_WRONLY|O_TRUNC, 0644); + put_temp_file(file); + if (fd < 0) + die("Failed to open", file); + + /* Start reading tracing metadata */ + if (tracecmd_msg_read_data(msg_handle, fd)) + die("Failed receiving metadata"); + close(fd); +} + +static void setup_agent(struct buffer_instance *instance, struct common_record_context *ctx) +{ + struct tracecmd_output *network_handle; + + network_handle = tracecmd_create_init_fd_msg(instance->msg_handle, + listed_events); + add_options(network_handle, ctx); + tracecmd_write_cpus(network_handle, instance->cpu_count); + tracecmd_write_options(network_handle); + tracecmd_msg_finish_sending_data(instance->msg_handle); + instance->network_handle = network_handle; +} + void start_threads(enum trace_type type, struct common_record_context *ctx) { struct buffer_instance *instance; - int *brass = NULL; int total_cpu_count = 0; int i = 0; int ret; - for_all_instances(instance) + for_all_instances(instance) { + /* Start the connection now to find out how many CPUs we need */ + if (instance->flags & BUFFER_FL_GUEST) + connect_to_agent(instance); total_cpu_count += instance->cpu_count; + } /* make a thread for every CPU we have */ - pids = malloc(sizeof(*pids) * total_cpu_count * (buffers + 1)); + pids = calloc(total_cpu_count * (buffers + 1), sizeof(*pids)); if (!pids) - die("Failed to allocat pids for %d cpus", total_cpu_count); - - memset(pids, 0, sizeof(*pids) * total_cpu_count * (buffers + 1)); + die("Failed to allocate pids for %d cpus", total_cpu_count); for_all_instances(instance) { + int *brass = NULL; int x, pid; - if (host) { + if (instance->flags & BUFFER_FL_AGENT) { + setup_agent(instance, ctx); + } else if (instance->flags & BUFFER_FL_GUEST) { + setup_guest(instance); + } else if (host) { instance->msg_handle = setup_connection(instance, ctx); if (!instance->msg_handle) die("Failed to make connection"); @@ -3139,13 +3440,14 @@ static void print_stat(struct buffer_instance *instance) { int cpu; + if (quiet) + return; + if (!is_top_instance(instance)) - if (!quiet) - printf("\nBuffer: %s\n\n", instance->name); + printf("\nBuffer: %s\n\n", instance->name); for (cpu = 0; cpu < instance->cpu_count; cpu++) - if (!quiet) - trace_seq_do_printf(&instance->s_print[cpu]); + trace_seq_do_printf(&instance->s_print[cpu]); } enum { @@ -3171,7 +3473,44 @@ static void add_options(struct tracecmd_output *handle, struct common_record_con tracecmd_add_option(handle, TRACECMD_OPTION_TRACECLOCK, 0, NULL); add_option_hooks(handle); add_uname(handle); +} + +static void write_guest_file(struct buffer_instance *instance) +{ + struct tracecmd_output *handle; + int cpu_count = instance->cpu_count; + char *file; + char **temp_files; + int i, fd; + + file = get_guest_file(output_file, instance->name); + fd = open(file, O_RDWR); + if (fd < 0) + die("error opening %s", file); + put_temp_file(file); + + handle = tracecmd_get_output_handle_fd(fd); + if (!handle) + die("error writing to %s", file); + temp_files = malloc(sizeof(*temp_files) * cpu_count); + if (!temp_files) + die("failed to allocate temp_files for %d cpus", + cpu_count); + + for (i = 0; i < cpu_count; i++) { + temp_files[i] = get_temp_file(instance, i); + if (!temp_files[i]) + die("failed to allocate memory"); + } + + if (tracecmd_write_cpu_data(handle, cpu_count, temp_files) < 0) + die("failed to write CPU data"); + tracecmd_output_close(handle); + + for (i = 0; i < cpu_count; i++) + put_temp_file(temp_files[i]); + free(temp_files); } static void record_data(struct common_record_context *ctx) @@ -3185,7 +3524,9 @@ static void record_data(struct common_record_context *ctx) int i; for_all_instances(instance) { - if (instance->msg_handle) + if (instance->flags & BUFFER_FL_GUEST) + write_guest_file(instance); + else if (host && instance->msg_handle) finish_network(instance->msg_handle); else local = true; @@ -4404,6 +4745,7 @@ void trace_stop(int argc, char **argv) c = getopt(argc-1, argv+1, "hatB:"); if (c == -1) break; + switch (c) { case 'h': usage(argv); @@ -4566,6 +4908,63 @@ static void init_common_record_context(struct common_record_context *ctx, #define IS_STREAM(ctx) ((ctx)->curr_cmd == CMD_stream) #define IS_PROFILE(ctx) ((ctx)->curr_cmd == CMD_profile) #define IS_RECORD(ctx) ((ctx)->curr_cmd == CMD_record) +#define IS_AGENT(ctx) ((ctx)->curr_cmd == CMD_record_agent) + +static void add_argv(struct buffer_instance *instance, char *arg, bool prepend) +{ + instance->argv = realloc(instance->argv, + (instance->argc + 1) * sizeof(char *)); + if (!instance->argv) + die("Can not allocate instance args"); + if (prepend) { + memmove(instance->argv + 1, instance->argv, + instance->argc * sizeof(*instance->argv)); + instance->argv[0] = arg; + } else { + instance->argv[instance->argc] = arg; + } + instance->argc++; +} + +static void add_arg(struct buffer_instance *instance, + int c, const char *opts, + struct option *long_options, char *optarg) +{ + char *ptr; + char *arg; + int ret; + int i; + + /* Short or long arg */ + if (!(c & 0x80)) { + ret = asprintf(&arg, "-%c", c); + if (ret < 0) + die("Can not allocate argument"); + ptr = strstr(opts, arg+1); + if (!ptr) + return; /* Not found? */ + add_argv(instance, arg, false); + if (ptr[1] == ':') + add_argv(instance, optarg, false); + return; + } + for (i = 0; long_options[i].name; i++) { + if (c == long_options[i].val) { + ret = asprintf(&arg, "--%s", long_options[i].name); + if (ret < 0) + die("Can not allocate argument"); + add_argv(instance, arg, false); + if (long_options[i].has_arg) { + arg = strdup(optarg); + if (!arg) + die("Can not allocate arguments"); + add_argv(instance, arg, false); + return; + } + } + } + /* Not found? */ +} static void parse_record_options(int argc, char **argv, @@ -4607,10 +5006,20 @@ static void parse_record_options(int argc, if (IS_EXTRACT(ctx)) opts = "+haf:Fp:co:O:sr:g:l:n:P:N:tb:B:ksiT"; else - opts = "+hae:f:Fp:cC:dDGo:O:s:r:vg:l:n:P:N:tb:R:B:ksSiTm:M:H:q"; + opts = "+hae:f:FA:p:cC:dDGo:O:s:r:vg:l:n:P:N:tb:R:B:ksSiTm:M:H:q"; c = getopt_long (argc-1, argv+1, opts, long_options, &option_index); if (c == -1) break; + + /* + * If the current instance is to record a guest, then save + * all the arguments for this instance. + */ + if (c != 'B' && c != 'A' && ctx->instance->flags & BUFFER_FL_GUEST) { + add_arg(ctx->instance, c, opts, long_options, optarg); + continue; + } + switch (c) { case 'h': usage(argv); @@ -4663,6 +5072,26 @@ static void parse_record_options(int argc, add_trigger(event, optarg); break; + case 'A': { + char *name = NULL; + int cid = -1, port = -1; + + if (!IS_RECORD(ctx)) + die("-A is only allowed for record operations"); + + name = parse_guest_name(optarg, &cid, &port); + if (!name || cid == -1) + die("guest %s not found", optarg); + if (port == -1) + port = TRACE_AGENT_DEFAULT_PORT; + + ctx->instance = create_instance(name); + ctx->instance->flags |= BUFFER_FL_GUEST; + ctx->instance->cid = cid; + ctx->instance->port = port; + add_instance(ctx->instance, 0); + break; + } case 'F': test_set_event_pid(); filter_task = 1; @@ -4733,6 +5162,8 @@ static void parse_record_options(int argc, ctx->disable = 1; break; case 'o': + if (IS_AGENT(ctx)) + die("-o incompatible with agent recording"); if (host) die("-o incompatible with -N"); if (IS_START(ctx)) @@ -4794,6 +5225,8 @@ static void parse_record_options(int argc, case 'N': if (!IS_RECORD(ctx)) die("-N only available with record"); + if (IS_AGENT(ctx)) + die("-N incompatible with agent recording"); if (ctx->output) die("-N incompatible with -o"); host = optarg; @@ -4890,6 +5323,16 @@ static void parse_record_options(int argc, } } + /* If --date is specified, prepend it to all guest VM flags */ + if (ctx->date) { + struct buffer_instance *instance; + + for_all_instances(instance) { + if (instance->flags & BUFFER_FL_GUEST) + add_argv(instance, "--date", true); + } + } + if (!ctx->filtered && ctx->instance->filter_mod) add_func(&ctx->instance->filter_funcs, ctx->instance->filter_mod, "*"); @@ -4920,7 +5363,8 @@ static enum trace_type get_trace_cmd_type(enum trace_cmd cmd) {CMD_stream, TRACE_TYPE_STREAM}, {CMD_extract, TRACE_TYPE_EXTRACT}, {CMD_profile, TRACE_TYPE_STREAM}, - {CMD_start, TRACE_TYPE_START} + {CMD_start, TRACE_TYPE_START}, + {CMD_record_agent, TRACE_TYPE_RECORD} }; for (int i = 0; i < ARRAY_SIZE(trace_type_per_command); i++) { @@ -4952,12 +5396,28 @@ static void finalize_record_trace(struct common_record_context *ctx) if (instance->flags & BUFFER_FL_KEEP) write_tracing_on(instance, instance->tracing_on_init_val); + if (instance->flags & BUFFER_FL_AGENT) + tracecmd_output_close(instance->network_handle); } if (host) tracecmd_output_close(ctx->instance->network_handle); } +static bool has_local_instances(void) +{ + struct buffer_instance *instance; + + for_all_instances(instance) { + if (instance->flags & BUFFER_FL_GUEST) + continue; + if (host && instance->msg_handle) + continue; + return true; + } + return false; +} + /* * This function contains common code for the following commands: * record, start, stream, profile. @@ -4986,7 +5446,6 @@ static void record_trace(int argc, char **argv, /* Save the state of tracing_on before starting */ for_all_instances(instance) { - if (!ctx->manual && instance->flags & BUFFER_FL_PROFILE) enable_profile(instance); @@ -5003,14 +5462,16 @@ static void record_trace(int argc, char **argv, page_size = getpagesize(); - fset = set_ftrace(!ctx->disable, ctx->total_disable); + if (!(ctx->instance->flags & BUFFER_FL_GUEST)) + fset = set_ftrace(!ctx->disable, ctx->total_disable); tracecmd_disable_all_tracing(1); for_all_instances(instance) set_clock(instance); /* Record records the date first */ - if (IS_RECORD(ctx) && ctx->date) + if (ctx->date && + ((IS_RECORD(ctx) && has_local_instances()) || IS_AGENT(ctx))) ctx->date2ts = get_date_to_ts(); for_all_instances(instance) { @@ -5045,9 +5506,13 @@ static void record_trace(int argc, char **argv, exit(0); } - if (ctx->run_command) + if (ctx->run_command) { run_cmd(type, (argc - optind) - 1, &argv[optind + 1]); - else { + } else if (ctx->instance && (ctx->instance->flags & BUFFER_FL_AGENT)) { + update_task_filter(); + tracecmd_enable_tracing(); + tracecmd_msg_wait_close(ctx->instance->msg_handle); + } else { update_task_filter(); tracecmd_enable_tracing(); /* We don't ptrace ourself */ @@ -5057,6 +5522,8 @@ static void record_trace(int argc, char **argv, printf("Hit Ctrl^C to stop recording\n"); while (!finished) trace_or_sleep(type); + + tell_guests_to_stop(); } tracecmd_disable_tracing(); @@ -5202,3 +5669,40 @@ void trace_record(int argc, char **argv) record_trace(argc, argv, &ctx); exit(0); } + +int trace_record_agent(struct tracecmd_msg_handle *msg_handle, + int cpus, int *fds, + int argc, char **argv) +{ + struct common_record_context ctx; + char **argv_plus; + + /* Reset optind for getopt_long */ + optind = 1; + /* + * argc is the number of elements in argv, but we need to convert + * argc and argv into "trace-cmd", "record", argv. + * where argc needs to grow by two. + */ + argv_plus = calloc(argc + 2, sizeof(char *)); + if (!argv_plus) + return -ENOMEM; + + argv_plus[0] = "trace-cmd"; + argv_plus[1] = "record"; + memmove(argv_plus + 2, argv, argc * sizeof(char *)); + argc += 2; + + parse_record_options(argc, argv_plus, CMD_record_agent, &ctx); + if (ctx.run_command) + return -EINVAL; + + ctx.instance->fds = fds; + ctx.instance->flags |= BUFFER_FL_AGENT; + ctx.instance->msg_handle = msg_handle; + msg_handle->version = V3_PROTOCOL; + record_trace(argc, argv, &ctx); + + free(argv_plus); + return 0; +} diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 9ea1906..e845f50 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -231,11 +231,22 @@ static struct usage_help usage_help[] = { "listen on a network socket for trace clients", " %s listen -p port[-D][-o file][-d dir][-l logfile]\n" " Creates a socket to listen for clients.\n" - " -D create it in daemon mode.\n" + " -p port number to listen on.\n" + " -D run in daemon mode.\n" " -o file name to use for clients.\n" " -d directory to store client files.\n" " -l logfile to write messages to.\n" }, +#ifdef VSOCK + { + "agent", + "listen on a vsocket for trace clients", + " %s agent -p port[-D]\n" + " Creates a vsocket to listen for clients.\n" + " -p port number to listen on.\n" + " -D run in daemon mode.\n" + }, +#endif { "list", "list the available events, plugins or options", From patchwork Mon Feb 4 06:58:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10794999 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B73226C2 for ; Mon, 4 Feb 2019 06:59:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A5C432B03C for ; Mon, 4 Feb 2019 06:59:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9A1CD2B03F; Mon, 4 Feb 2019 06:59:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 380C12B03C for ; Mon, 4 Feb 2019 06:59:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726720AbfBDG7G (ORCPT ); Mon, 4 Feb 2019 01:59:06 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:42463 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726066AbfBDG7F (ORCPT ); Mon, 4 Feb 2019 01:59:05 -0500 Received: by mail-wr1-f68.google.com with SMTP id q18so13122002wrx.9 for ; Sun, 03 Feb 2019 22:59:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e3J/W9wUPHGJUHak9160UZGjCuXyNtQd8t5Pjta35DA=; b=aPImPz4bHJ0JqlxsmS6moQ7SAfk5/yPcT1CJRfC51vGPKz4ydDZcIUwzwSyuOCXhoi WCx5rdc+YxBe2xum0U6fiyTFDDKIY164RJHH+F5Jui/PWEuKI1t4du9QO8fk8jRDgCaW E+q9Vw71YkQ3dJcT/Sg8aAtmgjMoN+8kV3unE3Bbf8GI9Rjs6Oq8u2s9lkXbVZO0jX2I hfK5Kuu6A4kGpDsbePlIqUMOAlFXMCEMHndBfrcz2KI/+4ousjGscqjXGz3micaKQZjY Ff9uVvRthWKy6H38Yv0qgJdRSX0b9yl3W8Qv16ojJEA5bh0XrjEktbMo//EHLrfPKdns OfLQ== X-Gm-Message-State: AJcUukcNxNC61XMoZWKEnCGykH7xhGZ623p4DmCdr4pWH0h8V6dpTDuP p6Bi3Ty5vvYb+ZHIjc6hHfdwcpJRLQ== X-Google-Smtp-Source: ALg8bN7+nDDmDl7Hk+Rjba0hSVGSn0mgljhQFS+4ayRDN2SAfMtjDay8Ihg9W3bHu9bvM4+sJ+ZdHA== X-Received: by 2002:a5d:4303:: with SMTP id h3mr48681616wrq.273.1549263543661; Sun, 03 Feb 2019 22:59:03 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:03 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 06/11] trace-cmd: Use splice(2) for vsockets if available Date: Mon, 4 Feb 2019 08:58:43 +0200 Message-Id: <20190204065848.8248-7-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Detect if splice(2) reading is supported for vsockets (Linux 4.20 and later) and use it, or fallback to read/write-ing otherwise. Signed-off-by: Slavomir Kaslev --- lib/trace-cmd/trace-recorder.c | 6 ++-- tracecmd/trace-record.c | 56 ++++++++++++++++++++++++++++++++-- 2 files changed, 56 insertions(+), 6 deletions(-) diff --git a/lib/trace-cmd/trace-recorder.c b/lib/trace-cmd/trace-recorder.c index 83a12f4..b750935 100644 --- a/lib/trace-cmd/trace-recorder.c +++ b/lib/trace-cmd/trace-recorder.c @@ -380,10 +380,8 @@ static long splice_data(struct tracecmd_recorder *recorder) ret = splice(recorder->brass[0], NULL, recorder->fd, NULL, read, recorder->fd_flags); if (ret < 0) { - if (errno != EAGAIN && errno != EINTR) { - warning("recorder error in splice output"); - return -1; - } + if (errno != EAGAIN && errno != EINTR) + return ret; return total_read; } else update_fd(recorder, ret); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 3c28314..71176d9 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -2656,12 +2656,61 @@ static int open_vsock(unsigned int cid, unsigned int port) return sd; } + +static int try_splice_read_vsock(void) +{ + int ret, sd, brass[2]; + + sd = socket(AF_VSOCK, SOCK_STREAM, 0); + if (sd < 0) + return sd; + + ret = pipe(brass); + if (ret < 0) + goto out_close_sd; + + /* + * On kernels that don't support splice reading from vsockets + * this will fail with EINVAL, or ENOTCONN otherwise. + * Technically, it should never succeed but if it does, claim splice + * reading is supported. + */ + ret = splice(sd, NULL, brass[1], NULL, 10, 0); + if (ret < 0) + ret = errno != EINVAL; + else + ret = 1; + + close(brass[0]); + close(brass[1]); +out_close_sd: + close(sd); + return ret; +} + +static bool can_splice_read_vsock(void) +{ + static bool initialized, res; + + if (initialized) + return res; + + res = try_splice_read_vsock() > 0; + initialized = true; + return res; +} + #else static inline int open_vsock(unsigned int cid, unsigned int port) { die("vsock is not supported"); return -1; } + +static bool can_splice_read_vsock(void) +{ + return false; +} #endif static int do_accept(int sd) @@ -2866,13 +2915,16 @@ create_recorder_instance(struct buffer_instance *instance, const char *file, int if (instance->flags & BUFFER_FL_GUEST) { int sd; + unsigned int flags; sd = open_vsock(instance->cid, instance->client_ports[cpu]); if (sd < 0) die("Failed to connect to agent"); - return tracecmd_create_recorder_virt( - file, cpu, recorder_flags | TRACECMD_RECORD_NOSPLICE, sd); + flags = recorder_flags; + if (!can_splice_read_vsock()) + flags |= TRACECMD_RECORD_NOSPLICE; + return tracecmd_create_recorder_virt(file, cpu, flags, sd); } if (brass) From patchwork Mon Feb 4 06:58:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10795001 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50DCF159A for ; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 405282B03C for ; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 32E8C2B042; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 996A82B03F for ; Mon, 4 Feb 2019 06:59:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726796AbfBDG7H (ORCPT ); Mon, 4 Feb 2019 01:59:07 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:35802 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726348AbfBDG7H (ORCPT ); Mon, 4 Feb 2019 01:59:07 -0500 Received: by mail-wr1-f66.google.com with SMTP id r17so7215990wrp.2 for ; Sun, 03 Feb 2019 22:59:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fH+1t0YUr2RANVRh+5kbl7TECMvwFPVNinSHp6aL2uk=; b=dA3Er7g56RGKfqOogBwEViXqvzdoy/X+rMaGwORU4JWhn1TYv9cw0CgV27g+tFPAoq D24qIhGYRBP/3NNMR3uaCmg3VJwlW+5NlxIGil5b9Xo+XtrGz1rJI0Bc9bVkJPFNiJcX TmpoJLspcpQqJ2MxWOJ3kuvkUAvnabxAxj/VDhnJ2S3Swdktgh4KcRFxzXkRTkrFqs8v XNkQykn8ffvek3YMeNNztUvSwLQkes8NsYMzahPe04ZFnvChIhb5o9LORIr5ga2qwcAy FlYYODHZan/ZIFMFpP0tXczwy5JpY4O9g7ZuEJ7iUb1LCJkJXZD6G33dhMsohj13pQ1u pJsg== X-Gm-Message-State: AHQUAuZNA1Ic3VFO+/FClBuJx/2GoTzgeElHC9KajBUErV+aaOtl6Fqx fNE4UJ+d38vNFOB/yJ2uSLRluW3mZA== X-Google-Smtp-Source: AHgI3IayAJvt1ZIRcnmo63XsX8/JW2I/A+vTeZOuZ4RcBPTUiOnZt0/+J1+jnizt1e7sTVtwtouJLA== X-Received: by 2002:adf:ff49:: with SMTP id u9mr6987268wrs.279.1549263544960; Sun, 03 Feb 2019 22:59:04 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:04 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 07/11] trace-cmd: Add `trace-cmd setup-guest` command Date: Mon, 4 Feb 2019 08:58:44 +0200 Message-Id: <20190204065848.8248-8-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add `trace-cmd setup-guest` command that creates the necessary FIFOs for tracing a guest over FIFOs instead of vsockets. Signed-off-by: Slavomir Kaslev --- tracecmd/Makefile | 1 + tracecmd/include/trace-local.h | 6 ++ tracecmd/trace-cmd.c | 1 + tracecmd/trace-setup-guest.c | 181 +++++++++++++++++++++++++++++++++ tracecmd/trace-usage.c | 8 ++ 5 files changed, 197 insertions(+) create mode 100644 tracecmd/trace-setup-guest.c diff --git a/tracecmd/Makefile b/tracecmd/Makefile index 865b1c6..d3e3080 100644 --- a/tracecmd/Makefile +++ b/tracecmd/Makefile @@ -35,6 +35,7 @@ TRACE_CMD_OBJS += trace-msg.o ifeq ($(VSOCK_DEFINED), 1) TRACE_CMD_OBJS += trace-agent.o +TRACE_CMD_OBJS += trace-setup-guest.o endif ALL_OBJS := $(TRACE_CMD_OBJS:%.o=$(bdir)/%.o) diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index 823d323..b23130e 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -14,6 +14,10 @@ #define TRACE_AGENT_DEFAULT_PORT 823 +#define GUEST_PIPE_NAME "trace-pipe-cpu" +#define GUEST_DIR_FMT "/var/lib/trace-cmd/virt/%s" +#define GUEST_FIFO_FMT GUEST_DIR_FMT "/" GUEST_PIPE_NAME "%d" + extern int debug; extern int quiet; @@ -68,6 +72,8 @@ void trace_listen(int argc, char **argv); void trace_agent(int argc, char **argv); +void trace_setup_guest(int argc, char **argv); + void trace_restore(int argc, char **argv); void trace_clear(int argc, char **argv); diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c index 3ae5e2e..4da82b4 100644 --- a/tracecmd/trace-cmd.c +++ b/tracecmd/trace-cmd.c @@ -85,6 +85,7 @@ struct command commands[] = { {"listen", trace_listen}, #ifdef VSOCK {"agent", trace_agent}, + {"setup-guest", trace_setup_guest}, #endif {"split", trace_split}, {"restore", trace_restore}, diff --git a/tracecmd/trace-setup-guest.c b/tracecmd/trace-setup-guest.c new file mode 100644 index 0000000..1a2c8d4 --- /dev/null +++ b/tracecmd/trace-setup-guest.c @@ -0,0 +1,181 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 VMware Inc, Slavomir Kaslev + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "trace-local.h" +#include "trace-msg.h" + +static int make_dir(const char *path, mode_t mode) +{ + char *buf, *end, *p; + int ret = 0; + + buf = strdup(path); + end = buf + strlen(buf); + + for (p = buf; p < end; p++) { + for (; p < end && *p == '/'; p++); + for (; p < end && *p != '/'; p++); + + *p = '\0'; + ret = mkdir(buf, mode); + if (ret < 0) { + if (errno != EEXIST) + break; + ret = 0; + } + *p = '/'; + } + + free(buf); + return ret; +} + +static int make_fifo(const char *path, mode_t mode) +{ + struct stat st; + int ret; + + ret = stat(path, &st); + if (ret == 0) { + if (S_ISFIFO(st.st_mode)) + return 0; + errno = EEXIST; + return -1; + } + + return mkfifo(path, mode); +} + +static int make_guest_dir(const char *guest) +{ + char path[PATH_MAX]; + + snprintf(path, sizeof(path), GUEST_DIR_FMT, guest); + return make_dir(path, 0750); +} + +static int make_guest_fifo(const char *guest, int cpu, mode_t mode) +{ + static const char *exts[] = {".in", ".out"}; + char path[PATH_MAX]; + int i, ret = 0; + + for (i = 0; i < ARRAY_SIZE(exts); i++) { + snprintf(path, sizeof(path), GUEST_FIFO_FMT "%s", + guest, cpu, exts[i]); + ret = make_fifo(path, mode); + if (ret < 0) + break; + } + + return ret; +} + +static int make_guest_fifos(const char *guest, int nr_cpus, mode_t mode) +{ + int i, ret = 0; + mode_t mask; + + mask = umask(0); + for (i = 0; i < nr_cpus; i++) { + ret = make_guest_fifo(guest, i, mode); + if (ret < 0) + break; + } + umask(mask); + + return ret; +} + +static void do_setup_guest(const char *guest, int nr_cpus, mode_t mode, gid_t gid) +{ + gid_t save_egid; + int ret; + + save_egid = getegid(); + ret = setegid(gid); + if (ret < 0) + pdie("failed to set effective group ID"); + + ret = make_guest_dir(guest); + if (ret < 0) + pdie("failed to create guest directory for %s", guest); + + ret = make_guest_fifos(guest, nr_cpus, mode); + if (ret < 0) + pdie("failed to create FIFOs for %s", guest); + + ret = setegid(save_egid); + if (ret < 0) + pdie("failed to restore effective group ID"); +} + +void trace_setup_guest(int argc, char **argv) +{ + struct group *group; + mode_t mode = 0660; + int nr_cpus = -1; + gid_t gid = -1; + char *guest; + + if (argc < 2) + usage(argv); + + if (strcmp(argv[1], "setup-guest") != 0) + usage(argv); + + for (;;) { + int c, option_index = 0; + static struct option long_options[] = { + {"help", no_argument, NULL, '?'}, + {NULL, 0, NULL, 0} + }; + + c = getopt_long(argc-1, argv+1, "+hc:p:g:", + long_options, &option_index); + if (c == -1) + break; + switch (c) { + case 'h': + usage(argv); + break; + case 'c': + nr_cpus = atoi(optarg); + break; + case 'p': + mode = strtol(optarg, NULL, 8); + break; + case 'g': + group = getgrnam(optarg); + if (!group) + pdie("group %s does not exist", optarg); + gid = group->gr_gid; + break; + default: + usage(argv); + } + } + + if (optind != argc-2) + usage(argv); + + guest = argv[optind+1]; + + if (nr_cpus <= 0) + pdie("invalid number of cpus for guest %s", guest); + + do_setup_guest(guest, nr_cpus, mode, gid); +} diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index e845f50..7a8002f 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -246,6 +246,14 @@ static struct usage_help usage_help[] = { " -p port number to listen on.\n" " -D run in daemon mode.\n" }, + { + "setup-guest", + "create FIFOs for tracing guest VMs", + " %s setup-guest -c cpus[-p perm][-g group] guest\n" + " -c number of guest virtual CPUs\n" + " -p FIFOs permissions (default: 0660)\n" + " -g FIFOs group owner\n" + }, #endif { "list", From patchwork Mon Feb 4 06:58:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10795003 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DFC156C2 for ; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC74E2B03C for ; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BF7D52B042; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6CBF32B03C for ; Mon, 4 Feb 2019 06:59:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726348AbfBDG7I (ORCPT ); Mon, 4 Feb 2019 01:59:08 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:34619 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG7I (ORCPT ); Mon, 4 Feb 2019 01:59:08 -0500 Received: by mail-wm1-f67.google.com with SMTP id y185so9296664wmd.1 for ; Sun, 03 Feb 2019 22:59:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oO5do8q1U5Vv4Twz8iY4p0exxjg6NdsA2d3EcWpAL5s=; b=M00OPCjymDZNmLKI0ceI+MI4b3k3byaRXk6CVCk4PfPfW5GvxG9Y3i0XB5LuHmteZn xjMn2o7efA215zEHEcSI0yiPxKbMkKlSH+ekj2jftdLa/4pMn6Q5PmltdAGSm+Ba3LCE weBFc+yY2/SMBUJmB4kFsC2wv6smSA4VWyDVXb0Sp1zFg0e+yIKk3Lzt3ENEkOwc1hQX PSDXHHrF26rTdE5HAIrUmkgOw3BBeruOrSyQJc9Did8KFJzAkLjlt7HaARwfdQvOhFcQ ruIRmwur5pn+k6MtJfSP36FV+h4Gh+NzztG9W08xyQv+wFEgkbP5JbTE8L4ufYDMhJvf 5q0A== X-Gm-Message-State: AHQUAubebNzLmg/9/tGQi1nkgq3sAmQbzrS4eEqpJUsZzkZUQsOy73wZ P30gSPlBpPZcmImc7VHRbRpSMbF4Yg== X-Google-Smtp-Source: AHgI3Ia2MdyNkNTNIcX05ufz1SUwLAytlYVBluTGCVlFvXy3I6ugTvC45stNlCZsSlRrRZlI+/iMWw== X-Received: by 2002:a1c:5a42:: with SMTP id o63mr11875915wmb.88.1549263546379; Sun, 03 Feb 2019 22:59:06 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:05 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 08/11] trace-cmd: Try to autodetect number of guest CPUs in setup-guest if not specified Date: Mon, 4 Feb 2019 08:58:45 +0200 Message-Id: <20190204065848.8248-9-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When no number of guest CPUs is provided with the -c flag to `trace-cmd setup-guest`, try to autodetect it using virsh for libvirt managed guests. Signed-off-by: Slavomir Kaslev --- tracecmd/trace-setup-guest.c | 21 +++++++++++++++++++++ tracecmd/trace-usage.c | 2 +- 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/tracecmd/trace-setup-guest.c b/tracecmd/trace-setup-guest.c index 1a2c8d4..bcd8f40 100644 --- a/tracecmd/trace-setup-guest.c +++ b/tracecmd/trace-setup-guest.c @@ -100,6 +100,24 @@ static int make_guest_fifos(const char *guest, int nr_cpus, mode_t mode) return ret; } +static int get_guest_cpu_count(const char *guest) +{ + const char *cmd_fmt = "virsh vcpucount --maximum '%s' 2>/dev/null"; + int nr_cpus = -1; + char cmd[1024]; + FILE *f; + + snprintf(cmd, sizeof(cmd), cmd_fmt, guest); + f = popen(cmd, "r"); + if (!f) + return -1; + + fscanf(f, "%d", &nr_cpus); + pclose(f); + + return nr_cpus; +} + static void do_setup_guest(const char *guest, int nr_cpus, mode_t mode, gid_t gid) { gid_t save_egid; @@ -174,6 +192,9 @@ void trace_setup_guest(int argc, char **argv) guest = argv[optind+1]; + if (nr_cpus <= 0) + nr_cpus = get_guest_cpu_count(guest); + if (nr_cpus <= 0) pdie("invalid number of cpus for guest %s", guest); diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c index 7a8002f..9a13d93 100644 --- a/tracecmd/trace-usage.c +++ b/tracecmd/trace-usage.c @@ -249,7 +249,7 @@ static struct usage_help usage_help[] = { { "setup-guest", "create FIFOs for tracing guest VMs", - " %s setup-guest -c cpus[-p perm][-g group] guest\n" + " %s setup-guest [-c cpus][-p perm][-g group] guest\n" " -c number of guest virtual CPUs\n" " -p FIFOs permissions (default: 0660)\n" " -g FIFOs group owner\n" From patchwork Mon Feb 4 06:58:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10795007 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88CB3159A for ; Mon, 4 Feb 2019 06:59:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 73FC82B03F for ; Mon, 4 Feb 2019 06:59:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 656072B043; Mon, 4 Feb 2019 06:59:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00FDD2B042 for ; Mon, 4 Feb 2019 06:59:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726894AbfBDG7K (ORCPT ); Mon, 4 Feb 2019 01:59:10 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:52832 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG7K (ORCPT ); Mon, 4 Feb 2019 01:59:10 -0500 Received: by mail-wm1-f66.google.com with SMTP id m1so11917311wml.2 for ; Sun, 03 Feb 2019 22:59:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lKtnP1A7c1Mnvfk3XOJyOhHm8nmBusLfYbVmELEmxE8=; b=igyi05MdUdyxNZkizm9bBtJyBbORQxG8Jt9orJC9N6shFAqsE03qV7GyrV4ClOPxii /sEnVeSTBmifm11B2zBu+MAlOBsXPltDssDwOEokta1bUHBoGNPbXjhF3BSgCRrCw//o 4AL4oVignDRQV+ur6ZYeVZA6NAdFZnHgzqFAnh47AU8oQ6bKgx8ir78qRLHmc0GUmTdt q9/zIYUsPVCSfq79xFSjyZhdJBOAjJKxvfbtvRyekytbJk425o7JPvHwN3RlDYSVVwHa JWMwqMeJeb9QkPPXQwXiK7bR9ZIugNxCoUitzw3QkM7Z8sJcsczZTmvco93WWhxeutpc Wydg== X-Gm-Message-State: AHQUAualcU5FNmgH7pm/pfsUFIxM31YRT3H+vMKG2ZrUMvaOyF1jtTR4 KylvMY95E0Ky4576xgkp1DAh6ihsAQ== X-Google-Smtp-Source: AHgI3IZSS+PHcsx+zXXK2NtxwdKR8qMricUdZI8YLaHCJ8MpvhHyr7YrhoZ4oeqaBtNhYO6XM0GC8g== X-Received: by 2002:a1c:23cb:: with SMTP id j194mr11259043wmj.91.1549263547689; Sun, 03 Feb 2019 22:59:07 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:07 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 09/11] trace-cmd: Make setup-guest auto attach FIFOs to the guest VM config Date: Mon, 4 Feb 2019 08:58:46 +0200 Message-Id: <20190204065848.8248-10-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch tries to attach the newly created FIFOs for guest tracing as virtio serial devices to libvirt managed guests. Signed-off-by: Slavomir Kaslev --- tracecmd/trace-setup-guest.c | 41 ++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/tracecmd/trace-setup-guest.c b/tracecmd/trace-setup-guest.c index bcd8f40..3176b7e 100644 --- a/tracecmd/trace-setup-guest.c +++ b/tracecmd/trace-setup-guest.c @@ -118,6 +118,43 @@ static int get_guest_cpu_count(const char *guest) return nr_cpus; } +static int attach_guest_fifos(const char *guest, int nr_cpus) +{ + const char *cmd_fmt = + "virsh attach-device --config '%s' '%s' >/dev/null 2>/dev/null"; + const char *xml_fmt = + "\n" + " \n" + " \n" + ""; + char tmp_path[PATH_MAX], path[PATH_MAX]; + char cmd[PATH_MAX], xml[PATH_MAX]; + int i, fd, ret = 0; + + strcpy(tmp_path, "/tmp/pipexmlXXXXXX"); + fd = mkstemp(tmp_path); + if (fd < 0) + return fd; + + for (i = 0; i < nr_cpus; i++) { + snprintf(path, sizeof(path), GUEST_FIFO_FMT, guest, i); + snprintf(xml, sizeof(xml), xml_fmt, path, GUEST_PIPE_NAME, i); + pwrite(fd, xml, strlen(xml), 0); + + snprintf(cmd, sizeof(cmd), cmd_fmt, guest, tmp_path); + errno = 0; + if (system(cmd) != 0) { + ret = -1; + break; + } + } + + close(fd); + unlink(tmp_path); + + return ret; +} + static void do_setup_guest(const char *guest, int nr_cpus, mode_t mode, gid_t gid) { gid_t save_egid; @@ -136,6 +173,10 @@ static void do_setup_guest(const char *guest, int nr_cpus, mode_t mode, gid_t gi if (ret < 0) pdie("failed to create FIFOs for %s", guest); + ret = attach_guest_fifos(guest, nr_cpus); + if (ret < 0) + warning("failed to attach FIFOs to %s", guest); + ret = setegid(save_egid); if (ret < 0) pdie("failed to restore effective group ID"); From patchwork Mon Feb 4 06:58:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10795009 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 697396C2 for ; Mon, 4 Feb 2019 06:59:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59D742B03F for ; Mon, 4 Feb 2019 06:59:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4E1B12B043; Mon, 4 Feb 2019 06:59:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA5732B03F for ; Mon, 4 Feb 2019 06:59:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726916AbfBDG7Q (ORCPT ); Mon, 4 Feb 2019 01:59:16 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:54650 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725906AbfBDG7Q (ORCPT ); Mon, 4 Feb 2019 01:59:16 -0500 Received: by mail-wm1-f67.google.com with SMTP id a62so11910836wmh.4 for ; Sun, 03 Feb 2019 22:59:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BLuQN/bAKgJ8I0YFAGQgYKvqZfR8uQV6+xYGJ04W4N8=; b=ldObPestTtdgbg71RO//nJ2k5ieV+dgyWtmWoyvjCc3M6asc8LGY3I9L11QUx8Sg/9 jSetCygcPN+Gqtg/ykbwBD1PtUoFQseCep4J/nsCzLXH+1YVuyeOfO07NoaT5MBmA8gR EMXp7fymKfABJ8HiB3OCayurqY/KSVqCYPLo/0slRb17RfWtvNIfMjhcpiCzEu88MHTN 1X4gQmX1mb2yWf0ySSWNJWT4wV5tpwVXOVJWC7sns3/SQphD+zJXAwNFXFYg37borPLm uTkFgE7jh4QKK96VLujiod0Chy3GTQbPd5qflhqwBmiY0kYtxNRAVimZlHsI3eJJmhd+ ZcVA== X-Gm-Message-State: AHQUAuY35Tm38vVu9o2LL3/ldAjVFtiFON7gC0no2z7SkrpiBuur2gre dt4GyAMBrcl5c4Iu6Sz1IiD7c1sRZQ== X-Google-Smtp-Source: AHgI3IZYa1ELp9qfNHjPtFQ9TwKg04XyrVLzi3g6lnkqRWkKlQiTSHWJv6h54B1DORyhhnlZG3FjdA== X-Received: by 2002:a1c:a8d2:: with SMTP id r201mr11492610wme.81.1549263554010; Sun, 03 Feb 2019 22:59:14 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:13 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 10/11] trace-cmd: Set both input and output to non-blocking when recording is stopped Date: Mon, 4 Feb 2019 08:58:47 +0200 Message-Id: <20190204065848.8248-11-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When tracing VMs over FIFOs both the agent and the host recording process might get blocked doing IO over a FIFO when the other side is gone. In this case we won't get EPIPE error (as in the vsockets case) since the FIFOs will always be opened by the hypervisor. Signed-off-by: Slavomir Kaslev --- lib/trace-cmd/trace-recorder.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/lib/trace-cmd/trace-recorder.c b/lib/trace-cmd/trace-recorder.c index b750935..1c12c9e 100644 --- a/lib/trace-cmd/trace-recorder.c +++ b/lib/trace-cmd/trace-recorder.c @@ -40,6 +40,7 @@ struct tracecmd_recorder { int pages; int count; unsigned fd_flags; + unsigned trace_fd_flags; unsigned flags; }; @@ -121,6 +122,8 @@ tracecmd_create_buffer_recorder_fd2(int fd, int fd2, int cpu, unsigned flags, if (!(recorder->flags & TRACECMD_RECORD_BLOCK)) recorder->fd_flags |= 2; /* and NON_BLOCK */ + recorder->trace_fd_flags = 1; /* SPLICE_F_MOVE */ + /* Init to know what to free and release */ recorder->trace_fd = -1; recorder->brass[0] = -1; @@ -366,7 +369,7 @@ static long splice_data(struct tracecmd_recorder *recorder) long ret; read = splice(recorder->trace_fd, NULL, recorder->brass[1], NULL, - recorder->pipe_size, 1 /* SPLICE_F_MOVE */); + recorder->pipe_size, recorder->trace_fd_flags); if (read < 0) { if (errno != EAGAIN && errno != EINTR) { warning("recorder error in splice input"); @@ -434,9 +437,12 @@ static void set_nonblock(struct tracecmd_recorder *recorder) /* Do not block on reads for flushing */ flags = fcntl(recorder->trace_fd, F_GETFL); fcntl(recorder->trace_fd, F_SETFL, flags | O_NONBLOCK); + recorder->trace_fd_flags |= 2; /* SPLICE_F_NONBLOCK */ /* Do not block on streams for write */ - recorder->fd_flags |= 2; /* NON_BLOCK */ + flags = fcntl(recorder->fd, F_GETFL); + fcntl(recorder->fd, F_SETFL, flags | O_NONBLOCK); + recorder->fd_flags |= 2; /* SPLICE_F_NONBLOCK */ } long tracecmd_flush_recording(struct tracecmd_recorder *recorder) From patchwork Mon Feb 4 06:58:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slavomir Kaslev X-Patchwork-Id: 10795011 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE4686C2 for ; Mon, 4 Feb 2019 06:59:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAABB2B08A for ; Mon, 4 Feb 2019 06:59:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B6832B096; Mon, 4 Feb 2019 06:59:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA81A2B08A for ; Mon, 4 Feb 2019 06:59:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726918AbfBDG7T (ORCPT ); Mon, 4 Feb 2019 01:59:19 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:55310 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726883AbfBDG7T (ORCPT ); Mon, 4 Feb 2019 01:59:19 -0500 Received: by mail-wm1-f67.google.com with SMTP id y139so11840271wmc.5 for ; Sun, 03 Feb 2019 22:59:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mBlvW4CmqBj+ocrg5C0wUec027NrzRu734wMyiTXNig=; b=N9Sw35lU3X9dPqf9VCFsj/tJZMY1YiqMbgwObqAE6hdSaE7prmWnkZJsndmkz1e/I0 IBUa+ZRKAjQ0sT187yqwQuAJSkFX6eFBXK2J4N0d342/VM5ITNqNbHPPjebKas2YJ7bF lA9OeL0IFIGxU2GLnxfxHLChPTt2YlLJ9YMbZBhoIT6f3NGT242E9v13CgVTrdV6/LPH 49LtGQw5EXw5It5nZTLFCxybZRo4fJWTpLd735mBvLRcTLWKKl045/fFFn1EFbDumiqy EqAqAWtygz8BLFXE6HcLmeglJrhHe6TzRDLg4vlqtD5tDiZX4tVohWxBcuom1KR3DCg4 xWJA== X-Gm-Message-State: AHQUAuZTMuClyo+PxC+WQz7Hmsdxz72aEfM8SeNR2ccnH7kYA/azJLzH 0LGqDa7yqWKwRcL+cFit5+m3WYhgDg== X-Google-Smtp-Source: AHgI3IbModxaIbQPmeL989zKOqnlKlSoaedFT3dv7P4CqHxXK1he4S8+9Bev5Wh102O1G06yT8UBRg== X-Received: by 2002:a7b:c76b:: with SMTP id x11mr11602771wmk.37.1549263555427; Sun, 03 Feb 2019 22:59:15 -0800 (PST) Received: from localhost.localdomain ([213.145.108.55]) by smtp.gmail.com with ESMTPSA id q12sm12596809wrx.31.2019.02.03.22.59.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 22:59:14 -0800 (PST) From: Slavomir Kaslev To: linux-trace-devel@vger.kernel.org Cc: rostedt@goodmis.org, slavomir.kaslev@gmail.com, tstoyanov@vmware.com, ykaradzhov@vmware.com Subject: [RFC PATCH v5 11/11] trace-cmd: Add VM tracing over FIFOs transport Date: Mon, 4 Feb 2019 08:58:48 +0200 Message-Id: <20190204065848.8248-12-kaslevs@vmware.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190204065848.8248-1-kaslevs@vmware.com> References: <20190204065848.8248-1-kaslevs@vmware.com> MIME-Version: 1.0 Sender: linux-trace-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP FIFOs offer ~3x the throughput of vsockets. This patch adds support for using FIFOs to stream tracing data back to the host when tracing VMs. Signed-off-by: Slavomir Kaslev --- include/trace-cmd/trace-cmd.h | 8 ++-- tracecmd/include/trace-local.h | 4 +- tracecmd/trace-agent.c | 40 ++++++++++++++-- tracecmd/trace-msg.c | 26 +++++++---- tracecmd/trace-record.c | 84 ++++++++++++++++++++++++++++------ 5 files changed, 131 insertions(+), 31 deletions(-) diff --git a/include/trace-cmd/trace-cmd.h b/include/trace-cmd/trace-cmd.h index 7664074..84dc95a 100644 --- a/include/trace-cmd/trace-cmd.h +++ b/include/trace-cmd/trace-cmd.h @@ -338,16 +338,16 @@ bool tracecmd_msg_done(struct tracecmd_msg_handle *msg_handle); void tracecmd_msg_set_done(struct tracecmd_msg_handle *msg_handle); int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, - int argc, char **argv); + int argc, char **argv, bool use_fifos); int tracecmd_msg_recv_trace_req(struct tracecmd_msg_handle *msg_handle, - int *argc, char ***argv); + int *argc, char ***argv, bool *use_fifos); int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, int nr_cpus, int page_size, - unsigned int *ports); + unsigned int *ports, bool use_fifos); int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, int *nr_cpus, int *page_size, - unsigned int **ports); + unsigned int **ports, bool *use_fifos); int tracecmd_msg_wait_close(struct tracecmd_msg_handle *msg_handle); diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h index b23130e..8bf7c8e 100644 --- a/tracecmd/include/trace-local.h +++ b/tracecmd/include/trace-local.h @@ -17,6 +17,7 @@ #define GUEST_PIPE_NAME "trace-pipe-cpu" #define GUEST_DIR_FMT "/var/lib/trace-cmd/virt/%s" #define GUEST_FIFO_FMT GUEST_DIR_FMT "/" GUEST_PIPE_NAME "%d" +#define VIRTIO_FIFO_FMT "/dev/virtio-ports/" GUEST_PIPE_NAME "%d" extern int debug; extern int quiet; @@ -100,7 +101,7 @@ void trace_usage(int argc, char **argv); int trace_record_agent(struct tracecmd_msg_handle *msg_handle, int cpus, int *fds, - int argc, char **argv); + int argc, char **argv, bool use_fifos); struct hook_list; @@ -212,6 +213,7 @@ struct buffer_instance { int cid; int port; int *fds; + bool use_fifos; }; extern struct buffer_instance top_instance; diff --git a/tracecmd/trace-agent.c b/tracecmd/trace-agent.c index b4b3bbb..34e6c0c 100644 --- a/tracecmd/trace-agent.c +++ b/tracecmd/trace-agent.c @@ -83,12 +83,39 @@ static void make_vsocks(int nr, int *fds, unsigned int *ports) } } +static int open_agent_fifos(int nr_cpus, int *fds) +{ + char path[PATH_MAX]; + int i, fd, ret = 0; + + for (i = 0; i < nr_cpus; i++) { + snprintf(path, sizeof(path), VIRTIO_FIFO_FMT, i); + fd = open(path, O_WRONLY); + if (fd < 0) { + ret = -1; + break; + } + + fds[i] = fd; + } + + if (!ret) + return ret; + + /* We failed to open all FIFOs so clean up and return error */ + while (--i >= 0) + close(fds[i]); + + return ret; +} + static void agent_handle(int sd, int nr_cpus, int page_size) { struct tracecmd_msg_handle *msg_handle; unsigned int *ports; char **argv = NULL; int argc = 0; + bool use_fifos; int *fds; int ret; @@ -101,17 +128,22 @@ static void agent_handle(int sd, int nr_cpus, int page_size) if (!msg_handle) die("Failed to allocate message handle"); - ret = tracecmd_msg_recv_trace_req(msg_handle, &argc, &argv); + ret = tracecmd_msg_recv_trace_req(msg_handle, &argc, &argv, &use_fifos); if (ret < 0) die("Failed to receive trace request"); - make_vsocks(nr_cpus, fds, ports); + if (use_fifos && open_agent_fifos(nr_cpus, fds)) + use_fifos = false; + + if (!use_fifos) + make_vsocks(nr_cpus, fds, ports); - ret = tracecmd_msg_send_trace_resp(msg_handle, nr_cpus, page_size, ports); + ret = tracecmd_msg_send_trace_resp(msg_handle, nr_cpus, page_size, + ports, use_fifos); if (ret < 0) die("Failed to send trace response"); - trace_record_agent(msg_handle, nr_cpus, fds, argc, argv); + trace_record_agent(msg_handle, nr_cpus, fds, argc, argv, use_fifos); free(argv[0]); free(argv); diff --git a/tracecmd/trace-msg.c b/tracecmd/trace-msg.c index 7ffb8a7..c28d294 100644 --- a/tracecmd/trace-msg.c +++ b/tracecmd/trace-msg.c @@ -175,6 +175,10 @@ static int msg_write(int fd, struct tracecmd_msg *msg) return __do_write_check(fd, msg->buf, data_size); } +enum msg_trace_flags { + MSG_TRACE_USE_FIFOS = 1 << 0, +}; + enum msg_opt_command { MSGOPT_USETCP = 1, }; @@ -732,7 +736,7 @@ error: return ret; } -static int make_trace_req(struct tracecmd_msg *msg, int argc, char **argv) +static int make_trace_req(struct tracecmd_msg *msg, int argc, char **argv, bool use_fifos) { size_t args_size = 0; char *p; @@ -742,6 +746,7 @@ static int make_trace_req(struct tracecmd_msg *msg, int argc, char **argv) args_size += strlen(argv[i]) + 1; msg->hdr.size = htonl(ntohl(msg->hdr.size) + args_size); + msg->trace_req.flags = use_fifos ? htonl(MSG_TRACE_USE_FIFOS) : htonl(0); msg->trace_req.argc = htonl(argc); msg->buf = calloc(args_size, 1); if (!msg->buf) @@ -755,13 +760,13 @@ static int make_trace_req(struct tracecmd_msg *msg, int argc, char **argv) } int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, - int argc, char **argv) + int argc, char **argv, bool use_fifos) { struct tracecmd_msg msg; int ret; tracecmd_msg_init(MSG_TRACE_REQ, &msg); - ret = make_trace_req(&msg, argc, argv); + ret = make_trace_req(&msg, argc, argv, use_fifos); if (ret < 0) return ret; @@ -774,7 +779,7 @@ int tracecmd_msg_send_trace_req(struct tracecmd_msg_handle *msg_handle, * free(argv); */ int tracecmd_msg_recv_trace_req(struct tracecmd_msg_handle *msg_handle, - int *argc, char ***argv) + int *argc, char ***argv, bool *use_fifos) { struct tracecmd_msg msg; char *p, *buf_end, **args; @@ -819,6 +824,7 @@ int tracecmd_msg_recv_trace_req(struct tracecmd_msg_handle *msg_handle, msg.buf = NULL; *argc = nr_args; *argv = args; + *use_fifos = ntohl(msg.trace_req.flags) & MSG_TRACE_USE_FIFOS; ret = 0; out: @@ -826,13 +832,14 @@ out: return ret; } -static int make_trace_resp(struct tracecmd_msg *msg, - int page_size, int nr_cpus, unsigned int *ports) +static int make_trace_resp(struct tracecmd_msg *msg, int page_size, int nr_cpus, + unsigned int *ports, bool use_fifos) { int ports_size = nr_cpus * sizeof(*msg->port_array); int i; msg->hdr.size = htonl(ntohl(msg->hdr.size) + ports_size); + msg->trace_resp.flags = use_fifos ? htonl(MSG_TRACE_USE_FIFOS) : htonl(0); msg->trace_resp.cpus = htonl(nr_cpus); msg->trace_resp.page_size = htonl(page_size); @@ -848,13 +855,13 @@ static int make_trace_resp(struct tracecmd_msg *msg, int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, int nr_cpus, int page_size, - unsigned int *ports) + unsigned int *ports, bool use_fifos) { struct tracecmd_msg msg; int ret; tracecmd_msg_init(MSG_TRACE_RESP, &msg); - ret = make_trace_resp(&msg, page_size, nr_cpus, ports); + ret = make_trace_resp(&msg, page_size, nr_cpus, ports, use_fifos); if (ret < 0) return ret; @@ -863,7 +870,7 @@ int tracecmd_msg_send_trace_resp(struct tracecmd_msg_handle *msg_handle, int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, int *nr_cpus, int *page_size, - unsigned int **ports) + unsigned int **ports, bool *use_fifos) { struct tracecmd_msg msg; size_t buf_len; @@ -885,6 +892,7 @@ int tracecmd_msg_recv_trace_resp(struct tracecmd_msg_handle *msg_handle, goto out; } + *use_fifos = ntohl(msg.trace_resp.flags) & MSG_TRACE_USE_FIFOS; *nr_cpus = ntohl(msg.trace_resp.cpus); *page_size = ntohl(msg.trace_resp.page_size); *ports = calloc(*nr_cpus, sizeof(**ports)); diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c index 71176d9..43d782e 100644 --- a/tracecmd/trace-record.c +++ b/tracecmd/trace-record.c @@ -76,6 +76,8 @@ static int buffers; /* Clear all function filters */ static int clear_function_filters; +static bool no_fifos; + static char *host; /* Max size to let a per cpu file get */ @@ -2914,17 +2916,20 @@ create_recorder_instance(struct buffer_instance *instance, const char *file, int char *path; if (instance->flags & BUFFER_FL_GUEST) { - int sd; + int fd; unsigned int flags; - sd = open_vsock(instance->cid, instance->client_ports[cpu]); - if (sd < 0) + if (instance->use_fifos) + fd = instance->fds[cpu]; + else + fd = open_vsock(instance->cid, instance->client_ports[cpu]); + if (fd < 0) die("Failed to connect to agent"); flags = recorder_flags; - if (!can_splice_read_vsock()) + if (!instance->use_fifos && !can_splice_read_vsock()) flags |= TRACECMD_RECORD_NOSPLICE; - return tracecmd_create_recorder_virt(file, cpu, flags, sd); + return tracecmd_create_recorder_virt(file, cpu, flags, fd); } if (brass) @@ -2976,10 +2981,14 @@ static int create_recorder(struct buffer_instance *instance, int cpu, char *path = NULL; int fd; - if (instance->flags & BUFFER_FL_AGENT) - fd = do_accept(instance->fds[cpu]); - else + if (instance->flags & BUFFER_FL_AGENT) { + if (instance->use_fifos) + fd = instance->fds[cpu]; + else + fd = do_accept(instance->fds[cpu]); + } else { fd = connect_port(host, instance->client_ports[cpu]); + } if (fd < 0) die("Failed connecting to client"); if (instance->name && !(instance->flags & BUFFER_FL_AGENT)) @@ -3260,11 +3269,36 @@ static void finish_network(struct tracecmd_msg_handle *msg_handle) free(host); } +static int open_guest_fifos(const char *guest, int **fds) +{ + char path[PATH_MAX]; + int i, fd; + + for (i = 0; ; i++) { + snprintf(path, sizeof(path), GUEST_FIFO_FMT ".out", guest, i); + fd = open(path, O_RDONLY); + if (fd < 0) + break; + + *fds = realloc(*fds, i + 1); + (*fds)[i] = fd; + } + + return i; +} + static void connect_to_agent(struct buffer_instance *instance) { struct tracecmd_msg_handle *msg_handle; - int sd, ret, nr_cpus, page_size; + int sd, ret, nr_fifos, nr_cpus, page_size; unsigned int *ports; + int i, *fds = NULL; + bool use_fifos = false; + + if (!no_fifos) { + nr_fifos = open_guest_fifos(instance->name, &fds); + use_fifos = nr_fifos > 0; + } sd = open_vsock(instance->cid, instance->port); if (sd < 0) @@ -3275,15 +3309,32 @@ static void connect_to_agent(struct buffer_instance *instance) if (!msg_handle) die("Failed to allocate message handle"); - ret = tracecmd_msg_send_trace_req(msg_handle, instance->argc, instance->argv); + ret = tracecmd_msg_send_trace_req(msg_handle, instance->argc, + instance->argv, use_fifos); if (ret < 0) die("Failed to send trace request"); - ret = tracecmd_msg_recv_trace_resp(msg_handle, &nr_cpus, &page_size, &ports); + ret = tracecmd_msg_recv_trace_resp(msg_handle, &nr_cpus, &page_size, + &ports, &use_fifos); if (ret < 0) die("Failed to receive trace response"); - instance->client_ports = ports; + if (use_fifos) { + if (nr_cpus != nr_fifos) { + warning("number of FIFOs for guest %s differs " + "from number of cpus", instance->name); + nr_cpus = nr_cpus < nr_fifos ? nr_cpus : nr_fifos; + } + free(ports); + instance->fds = fds; + } else { + for (i = 0; i < nr_fifos; i++) + close(fds[i]); + free(fds); + instance->client_ports = ports; + } + + instance->use_fifos = use_fifos; instance->cpu_count = nr_cpus; /* the msg_handle now points to the guest fd */ @@ -4782,6 +4833,7 @@ enum { OPT_funcstack = 254, OPT_date = 255, OPT_module = 256, + OPT_nofifos = 257, }; void trace_stop(int argc, char **argv) @@ -5043,6 +5095,7 @@ static void parse_record_options(int argc, {"date", no_argument, NULL, OPT_date}, {"func-stack", no_argument, NULL, OPT_funcstack}, {"nosplice", no_argument, NULL, OPT_nosplice}, + {"nofifos", no_argument, NULL, OPT_nofifos}, {"profile", no_argument, NULL, OPT_profile}, {"stderr", no_argument, NULL, OPT_stderr}, {"by-comm", no_argument, NULL, OPT_bycomm}, @@ -5328,6 +5381,9 @@ static void parse_record_options(int argc, case OPT_nosplice: recorder_flags |= TRACECMD_RECORD_NOSPLICE; break; + case OPT_nofifos: + no_fifos = true; + break; case OPT_profile: handle_init = trace_init_profile; ctx->instance->flags |= BUFFER_FL_PROFILE; @@ -5724,7 +5780,8 @@ void trace_record(int argc, char **argv) int trace_record_agent(struct tracecmd_msg_handle *msg_handle, int cpus, int *fds, - int argc, char **argv) + int argc, char **argv, + bool use_fifos) { struct common_record_context ctx; char **argv_plus; @@ -5750,6 +5807,7 @@ int trace_record_agent(struct tracecmd_msg_handle *msg_handle, return -EINVAL; ctx.instance->fds = fds; + ctx.instance->use_fifos = use_fifos; ctx.instance->flags |= BUFFER_FL_AGENT; ctx.instance->msg_handle = msg_handle; msg_handle->version = V3_PROTOCOL;