From patchwork Mon Sep 5 18:56:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Llu=C3=ADs_Vilanova?= X-Patchwork-Id: 9315073 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 40BB5600CA for ; Mon, 5 Sep 2016 19:06:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31F1F28A0E for ; Mon, 5 Sep 2016 19:06:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 26BFB28A22; Mon, 5 Sep 2016 19:06:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5DFB528A0E for ; Mon, 5 Sep 2016 19:06:37 +0000 (UTC) Received: from localhost ([::1]:56444 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bgzE4-0004bj-DT for patchwork-qemu-devel@patchwork.kernel.org; Mon, 05 Sep 2016 15:06:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44756) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bgz4O-0005FF-82 for qemu-devel@nongnu.org; Mon, 05 Sep 2016 14:56:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bgz4H-0008LA-J2 for qemu-devel@nongnu.org; Mon, 05 Sep 2016 14:56:36 -0400 Received: from roura.ac.upc.edu ([147.83.33.10]:57344 helo=roura.ac.upc.es) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bgz4H-0008Kn-2P for qemu-devel@nongnu.org; Mon, 05 Sep 2016 14:56:29 -0400 Received: from gw-2.ac.upc.es (gw-2.ac.upc.es [147.83.30.8]) by roura.ac.upc.es (8.13.8/8.13.8) with ESMTP id u85IuStZ001947; Mon, 5 Sep 2016 20:56:28 +0200 Received: from localhost (unknown [84.88.51.85]) by gw-2.ac.upc.es (Postfix) with ESMTPSA id 168903B1; Mon, 5 Sep 2016 20:56:28 +0200 (CEST) From: =?utf-8?b?TGx1w61z?= Vilanova To: qemu-devel@nongnu.org Date: Mon, 5 Sep 2016 20:56:27 +0200 Message-Id: <147310178781.10840.12547600646529892944.stgit@fimbulvetr.bsc.es> X-Mailer: git-send-email 2.9.3 In-Reply-To: <147310178240.10840.14758930096407696981.stgit@fimbulvetr.bsc.es> References: <147310178240.10840.14758930096407696981.stgit@fimbulvetr.bsc.es> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-MIME-Autoconverted: from 8bit to quoted-printable by roura.ac.upc.es id u85IuStZ001947 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 147.83.33.10 Subject: [Qemu-devel] [PATCH v2 1/6] hypertrace: Add documentation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stefan Hajnoczi , Luiz Capitulino Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Lluís Vilanova --- docs/hypertrace.txt | 227 +++++++++++++++++++++++++++++++++++++++++++++++++++ docs/tracing.txt | 3 + 2 files changed, 230 insertions(+) create mode 100644 docs/hypertrace.txt diff --git a/docs/hypertrace.txt b/docs/hypertrace.txt new file mode 100644 index 0000000..f061036 --- /dev/null +++ b/docs/hypertrace.txt @@ -0,0 +1,227 @@ += Hypertrace channel = + +Copyright (C) 2016 Lluís Vilanova + +This work is licensed under the terms of the GNU GPL, version 2 or later. +See the COPYING file in the top-level directory. + + +The hypertrace channel allows guest code to emit events in QEMU (the host) using +its tracing infrastructure (see "docs/trace.txt"). This works in both 'system' +and 'user' modes. Therefore, hypertrace is to tracing what hypercalls are to +system calls. + +The hypertrace channel can be used for various purposes: + +* The example below shows how to use this on your guest applications to identify + regions of interest where you want to selectively enable certain tracing + events, without paying the price of tracing them outside the interest + regions. + +* You can also invoke hypertrace in your guest code to time the performance of + new TCG optimizations. Each hypertrace event comes with a host timestamp, + making it easy to compare execution times of interesting guest code regions. + +* Calls to hypertrace can also be added along traces generated in the guest to + more easily correlate and merge guest and host traces. + +Hypertrace highlights: + +* Works with 'system' and 'user' mode. + +* Minimal setup for the guest; QEMU provides support guest code libraries to + make 'system' mode with Linux and 'user' mode work out of the box. + +* Independent of guest architecture; the guest code uses accesses to special + memory regions, as opposed to redefining instruction semantics. + +* Negligible guest overhead; emitting a hypertrace event requires a single guest + memory access, making it as unobtrusive as possible. + +Warning: The hypertrace channel in 'system' mode is presented as a PCI device, +and thus will only be available on systems with support for PCI. You can get the +list of guests with PCI support with 'grep pci.mak default-configs/*'. + + +== Quick guide == + +This shows an example of using the hypertrace channel to trace events only in a +region of the guest code, which is identified by calls to the hypertrace +channel. + +We are going to trace memory accesses to disk using QEMU's "log" backend, and +will use QEMU's "dtrace" (SystemTap) backend to control when we want to trace +them. Tracing is done with "log" because it is more efficient than using +"dtrace" in high-volume events like memory accesses. The first time the guest +code invokes the hypertrace channel, we will start tracing the +"guest_mem_before_exec" event, and then will disable it the second time around. + +1. Set the tracing backends and number of arguments for the hypertrace events: + + mkdir /tmp/qemu-build + cd /tmp/qemu-build + /path/to/qemu-source/configure \ + --enable-trace-backends=dtrace,log \ + --with-hypertrace-args=1 \ + --prefix=/tmp/qemu-install + make -j install + +2. Enable the "guest_mem_before_exec" event: + + sed -i -e 's/disable vcpu tcg guest_mem_before/vcpu tcg guest_mem_before/g' /path/to/qemu-source/trace-events + +3. Compile the guest support code: + + make -C /tmp/qemu-build/x86_64-linux-user/hypertrace/guest/user + make -C /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/user + make -C /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/linux-module + + If you need to cross-compile the guest library, set the 'CC' variable: + + make -C /tmp/qemu-build/mipsel-linux-user/hypertrace/guest/user CC=mipsel-gnu-linux-gcc + +3. Create a guest application using "qemu-hypertrace.h" to interact with the + hypertrace channel: + + cat > /tmp/my-hypertrace.c <<\EOF + #include + #include + #include + #include + #include + + + int main(int argc, char **argv) + { + char *base = NULL; + if (argc > 1) { + base = argv[1]; + } + + /* In 'user' mode this path must be the same we will use to start QEMU. */ + if (qemu_hypertrace_init(base) != 0) { + perror("error: qemu_hypertrace_init"); + abort(); + } + + /* Set additional event arguments */ + uint64_t client = 0; + uint64_t *data = qemu_hypertrace_data(client); + data[0] = 0xbabe; + data[1] = 0xdead; + data[2] = 0xbeef; + + /* Emit 1st event */ + qemu_hypertrace(client, 0xcafe); + + /* Computation in between */ + printf("Some computation...\n"); + + /* Emit 2nd event */ + qemu_hypertrace(client, 0xcafe); + } + EOF + + gcc -o /tmp/my-hypertrace-user /tmp/my-hypertrace.c \ + /tmp/qemu-build/x86_64-linux-user/hypertrace/guest/user/libqemu-hypertrace-guest.a \ + -I/tmp/qemu-install/include -lpthread + + gcc -o /tmp/my-hypertrace-softmmu /tmp/my-hypertrace.c \ + /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/user/libqemu-hypertrace-guest.a \ + -I/tmp/qemu-install/include -lpthread + +4. Create a SystemTap script to control event tracing: + + cat > /tmp/my-hypertrace-script.stp <<\EOF + #!/usr/bin/env stap + + global hypercount + + %{ + #include + %} + + function enable_mem:long() + %{ + char *argv[4] = {"/bin/sh", "-c", "echo 'trace-event guest_mem_before_exec on' | telnet localhost 1234", NULL}; + printk(KERN_ERR "enable\n"); + call_usermodehelper(argv[0], argv, NULL, UMH_WAIT_EXEC); + /* Wait for changes to apply */ + msleep(1000); + printk(KERN_ERR "enabled\n"); + STAP_RETURN(0); + %} + + function disable_mem:long() + %{ + char *argv[4] = {"/bin/sh", "-c", "echo 'trace-event guest_mem_before_exec off' | telnet localhost 1234", NULL}; + printk(KERN_ERR "disable\n"); + call_usermodehelper(argv[0], argv, NULL, UMH_WAIT_EXEC); + msleep(1000); + printk(KERN_ERR "disabled\n"); + STAP_RETURN(0); + %} + + probe process("./install/vanilla/bin/qemu-*").mark("guest_hypertrace") + { + hypercount++ + printf("guest_hypertrace: %x\n", $arg1) + if (hypercount == 1) { + enable_mem() + } else if (hypercount == 2) { + disable_mem() + exit() + } + } + EOF + +4. Run a guest system with access to QEMU's hypertrace: + + stap -g /tmp/my-hypertrace-script.stp + /tmp/qemu-install/x86_64-softmmu/qemu-system-x86_64 \ + -device hypertrace \ + -monitor tcp:localhost:1234,server,nowait \ + -trace enable=guest_hypertrace -D /dev/stdout \ + ...' + + And inside the VM: + + sudo /tmp/my-hypertrace-softmmu + + The result will be something like this: + + VNC server running on ::1:5900 + 23071@1473096085.744211:guest_hypertrace cpu=0x5602e1f49c10 arg1=0x000000000000cafe arg2=0x000000000000babe arg3=0x000000000000dead arg4=0x000000000000beef + 23071@1473096085.745763:guest_mem_before_trans cpu=0x5602e1f49c10 info=19 + 23071@1473096085.745907:guest_mem_before_trans cpu=0x5602e1f49c10 info=3 + 23071@1473096085.752368:guest_mem_before_trans cpu=0x5602e1f49c10 info=3 + 23071@1473096085.752384:guest_mem_before_trans cpu=0x5602e1f49c10 info=19 + 23071@1473096086.756117:guest_hypertrace cpu=0x5602e1f49c10 arg1=0x000000000000cafe arg2=0x000000000000babe arg3=0x000000000000dead arg4=0x000000000000beef + +Similarly, you can enable hypertrace when running standalone guest applications: + + /tmp/qemu-install/bin/qemu-x86_64 \ + -hypertrace /tmp/hypertrace \ + -trace enable=guest* -D /dev/stdout \ + /tmp/my-hypertrace-user /tmp/hypertrace + +You can also use hypertrace inside the Linux's kernel code with the provided +guest module (see "/tmp/qemu-install/include/linux/qemu-hypertrace.h"): + + sudo insmod /tmp/qemu-build/x86_64-softmmu/hypertrace/guest/linux-module/qemu-hypertrace.ko + + +== Details == + +To make it more efficient in terms of guest and host time, hypertrace provides +two different memory areas (channels). + +The control channel is used by the guest to tell QEMU that new data is ready to +be processed in the data channel. Writes to the control channel are intercepted +by QEMU, which emits the "hypertrace" tracing event. + +The data channel is a regular memory buffer used by the guest to write +additional event arguments before raising the event through the control channel. + +Both channels accept different "per-client offsets" to enable multiple guest +threads or CPUs to use the hypertrace channel without having to synchronize. diff --git a/docs/tracing.txt b/docs/tracing.txt index 29f2f9a..f312596 100644 --- a/docs/tracing.txt +++ b/docs/tracing.txt @@ -5,6 +5,9 @@ This document describes the tracing infrastructure in QEMU and how to use it for debugging, profiling, and observing execution. +See "docs/hypertrace.txt" to correlate guest tracing events with those in the +QEMU host. + == Quickstart == 1. Build with the 'simple' trace backend: