From patchwork Thu Nov 17 11:13:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Palatin X-Patchwork-Id: 9434097 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C41BE60469 for ; Thu, 17 Nov 2016 11:15:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B43602940A for ; Thu, 17 Nov 2016 11:15:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A74AA2940C; Thu, 17 Nov 2016 11:15:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D51422940A for ; Thu, 17 Nov 2016 11:15:09 +0000 (UTC) Received: from localhost ([::1]:58183 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c7Ker-0005Cb-2X for patchwork-qemu-devel@patchwork.kernel.org; Thu, 17 Nov 2016 06:15:09 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46657) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c7KeI-0005AB-UQ for qemu-devel@nongnu.org; Thu, 17 Nov 2016 06:14:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c7KeE-0004fP-7X for qemu-devel@nongnu.org; Thu, 17 Nov 2016 06:14:34 -0500 Received: from mail-wm0-x232.google.com ([2a00:1450:400c:c09::232]:36907) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1c7KeD-0004f2-Sz for qemu-devel@nongnu.org; Thu, 17 Nov 2016 06:14:30 -0500 Received: by mail-wm0-x232.google.com with SMTP id t79so137802321wmt.0 for ; Thu, 17 Nov 2016 03:14:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=uq65QBhQjVjU84E7J3UeT2L+Og23eceKj00CZsd65C4=; b=Eg/cEjxskz0HnTBj7bNuZi6+hn//hKZP/Zq8ov/g9IswcLRMKR9pgNW2O5aJ7R+zgJ dFqqFCny2cnAsJmVoSLN2B/MFBbCdK6z9GnFNODI8Av9aioD1uSDEzZabWUXwcr9rXUj TMsEVipuIeOzbJKo+B4yfPdhoMmlnAAhOHi/RKLgczkaYBYTlhj2Q/MQEwn/h8F6/UZd XqVfHoc6mxQ7hMLlcVjD9nwQQQbUbyD36KLDK5ztacqm9F//LHsvY7YInCftFJVxc+4E j7B+fcdFyv9ejMAS33cPHicRUGukYZ/3KBa5BZiY9xJdtIEvUJiw89APWOzTnzk4Oh8Z AhrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:in-reply-to:references; bh=uq65QBhQjVjU84E7J3UeT2L+Og23eceKj00CZsd65C4=; b=e+4Tbt3kDjuZrnQ26tU5yli2GutZb7EsT8lI5RibPQmPNf55SZswWVYZHtGZwEtnil hTf2k8rrBwcAXNZU6Ucuqqzb0HCAgqFyOBwlFk8MuD8TgnoiK2vbGmfw3pQGbkJ1LBYw 9jTPg86IIuIOHOpfYFzNS8dLyIzwIjqCPevz+AG0HKqqcu53+FDG8mS6tjHc7vSNlmp5 tZGdho4cnX7NmT4q++nPza5knjAD3k6j2/wg3gIQXyTrZoftRxw6iyGiIjiM+nEOKkLm IXf9P9Wx5QE2SuZDIcUkvCAdxjPGqhgIYdH5TX0j4qSgLzkwgUBu4pmeDl0ddSxIIGGf 3m1w== X-Gm-Message-State: ABUngvcCrKPI2dMwMwZNnR36pzNe4a/mB7fDrYjOe0acoxrwWpNEczd+QFqFLOCpFeKKgAgL X-Received: by 10.28.16.70 with SMTP id 67mr3238652wmq.53.1479381268568; Thu, 17 Nov 2016 03:14:28 -0800 (PST) Received: from vpalatin.zrh.corp.google.com ([172.16.114.40]) by smtp.gmail.com with ESMTPSA id j6sm2683757wjk.25.2016.11.17.03.14.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 17 Nov 2016 03:14:27 -0800 (PST) From: Vincent Palatin To: qemu-devel Date: Thu, 17 Nov 2016 12:13:52 +0100 Message-Id: X-Mailer: git-send-email 2.8.0.rc3.226.g39d4020 In-Reply-To: References: In-Reply-To: References: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::232 Subject: [Qemu-devel] [PATCH v3 3/3] Plumb the HAXM-based hardware acceleration support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vincent Palatin Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Use the Intel HAX is kernel-based hardware acceleration module for Windows (similar to KVM on Linux). Based on the "target-i386: Add Intel HAX to android emulator" patch from David Chou Signed-off-by: Vincent Palatin --- Makefile.target | 1 + configure | 18 ++++++++++ cpus.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++- exec.c | 16 +++++++++ hw/intc/apic_common.c | 3 +- include/qom/cpu.h | 5 +++ include/sysemu/hw_accel.h | 9 +++++ qemu-options.hx | 11 ++++++ target-i386/Makefile.objs | 4 +++ vl.c | 15 ++++++-- 10 files changed, 164 insertions(+), 5 deletions(-) diff --git a/Makefile.target b/Makefile.target index 7a5080e..dab81e7 100644 --- a/Makefile.target +++ b/Makefile.target @@ -96,6 +96,7 @@ obj-y += target-$(TARGET_BASE_ARCH)/ obj-y += disas.o obj-y += tcg-runtime.o obj-$(call notempty,$(TARGET_XML_FILES)) += gdbstub-xml.o +obj-$(call lnot,$(CONFIG_HAX)) += hax-stub.o obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o obj-$(CONFIG_LIBDECNUMBER) += libdecnumber/decContext.o diff --git a/configure b/configure index 7d2a34e..236a7b7 100755 --- a/configure +++ b/configure @@ -230,6 +230,7 @@ vhost_net="no" vhost_scsi="no" vhost_vsock="no" kvm="no" +hax="no" colo="yes" rdma="" gprof="no" @@ -563,6 +564,7 @@ CYGWIN*) ;; MINGW32*) mingw32="yes" + hax="yes" audio_possible_drivers="dsound sdl" if check_include dsound.h; then audio_drv_list="dsound" @@ -610,6 +612,7 @@ OpenBSD) Darwin) bsd="yes" darwin="yes" + hax="yes" LDFLAGS_SHARED="-bundle -undefined dynamic_lookup" if [ "$cpu" = "x86_64" ] ; then QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS" @@ -919,6 +922,10 @@ for opt do ;; --enable-kvm) kvm="yes" ;; + --disable-hax) hax="no" + ;; + --enable-hax) hax="yes" + ;; --disable-colo) colo="no" ;; --enable-colo) colo="yes" @@ -1371,6 +1378,7 @@ disabled with --disable-FEATURE, default is enabled if available: fdt fdt device tree bluez bluez stack connectivity kvm KVM acceleration support + hax HAX acceleration support colo COarse-grain LOck-stepping VM for Non-stop Service rdma RDMA-based migration support vde support for vde network @@ -5043,6 +5051,7 @@ echo "ATTR/XATTR support $attr" echo "Install blobs $blobs" echo "KVM support $kvm" echo "COLO support $colo" +echo "HAX support $hax" echo "RDMA support $rdma" echo "TCG interpreter $tcg_interpreter" echo "fdt support $fdt" @@ -6027,6 +6036,15 @@ case "$target_name" in fi fi esac +if test "$hax" = "yes" ; then + if test "$target_softmmu" = "yes" ; then + case "$target_name" in + i386|x86_64) + echo "CONFIG_HAX=y" >> $config_target_mak + ;; + esac + fi +fi if test "$target_bigendian" = "yes" ; then echo "TARGET_WORDS_BIGENDIAN=y" >> $config_target_mak fi diff --git a/cpus.c b/cpus.c index fc78502..0e01791 100644 --- a/cpus.c +++ b/cpus.c @@ -35,6 +35,7 @@ #include "sysemu/dma.h" #include "sysemu/hw_accel.h" #include "sysemu/kvm.h" +#include "sysemu/hax.h" #include "qmp-commands.h" #include "exec/exec-all.h" @@ -1221,6 +1222,39 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) return NULL; } +static void *qemu_hax_cpu_thread_fn(void *arg) +{ + CPUState *cpu = arg; + int r; + qemu_thread_get_self(cpu->thread); + qemu_mutex_lock(&qemu_global_mutex); + + cpu->thread_id = qemu_get_thread_id(); + cpu->created = true; + cpu->halted = 0; + current_cpu = cpu; + + hax_init_vcpu(cpu); + qemu_cond_signal(&qemu_cpu_cond); + + while (1) { + if (cpu_can_run(cpu)) { + r = hax_smp_cpu_exec(cpu); + if (r == EXCP_DEBUG) { + cpu_handle_guest_debug(cpu); + } + } + + while (cpu_thread_is_idle(cpu)) { + qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + } + + qemu_wait_io_event_common(cpu); + } + return NULL; +} + + static void qemu_cpu_kick_thread(CPUState *cpu) { #ifndef _WIN32 @@ -1236,7 +1270,33 @@ static void qemu_cpu_kick_thread(CPUState *cpu) exit(1); } #else /* _WIN32 */ - abort(); + if (!qemu_cpu_is_self(cpu)) { + CONTEXT context; + + if (SuspendThread(cpu->hThread) == (DWORD)(-1)) { + fprintf(stderr, "qemu:%s: GetLastError:%lu\n", __func__, + GetLastError()); + exit(1); + } + + /* On multi-core systems, we are not sure that the thread is actually + * suspended until we can get the context. + */ + context.ContextFlags = CONTEXT_CONTROL; + while (GetThreadContext(cpu->hThread, &context) != 0) { + continue; + } + + if (hax_enabled()) { + cpu->exit_request = 1; + } + + if (ResumeThread(cpu->hThread) == (DWORD)(-1)) { + fprintf(stderr, "qemu:%s: GetLastError:%lu\n", __func__, + GetLastError()); + exit(1); + } + } #endif } @@ -1396,6 +1456,9 @@ static void qemu_tcg_init_vcpu(CPUState *cpu) static QemuCond *tcg_halt_cond; static QemuThread *tcg_cpu_thread; + if (hax_enabled()) + hax_init_vcpu(cpu); + /* share a single thread for all cpus with TCG */ if (!tcg_cpu_thread) { cpu->thread = g_malloc0(sizeof(QemuThread)); @@ -1419,6 +1482,26 @@ static void qemu_tcg_init_vcpu(CPUState *cpu) } } +static void qemu_hax_start_vcpu(CPUState *cpu) +{ + char thread_name[VCPU_THREAD_NAME_SIZE]; + + cpu->thread = g_malloc0(sizeof(QemuThread)); + cpu->halt_cond = g_malloc0(sizeof(QemuCond)); + qemu_cond_init(cpu->halt_cond); + + snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/HAX", + cpu->cpu_index); + qemu_thread_create(cpu->thread, thread_name, qemu_hax_cpu_thread_fn, + cpu, QEMU_THREAD_JOINABLE); +#ifdef _WIN32 + cpu->hThread = qemu_thread_get_handle(cpu->thread); +#endif + while (!cpu->created) { + qemu_cond_wait(&qemu_cpu_cond, &qemu_global_mutex); + } +} + static void qemu_kvm_start_vcpu(CPUState *cpu) { char thread_name[VCPU_THREAD_NAME_SIZE]; @@ -1469,6 +1552,8 @@ void qemu_init_vcpu(CPUState *cpu) if (kvm_enabled()) { qemu_kvm_start_vcpu(cpu); + } else if (hax_enabled()) { + qemu_hax_start_vcpu(cpu); } else if (tcg_enabled()) { qemu_tcg_init_vcpu(cpu); } else { diff --git a/exec.c b/exec.c index 3d867f1..fa24cd1 100644 --- a/exec.c +++ b/exec.c @@ -31,6 +31,7 @@ #include "hw/xen/xen.h" #endif #include "sysemu/kvm.h" +#include "sysemu/hax.h" #include "sysemu/sysemu.h" #include "qemu/timer.h" #include "qemu/config-file.h" @@ -1624,6 +1625,21 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) qemu_mutex_unlock_ramlist(); return; } + /* + * In Hax, the qemu allocate the virtual address, and HAX kernel + * populate the memory with physical memory. Currently we have no + * paging, so user should make sure enough free memory in advance + */ + if (hax_enabled()) { + int ret; + ret = hax_populate_ram((uint64_t)(uintptr_t)new_block->host, + new_block->max_length); + if (ret < 0) { + error_setg(errp, "Hax failed to populate ram"); + return; + } + } + memory_try_enable_merging(new_block->host, new_block->max_length); } } diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c index d78c885..3945dfd 100644 --- a/hw/intc/apic_common.c +++ b/hw/intc/apic_common.c @@ -26,6 +26,7 @@ #include "hw/i386/apic.h" #include "hw/i386/apic_internal.h" #include "trace.h" +#include "sysemu/hax.h" #include "sysemu/kvm.h" #include "hw/qdev.h" #include "hw/sysbus.h" @@ -316,7 +317,7 @@ static void apic_common_realize(DeviceState *dev, Error **errp) /* Note: We need at least 1M to map the VAPIC option ROM */ if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK && - ram_size >= 1024 * 1024) { + !hax_enabled() && ram_size >= 1024 * 1024) { vapic = sysbus_create_simple("kvmvapic", -1, NULL); } s->vapic = vapic; diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 3f79a8e..ca4d0fb 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -227,6 +227,8 @@ struct CPUWatchpoint { struct KVMState; struct kvm_run; +struct hax_vcpu_state; + #define TB_JMP_CACHE_BITS 12 #define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS) @@ -392,6 +394,9 @@ struct CPUState { (absolute value) offset as small as possible. This reduces code size, especially for hosts without large memory offsets. */ uint32_t tcg_exit_req; + + bool hax_vcpu_dirty; + struct hax_vcpu_state *hax_vcpu; }; QTAILQ_HEAD(CPUTailQ, CPUState); diff --git a/include/sysemu/hw_accel.h b/include/sysemu/hw_accel.h index 03812cf..c9b3105 100644 --- a/include/sysemu/hw_accel.h +++ b/include/sysemu/hw_accel.h @@ -20,6 +20,9 @@ static inline void cpu_synchronize_state(CPUState *cpu) if (kvm_enabled()) { kvm_cpu_synchronize_state(cpu); } + if (hax_enabled()) { + hax_cpu_synchronize_state(cpu); + } } static inline void cpu_synchronize_post_reset(CPUState *cpu) @@ -27,6 +30,9 @@ static inline void cpu_synchronize_post_reset(CPUState *cpu) if (kvm_enabled()) { kvm_cpu_synchronize_post_reset(cpu); } + if (hax_enabled()) { + hax_cpu_synchronize_post_reset(cpu); + } } static inline void cpu_synchronize_post_init(CPUState *cpu) @@ -34,6 +40,9 @@ static inline void cpu_synchronize_post_init(CPUState *cpu) if (kvm_enabled()) { kvm_cpu_synchronize_post_init(cpu); } + if (hax_enabled()) { + hax_cpu_synchronize_post_init(cpu); + } } #endif /* QEMU_HW_ACCEL_H */ diff --git a/qemu-options.hx b/qemu-options.hx index 4536e18..bd28219 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -3259,6 +3259,17 @@ Enable KVM full virtualization support. This option is only available if KVM support is enabled when compiling. ETEXI +DEF("enable-hax", 0, QEMU_OPTION_enable_hax, \ + "-enable-hax enable HAX virtualization support\n", QEMU_ARCH_I386) +STEXI +@item -enable-hax +@findex -enable-hax +Enable HAX (Hardware-based Acceleration eXecution) support. This option +is only available if HAX support is enabled when compiling. HAX is only +applicable to MAC and Windows platform, and thus does not conflict with +KVM. +ETEXI + DEF("xen-domid", HAS_ARG, QEMU_OPTION_xen_domid, "-xen-domid id specify xen guest domain id\n", QEMU_ARCH_ALL) DEF("xen-create", 0, QEMU_OPTION_xen_create, diff --git a/target-i386/Makefile.objs b/target-i386/Makefile.objs index b223d79..f14cda9 100644 --- a/target-i386/Makefile.objs +++ b/target-i386/Makefile.objs @@ -5,3 +5,7 @@ obj-y += gdbstub.o obj-$(CONFIG_SOFTMMU) += machine.o arch_memory_mapping.o arch_dump.o monitor.o obj-$(CONFIG_KVM) += kvm.o hyperv.o obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o +# HAX support +ifdef CONFIG_WIN32 +obj-$(CONFIG_HAX) += hax-all.o hax-slot.o hax-windows.o +endif diff --git a/vl.c b/vl.c index 319f641..b11fa50 100644 --- a/vl.c +++ b/vl.c @@ -92,6 +92,7 @@ int main(int argc, char **argv) #include "sysemu/cpus.h" #include "migration/colo.h" #include "sysemu/kvm.h" +#include "sysemu/hax.h" #include "qapi/qmp/qjson.h" #include "qemu/option.h" #include "qemu/config-file.h" @@ -1959,7 +1960,7 @@ static void main_loop(void) int64_t ti; #endif do { - nonblocking = !kvm_enabled() && !xen_enabled() && last_io > 0; + nonblocking = tcg_enabled() && last_io > 0; #ifdef CONFIG_PROFILER ti = profile_getclock(); #endif @@ -3724,6 +3725,10 @@ int main(int argc, char **argv, char **envp) olist = qemu_find_opts("machine"); qemu_opts_parse_noisily(olist, "accel=kvm", false); break; + case QEMU_OPTION_enable_hax: + olist = qemu_find_opts("machine"); + qemu_opts_parse_noisily(olist, "accel=hax", false); + break; case QEMU_OPTION_M: case QEMU_OPTION_machine: olist = qemu_find_opts("machine"); @@ -4418,8 +4423,8 @@ int main(int argc, char **argv, char **envp) cpu_ticks_init(); if (icount_opts) { - if (kvm_enabled() || xen_enabled()) { - error_report("-icount is not allowed with kvm or xen"); + if (!tcg_enabled()) { + error_report("-icount is not allowed with hardware virtualization"); exit(1); } configure_icount(icount_opts, &error_abort); @@ -4555,6 +4560,10 @@ int main(int argc, char **argv, char **envp) numa_post_machine_init(); + if (hax_enabled()) { + hax_sync_vcpus(); + } + if (qemu_opts_foreach(qemu_find_opts("fw_cfg"), parse_fw_cfg, fw_cfg_find(), NULL) != 0) { exit(1);