diff mbox series

hax: Support for Linux hosts

Message ID 20181114130428.32326-1-asanchez@kryptoslogic.com (mailing list archive)
State New, archived
Headers show
Series hax: Support for Linux hosts | expand

Commit Message

Alexandro Sanchez Nov. 14, 2018, 1:04 p.m. UTC
Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
the corresponding userland changes.

Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
files have been renamed to hax-posix.{c,h}. This prefix is consistent with
the naming used in the rest of QEMU.

Signed-off-by: Alexandro Sanchez Bach <asanchez@kryptoslogic.com>
---
 target/i386/Makefile.objs                 |  5 ++++-
 target/i386/hax-i386.h                    | 10 +++++++++-
 target/i386/{hax-darwin.c => hax-posix.c} |  0
 target/i386/{hax-darwin.h => hax-posix.h} |  0
 4 files changed, 13 insertions(+), 2 deletions(-)
 rename target/i386/{hax-darwin.c => hax-posix.c} (100%)
 rename target/i386/{hax-darwin.h => hax-posix.h} (100%)

Comments

Alex Bennée Nov. 14, 2018, 3:04 p.m. UTC | #1
Alexandro Sanchez Bach <asanchez@kryptoslogic.com> writes:

> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
> the corresponding userland changes.
>
> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
> the naming used in the rest of QEMU.
>
> Signed-off-by: Alexandro Sanchez Bach <asanchez@kryptoslogic.com>
> ---
>  target/i386/Makefile.objs                 |  5 ++++-
>  target/i386/hax-i386.h                    | 10 +++++++++-
>  target/i386/{hax-darwin.c => hax-posix.c} |  0
>  target/i386/{hax-darwin.h => hax-posix.h} |  0
>  4 files changed, 13 insertions(+), 2 deletions(-)
>  rename target/i386/{hax-darwin.c => hax-posix.c} (100%)
>  rename target/i386/{hax-darwin.h => hax-posix.h} (100%)
>
> diff --git a/target/i386/Makefile.objs b/target/i386/Makefile.objs
> index 04678f5503..be36d31d23 100644
> --- a/target/i386/Makefile.objs
> +++ b/target/i386/Makefile.objs
> @@ -12,8 +12,11 @@ obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
>  ifdef CONFIG_WIN32
>  obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-windows.o
>  endif
> +ifdef CONFIG_LINUX
> +obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
> +endif
>  ifdef CONFIG_DARWIN
> -obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-darwin.o
> +obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o

This is starting to get repetitive, perhaps:

  obj-$(CONFIG_HAX) += hax-all.o hax-mem.o
  ifneq ($(CONFIG_LINUX)$(CONFIG_DARWIN),)
  obj-$(CONFIG_HAX) += hax-posix.o
  endif
  ifdef CONFIG_WIN32
  obj-$(CONFIG_HAX) += hax-windows.o
  endif

Would be better?

>  obj-$(CONFIG_HVF) += hvf/
>  endif
>  obj-$(CONFIG_WHPX) += whpx-all.o
> diff --git a/target/i386/hax-i386.h b/target/i386/hax-i386.h
> index 6abc156f88..f87a64c9df 100644
> --- a/target/i386/hax-i386.h
> +++ b/target/i386/hax-i386.h
> @@ -20,6 +20,10 @@
>  typedef int hax_fd;
>  #endif
>
> +#ifdef CONFIG_LINUX
> +typedef int hax_fd;
> +#endif

Again this could be merged with the other POSIX like above.

> +
>  #ifdef CONFIG_WIN32
>  typedef HANDLE hax_fd;
>  #endif
> @@ -83,7 +87,11 @@ void hax_memory_init(void);
>
>
>  #ifdef CONFIG_DARWIN
> -#include "target/i386/hax-darwin.h"
> +#include "target/i386/hax-posix.h"
> +#endif
> +
> +#ifdef CONFIG_LINUX
> +#include "target/i386/hax-posix.h"
>  #endif

And here.

In fact looking at my config-host.mak we have a CONFIG_POSIX so you
could use that.

>
>  #ifdef CONFIG_WIN32
> diff --git a/target/i386/hax-darwin.c b/target/i386/hax-posix.c
> similarity index 100%
> rename from target/i386/hax-darwin.c
> rename to target/i386/hax-posix.c
> diff --git a/target/i386/hax-darwin.h b/target/i386/hax-posix.h
> similarity index 100%
> rename from target/i386/hax-darwin.h
> rename to target/i386/hax-posix.h


--
Alex Bennée
Paolo Bonzini Nov. 16, 2018, 12:52 p.m. UTC | #2
On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
> the corresponding userland changes.
> 
> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
> the naming used in the rest of QEMU.

What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
avoiding bitrot?

Paolo
Alexandro Sanchez Nov. 16, 2018, 3 p.m. UTC | #3
Hey Paolo,

> What's the advantage of HAXM when Linux hosts can just run KVM?

For end-users, it's true that there's several disadvantages and barely any
advantages: There's some guests run only on HAXM, since it can handle MMIO
accesses using BMI instructions ...and that's about it. :-) The HAXM API
also offers protecting/unprotecting arbitrary chunks of guest memory, which
has some advantages over KVM_GET_DIRTY_LOG. It's used in [1], but has not
been upstreamed yet.

For developers, there's few advantages over KVM that make work slightly
easier/simpler:
- No need to recompile+install a kernel, but just build+load a kernel
module, which takes few seconds, e.g. [2]. I'm aware of your kvm-kmod
(really helpful!), but Linux updates require patching it, which in turn
makes it incompatible with older Linux releases.
- Automated building, testing (and soon nightly builds as well).
- Extensive emulator tests [3] (generic virtualization tests also on the
way).
- Inspired by Bareflank, support for 3rd party "extensions". [4]

Of course, I'm aware that KVM is much more mature/compatible/performant
that HAXM and many other alternatives. My point is that it might be a bit
more comfortable to work with HAXM (and we are trying to improve this
further every day).

Another point concerns what I feel is the raison d'etre for HAXM:

There seems to be an unnecessary fragmentation of hypervisor APIs: KVM
(Linux), HVF (Mac), WHPX (Windows), which resembles the fragmentation of
graphics APIs. Afaik, no cross-platform type-2 hypervisor exists aside from
HAXM and Bareflank. This is fine for QEMU, since all major accelerators are
implemented, but other smaller projects might suffer from this. Of course,
we could wrap all APIs under a single userland library, but aside from
adding another layer of abstraction/translation, the underlying problem
persists: Each new feature needs to be re-implemented inside each
hypervisor (and HVF/WHPX are closed-source!).

Type-2 hypervisors don't *need* to be tightly coupled to a specific kernel:
If you look at HAXM, a thin platform-specific layer exists, the rest is
shared code. I believe that's much more convenient.

Cheers,
Alexandro

[1]
https://android-review.googlesource.com/c/platform/external/qemu/+/643674/
[2] https://travis-ci.com/intel/haxm/builds/91615353
[3] https://github.com/intel/haxm/blob/master/tests/test_emulator.cpp
[4] https://github.com/intel/haxm/pull/121


On Fri, Nov 16, 2018 at 1:53 PM Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
> > Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch
> includes
> > the corresponding userland changes.
> >
> > Since the Darwin userland backend is POSIX-compliant, the
> hax-darwin.{c,h}
> > files have been renamed to hax-posix.{c,h}. This prefix is consistent
> with
> > the naming used in the rest of QEMU.
>
> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
> avoiding bitrot?
>
> Paolo
>
Kamil Rytarowski Nov. 22, 2018, 7:24 a.m. UTC | #4
On 16.11.2018 13:52, Paolo Bonzini wrote:
> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>> the corresponding userland changes.
>>
>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>> the naming used in the rest of QEMU.
> 
> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
> avoiding bitrot?
> 
> Paolo
> 

This patch is also useful for NetBSD, even if it's not a Linux host.
There is a driver in progress again (thanks to the newly added Linux
port, it's now much easier to get done).

I recommend to merge this patch.
Kamil Rytarowski Nov. 24, 2018, 11:50 p.m. UTC | #5
On 22.11.2018 08:24, Kamil Rytarowski wrote:
> On 16.11.2018 13:52, Paolo Bonzini wrote:
>> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>>> the corresponding userland changes.
>>>
>>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>>> the naming used in the rest of QEMU.
>>
>> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
>> avoiding bitrot?
>>
>> Paolo
>>
> 
> This patch is also useful for NetBSD, even if it's not a Linux host.
> There is a driver in progress again (thanks to the newly added Linux
> port, it's now much easier to get done).
> 
> I recommend to merge this patch.
> 

For the record, I've a functional version of HAXM for NetBSD as host.
Once you will merge this patch, I will submit another one to configure
to enable haxm for NetBSD.

I need to keep the patch by Alexandro in a local copy of qemu.
Paolo Bonzini Nov. 25, 2018, 5:14 p.m. UTC | #6
On 25/11/18 00:50, Kamil Rytarowski wrote:
> On 22.11.2018 08:24, Kamil Rytarowski wrote:
>> On 16.11.2018 13:52, Paolo Bonzini wrote:
>>> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>>>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>>>> the corresponding userland changes.
>>>>
>>>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>>>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>>>> the naming used in the rest of QEMU.
>>>
>>> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
>>> avoiding bitrot?
>>>
>>> Paolo
>>>
>>
>> This patch is also useful for NetBSD, even if it's not a Linux host.
>> There is a driver in progress again (thanks to the newly added Linux
>> port, it's now much easier to get done).
>>
>> I recommend to merge this patch.
>>
> 
> For the record, I've a functional version of HAXM for NetBSD as host.
> Once you will merge this patch, I will submit another one to configure
> to enable haxm for NetBSD.
> 
> I need to keep the patch by Alexandro in a local copy of qemu.

Sure, it will be accepted for the release after 3.1.

Paolo
Kamil Rytarowski Jan. 8, 2019, 5:12 p.m. UTC | #7
On 25.11.2018 18:14, Paolo Bonzini wrote:
> On 25/11/18 00:50, Kamil Rytarowski wrote:
>> On 22.11.2018 08:24, Kamil Rytarowski wrote:
>>> On 16.11.2018 13:52, Paolo Bonzini wrote:
>>>> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>>>>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>>>>> the corresponding userland changes.
>>>>>
>>>>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>>>>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>>>>> the naming used in the rest of QEMU.
>>>>
>>>> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
>>>> avoiding bitrot?
>>>>
>>>> Paolo
>>>>
>>>
>>> This patch is also useful for NetBSD, even if it's not a Linux host.
>>> There is a driver in progress again (thanks to the newly added Linux
>>> port, it's now much easier to get done).
>>>
>>> I recommend to merge this patch.
>>>
>>
>> For the record, I've a functional version of HAXM for NetBSD as host.
>> Once you will merge this patch, I will submit another one to configure
>> to enable haxm for NetBSD.
>>
>> I need to keep the patch by Alexandro in a local copy of qemu.
> 
> Sure, it will be accepted for the release after 3.1.
> 
> Paolo
> 
> 

Ping.
Paolo Bonzini Jan. 8, 2019, 6:27 p.m. UTC | #8
On 08/01/19 18:12, Kamil Rytarowski wrote:
> On 25.11.2018 18:14, Paolo Bonzini wrote:
>> On 25/11/18 00:50, Kamil Rytarowski wrote:
>>> On 22.11.2018 08:24, Kamil Rytarowski wrote:
>>>> On 16.11.2018 13:52, Paolo Bonzini wrote:
>>>>> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>>>>>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>>>>>> the corresponding userland changes.
>>>>>>
>>>>>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>>>>>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>>>>>> the naming used in the rest of QEMU.
>>>>>
>>>>> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
>>>>> avoiding bitrot?
>>>>>
>>>>> Paolo
>>>>>
>>>>
>>>> This patch is also useful for NetBSD, even if it's not a Linux host.
>>>> There is a driver in progress again (thanks to the newly added Linux
>>>> port, it's now much easier to get done).
>>>>
>>>> I recommend to merge this patch.
>>>>
>>>
>>> For the record, I've a functional version of HAXM for NetBSD as host.
>>> Once you will merge this patch, I will submit another one to configure
>>> to enable haxm for NetBSD.
>>>
>>> I need to keep the patch by Alexandro in a local copy of qemu.
>>
>> Sure, it will be accepted for the release after 3.1.
>>
>> Paolo
>>
>>
> 
> Ping.

It's already queued, but the pull request was pretty big and it had some
issues (quite an understatement).

Paolo
Philippe Mathieu-Daudé Jan. 9, 2019, 11 a.m. UTC | #9
On 1/8/19 7:27 PM, Paolo Bonzini wrote:
> On 08/01/19 18:12, Kamil Rytarowski wrote:
>> On 25.11.2018 18:14, Paolo Bonzini wrote:
>>> On 25/11/18 00:50, Kamil Rytarowski wrote:
>>>> On 22.11.2018 08:24, Kamil Rytarowski wrote:
>>>>> On 16.11.2018 13:52, Paolo Bonzini wrote:
>>>>>> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>>>>>>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>>>>>>> the corresponding userland changes.
>>>>>>>
>>>>>>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>>>>>>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>>>>>>> the naming used in the rest of QEMU.
>>>>>>
>>>>>> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
>>>>>> avoiding bitrot?
>>>>>>
>>>>>> Paolo
>>>>>>
>>>>>
>>>>> This patch is also useful for NetBSD, even if it's not a Linux host.
>>>>> There is a driver in progress again (thanks to the newly added Linux
>>>>> port, it's now much easier to get done).
>>>>>
>>>>> I recommend to merge this patch.
>>>>>
>>>>
>>>> For the record, I've a functional version of HAXM for NetBSD as host.
>>>> Once you will merge this patch, I will submit another one to configure
>>>> to enable haxm for NetBSD.
>>>>
>>>> I need to keep the patch by Alexandro in a local copy of qemu.
>>>
>>> Sure, it will be accepted for the release after 3.1.
>>>
>>> Paolo
>>>
>>>
>>
>> Ping.
> 
> It's already queued, but the pull request was pretty big and it had some
> issues (quite an understatement).

Hopefully you queued Alexandro's v2 (which isn't named v2) with Alex
comments fixed, thanks!

Phil.
Kamil Rytarowski Feb. 2, 2019, 3:03 p.m. UTC | #10
On 25.11.2018 18:14, Paolo Bonzini wrote:
> On 25/11/18 00:50, Kamil Rytarowski wrote:
>> On 22.11.2018 08:24, Kamil Rytarowski wrote:
>>> On 16.11.2018 13:52, Paolo Bonzini wrote:
>>>> On 14/11/18 14:04, Alexandro Sanchez Bach wrote:
>>>>> Intel HAXM supports now 32-bit and 64-bit Linux hosts. This patch includes
>>>>> the corresponding userland changes.
>>>>>
>>>>> Since the Darwin userland backend is POSIX-compliant, the hax-darwin.{c,h}
>>>>> files have been renamed to hax-posix.{c,h}. This prefix is consistent with
>>>>> the naming used in the rest of QEMU.
>>>>
>>>> What's the advantage of HAXM when Linux hosts can just run KVM?  I guess
>>>> avoiding bitrot?
>>>>
>>>> Paolo
>>>>
>>>
>>> This patch is also useful for NetBSD, even if it's not a Linux host.
>>> There is a driver in progress again (thanks to the newly added Linux
>>> port, it's now much easier to get done).
>>>
>>> I recommend to merge this patch.
>>>
>>
>> For the record, I've a functional version of HAXM for NetBSD as host.
>> Once you will merge this patch, I will submit another one to configure
>> to enable haxm for NetBSD.
>>
>> I need to keep the patch by Alexandro in a local copy of qemu.
> 
> Sure, it will be accepted for the release after 3.1.
> 
> Paolo
> 
> 

I've pushed the haxm patch for NetBSD through qemu-trivial.

https://lists.gnu.org/archive/html/qemu-trivial/2019-01/msg00161.html

A proof that it is usable:

http://blog.netbsd.org/tnf/entry/the_hardware_assisted_virtualization_challenge
diff mbox series

Patch

diff --git a/target/i386/Makefile.objs b/target/i386/Makefile.objs
index 04678f5503..be36d31d23 100644
--- a/target/i386/Makefile.objs
+++ b/target/i386/Makefile.objs
@@ -12,8 +12,11 @@  obj-$(call lnot,$(CONFIG_SEV)) += sev-stub.o
 ifdef CONFIG_WIN32
 obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-windows.o
 endif
+ifdef CONFIG_LINUX
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
+endif
 ifdef CONFIG_DARWIN
-obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-darwin.o
+obj-$(CONFIG_HAX) += hax-all.o hax-mem.o hax-posix.o
 obj-$(CONFIG_HVF) += hvf/
 endif
 obj-$(CONFIG_WHPX) += whpx-all.o
diff --git a/target/i386/hax-i386.h b/target/i386/hax-i386.h
index 6abc156f88..f87a64c9df 100644
--- a/target/i386/hax-i386.h
+++ b/target/i386/hax-i386.h
@@ -20,6 +20,10 @@ 
 typedef int hax_fd;
 #endif
 
+#ifdef CONFIG_LINUX
+typedef int hax_fd;
+#endif
+
 #ifdef CONFIG_WIN32
 typedef HANDLE hax_fd;
 #endif
@@ -83,7 +87,11 @@  void hax_memory_init(void);
 
 
 #ifdef CONFIG_DARWIN
-#include "target/i386/hax-darwin.h"
+#include "target/i386/hax-posix.h"
+#endif
+
+#ifdef CONFIG_LINUX
+#include "target/i386/hax-posix.h"
 #endif
 
 #ifdef CONFIG_WIN32
diff --git a/target/i386/hax-darwin.c b/target/i386/hax-posix.c
similarity index 100%
rename from target/i386/hax-darwin.c
rename to target/i386/hax-posix.c
diff --git a/target/i386/hax-darwin.h b/target/i386/hax-posix.h
similarity index 100%
rename from target/i386/hax-darwin.h
rename to target/i386/hax-posix.h