mbox series

[v8,0/7] KVM: x86: Allow Qemu/KVM to use PVH entry point

Message ID 1544076152-21637-1-git-send-email-maran.wilson@oracle.com (mailing list archive)
Headers show
Series KVM: x86: Allow Qemu/KVM to use PVH entry point | expand

Message

Maran Wilson Dec. 6, 2018, 6:02 a.m. UTC
For certain applications it is desirable to rapidly boot a KVM virtual
machine. In cases where legacy hardware and software support within the
guest is not needed, Qemu should be able to boot directly into the
uncompressed Linux kernel binary without the need to run firmware.

There already exists an ABI to allow this for Xen PVH guests and the ABI
is supported by Linux and FreeBSD:

   https://xenbits.xen.org/docs/unstable/misc/pvh.html

This patch series would enable Qemu to use that same entry point for
booting KVM guests.

Changes from v7:

 (No functional changes from v7 other than rebasing to latest upstream) 
 * Added Review-by tags as provided by Juergen Gross (1,2,3,6,7)
 * Rebasing to upstream 4.18 caused a minor conflict in patch 4 that had
   to be hand merged due to this patch:
      1fe8388 xen: share start flags between PV and PVH
   I just had to make sure we were accounting for the xen_start_flags
   in the new code path.
 * Rebasing to upstream 4.20-rc4 caused a few minor conflicts in patches
   2,3,5,7 that needed to be resolved by hand. The conflicts were due to
   upstream non-functional code cleanup changes in arch/x86/xen/Makefile
   and arch/x86/platform/pvh/enlighten.c due to these patches:
      28c11b0 x86/xen: Move pv irq related functions under CONFIG_XEN_PV
              umbrella
      357d291 x86/xen: Fix boot loader version reported for PVH guests
      3cfa210 xen: don't include <xen/xen.h> from <asm/io.h> and
              <asm/dma-mapping.h>
 * Qemu and qboot RFC patches have been posted to show one example of how
   this functionality can be used. Some preliminary numbers are available
   in those cover letters showing the KVM guest boot time improvement.
      Qemu:
      http://lists.nongnu.org/archive/html/qemu-devel/2018-12/msg00957.html
      qboot:
      http://lists.nongnu.org/archive/html/qemu-devel/2018-12/msg00953.html

Changes from v6:

 * Addressed issues caught by the kbuild test robot:
    - Restored an #include line that had been dropped by mistake (patch 4)
    - Removed a pair of #include lines that were no longer needed in a
      common code file and causing problems for certain 32-bit configs
      (patchs 4 and 7)

Changes from v5:

 * The interface changes to the x86/HVM start info layout have
   now been accepted into the Xen tree.
 * Rebase and merge upstream PVH file changes.
 * (Patch 6) Synced up to the final version of the header file that was
             acked and pulled into the Xen tree.
 * (Patch 1) Fixed typo and removed redundant "def_bool n" line.

Changes from v4:

Note: I've withheld Juergen's earlier "Reviewed-by" tags from patches
1 and 7 since there were minor changes (mostly just addition of
CONFIG_KVM_GUEST_PVH as requested) that came afterwards.

 * Changed subject prefix from RFC to PATCH
 * Added CONFIG_KVM_GUEST_PVH as suggested
 * Relocated the PVH common files to
   arch/x86/platform/pvh/{enlighten.c,head.S}
 * Realized I also needed to move the objtool override for those files
 * Updated a few code comments per reviewer feedback
 * Sent out a patch of the hvm_start_info struct changes against the Xen
   tree since that is the canonical copy of the header. Discussions on
   that thread have resulted in some (non-functional) updates to
   start_info.h (patch 6/7) and those changes are reflected here as well
   in order to keep the files in sync. The header file has since been
   ack'ed for the Xen tree by Jan Beulich.

Changes from v3:

 * Implemented Juergen's suggestion for refactoring and moving the PVH
   code so that CONFIG_XEN is no longer required for booting KVM guests
   via the PVH entry point.
   Functionally, nothing has changed from V3 really, but the patches
   look completely different now because of all the code movement and
   refactoring. Some of these patches can be combined, but I've left
   them very small in some cases to make the refactoring and code
   movement easier to review.
   My approach for refactoring has been to create a PVH entry layer that
   still has understanding and knowledge about Xen vs non-Xen guest types
   so that it can make run time decisions to handle either case, as
   opposed to going all the way and re-writing it to be a completely
   hypervisor agnostic and architecturally pure layer that is separate
   from guest type details. The latter seemed a bit overkill in this
   situation. And I've handled the complexity of having to support
   Qemu/KVM boot of kernels compiled with or without CONFIG_XEN via a
   pair of xen specific __weak routines that can be overridden in kernels
   that support Xen guests. Importantly, the __weak routines are for
   xen specific code only (not generic "guest type" specific code) so
   there is no clashing between xen version of the strong routine and,
   say, a KVM version of the same routine. But I'm sure there are many
   ways to skin this cat, so I'm open to alternate suggestions if there
   is a compelling reason for not using __weak in this situation.

Changes from v2:

 * All structures (including memory map table entries) are padded and
   aligned to an 8 byte boundary.

 * Removed the "packed" attributes and made changes to comments as
   suggested by Jan.

Changes from v1:

 * Adopted Paolo's suggestion for defining a v2 PVH ABI that includes the
   e820 map instead of using the second module entry to pass the table.

 * Cleaned things up a bit to reduce the number of xen vs non-xen special
   cases.


Maran Wilson (7):
  xen/pvh: Split CONFIG_XEN_PVH into CONFIG_PVH and CONFIG_XEN_PVH
  xen/pvh: Move PVH entry code out of Xen specific tree
  xen/pvh: Create a new file for Xen specific PVH code
  xen/pvh: Move Xen specific PVH VM initialization out of common file
  xen/pvh: Move Xen code for getting mem map via hcall out of common
    file
  xen/pvh: Add memory map pointer to hvm_start_info struct
  KVM: x86: Allow Qemu/KVM to use PVH entry point

 MAINTAINERS                                     |   1 +
 arch/x86/Kbuild                                 |   2 +
 arch/x86/Kconfig                                |  14 +++
 arch/x86/kernel/head_64.S                       |   2 +-
 arch/x86/platform/pvh/Makefile                  |   5 +
 arch/x86/platform/pvh/enlighten.c               | 137 ++++++++++++++++++++++++
 arch/x86/{xen/xen-pvh.S => platform/pvh/head.S} |   0
 arch/x86/xen/Kconfig                            |   3 +-
 arch/x86/xen/Makefile                           |   2 -
 arch/x86/xen/enlighten_pvh.c                    |  94 ++++------------
 include/xen/interface/hvm/start_info.h          |  63 ++++++++++-
 include/xen/xen.h                               |   3 +
 12 files changed, 245 insertions(+), 81 deletions(-)
 create mode 100644 arch/x86/platform/pvh/Makefile
 create mode 100644 arch/x86/platform/pvh/enlighten.c
 rename arch/x86/{xen/xen-pvh.S => platform/pvh/head.S} (100%)

Comments

Paolo Bonzini Dec. 6, 2018, 9:21 p.m. UTC | #1
On 06/12/18 07:02, Maran Wilson wrote:
> For certain applications it is desirable to rapidly boot a KVM virtual
> machine. In cases where legacy hardware and software support within the
> guest is not needed, Qemu should be able to boot directly into the
> uncompressed Linux kernel binary without the need to run firmware.
> 
> There already exists an ABI to allow this for Xen PVH guests and the ABI
> is supported by Linux and FreeBSD:
> 
>    https://xenbits.xen.org/docs/unstable/misc/pvh.html
> 
> This patch series would enable Qemu to use that same entry point for
> booting KVM guests.

Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
you going to pick it up for 4.21?

Paolo
Borislav Petkov Dec. 6, 2018, 9:37 p.m. UTC | #2
On Thu, Dec 06, 2018 at 10:21:12PM +0100, Paolo Bonzini wrote:
> Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
> you going to pick it up for 4.21?

Boris me or Boris O.?

:-)
Boris Ostrovsky Dec. 6, 2018, 9:58 p.m. UTC | #3
On 12/6/18 4:37 PM, Borislav Petkov wrote:
> On Thu, Dec 06, 2018 at 10:21:12PM +0100, Paolo Bonzini wrote:
>> Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
>> you going to pick it up for 4.21?
> Boris me or Boris O.?
>
> :-)
>

O. ;-)

There are some minor changes in non-xen x86 code so it would be good to
get x86 maintainers' ack.

And as far as qemu/qboot changes, should we assume that the general
approach is acceptable? I understand that the patches will probably need
to go through some iterations but I want to make sure we have a path
forward there.


-boris
Paolo Bonzini Dec. 6, 2018, 10:14 p.m. UTC | #4
On 06/12/18 22:58, Boris Ostrovsky wrote:
> On 12/6/18 4:37 PM, Borislav Petkov wrote:
>> On Thu, Dec 06, 2018 at 10:21:12PM +0100, Paolo Bonzini wrote:
>>> Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
>>> you going to pick it up for 4.21?
>> Boris me or Boris O.?
>>
>> :-)
>>
> 
> O. ;-)
> 
> There are some minor changes in non-xen x86 code so it would be good to
> get x86 maintainers' ack.

It's not really code, only Kconfig (and I remarked on it just now), but
it doesn't hurt of course.

> And as far as qemu/qboot changes, should we assume that the general
> approach is acceptable? I understand that the patches will probably need
> to go through some iterations but I want to make sure we have a path
> forward there.

Yes, the general approach is fine.  I have already reviewed the qboot
parts, I guess we will also want an option ROM similar to
linuxboot/multiboot for SeaBIOS support but that's simple matter of
programming. :)

Paolo
Borislav Petkov Dec. 7, 2018, 10:25 a.m. UTC | #5
On Thu, Dec 06, 2018 at 11:14:34PM +0100, Paolo Bonzini wrote:
> > There are some minor changes in non-xen x86 code so it would be good to
> > get x86 maintainers' ack.
> 
> It's not really code, only Kconfig (and I remarked on it just now), but
> it doesn't hurt of course.

Yeah, I don't see anything objectionable.

Thx.
Boris Ostrovsky Dec. 7, 2018, 4:07 p.m. UTC | #6
On 12/7/18 5:25 AM, Borislav Petkov wrote:
> On Thu, Dec 06, 2018 at 11:14:34PM +0100, Paolo Bonzini wrote:
>>> There are some minor changes in non-xen x86 code so it would be good to
>>> get x86 maintainers' ack.
>> It's not really code, only Kconfig (and I remarked on it just now), but
>> it doesn't hurt of course.
> Yeah, I don't see anything objectionable.


Can this be considered as an ACK from you?

-boris
Borislav Petkov Dec. 7, 2018, 7:22 p.m. UTC | #7
On Fri, Dec 07, 2018 at 11:07:54AM -0500, Boris Ostrovsky wrote:
> Can this be considered as an ACK from you?

I'll look at v9 next week and add tags, assuming v9 is going to be the
final one, of course.
Maran Wilson Dec. 12, 2018, 6:09 p.m. UTC | #8
On 12/6/2018 1:21 PM, Paolo Bonzini wrote:
> On 06/12/18 07:02, Maran Wilson wrote:
>> For certain applications it is desirable to rapidly boot a KVM virtual
>> machine. In cases where legacy hardware and software support within the
>> guest is not needed, Qemu should be able to boot directly into the
>> uncompressed Linux kernel binary without the need to run firmware.
>>
>> There already exists an ABI to allow this for Xen PVH guests and the ABI
>> is supported by Linux and FreeBSD:
>>
>>     https://xenbits.xen.org/docs/unstable/misc/pvh.html
>>
>> This patch series would enable Qemu to use that same entry point for
>> booting KVM guests.
> Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
> you going to pick it up for 4.21?

Hi Paolo,

Are you still planning on running some testing of your own for these 
patches? Should Boris wait to hear from you before moving forward?

Thanks,
-Maran

> Paolo
Boris Ostrovsky Dec. 12, 2018, 6:17 p.m. UTC | #9
On 12/12/18 1:09 PM, Maran Wilson wrote:
> On 12/6/2018 1:21 PM, Paolo Bonzini wrote:
>> On 06/12/18 07:02, Maran Wilson wrote:
>>> For certain applications it is desirable to rapidly boot a KVM virtual
>>> machine. In cases where legacy hardware and software support within the
>>> guest is not needed, Qemu should be able to boot directly into the
>>> uncompressed Linux kernel binary without the need to run firmware.
>>>
>>> There already exists an ABI to allow this for Xen PVH guests and the
>>> ABI
>>> is supported by Linux and FreeBSD:
>>>
>>>     https://xenbits.xen.org/docs/unstable/misc/pvh.html
>>>
>>> This patch series would enable Qemu to use that same entry point for
>>> booting KVM guests.
>> Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
>> you going to pick it up for 4.21?


Just realized I never answered this --- yes, I am. Now that BorisP.
acked this, having your T-b would be good.

-boris


>
> Hi Paolo,
>
> Are you still planning on running some testing of your own for these
> patches? Should Boris wait to hear from you before moving forward?
>
> Thanks,
> -Maran
>
>> Paolo
>
Paolo Bonzini Dec. 13, 2018, 1:13 p.m. UTC | #10
On 12/12/18 19:09, Maran Wilson wrote:
> On 12/6/2018 1:21 PM, Paolo Bonzini wrote:
>> On 06/12/18 07:02, Maran Wilson wrote:
>>> For certain applications it is desirable to rapidly boot a KVM virtual
>>> machine. In cases where legacy hardware and software support within the
>>> guest is not needed, Qemu should be able to boot directly into the
>>> uncompressed Linux kernel binary without the need to run firmware.
>>>
>>> There already exists an ABI to allow this for Xen PVH guests and the ABI
>>> is supported by Linux and FreeBSD:
>>>
>>>     https://xenbits.xen.org/docs/unstable/misc/pvh.html
>>>
>>> This patch series would enable Qemu to use that same entry point for
>>> booting KVM guests.
>> Thanks!  I should be able to post a Tested-by next Monday.  Boris, are
>> you going to pick it up for 4.21?
> 
> Hi Paolo,
> 
> Are you still planning on running some testing of your own for these
> patches? Should Boris wait to hear from you before moving forward?

He can go ahead with v9, I was just curious about it.

Paolo