[v14,4/9] x86: add multiboot2 protocol support for EFI platforms
diff mbox

Message ID 1486072879-31637-5-git-send-email-daniel.kiper@oracle.com
State New, archived
Headers show

Commit Message

Daniel Kiper Feb. 2, 2017, 10:01 p.m. UTC
This way Xen can be loaded on EFI platforms using GRUB2 and
other boot loaders which support multiboot2 protocol.

Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
---
v14 - suggestions/fixes:
    - mark .init.data section as writable; by the way we must change
      similar definition in xen/arch/x86/boot/x86_64.S because otherwise
      compiler complains about section types conflicts
      (suggested by Jan Beulich),
    - use %r15 instead of %r15d
      (suggested by Jan Beulich),
    - remove redundant add from UEFI stack alignment
      (suggested by Jan Beulich),
    - use "mov (%rsp),%rdi" instead of "pop %rdi/push %rdi"
      (suggested by Jan Beulich),
    - return void instead of paddr_t from efi_multiboot2()
      and then simplify a bit trampoline setup assembly
      (suggested by Jan Beulich),
    - remove "(XEN)" from efi_multiboot2() stub error messages
      (suggested by Jan Beulich),
    - move err from inline assembly OutputOperands to InputOperands in
      stub.c:efi_multiboot2(); this way we avoid following compile time error:
      stub.c: In function ‘efi_multiboot2’:
      stub.c:36:5: error: read-only variable ‘err’ used as ‘asm’ output
           asm volatile(
           ^~~
      issue was introduced by changing err type to "static const CHAR16 __initconst";
      potentially we can revert this change but move to InputOperands looks better IMO;
      even if we are not able to specify %rdx in Clobbers; simply we do not care
      because next instruction after call is hlt
      (discovered by Konrad Rzeszutek Wilk and Marcos Matsunaga),
    - take into account MBI_SPACE_MIN in ASSERT((trampoline_end - trampoline_start) < ...)
      (suggested by Jan Beulich),
    - improve comments
      (suggested by Jan Beulich).

v13 - suggestions/fixes:
    - move vga_text_buffer and efi_platform to .init.data section
      (suggested by Jan Beulich),
    - reduce number of error branches in EFI code in xen/arch/x86/boot/head.S
      (suggested by Jan Beulich),
    - rename run_bs label to .Lrun_bs
      (suggested by Jan Beulich),
    - align the stack as UEFI spec requires
      (suggested by Jan Beulich),
    - change trampoline region memory layout
      (suggested by Jan Beulich),
    - revert changes in efi_arch_pre_exit_boot()
      (suggested by Jan Beulich),
    - relocate_trampoline() must set trampoline_phys for all bootloaders;
      otherwise fallback allocator is always used if Xen was loaded with
      Multiboot2 protocol,
    - change err type in efi_multiboot2() to "static const CHAR16 __initconst"
      (suggested by Jan Beulich),
    - change asm "g" constraint to "rm" in efi_multiboot2()
      (suggested by Jan Beulich),
    - improve comments
      (suggested by Jan Beulich and Doug Goldstein).

v12 - suggestions/fixes:
    - rename __efi64_start to __efi64_mb2_start
      (suggested by Andrew Cooper),
    - use efi_arch_memory_setup() machinery as trampoline
      et consortes main memory allocator
      (suggested by Doug Goldstein),
    - allocate space for mbi struct in efi_arch_memory_setup() too;
      this thing was not taken into account in earlier releases,
    - revert trampoline et consortes fallback memory allocator code
      in efi_arch_process_memory_map() to current upstream state
      (suggested by Doug Goldstein),
    - further simplify efi_arch_pre_exit_boot() code,
    - call efi_arch_memory_setup() from efi_multiboot2()
      (suggested by Doug Goldstein),
    - fix asembly call argument in xen/arch/x86/efi/stub.c
      (suggested by Andrew Cooper),
    - add ASSERT() for trampoline size
      (suggested by Doug Goldstein),
    - add KB() macro
      (suggested by Doug Goldstein),
    - improve comments
      (suggested by Andrew Cooper and Doug Goldstein).

v10 - suggestions/fixes:
    - replace ljmpl with lretq
      (suggested by Andrew Cooper),
    - introduce efi_platform to increase code readability
      (suggested by Andrew Cooper).

v9 - suggestions/fixes:
   - use .L labels instead of numeric ones in multiboot2 data scanning loops
     (suggested by Jan Beulich).

v8 - suggestions/fixes:
   - use __bss_start(%rip)/__bss_end(%rip) instead of
     of .startof.(.bss)(%rip)/$.sizeof.(.bss) because
     latter is not tested extensively in different
     built environments yet
     (suggested by Andrew Cooper),
   - fix multiboot2 data scanning loop in x86_32 code
     (suggested by Jan Beulich),
   - add check for extra mem for mbi data if Xen is loaded
     via multiboot2 protocol on EFI platform
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Jan Beulich).

v7 - suggestions/fixes:
   - do not allocate twice memory for trampoline if we were
     loaded via multiboot2 protocol on EFI platform,
   - wrap long line
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Jan Beulich).

v6 - suggestions/fixes:
   - improve label names in assembly
     error printing code
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Jan Beulich),
   - various minor cleanups and fixes
     (suggested by Jan Beulich).

v4 - suggestions/fixes:
   - remove redundant BSS alignment,
   - update BSS alignment check,
   - use __set_bit() instead of set_bit() if possible
     (suggested by Jan Beulich),
   - call efi_arch_cpu() from efi_multiboot2()
     even if the same work is done later in
     other place right now
     (suggested by Jan Beulich),
   - xen/arch/x86/efi/stub.c:efi_multiboot2()
     fail properly on EFI platforms,
   - do not read data beyond the end of multiboot2
     information in xen/arch/x86/boot/head.S
     (suggested by Jan Beulich),
   - use 32-bit registers in x86_64 code if possible
     (suggested by Jan Beulich),
   - multiboot2 information address is 64-bit
     in x86_64 code, so, treat it is as is
     (suggested by Jan Beulich),
   - use cmovcc if possible,
   - leave only one space between rep and stosq
     (suggested by Jan Beulich),
   - improve error handling,
   - improve early error messages,
     (suggested by Jan Beulich),
   - improve early error messages printing code,
   - improve label names
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Jan Beulich),
   - various minor cleanups.

v3 - suggestions/fixes:
   - take into account alignment when skipping multiboot2 fixed part
     (suggested by Konrad Rzeszutek Wilk),
   - improve segment registers initialization
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Jan Beulich and Konrad Rzeszutek Wilk),
   - improve commit message
     (suggested by Jan Beulich).

v2 - suggestions/fixes:
   - generate multiboot2 header using macros
     (suggested by Jan Beulich),
   - switch CPU to x86_32 mode before
     jumping to 32-bit code
     (suggested by Andrew Cooper),
   - reduce code changes to increase patch readability
     (suggested by Jan Beulich),
   - improve comments
     (suggested by Jan Beulich),
   - ignore MULTIBOOT2_TAG_TYPE_BASIC_MEMINFO tag on EFI platform
     and find on my own multiboot2.mem_lower value,
   - stop execution if EFI platform is detected
     in legacy BIOS path.
---
 xen/arch/x86/boot/head.S          |  297 ++++++++++++++++++++++++++++++++++---
 xen/arch/x86/boot/reloc.c         |    2 +-
 xen/arch/x86/boot/x86_64.S        |    2 +-
 xen/arch/x86/efi/efi-boot.h       |   49 +++++-
 xen/arch/x86/efi/stub.c           |   39 +++++
 xen/arch/x86/x86_64/asm-offsets.c |    2 +
 xen/arch/x86/xen.lds.S            |    7 +-
 xen/common/efi/boot.c             |   11 ++
 xen/include/asm-x86/config.h      |    5 +
 xen/include/xen/config.h          |    1 +
 10 files changed, 392 insertions(+), 23 deletions(-)

Comments

Jan Beulich Feb. 6, 2017, 10:18 a.m. UTC | #1
>>> On 02.02.17 at 23:01, <daniel.kiper@oracle.com> wrote:
> This way Xen can be loaded on EFI platforms using GRUB2 and
> other boot loaders which support multiboot2 protocol.
> 
> Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
> ---
> v14 - suggestions/fixes:
>     - mark .init.data section as writable; by the way we must change
>       similar definition in xen/arch/x86/boot/x86_64.S because otherwise
>       compiler complains about section types conflicts

Did you observe a problem here, or was this just based on memory
from having seen such in other cases? I ask because I doubt there
would be any problem here, as the compiler isn't involved: This is
assembly code, and iirc you validly talk about a compiler diagnostic.
Hence the wrong attributes want correcting, but in a separate
patch.

>          .section .init.text, "ax", @progbits
>  
>  bad_cpu:
>          mov     $(sym_phys(.Lbad_cpu_msg)),%esi # Error message
> -        jmp     print_err
> +        jmp     .Lget_vtb
>  not_multiboot:
>          mov     $(sym_phys(.Lbad_ldr_msg)),%esi # Error message
> -print_err:
> -        mov     $0xB8000,%edi  # VGA framebuffer
> -1:      mov     (%esi),%bl
> +        jmp     .Lget_vtb
> +.Lmb2_no_st:
> +        mov     $(sym_phys(.Lbad_ldr_nst)),%esi # Error message
> +        jmp     .Lget_vtb
> +.Lmb2_no_ih:
> +        mov     $(sym_phys(.Lbad_ldr_nih)),%esi # Error message
> +        jmp     .Lget_vtb
> +.Lmb2_no_bs:
> +        mov     $(sym_phys(.Lbad_ldr_nbs)),%esi # Error message
> +        xor     %edi,%edi                       # No VGA text buffer
> +        jmp     .Lsend_chr
> +.Lmb2_efi_ia_32:
> +        mov     $(sym_phys(.Lbad_efi_msg)),%esi # Error message
> +        xor     %edi,%edi                       # No VGA text buffer
> +        jmp     .Lsend_chr

I continue to have problems to understand when you use / don't use
the VGA buffer here: I think at the very least we want a comment as
to when which of the two models applies. The situation isn't being
helped by .Lmb2_no_bs reachable via two paths (one neighboring a
use of .Lmb2_efi_ia_32, and another neighboring uses of
.Lmb2_no_st and .Lmb2_no_ih). Since you zap vga_text_buffer in
the EFI case, couldn't you simply use that one instead of clearing
%edi here?

> +void __init noreturn efi_multiboot2(EFI_HANDLE ImageHandle,
> +                                    EFI_SYSTEM_TABLE *SystemTable)
> +{
> +    static const CHAR16 __initconst err[] =
> +        L"Xen does not have EFI code build in!\r\nSystem halted!\r\n";
> +    SIMPLE_TEXT_OUTPUT_INTERFACE *StdErr;
> +
> +    StdErr = SystemTable->StdErr ? SystemTable->StdErr : SystemTable->ConOut;
> +
> +    /*
> +     * Print error message and halt the system.
> +     *
> +     * We have to open code MS x64 calling convention
> +     * in assembly because here this convention may
> +     * not be directly supported by C compiler.
> +     */
> +    asm volatile(
> +    "    call *%2                     \n"
> +    "0:  hlt                          \n"
> +    "    jmp  0b                      \n"
> +       : "+c" (StdErr) : "d" (err), "rm" (StdErr->OutputString)
> +       : "rax", "r8", "r9", "r10", "r11", "memory");

Regardless of the hlt I think this would better be correct: As done
in various other places, you simply need to indicate "=d" as a
(dummy) output; since you clobber StdErr anyway, you could
simply use that one:

    asm volatile(
    "    call *%2                     \n"
    "0:  hlt                          \n"
    "    jmp  0b                      \n"
       : "+c" (StdErr), "=d" (StdErr) : "1" (err), "rm" (StdErr->OutputString)
       : "rax", "r8", "r9", "r10", "r11", "memory");

Otherwise I'd ask for consistency, i.e. for %rcx to be an input only,
too.

Jan
Daniel Kiper Feb. 8, 2017, 1:44 p.m. UTC | #2
On Mon, Feb 06, 2017 at 03:18:53AM -0700, Jan Beulich wrote:
> >>> On 02.02.17 at 23:01, <daniel.kiper@oracle.com> wrote:
> > This way Xen can be loaded on EFI platforms using GRUB2 and
> > other boot loaders which support multiboot2 protocol.
> >
> > Signed-off-by: Daniel Kiper <daniel.kiper@oracle.com>
> > ---
> > v14 - suggestions/fixes:
> >     - mark .init.data section as writable; by the way we must change
> >       similar definition in xen/arch/x86/boot/x86_64.S because otherwise
> >       compiler complains about section types conflicts
>
> Did you observe a problem here, or was this just based on memory
> from having seen such in other cases? I ask because I doubt there

It happened here. Though I am not able to reproduce it now... Hmmm...

> would be any problem here, as the compiler isn't involved: This is
> assembly code, and iirc you validly talk about a compiler diagnostic.
> Hence the wrong attributes want correcting, but in a separate
> patch.

OK.

> >          .section .init.text, "ax", @progbits
> >
> >  bad_cpu:
> >          mov     $(sym_phys(.Lbad_cpu_msg)),%esi # Error message
> > -        jmp     print_err
> > +        jmp     .Lget_vtb
> >  not_multiboot:
> >          mov     $(sym_phys(.Lbad_ldr_msg)),%esi # Error message
> > -print_err:
> > -        mov     $0xB8000,%edi  # VGA framebuffer
> > -1:      mov     (%esi),%bl
> > +        jmp     .Lget_vtb
> > +.Lmb2_no_st:
> > +        mov     $(sym_phys(.Lbad_ldr_nst)),%esi # Error message
> > +        jmp     .Lget_vtb
> > +.Lmb2_no_ih:
> > +        mov     $(sym_phys(.Lbad_ldr_nih)),%esi # Error message
> > +        jmp     .Lget_vtb
> > +.Lmb2_no_bs:
> > +        mov     $(sym_phys(.Lbad_ldr_nbs)),%esi # Error message
> > +        xor     %edi,%edi                       # No VGA text buffer
> > +        jmp     .Lsend_chr
> > +.Lmb2_efi_ia_32:
> > +        mov     $(sym_phys(.Lbad_efi_msg)),%esi # Error message
> > +        xor     %edi,%edi                       # No VGA text buffer
> > +        jmp     .Lsend_chr
>
> I continue to have problems to understand when you use / don't use
> the VGA buffer here: I think at the very least we want a comment as
> to when which of the two models applies. The situation isn't being
> helped by .Lmb2_no_bs reachable via two paths (one neighboring a
> use of .Lmb2_efi_ia_32, and another neighboring uses of
> .Lmb2_no_st and .Lmb2_no_ih). Since you zap vga_text_buffer in
> the EFI case, couldn't you simply use that one instead of clearing
> %edi here?

Here it is much simpler to do. Otherwise we must move jumps to
.Lmb2_no_bs and .Lmb2_efi_ia_32 behind Multiboot2 scanning loop.
Earlier we could not have MB2_load_base_addr value which is needed
to calculate real vga_text_buffer address in 32-bit mode. Ahhh...
This is not obvious when you look at this patch alone because
MB2_load_base_addr is added by "x86: add multiboot2 protocol
support for relocatable images" patch.

> > +void __init noreturn efi_multiboot2(EFI_HANDLE ImageHandle,
> > +                                    EFI_SYSTEM_TABLE *SystemTable)
> > +{
> > +    static const CHAR16 __initconst err[] =
> > +        L"Xen does not have EFI code build in!\r\nSystem halted!\r\n";
> > +    SIMPLE_TEXT_OUTPUT_INTERFACE *StdErr;
> > +
> > +    StdErr = SystemTable->StdErr ? SystemTable->StdErr : SystemTable->ConOut;
> > +
> > +    /*
> > +     * Print error message and halt the system.
> > +     *
> > +     * We have to open code MS x64 calling convention
> > +     * in assembly because here this convention may
> > +     * not be directly supported by C compiler.
> > +     */
> > +    asm volatile(
> > +    "    call *%2                     \n"
> > +    "0:  hlt                          \n"
> > +    "    jmp  0b                      \n"
> > +       : "+c" (StdErr) : "d" (err), "rm" (StdErr->OutputString)
> > +       : "rax", "r8", "r9", "r10", "r11", "memory");
>
> Regardless of the hlt I think this would better be correct: As done
> in various other places, you simply need to indicate "=d" as a
> (dummy) output; since you clobber StdErr anyway, you could
> simply use that one:
>
>     asm volatile(
>     "    call *%2                     \n"
>     "0:  hlt                          \n"
>     "    jmp  0b                      \n"
>        : "+c" (StdErr), "=d" (StdErr) : "1" (err), "rm" (StdErr->OutputString)
>        : "rax", "r8", "r9", "r10", "r11", "memory");
>
> Otherwise I'd ask for consistency, i.e. for %rcx to be an input only,
> too.

OK, I will try.

Daniel
Jan Beulich Feb. 8, 2017, 1:52 p.m. UTC | #3
>>> On 08.02.17 at 14:44, <daniel.kiper@oracle.com> wrote:
> On Mon, Feb 06, 2017 at 03:18:53AM -0700, Jan Beulich wrote:
>> >>> On 02.02.17 at 23:01, <daniel.kiper@oracle.com> wrote:
>> >  bad_cpu:
>> >          mov     $(sym_phys(.Lbad_cpu_msg)),%esi # Error message
>> > -        jmp     print_err
>> > +        jmp     .Lget_vtb
>> >  not_multiboot:
>> >          mov     $(sym_phys(.Lbad_ldr_msg)),%esi # Error message
>> > -print_err:
>> > -        mov     $0xB8000,%edi  # VGA framebuffer
>> > -1:      mov     (%esi),%bl
>> > +        jmp     .Lget_vtb
>> > +.Lmb2_no_st:
>> > +        mov     $(sym_phys(.Lbad_ldr_nst)),%esi # Error message
>> > +        jmp     .Lget_vtb
>> > +.Lmb2_no_ih:
>> > +        mov     $(sym_phys(.Lbad_ldr_nih)),%esi # Error message
>> > +        jmp     .Lget_vtb
>> > +.Lmb2_no_bs:
>> > +        mov     $(sym_phys(.Lbad_ldr_nbs)),%esi # Error message
>> > +        xor     %edi,%edi                       # No VGA text buffer
>> > +        jmp     .Lsend_chr
>> > +.Lmb2_efi_ia_32:
>> > +        mov     $(sym_phys(.Lbad_efi_msg)),%esi # Error message
>> > +        xor     %edi,%edi                       # No VGA text buffer
>> > +        jmp     .Lsend_chr
>>
>> I continue to have problems to understand when you use / don't use
>> the VGA buffer here: I think at the very least we want a comment as
>> to when which of the two models applies. The situation isn't being
>> helped by .Lmb2_no_bs reachable via two paths (one neighboring a
>> use of .Lmb2_efi_ia_32, and another neighboring uses of
>> .Lmb2_no_st and .Lmb2_no_ih). Since you zap vga_text_buffer in
>> the EFI case, couldn't you simply use that one instead of clearing
>> %edi here?
> 
> Here it is much simpler to do. Otherwise we must move jumps to
> .Lmb2_no_bs and .Lmb2_efi_ia_32 behind Multiboot2 scanning loop.
> Earlier we could not have MB2_load_base_addr value which is needed
> to calculate real vga_text_buffer address in 32-bit mode. Ahhh...
> This is not obvious when you look at this patch alone because
> MB2_load_base_addr is added by "x86: add multiboot2 protocol
> support for relocatable images" patch.

Well, it the shape things are in is required only by a later patch,
either the respective pieces should be moved there, or some
enlightening comment should be added here.

Jan

Patch
diff mbox

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 84cf44d..5147204 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -89,6 +89,13 @@  multiboot2_header_start:
                    0, /* Number of the lines - no preference. */ \
                    0  /* Number of bits per pixel - no preference. */
 
+        /* Request that ExitBootServices() not be called. */
+        mb2ht_init MB2_HT(EFI_BS), MB2_HT(OPTIONAL)
+
+        /* EFI64 Multiboot2 entry point. */
+        mb2ht_init MB2_HT(ENTRY_ADDRESS_EFI64), MB2_HT(OPTIONAL), \
+                   sym_phys(__efi64_mb2_start)
+
         /* Multiboot2 header end tag. */
         mb2ht_init MB2_HT(END), MB2_HT(REQUIRED)
 .Lmultiboot2_header_end:
@@ -100,20 +107,50 @@  multiboot2_header_start:
 gdt_boot_descr:
         .word   6*8-1
         .long   sym_phys(trampoline_gdt)
+        .long   0 /* Needed for 64-bit lgdt */
 
 .Lbad_cpu_msg: .asciz "ERR: Not a 64-bit CPU!"
 .Lbad_ldr_msg: .asciz "ERR: Not a Multiboot bootloader!"
+.Lbad_ldr_nbs: .asciz "ERR: Bootloader shutdown EFI x64 boot services!"
+.Lbad_ldr_nst: .asciz "ERR: EFI SystemTable is not provided by bootloader!"
+.Lbad_ldr_nih: .asciz "ERR: EFI ImageHandle is not provided by bootloader!"
+.Lbad_efi_msg: .asciz "ERR: EFI IA-32 platforms are not supported!"
+
+        .section .init.data, "aw", @progbits
+        .align 4
+
+vga_text_buffer:
+        .long   0xb8000
+
+efi_platform:
+        .byte   0
 
         .section .init.text, "ax", @progbits
 
 bad_cpu:
         mov     $(sym_phys(.Lbad_cpu_msg)),%esi # Error message
-        jmp     print_err
+        jmp     .Lget_vtb
 not_multiboot:
         mov     $(sym_phys(.Lbad_ldr_msg)),%esi # Error message
-print_err:
-        mov     $0xB8000,%edi  # VGA framebuffer
-1:      mov     (%esi),%bl
+        jmp     .Lget_vtb
+.Lmb2_no_st:
+        mov     $(sym_phys(.Lbad_ldr_nst)),%esi # Error message
+        jmp     .Lget_vtb
+.Lmb2_no_ih:
+        mov     $(sym_phys(.Lbad_ldr_nih)),%esi # Error message
+        jmp     .Lget_vtb
+.Lmb2_no_bs:
+        mov     $(sym_phys(.Lbad_ldr_nbs)),%esi # Error message
+        xor     %edi,%edi                       # No VGA text buffer
+        jmp     .Lsend_chr
+.Lmb2_efi_ia_32:
+        mov     $(sym_phys(.Lbad_efi_msg)),%esi # Error message
+        xor     %edi,%edi                       # No VGA text buffer
+        jmp     .Lsend_chr
+.Lget_vtb:
+        mov     sym_phys(vga_text_buffer),%edi
+.Lsend_chr:
+        mov     (%esi),%bl
         test    %bl,%bl        # Terminate on '\0' sentinel
         je      .Lhalt
         mov     $0x3f8+5,%dx   # UART Line Status Register
@@ -123,13 +160,192 @@  print_err:
         mov     $0x3f8+0,%dx   # UART Transmit Holding Register
         mov     %bl,%al
         out     %al,%dx        # Send a character over the serial line
-        movsb                  # Write a character to the VGA framebuffer
+        test    %edi,%edi      # Is the VGA text buffer available?
+        jz      .Lsend_chr
+        movsb                  # Write a character to the VGA text buffer
         mov     $7,%al
-        stosb                  # Write an attribute to the VGA framebuffer
-        jmp     1b
+        stosb                  # Write an attribute to the VGA text buffer
+        jmp     .Lsend_chr
 .Lhalt: hlt
         jmp     .Lhalt
 
+        .code64
+
+__efi64_mb2_start:
+        /*
+         * Multiboot2 spec says that here CPU is in 64-bit mode. However,
+         * there is also guarantee that all code and data is always put
+         * by the bootloader below 4 GiB. Hence, we can safely truncate
+         * addresses to 32-bits in most cases below.
+         */
+
+        cld
+
+        /* VGA is not available on EFI platforms. */
+        movl   $0,vga_text_buffer(%rip)
+
+        /* Check for Multiboot2 bootloader. */
+        cmp     $MULTIBOOT2_BOOTLOADER_MAGIC,%eax
+        je      .Lefi_multiboot2_proto
+
+        /* Jump to not_multiboot after switching CPU to x86_32 mode. */
+        lea     not_multiboot(%rip),%r15
+        jmp     x86_32_switch
+
+.Lefi_multiboot2_proto:
+        /* Zero EFI SystemTable and EFI ImageHandle addresses. */
+        xor     %esi,%esi
+        xor     %edi,%edi
+
+        /* Skip Multiboot2 information fixed part. */
+        lea     (MB2_fixed_sizeof+MULTIBOOT2_TAG_ALIGN-1)(%rbx),%ecx
+        and     $~(MULTIBOOT2_TAG_ALIGN-1),%ecx
+
+.Lefi_mb2_tsize:
+        /* Check Multiboot2 information total size. */
+        mov     %ecx,%r8d
+        sub     %ebx,%r8d
+        cmp     %r8d,MB2_fixed_total_size(%rbx)
+        jbe     .Lrun_bs
+
+        /* Are EFI boot services available? */
+        cmpl    $MULTIBOOT2_TAG_TYPE_EFI_BS,MB2_tag_type(%rcx)
+        jne     .Lefi_mb2_st
+
+        /* We are on EFI platform and EFI boot services are available. */
+        incb    efi_platform(%rip)
+
+        /*
+         * Disable real mode and other legacy stuff which should not
+         * be run on EFI platforms.
+         */
+        incb    skip_realmode(%rip)
+        jmp     .Lefi_mb2_next_tag
+
+.Lefi_mb2_st:
+        /* Get EFI SystemTable address from Multiboot2 information. */
+        cmpl    $MULTIBOOT2_TAG_TYPE_EFI64,MB2_tag_type(%rcx)
+        cmove   MB2_efi64_st(%rcx),%rsi
+        je      .Lefi_mb2_next_tag
+
+        /* Get EFI ImageHandle address from Multiboot2 information. */
+        cmpl    $MULTIBOOT2_TAG_TYPE_EFI64_IH,MB2_tag_type(%rcx)
+        cmove   MB2_efi64_ih(%rcx),%rdi
+        je      .Lefi_mb2_next_tag
+
+        /* Is it the end of Multiboot2 information? */
+        cmpl    $MULTIBOOT2_TAG_TYPE_END,MB2_tag_type(%rcx)
+        je      .Lrun_bs
+
+.Lefi_mb2_next_tag:
+        /* Go to next Multiboot2 information tag. */
+        add     MB2_tag_size(%rcx),%ecx
+        add     $(MULTIBOOT2_TAG_ALIGN-1),%ecx
+        and     $~(MULTIBOOT2_TAG_ALIGN-1),%ecx
+        jmp     .Lefi_mb2_tsize
+
+.Lrun_bs:
+        /* Are EFI boot services available? */
+        cmpb    $0,efi_platform(%rip)
+
+        /* Jump to .Lmb2_no_bs after switching CPU to x86_32 mode. */
+        lea     .Lmb2_no_bs(%rip),%r15
+        jz      x86_32_switch
+
+        /* Is EFI SystemTable address provided by boot loader? */
+        test    %rsi,%rsi
+
+        /* Jump to .Lmb2_no_st after switching CPU to x86_32 mode. */
+        lea     .Lmb2_no_st(%rip),%r15
+        jz      x86_32_switch
+
+        /* Is EFI ImageHandle address provided by boot loader? */
+        test    %rdi,%rdi
+
+        /* Jump to .Lmb2_no_ih after switching CPU to x86_32 mode. */
+        lea     .Lmb2_no_ih(%rip),%r15
+        jz      x86_32_switch
+
+        /*
+         * Align the stack as UEFI spec requires. Keep it aligned
+         * before efi_multiboot2() call by pushing/popping even
+         * numbers of items on it.
+         */
+        and     $~15,%rsp
+
+        /* Save Multiboot2 magic on the stack. */
+        push    %rax
+
+        /* Save EFI ImageHandle on the stack. */
+        push    %rdi
+
+        /*
+         * Initialize BSS (no nasty surprises!).
+         * It must be done earlier than in BIOS case
+         * because efi_multiboot2() touches it.
+         */
+        lea     __bss_start(%rip),%edi
+        lea     __bss_end(%rip),%ecx
+        sub     %edi,%ecx
+        shr     $3,%ecx
+        xor     %eax,%eax
+        rep stosq
+
+        /* Keep the stack aligned. Do not pop a single item off it. */
+        mov     (%rsp),%rdi
+
+        /*
+         * efi_multiboot2() is called according to System V AMD64 ABI:
+         *   - IN:  %rdi - EFI ImageHandle, %rsi - EFI SystemTable.
+         */
+        call    efi_multiboot2
+
+        /* Just pop an item from the stack. */
+        pop     %rax
+
+        /* Restore Multiboot2 magic. */
+        pop     %rax
+
+        /* Jump to trampoline_setup after switching CPU to x86_32 mode. */
+        lea     trampoline_setup(%rip),%r15
+
+x86_32_switch:
+        mov     %r15,%rdi
+
+        cli
+
+        /* Initialize GDTR. */
+        lgdt    gdt_boot_descr(%rip)
+
+        /* Reload code selector. */
+        pushq   $BOOT_CS32
+        lea     cs32_switch(%rip),%edx
+        push    %rdx
+        lretq
+
+        .code32
+
+cs32_switch:
+        /* Initialize basic data segments. */
+        mov     $BOOT_DS,%edx
+        mov     %edx,%ds
+        mov     %edx,%es
+        mov     %edx,%ss
+        /* %esp is initialized later. */
+
+        /* Load null descriptor to unused segment registers. */
+        xor     %edx,%edx
+        mov     %edx,%fs
+        mov     %edx,%gs
+
+        /* Disable paging. */
+        mov     %cr0,%edx
+        and     $(~X86_CR0_PG),%edx
+        mov     %edx,%cr0
+
+        /* Jump to earlier loaded address. */
+        jmp     *%edi
+
 __start:
         cld
         cli
@@ -157,7 +373,7 @@  __start:
 
         /* Not available? BDA value will be fine. */
         cmovnz  MB_mem_lower(%ebx),%edx
-        jmp     trampoline_setup
+        jmp     trampoline_bios_setup
 
 .Lmultiboot2_proto:
         /* Skip Multiboot2 information fixed part. */
@@ -169,24 +385,33 @@  __start:
         mov     %ecx,%edi
         sub     %ebx,%edi
         cmp     %edi,MB2_fixed_total_size(%ebx)
-        jbe     trampoline_setup
+        jbe     trampoline_bios_setup
 
         /* Get mem_lower from Multiboot2 information. */
         cmpl    $MULTIBOOT2_TAG_TYPE_BASIC_MEMINFO,MB2_tag_type(%ecx)
         cmove   MB2_mem_lower(%ecx),%edx
-        je      trampoline_setup
+        je      .Lmb2_next_tag
+
+        /* EFI IA-32 platforms are not supported. */
+        cmpl    $MULTIBOOT2_TAG_TYPE_EFI32,MB2_tag_type(%ecx)
+        je      .Lmb2_efi_ia_32
+
+        /* Bootloader shutdown EFI x64 boot services. */
+        cmpl    $MULTIBOOT2_TAG_TYPE_EFI64,MB2_tag_type(%ecx)
+        je      .Lmb2_no_bs
 
         /* Is it the end of Multiboot2 information? */
         cmpl    $MULTIBOOT2_TAG_TYPE_END,MB2_tag_type(%ecx)
-        je      trampoline_setup
+        je      trampoline_bios_setup
 
+.Lmb2_next_tag:
         /* Go to next Multiboot2 information tag. */
         add     MB2_tag_size(%ecx),%ecx
         add     $(MULTIBOOT2_TAG_ALIGN-1),%ecx
         and     $~(MULTIBOOT2_TAG_ALIGN-1),%ecx
         jmp     .Lmb2_tsize
 
-trampoline_setup:
+trampoline_bios_setup:
         /* Set up trampoline segment 64k below EBDA */
         movzwl  0x40e,%ecx          /* EBDA segment */
         cmp     $0xa000,%ecx        /* sanity check (high) */
@@ -207,23 +432,54 @@  trampoline_setup:
         cmp     %ecx,%edx           /* compare with BDA value */
         cmovb   %edx,%ecx           /* and use the smaller */
 
-2:      /* Reserve 64kb for the trampoline */
-        sub     $0x1000,%ecx
+2:
+        /* Reserve memory for the trampoline and the low-memory stack. */
+        sub     $((TRAMPOLINE_SPACE+TRAMPOLINE_STACK_SPACE)>>4),%ecx
 
         /* From arch/x86/smpboot.c: start_eip had better be page-aligned! */
         xor     %cl, %cl
         shl     $4, %ecx
         mov     %ecx,sym_phys(trampoline_phys)
 
+trampoline_setup:
+        mov     sym_phys(trampoline_phys),%ecx
+
+        /* Get bottom-most low-memory stack address. */
+        add     $TRAMPOLINE_SPACE,%ecx
+
         /* Save the Multiboot info struct (after relocation) for later use. */
         mov     $sym_phys(cpu0_stack)+1024,%esp
-        push    %ecx                /* Boot trampoline address. */
+        push    %ecx                /* Bottom-most low-memory stack address. */
         push    %ebx                /* Multiboot information address. */
         push    %eax                /* Multiboot magic. */
         call    reloc
         mov     %eax,sym_phys(multiboot_ptr)
 
-        /* Initialize BSS (no nasty surprises!) */
+        /*
+         * Now trampoline_phys points to the following structure (lowest address
+         * is at the bottom):
+         *
+         * +------------------------+
+         * | TRAMPOLINE_STACK_SPACE |
+         * +------------------------+
+         * |        mbi data        |
+         * +- - - - - - - - - - - - +
+         * |    TRAMPOLINE_SPACE    |
+         * +------------------------+
+         *
+         * mbi data grows downwards from the highest address of TRAMPOLINE_SPACE
+         * region to the end of the trampoline. The rest of TRAMPOLINE_SPACE is
+         * reserved for trampoline code and data.
+         */
+
+        /*
+         * Do not zero BSS on EFI platform here.
+         * It was initialized earlier.
+         */
+        cmpb    $0,sym_phys(efi_platform)
+        jnz     1f
+
+        /* Initialize BSS (no nasty surprises!). */
         mov     $sym_phys(__bss_start),%edi
         mov     $sym_phys(__bss_end),%ecx
         sub     %edi,%ecx
@@ -231,6 +487,7 @@  trampoline_setup:
         shr     $2,%ecx
         rep stosl
 
+1:
         /* Interrogate CPU extended features via CPUID. */
         mov     $0x80000000,%eax
         cpuid
@@ -282,6 +539,10 @@  trampoline_setup:
         cmp     $sym_phys(__trampoline_seg_stop),%edi
         jb      1b
 
+        /* Do not parse command line on EFI platform here. */
+        cmpb    $0,sym_phys(efi_platform)
+        jnz     1f
+
         /* Bail if there is no command line to parse. */
         mov     sym_phys(multiboot_ptr),%ebx
         testl   $MBI_CMDLINE,MB_flags(%ebx)
@@ -292,9 +553,9 @@  trampoline_setup:
         call    cmdline_parse_early
 
 1:
-        /* Switch to low-memory stack.  */
+        /* Switch to low-memory stack which lives at the end of trampoline region. */
         mov     sym_phys(trampoline_phys),%edi
-        lea     0x10000(%edi),%esp
+        lea     TRAMPOLINE_SPACE+TRAMPOLINE_STACK_SPACE(%edi),%esp
         lea     trampoline_boot_cpu_entry-trampoline_start(%edi),%eax
         pushl   $BOOT_CS32
         push    %eax
diff --git a/xen/arch/x86/boot/reloc.c b/xen/arch/x86/boot/reloc.c
index 91fab9d..b992678 100644
--- a/xen/arch/x86/boot/reloc.c
+++ b/xen/arch/x86/boot/reloc.c
@@ -16,7 +16,7 @@ 
  * This entry point is entered from xen/arch/x86/boot/head.S with:
  *   - 0x4(%esp) = MULTIBOOT_MAGIC,
  *   - 0x8(%esp) = MULTIBOOT_INFORMATION_ADDRESS,
- *   - 0xc(%esp) = BOOT_TRAMPOLINE_ADDRESS.
+ *   - 0xc(%esp) = TOPMOST_LOW_MEMORY_STACK_ADDRESS.
  */
 asm (
     "    .text                         \n"
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 139b2ca..4d507fb 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -186,7 +186,7 @@  GLOBAL(idle_pg_table)
 GLOBAL(__page_tables_end)
 
 /* Init pagetables.  Enough page directories to map into the bottom 1GB. */
-        .section .init.data, "a", @progbits
+        .section .init.data, "aw", @progbits
         .align PAGE_SIZE, 0
 
 GLOBAL(l2_bootmap)
diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
index 62c010e..94418bf 100644
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -101,6 +101,10 @@  static void __init relocate_trampoline(unsigned long phys)
     const s32 *trampoline_ptr;
 
     trampoline_phys = phys;
+
+    if ( !efi_enabled(EFI_LOADER) )
+        return;
+
     /* Apply relocations to trampoline. */
     for ( trampoline_ptr = __trampoline_rel_start;
           trampoline_ptr < __trampoline_rel_stop;
@@ -550,7 +554,12 @@  static void __init efi_arch_memory_setup(void)
 
     /* Allocate space for trampoline (in first Mb). */
     cfg.addr = 0x100000;
-    cfg.size = trampoline_end - trampoline_start;
+
+    if ( efi_enabled(EFI_LOADER) )
+        cfg.size = trampoline_end - trampoline_start;
+    else
+        cfg.size = TRAMPOLINE_SPACE + TRAMPOLINE_STACK_SPACE;
+
     status = efi_bs->AllocatePages(AllocateMaxAddress, EfiLoaderData,
                                    PFN_UP(cfg.size), &cfg.addr);
     if ( status == EFI_SUCCESS )
@@ -561,6 +570,9 @@  static void __init efi_arch_memory_setup(void)
         PrintStr(L"Trampoline space cannot be allocated; will try fallback.\r\n");
     }
 
+    if ( !efi_enabled(EFI_LOADER) )
+        return;
+
     /* Initialise L2 identity-map and boot-map page table entries (16MB). */
     for ( i = 0; i < 8; ++i )
     {
@@ -653,6 +665,41 @@  static bool_t __init efi_arch_use_config_file(EFI_SYSTEM_TABLE *SystemTable)
 
 static void efi_arch_flush_dcache_area(const void *vaddr, UINTN size) { }
 
+void __init efi_multiboot2(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
+{
+    EFI_GRAPHICS_OUTPUT_PROTOCOL *gop;
+    UINTN cols, gop_mode = ~0, rows;
+
+    __set_bit(EFI_BOOT, &efi_flags);
+    __set_bit(EFI_RS, &efi_flags);
+
+    efi_init(ImageHandle, SystemTable);
+
+    efi_console_set_mode();
+
+    if ( StdOut->QueryMode(StdOut, StdOut->Mode->Mode,
+                           &cols, &rows) == EFI_SUCCESS )
+        efi_arch_console_init(cols, rows);
+
+    gop = efi_get_gop();
+
+    if ( gop )
+        gop_mode = efi_find_gop_mode(gop, 0, 0, 0);
+
+    efi_arch_edd();
+    efi_arch_cpu();
+
+    efi_tables();
+    setup_efi_pci();
+    efi_variables();
+    efi_arch_memory_setup();
+
+    if ( gop )
+        efi_set_gop_mode(gop, gop_mode);
+
+    efi_exit_boot(ImageHandle, SystemTable);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/arch/x86/efi/stub.c b/xen/arch/x86/efi/stub.c
index 4158124..2127cce 100644
--- a/xen/arch/x86/efi/stub.c
+++ b/xen/arch/x86/efi/stub.c
@@ -3,6 +3,45 @@ 
 #include <xen/init.h>
 #include <xen/lib.h>
 #include <asm/page.h>
+#include <asm/efibind.h>
+#include <efi/efidef.h>
+#include <efi/eficapsule.h>
+#include <efi/eficon.h>
+#include <efi/efidevp.h>
+#include <efi/efiapi.h>
+
+/*
+ * Here we are in EFI stub. EFI calls are not supported due to lack
+ * of relevant functionality in compiler and/or linker.
+ *
+ * efi_multiboot2() is an exception. Please look below for more details.
+ */
+
+void __init noreturn efi_multiboot2(EFI_HANDLE ImageHandle,
+                                    EFI_SYSTEM_TABLE *SystemTable)
+{
+    static const CHAR16 __initconst err[] =
+        L"Xen does not have EFI code build in!\r\nSystem halted!\r\n";
+    SIMPLE_TEXT_OUTPUT_INTERFACE *StdErr;
+
+    StdErr = SystemTable->StdErr ? SystemTable->StdErr : SystemTable->ConOut;
+
+    /*
+     * Print error message and halt the system.
+     *
+     * We have to open code MS x64 calling convention
+     * in assembly because here this convention may
+     * not be directly supported by C compiler.
+     */
+    asm volatile(
+    "    call *%2                     \n"
+    "0:  hlt                          \n"
+    "    jmp  0b                      \n"
+       : "+c" (StdErr) : "d" (err), "rm" (StdErr->OutputString)
+       : "rax", "r8", "r9", "r10", "r11", "memory");
+
+    unreachable();
+}
 
 bool efi_enabled(unsigned int feature)
 {
diff --git a/xen/arch/x86/x86_64/asm-offsets.c b/xen/arch/x86/x86_64/asm-offsets.c
index 92f5d81..f135654 100644
--- a/xen/arch/x86/x86_64/asm-offsets.c
+++ b/xen/arch/x86/x86_64/asm-offsets.c
@@ -175,6 +175,8 @@  void __dummy__(void)
     OFFSET(MB2_tag_type, multiboot2_tag_t, type);
     OFFSET(MB2_tag_size, multiboot2_tag_t, size);
     OFFSET(MB2_mem_lower, multiboot2_tag_basic_meminfo_t, mem_lower);
+    OFFSET(MB2_efi64_st, multiboot2_tag_efi64_t, pointer);
+    OFFSET(MB2_efi64_ih, multiboot2_tag_efi64_ih_t, pointer);
     BLANK();
 
     OFFSET(DOMAIN_vm_assist, struct domain, vm_assist);
diff --git a/xen/arch/x86/xen.lds.S b/xen/arch/x86/xen.lds.S
index b0b1c9b..76e18ab 100644
--- a/xen/arch/x86/xen.lds.S
+++ b/xen/arch/x86/xen.lds.S
@@ -331,5 +331,8 @@  ASSERT(IS_ALIGNED(__init_end,   PAGE_SIZE), "__init_end misaligned")
 
 ASSERT(IS_ALIGNED(trampoline_start, 4), "trampoline_start misaligned")
 ASSERT(IS_ALIGNED(trampoline_end,   4), "trampoline_end misaligned")
-ASSERT(IS_ALIGNED(__bss_start,      4), "__bss_start misaligned")
-ASSERT(IS_ALIGNED(__bss_end,        4), "__bss_end misaligned")
+ASSERT(IS_ALIGNED(__bss_start,      8), "__bss_start misaligned")
+ASSERT(IS_ALIGNED(__bss_end,        8), "__bss_end misaligned")
+
+ASSERT((trampoline_end - trampoline_start) < TRAMPOLINE_SPACE - MBI_SPACE_MIN,
+    "not enough room for trampoline and mbi data")
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index 36dbb71..b6cbdad 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -79,6 +79,17 @@  static size_t wstrlen(const CHAR16 * s);
 static int set_color(u32 mask, int bpp, u8 *pos, u8 *sz);
 static bool_t match_guid(const EFI_GUID *guid1, const EFI_GUID *guid2);
 
+static void efi_init(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable);
+static void efi_console_set_mode(void);
+static EFI_GRAPHICS_OUTPUT_PROTOCOL *efi_get_gop(void);
+static UINTN efi_find_gop_mode(EFI_GRAPHICS_OUTPUT_PROTOCOL *gop,
+                               UINTN cols, UINTN rows, UINTN depth);
+static void efi_tables(void);
+static void setup_efi_pci(void);
+static void efi_variables(void);
+static void efi_set_gop_mode(EFI_GRAPHICS_OUTPUT_PROTOCOL *gop, UINTN gop_mode);
+static void efi_exit_boot(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable);
+
 static const EFI_BOOT_SERVICES *__initdata efi_bs;
 static UINT32 __initdata efi_bs_revision;
 static EFI_HANDLE __initdata efi_ih;
diff --git a/xen/include/asm-x86/config.h b/xen/include/asm-x86/config.h
index 6fd84e7..b9a6d94 100644
--- a/xen/include/asm-x86/config.h
+++ b/xen/include/asm-x86/config.h
@@ -73,6 +73,11 @@ 
 #define STACK_ORDER 3
 #define STACK_SIZE  (PAGE_SIZE << STACK_ORDER)
 
+#define TRAMPOLINE_STACK_SPACE  PAGE_SIZE
+#define TRAMPOLINE_SPACE        (KB(64) - TRAMPOLINE_STACK_SPACE)
+
+#define MBI_SPACE_MIN           (2 * PAGE_SIZE)
+
 /* Primary stack is restricted to 8kB by guard pages. */
 #define PRIMARY_STACK_SIZE 8192
 
diff --git a/xen/include/xen/config.h b/xen/include/xen/config.h
index 473c5e8..04e4da5 100644
--- a/xen/include/xen/config.h
+++ b/xen/include/xen/config.h
@@ -70,6 +70,7 @@ 
 #define __force
 #define __bitwise
 
+#define KB(_kb)     (_AC(_kb, ULL) << 10)
 #define MB(_mb)     (_AC(_mb, ULL) << 20)
 #define GB(_gb)     (_AC(_gb, ULL) << 30)