From patchwork Thu Mar 24 20:00:22 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konrad Rzeszutek Wilk X-Patchwork-Id: 8664361 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C840DC0554 for ; Thu, 24 Mar 2016 20:03:57 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B5DD82037E for ; Thu, 24 Mar 2016 20:03:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8308720380 for ; Thu, 24 Mar 2016 20:03:53 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajBRZ-0007Qv-6C; Thu, 24 Mar 2016 20:01:21 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ajBRX-0007OE-Mv for xen-devel@lists.xenproject.org; Thu, 24 Mar 2016 20:01:19 +0000 Received: from [193.109.254.147] by server-15.bemta-14.messagelabs.com id 6C/0F-02980-F8744F65; Thu, 24 Mar 2016 20:01:19 +0000 X-Env-Sender: konrad@char.us.oracle.com X-Msg-Ref: server-3.tower-27.messagelabs.com!1458849675!33382150!1 X-Originating-IP: [156.151.31.81] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTU2LjE1MS4zMS44MSA9PiAyODgzMzk=\n X-StarScan-Received: X-StarScan-Version: 8.11; banners=-,-,- X-VirusChecked: Checked Received: (qmail 34434 invoked from network); 24 Mar 2016 20:01:17 -0000 Received: from userp1040.oracle.com (HELO userp1040.oracle.com) (156.151.31.81) by server-3.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 24 Mar 2016 20:01:17 -0000 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u2OK16gn011505 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 24 Mar 2016 20:01:06 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u2OK169N013728 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 24 Mar 2016 20:01:06 GMT Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by aserv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u2OK15L6023714; Thu, 24 Mar 2016 20:01:05 GMT Received: from char.us.oracle.com (/10.137.176.158) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 24 Mar 2016 13:01:05 -0700 Received: by char.us.oracle.com (Postfix, from userid 1000) id 3A7BE6A00DD; Thu, 24 Mar 2016 16:01:03 -0400 (EDT) From: Konrad Rzeszutek Wilk To: xen-devel@lists.xenproject.org, ross.lagerwall@citrix.com, konrad@kernel.org, andrew.cooper3@citrix.com, mpohlack@amazon.de, sasha.levin@oracle.com Date: Thu, 24 Mar 2016 16:00:22 -0400 Message-Id: <1458849640-22588-11-git-send-email-konrad.wilk@oracle.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1458849640-22588-1-git-send-email-konrad.wilk@oracle.com> References: <1458849640-22588-1-git-send-email-konrad.wilk@oracle.com> X-Source-IP: aserv0021.oracle.com [141.146.126.233] Cc: Keir Fraser , Julien Grall , Stefano Stabellini , Jan Beulich , Konrad Rzeszutek Wilk Subject: [Xen-devel] [PATCH v5 10/28] xsplice: Implement payload loading X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ross Lagerwall Add support for loading xsplice payloads. This is somewhat similar to the Linux kernel module loader, implementing the following steps: - Verify the elf file. - Parse the elf file. - Allocate a region of memory mapped within a free area of [xen_virt_end, XEN_VIRT_END]. - Copy allocated sections into the new region. Split them in three regions - .text, .data, and .rodata. MUST have at least .text. - Resolve section symbols. All other symbols must be absolute addresses. (Note that patch titled "xsplice,symbols: Implement symbol name resolution on address" implements that) - Perform relocations. - Secure the the regions (.text,.data,.rodata) with proper permissions. We capitalize on the vmalloc callback API (see patch titled: "vmap: Add vmalloc_cb and vfree_cb") to allocate a region of memory within the [xen_virt_end, XEN_VIRT_END] for the code. Signed-off-by: Ross Lagerwall Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Stefano Stabellini Cc: Julien Grall Cc: Keir Fraser Cc: Jan Beulich Cc: Andrew Cooper v2: - Change the 'xsplice_patch_func' structure layout/size. - Add more error checking. Fix memory leak. - Move elf_resolve and elf_perform relocs in elf file. - Print the payload address and pages in keyhandler. v3: - Make it build under ARM - Build it without using the return_ macro. - Add fixes from Ross. - Add the _return macro back - but only use it during debug builds. - Remove the macro, prefix arch_ on arch specific calls. v4: - Move alloc_payload to arch specific file. - Use void* instead of uint8_t, use const - Add copyrights - Unroll the vmap code to add ASSERT. Change while to not incur potential long error loop - Use vmalloc/vfree cb APIs - Secure .text pages to be RX instead of RWX. v5: - Fix allocation of virtual addresses only allowing one page to be allocated. - Create .text, .data, and .rodata regions with different permissions. - Make the find_space_t not a typedef to pointer to a function. - Allocate memory in here. --- --- xen/arch/arm/Makefile | 1 + xen/arch/arm/xsplice.c | 55 ++++++++ xen/arch/x86/Makefile | 1 + xen/arch/x86/setup.c | 7 + xen/arch/x86/xsplice.c | 258 +++++++++++++++++++++++++++++++++++++ xen/common/xsplice.c | 262 +++++++++++++++++++++++++++++++++++++- xen/common/xsplice_elf.c | 91 +++++++++++++ xen/include/asm-x86/x86_64/page.h | 2 + xen/include/xen/xsplice.h | 46 +++++++ xen/include/xen/xsplice_elf.h | 5 + 10 files changed, 725 insertions(+), 3 deletions(-) create mode 100644 xen/arch/arm/xsplice.c create mode 100644 xen/arch/x86/xsplice.c diff --git a/xen/arch/arm/Makefile b/xen/arch/arm/Makefile index 0328b50..eae5cb3 100644 --- a/xen/arch/arm/Makefile +++ b/xen/arch/arm/Makefile @@ -40,6 +40,7 @@ obj-y += device.o obj-y += decode.o obj-y += processor.o obj-y += smc.o +obj-$(CONFIG_XSPLICE) += xsplice.o #obj-bin-y += ....o diff --git a/xen/arch/arm/xsplice.c b/xen/arch/arm/xsplice.c new file mode 100644 index 0000000..e9c49ab --- /dev/null +++ b/xen/arch/arm/xsplice.c @@ -0,0 +1,55 @@ +/* + * Copyright (C) 2016 Citrix Systems R&D Ltd. + */ +#include +#include +#include +#include + +int arch_xsplice_verify_elf(const struct xsplice_elf *elf, void *data) +{ + return -ENOSYS; +} + +int arch_xsplice_perform_rel(struct xsplice_elf *elf, + const struct xsplice_elf_sec *base, + const struct xsplice_elf_sec *rela) +{ + return -ENOSYS; +} + +int arch_xsplice_perform_rela(struct xsplice_elf *elf, + const struct xsplice_elf_sec *base, + const struct xsplice_elf_sec *rela) +{ + return -ENOSYS; +} + +void *arch_xsplice_alloc_payload(unsigned int pages, mfn_t **mfn) +{ + return NULL; +} + +int arch_xsplice_secure(void *va, unsigned int pages, enum va_type type, + const mfn_t *mfn) +{ + return -ENOSYS; +} + +void arch_xsplice_register_find_space(find_space_t *cb) +{ +} + +void arch_xsplice_free_payload(void *va, unsigned int pages) +{ +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile index 729065b..8a6a7d5 100644 --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -64,6 +64,7 @@ obj-y += vm_event.o obj-y += xstate.o obj-$(crash_debug) += gdbstub.o +obj-$(CONFIG_XSPLICE) += xsplice.o x86_emulate.o: x86_emulate/x86_emulate.c x86_emulate/x86_emulate.h diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index c8a5adb..b306da8 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -100,6 +100,9 @@ unsigned long __read_mostly xen_phys_start; unsigned long __read_mostly xen_virt_end; +unsigned long __read_mostly avail_virt_start; +unsigned long __read_mostly avail_virt_end; + DEFINE_PER_CPU(struct tss_struct, init_tss); char __section(".bss.stack_aligned") cpu0_stack[STACK_SIZE]; @@ -1211,6 +1214,10 @@ void __init noreturn __start_xen(unsigned long mbi_p) ~((1UL << L2_PAGETABLE_SHIFT) - 1); destroy_xen_mappings(xen_virt_end, XEN_VIRT_START + BOOTSTRAP_MAP_BASE); + avail_virt_start = xen_virt_end; + avail_virt_end = XEN_VIRT_END - NR_CPUS * PAGE_SIZE; + BUG_ON(avail_virt_end <= avail_virt_start); + /* * If not using 2M mappings to gain suitable pagetable permissions * directly from the relocation above, remap the code/data diff --git a/xen/arch/x86/xsplice.c b/xen/arch/x86/xsplice.c new file mode 100644 index 0000000..7149540 --- /dev/null +++ b/xen/arch/x86/xsplice.c @@ -0,0 +1,258 @@ +/* + * Copyright (C) 2016 Citrix Systems R&D Ltd. + */ + +#include +#include +#include +#include +#include +#include +#include + +int arch_xsplice_verify_elf(const struct xsplice_elf *elf, void *data) +{ + + Elf_Ehdr *hdr = data; + + if ( !IS_ELF(*hdr) ) + { + printk(XENLOG_ERR "%s%s: Not an ELF payload!\n", XSPLICE, elf->name); + return -EINVAL; + } + + if ( elf->len < (sizeof *hdr) || + !IS_ELF(*hdr) || + hdr->e_ident[EI_CLASS] != ELFCLASS64 || + hdr->e_ident[EI_DATA] != ELFDATA2LSB || + hdr->e_ident[EI_OSABI] != ELFOSABI_SYSV || + hdr->e_machine != EM_X86_64 || + hdr->e_type != ET_REL || + hdr->e_phnum != 0 ) + { + printk(XENLOG_ERR "%s%s: Invalid ELF payload!\n", XSPLICE, elf->name); + return -EOPNOTSUPP; + } + + return 0; +} + +int arch_xsplice_perform_rel(struct xsplice_elf *elf, + const struct xsplice_elf_sec *base, + const struct xsplice_elf_sec *rela) +{ + dprintk(XENLOG_ERR, "%s%s: SHR_REL relocation unsupported\n", + XSPLICE, elf->name); + return -ENOSYS; +} + +int arch_xsplice_perform_rela(struct xsplice_elf *elf, + const struct xsplice_elf_sec *base, + const struct xsplice_elf_sec *rela) +{ + Elf_RelA *r; + unsigned int symndx, i; + uint64_t val; + uint8_t *dest; + + if ( !rela->sec->sh_entsize || !rela->sec->sh_size || + rela->sec->sh_entsize != sizeof(Elf_RelA) ) + { + dprintk(XENLOG_DEBUG, "%s%s: Section relative header is corrupted!\n", + XSPLICE, elf->name); + return -EINVAL; + } + + for ( i = 0; i < (rela->sec->sh_size / rela->sec->sh_entsize); i++ ) + { + r = (Elf_RelA *)(rela->data + i * rela->sec->sh_entsize); + if ( (unsigned long)r > (unsigned long)(elf->hdr + elf->len) ) + { + dprintk(XENLOG_DEBUG, "%s%s: Relative entry %u in %s is past end!\n", + XSPLICE, elf->name, i, rela->name); + return -EINVAL; + } + + symndx = ELF64_R_SYM(r->r_info); + if ( symndx > elf->nsym ) + { + dprintk(XENLOG_DEBUG, "%s%s: Relative symbol wants symbol@%u which is past end!\n", + XSPLICE, elf->name, symndx); + return -EINVAL; + } + + dest = base->load_addr + r->r_offset; + val = r->r_addend + elf->sym[symndx].sym->st_value; + + switch ( ELF64_R_TYPE(r->r_info) ) + { + case R_X86_64_NONE: + break; + + case R_X86_64_64: + *(uint64_t *)dest = val; + break; + + case R_X86_64_PLT32: + /* + * Xen uses -fpic which normally uses PLT relocations + * except that it sets visibility to hidden which means + * that they are not used. However, when gcc cannot + * inline memcpy it emits memcpy with default visibility + * which then creates a PLT relocation. It can just be + * treated the same as R_X86_64_PC32. + */ + /* Fall through */ + + case R_X86_64_PC32: + *(uint32_t *)dest = val - (uint64_t)dest; + break; + + default: + printk(XENLOG_ERR "%s%s: Unhandled relocation %lu\n", + XSPLICE, elf->name, ELF64_R_TYPE(r->r_info)); + return -EINVAL; + } + } + + return 0; +} + +static find_space_t *find_space_fnc = NULL; + +void arch_xsplice_register_find_space(find_space_t *cb) +{ + ASSERT(!find_space_fnc); + + find_space_fnc = cb; +} + +static void* xsplice_map_rwx(const mfn_t *mfn, unsigned int pages) +{ + unsigned long cur; + unsigned long start, end; + + start = (unsigned long)avail_virt_start; + end = start + pages * PAGE_SIZE; + + ASSERT(find_space_fnc); + + if ( find_space_fnc(pages, &start, &end) ) + return NULL; + + if ( end >= avail_virt_end ) + return NULL; + + for ( cur = start; pages--; ++mfn, cur += PAGE_SIZE ) + { + /* + * We would like to to RX, but we need to copy data in it first. + * See arch_xsplice_secure for how we lockdown. + */ + if ( map_pages_to_xen(cur, mfn_x(*mfn), 1, PAGE_HYPERVISOR_RWX) ) + { + if ( cur != start ) + destroy_xen_mappings(start, cur); + return NULL; + } + } + + return (void*)start; +} + +/* + * The function prepares an xSplice payload by allocating space which + * then can be used for loading the allocated sections, resolving symbols, + * performing relocations, etc. + */ +void *arch_xsplice_alloc_payload(unsigned int pages, mfn_t **mfn) +{ + unsigned int i; + void *p; + + ASSERT(pages); + ASSERT(mfn && !*mfn); + + *mfn = NULL; + /* + * We let the vmalloc allocate the pages we need, and use + * our callback. The callback must set the pages W otherwise we can't + * put anything in them. + */ + p = vmalloc_cb(pages * PAGE_SIZE, xsplice_map_rwx, mfn); + WARN_ON(!p); + if ( !p ) + return NULL; + + for ( i = 0; i < pages; i++ ) + clear_page(p + (i * PAGE_SIZE) ); + + /* Note that we do not free mfn. The caller is responsible for that. */ + return p; +} + +static void arch_xsplice_vfree_cb(void *va, unsigned int pages) +{ + unsigned long addr = (unsigned long)va; + + destroy_xen_mappings(addr, addr + pages * PAGE_SIZE); +} + +/* + * Once the resolving symbols, performing relocations, etc is complete + * we secure the memory by putting in the proper page table attributes + * for the desired type. + */ +int arch_xsplice_secure(void *va, unsigned int pages, enum va_type type, + const mfn_t *mfn) +{ + unsigned long cur; + unsigned long start = (unsigned long)va; + int flag; + + ASSERT(va); + ASSERT(pages); + + if ( type == XSPLICE_VA_RX ) + flag = PAGE_HYPERVISOR_RX; + else if ( type == XSPLICE_VA_RW ) + flag = PAGE_HYPERVISOR_RW; + else + flag = PAGE_HYPERVISOR_RO; + + /* + * We could walk the pagetable and do the pagetable manipulations + * (strip the _PAGE_RW), which would mean also not needing the mfn + * array, but there are no generic code for this yet (TODO). + * + * For right now tear down the pagetables and recreate them. + */ + arch_xsplice_vfree_cb(va, pages); + + for ( cur = start; pages--; ++mfn, cur += PAGE_SIZE ) + { + if ( map_pages_to_xen(cur, mfn_x(*mfn), 1, flag) ) + { + if ( cur != start ) + destroy_xen_mappings(start, cur); + return -EINVAL; + } + } + + return 0; +} + +void arch_xsplice_free_payload(void *va, unsigned int pages) +{ + vfree_cb(va, pages, arch_xsplice_vfree_cb); +} + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/common/xsplice.c b/xen/common/xsplice.c index 0047032..4d24898 100644 --- a/xen/common/xsplice.c +++ b/xen/common/xsplice.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -28,6 +29,15 @@ struct payload { uint32_t state; /* One of the XSPLICE_STATE_*. */ int32_t rc; /* 0 or -XEN_EXX. */ struct list_head list; /* Linked to 'payload_list'. */ + void *text_addr; /* Virtual address of .text. */ + size_t text_size; /* .. and its size. */ + void *rw_addr; /* Virtual address of .data. */ + size_t rw_size; /* .. and its size (if any). */ + void *ro_addr; /* Virtual address of .rodata. */ + size_t ro_size; /* .. and its size (if any). */ + size_t payload_pages; /* Nr of the pages for the text_addr; + rw_addr, and ro_addr (if any) */ + mfn_t *mfn; /* Array of MFNs of the pages. */ char name[XEN_XSPLICE_NAME_SIZE]; /* Name of it. */ }; @@ -99,6 +109,193 @@ static struct payload *find_payload(const xen_xsplice_name_t *name) return found; } +/* + * Functions related to XEN_SYSCTL_XSPLICE_UPLOAD (see xsplice_upload), and + * freeing payload (XEN_SYSCTL_XSPLICE_ACTION:XSPLICE_ACTION_UNLOAD). + */ + +static void free_payload_data(struct payload *payload) +{ + /* Set to zero until "move_payload". */ + if ( !payload->text_addr ) + return; + + xfree(payload->mfn); + payload->mfn = NULL; + + arch_xsplice_free_payload(payload->text_addr, + payload->payload_pages); + + payload->text_addr = NULL; + payload->ro_addr = NULL; + payload->rw_addr = NULL; + payload->payload_pages = 0; +} + +/* +* calc_section computes the size (taking into account section alignment). +* +* It also modifies sh_entsize with the offset of from the start of +* virtual address space. This is used in move_payload to figure out the +* destination location. +*/ +static void calc_section(struct xsplice_elf_sec *sec, size_t *size) +{ + size_t align_size = ROUNDUP(*size, sec->sec->sh_addralign); + + sec->sec->sh_entsize = align_size; + *size = sec->sec->sh_size + align_size; +} + +static int find_hole(size_t pages, unsigned long *hole_start, + unsigned long *hole_end) +{ + struct payload *data, *data2; + + spin_lock_recursive(&payload_lock); + list_for_each_entry ( data, &payload_list, list ) + { + list_for_each_entry ( data2, &payload_list, list ) + { + unsigned long start, end; + + start = (unsigned long)data2->text_addr; + end = start + data2->payload_pages * PAGE_SIZE; + if ( *hole_end > start && *hole_start < end ) + { + *hole_start = end; + *hole_end = end + pages * PAGE_SIZE; + break; + } + } + if ( &data2->list == &payload_list ) + break; + } + spin_unlock_recursive(&payload_lock); + + return 0; +} + +static int move_payload(struct payload *payload, struct xsplice_elf *elf) +{ + uint8_t *buf; + unsigned int i; + size_t size = 0; + + /* Compute text regions. */ + for ( i = 0; i < elf->hdr->e_shnum; i++ ) + { + if ( (elf->sec[i].sec->sh_flags & (SHF_ALLOC|SHF_EXECINSTR)) == + (SHF_ALLOC|SHF_EXECINSTR) ) + calc_section(&elf->sec[i], &payload->text_size); + } + + /* Compute rw data. */ + for ( i = 0; i < elf->hdr->e_shnum; i++ ) + { + if ( (elf->sec[i].sec->sh_flags & SHF_ALLOC) && + !(elf->sec[i].sec->sh_flags & SHF_EXECINSTR) && + (elf->sec[i].sec->sh_flags & SHF_WRITE) ) + calc_section(&elf->sec[i], &payload->rw_size); + } + + /* Compute ro data. */ + for ( i = 0; i < elf->hdr->e_shnum; i++ ) + { + if ( (elf->sec[i].sec->sh_flags & SHF_ALLOC) && + !(elf->sec[i].sec->sh_flags & SHF_EXECINSTR) && + !(elf->sec[i].sec->sh_flags & SHF_WRITE) ) + calc_section(&elf->sec[i], &payload->ro_size); + } + + /* + * Total of all three regions - RX, RW, and RO. We have to have + * keep them in seperate pages so we PAGE_ALIGN the RX and RW to have + * them on seperate pages. The last one will by default fall on its + * own page. + */ + size = PAGE_ALIGN(payload->text_size) + PAGE_ALIGN(payload->rw_size) + + payload->ro_size; + + size = PFN_UP(size); + buf = arch_xsplice_alloc_payload(size, &payload->mfn); + if ( !buf ) { + printk(XENLOG_ERR "%s%s: Could not allocate memory for payload!\n", + XSPLICE, elf->name); + return -ENOMEM; + } + + payload->payload_pages = size; + payload->text_addr = buf; + payload->rw_addr = payload->text_addr + PAGE_ALIGN(payload->text_size); + payload->ro_addr = payload->rw_addr + PAGE_ALIGN(payload->rw_size); + + for ( i = 0; i < elf->hdr->e_shnum; i++ ) + { + if ( elf->sec[i].sec->sh_flags & SHF_ALLOC ) + { + if ( (elf->sec[i].sec->sh_flags & SHF_EXECINSTR) ) + buf = payload->text_addr; + else if ( (elf->sec[i].sec->sh_flags & SHF_WRITE) ) + buf = payload->rw_addr; + else + buf = payload->ro_addr; + + elf->sec[i].load_addr = buf + elf->sec[i].sec->sh_entsize; + + /* Don't copy NOBITS - such as BSS. */ + if ( elf->sec[i].sec->sh_type != SHT_NOBITS ) + { + memcpy(elf->sec[i].load_addr, elf->sec[i].data, + elf->sec[i].sec->sh_size); + dprintk(XENLOG_DEBUG, "%s%s: Loaded %s at 0x%p\n", XSPLICE, + elf->name, elf->sec[i].name, elf->sec[i].load_addr); + } + } + } + + return 0; +} + +static int secure_payload(struct payload *payload, struct xsplice_elf *elf) +{ + int rc; + unsigned int text_pages, rw_pages, ro_pages; + + ASSERT(payload->mfn); + + text_pages = PFN_UP(payload->text_size); + ASSERT(text_pages); + + rc = arch_xsplice_secure(payload->text_addr, text_pages, XSPLICE_VA_RX, + payload->mfn); + if ( rc ) + return rc; + + rw_pages = PFN_UP(payload->rw_size); + if ( rw_pages ) + { + rc = arch_xsplice_secure(payload->rw_addr, rw_pages, XSPLICE_VA_RW, + payload->mfn + text_pages); + if ( rc ) + return rc; + } + + ro_pages = PFN_UP(payload->ro_size); + if ( ro_pages ) + { + rc = arch_xsplice_secure(payload->ro_addr, ro_pages, XSPLICE_VA_RO, + payload->mfn + text_pages + rw_pages); + } + + ASSERT(ro_pages + rw_pages + text_pages == payload->payload_pages); + + xfree(payload->mfn); + payload->mfn = NULL; + + return rc; +} + /* We MUST be holding the payload_lock spinlock. */ static void free_payload(struct payload *data) { @@ -106,12 +303,55 @@ static void free_payload(struct payload *data) list_del(&data->list); payload_cnt--; payload_version++; + free_payload_data(data); xfree(data); } +static int load_payload_data(struct payload *payload, void *raw, ssize_t len) +{ + struct xsplice_elf elf; + int rc = 0; + + memset(&elf, 0, sizeof(elf)); + elf.name = payload->name; + elf.len = len; + + rc = arch_xsplice_verify_elf(&elf, raw); + if ( rc ) + return rc; + + rc = xsplice_elf_load(&elf, raw); + if ( rc ) + goto out; + + rc = move_payload(payload, &elf); + if ( rc ) + goto out; + + rc = xsplice_elf_resolve_symbols(&elf); + if ( rc ) + goto out; + + rc = xsplice_elf_perform_relocs(&elf); + if ( rc ) + goto out; + + rc = secure_payload(payload, &elf); + + out: + if ( rc ) + free_payload_data(payload); + + /* Free our temporary data structure. */ + xsplice_elf_free(&elf); + + return rc; +} + static int xsplice_upload(xen_sysctl_xsplice_upload_t *upload) { struct payload *data; + void *raw_data = NULL; int rc; rc = verify_payload(upload); @@ -137,7 +377,19 @@ static int xsplice_upload(xen_sysctl_xsplice_upload_t *upload) if ( data->name[upload->name.size - 1] ) goto out; - rc = 0; + rc = -ENOMEM; + raw_data = vmalloc(upload->size); + if ( !raw_data ) + goto out; + + rc = -EFAULT; + if ( __copy_from_guest(raw_data, upload->payload, upload->size) ) + goto out; + + rc = load_payload_data(data, raw_data, upload->size); + if ( rc ) + goto out; + data->state = XSPLICE_STATE_CHECKED; INIT_LIST_HEAD(&data->list); @@ -148,6 +400,7 @@ static int xsplice_upload(xen_sysctl_xsplice_upload_t *upload) spin_unlock_recursive(&payload_lock); out: + vfree(raw_data); if ( rc ) xfree(data); @@ -382,8 +635,9 @@ static void xsplice_printall(unsigned char key) } list_for_each_entry ( data, &payload_list, list ) - printk(" name=%s state=%s(%d)\n", data->name, - state2str(data->state), data->state); + printk(" name=%s state=%s(%d) %p (.data=%p, .rodata=%p) using %zu pages.\n", + data->name, state2str(data->state), data->state, data->text_addr, + data->rw_addr, data->ro_addr, data->payload_pages); spin_unlock_recursive(&payload_lock); } @@ -391,6 +645,8 @@ static void xsplice_printall(unsigned char key) static int __init xsplice_init(void) { register_keyhandler('x', xsplice_printall, "print xsplicing info", 1); + + arch_xsplice_register_find_space(&find_hole); return 0; } __initcall(xsplice_init); diff --git a/xen/common/xsplice_elf.c b/xen/common/xsplice_elf.c index f22cede..e5c0b12 100644 --- a/xen/common/xsplice_elf.c +++ b/xen/common/xsplice_elf.c @@ -213,6 +213,97 @@ static int elf_get_sym(struct xsplice_elf *elf, const void *data) return 0; } +int xsplice_elf_resolve_symbols(struct xsplice_elf *elf) +{ + unsigned int i; + + /* + * The first entry of an ELF symbol table is the "undefined symbol index". + * aka reserved so we skip it. + */ + ASSERT( elf->sym ); + for ( i = 1; i < elf->nsym; i++ ) + { + uint16_t idx = elf->sym[i].sym->st_shndx; + + switch ( idx ) + { + case SHN_COMMON: + printk(XENLOG_ERR "%s%s: Unexpected common symbol: %s\n", + XSPLICE, elf->name, elf->sym[i].name); + return -EINVAL; + break; + + case SHN_UNDEF: + printk(XENLOG_ERR "%s%s: Unknown symbol: %s\n", + XSPLICE, elf->name, elf->sym[i].name); + return -ENOENT; + break; + + case SHN_ABS: + dprintk(XENLOG_DEBUG, "%s%s: Absolute symbol: %s => 0x%"PRIx64"\n", + XSPLICE, elf->name, elf->sym[i].name, + elf->sym[i].sym->st_value); + break; + + default: + if ( elf->sec[idx].sec->sh_flags & SHF_ALLOC ) + { + elf->sym[i].sym->st_value += + (unsigned long)elf->sec[idx].load_addr; + if ( elf->sym[i].name ) + printk(XENLOG_DEBUG "%s%s: Symbol resolved: %s => 0x%"PRIx64"(%s)\n", + XSPLICE, elf->name, elf->sym[i].name, + (uint64_t)elf->sym[i].sym->st_value, + elf->sec[idx].name); + } + } + } + + return 0; +} + +int xsplice_elf_perform_relocs(struct xsplice_elf *elf) +{ + struct xsplice_elf_sec *rela, *base; + unsigned int i; + int rc; + + /* + * The first entry of an ELF symbol table is the "undefined symbol index". + * aka reserved so we skip it. + */ + ASSERT( elf->sym ); + for ( i = 1; i < elf->hdr->e_shnum; i++ ) + { + rela = &elf->sec[i]; + + if ( (rela->sec->sh_type != SHT_RELA ) && + (rela->sec->sh_type != SHT_REL ) ) + continue; + + /* Is it a valid relocation section? */ + if ( rela->sec->sh_info >= elf->hdr->e_shnum ) + continue; + + base = &elf->sec[rela->sec->sh_info]; + + /* Don't relocate non-allocated sections. */ + if ( !(base->sec->sh_flags & SHF_ALLOC) ) + continue; + + if ( elf->sec[i].sec->sh_type == SHT_RELA ) + rc = arch_xsplice_perform_rela(elf, base, rela); + else /* SHT_REL */ + rc = arch_xsplice_perform_rel(elf, base, rela); + + if ( rc ) + return rc; + } + + return 0; +} + static int xsplice_header_check(const struct xsplice_elf *elf) { if ( sizeof(*elf->hdr) >= elf->len ) diff --git a/xen/include/asm-x86/x86_64/page.h b/xen/include/asm-x86/x86_64/page.h index 86abb94..a854e05 100644 --- a/xen/include/asm-x86/x86_64/page.h +++ b/xen/include/asm-x86/x86_64/page.h @@ -38,6 +38,8 @@ #include extern unsigned long xen_virt_end; +extern unsigned long avail_virt_start; +extern unsigned long avail_virt_end; #define spage_to_pdx(spg) (((spg) - spage_table)<<(SUPERPAGE_SHIFT-PAGE_SHIFT)) #define pdx_to_spage(pdx) (spage_table + ((pdx)>>(SUPERPAGE_SHIFT-PAGE_SHIFT))) diff --git a/xen/include/xen/xsplice.h b/xen/include/xen/xsplice.h index 00482d0..74850ea 100644 --- a/xen/include/xen/xsplice.h +++ b/xen/include/xen/xsplice.h @@ -6,6 +6,9 @@ #ifndef __XEN_XSPLICE_H__ #define __XEN_XSPLICE_H__ +struct xsplice_elf; +struct xsplice_elf_sec; +struct xsplice_elf_sym; struct xen_sysctl_xsplice_op; #ifdef CONFIG_XSPLICE @@ -15,6 +18,49 @@ struct xen_sysctl_xsplice_op; int xsplice_op(struct xen_sysctl_xsplice_op *); +/* Arch hooks. */ +int arch_xsplice_verify_elf(const struct xsplice_elf *elf, void *data); +int arch_xsplice_perform_rel(struct xsplice_elf *elf, + const struct xsplice_elf_sec *base, + const struct xsplice_elf_sec *rela); +int arch_xsplice_perform_rela(struct xsplice_elf *elf, + const struct xsplice_elf_sec *base, + const struct xsplice_elf_sec *rela); +enum va_type { + XSPLICE_VA_RX, /* .text */ + XSPLICE_VA_RW, /* .data */ + XSPLICE_VA_RO, /* .rodata */ +}; + +#include +void *arch_xsplice_alloc_payload(unsigned int pages, mfn_t **mfn); + +/* + * Function to secure the allocate pages (from arch_xsplice_alloc_payload) + * with the right page permissions. + */ +int arch_xsplice_secure(void *va, unsigned int pages, enum va_type types, + const mfn_t *mfn); + +void arch_xsplice_free_payload(void *va, unsigned int pages); + +/* + * Callback to find available virtual address space in which the + * payload could be put in. + * + * The arguments are: + * - The size of the payload in bytes. + * - The starting virtual address to search. To be updated by + * callback if space found. + * - The ending virtual address to search. To be updated by + * callback if space found. + * + * The return value is zero if search was done. -EXX values + * if errors were encountered. + */ +typedef int (find_space_t)(size_t, unsigned long *, unsigned long *); +void arch_xsplice_register_find_space(find_space_t *cb); + #else #include /* For -EOPNOTSUPP */ diff --git a/xen/include/xen/xsplice_elf.h b/xen/include/xen/xsplice_elf.h index e2dea18..f6ffcbe 100644 --- a/xen/include/xen/xsplice_elf.h +++ b/xen/include/xen/xsplice_elf.h @@ -15,6 +15,8 @@ struct xsplice_elf_sec { elf_resolve_section_names. */ const void *data; /* Pointer to the section (done by elf_resolve_sections). */ + uint8_t *load_addr; /* A pointer to the allocated destination. + Done by load_payload_data. */ }; struct xsplice_elf_sym { @@ -38,6 +40,9 @@ struct xsplice_elf_sec *xsplice_elf_sec_by_name(const struct xsplice_elf *elf, int xsplice_elf_load(struct xsplice_elf *elf, void *data); void xsplice_elf_free(struct xsplice_elf *elf); +int xsplice_elf_resolve_symbols(struct xsplice_elf *elf); +int xsplice_elf_perform_relocs(struct xsplice_elf *elf); + #endif /* __XEN_XSPLICE_ELF_H__ */ /*