From patchwork Mon Nov 14 17:17:48 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Kuznetsov X-Patchwork-Id: 9428057 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 66FF16047D for ; Mon, 14 Nov 2016 17:20:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5154F28A42 for ; Mon, 14 Nov 2016 17:20:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45FC528AA7; Mon, 14 Nov 2016 17:20:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EA39728A42 for ; Mon, 14 Nov 2016 17:20:20 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c6KtO-0003uE-I7; Mon, 14 Nov 2016 17:18:02 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c6KtN-0003tY-F1 for xen-devel@lists.xenproject.org; Mon, 14 Nov 2016 17:18:01 +0000 Received: from [85.158.139.211] by server-15.bemta-5.messagelabs.com id 80/07-13537-8C1F9285; Mon, 14 Nov 2016 17:18:00 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrELMWRWlGSWpSXmKPExsVysWW7jO7xj5o RBhenW1p83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBlbr51lKejdwFjxuO8uUwPj6YmMXYycHEIC e5kkdj+y6GLkArKPMEq8ndjKBJJgE9CR+P70FDOILSKgJHFv1WQmkCJmgT2MEls/LAPrFhaIl Nix6gULiM0ioCqx6PQqdhCbV8BZ4tXiX2A2p4CLxOwps9ghtjlLLPm8DMyWENCWeH5hMjvIUA mBPkaJHa+2Mk1g5FnAyLCKUaM4tagstUjXyEgvqSgzPaMkNzEzR9fQwFQvN7W4ODE9NScxqVg vOT93EyPQ//UMDIw7GPe0+x1ilORgUhLl7byhGSHEl5SfUpmRWJwRX1Sak1p8iFGGg0NJgjf3 A1BOsCg1PbUiLTMHGIgwaQkOHiURXklgMArxFhck5hZnpkOkTjHqcrzZ9fIBkxBLXn5eqpQ4b yfIDAGQoozSPLgRsKi4xCgrJczLyMDAIMRTkFqUm1mCKv+KUZyDUUmY9wTIFJ7MvBK4Ta+Ajm ACOmKXuQbIESWJCCmpBsbVTxtm7Hgm+CY4aP9kyxw35YkbGvdF3Nw/R/uoxeUw+43XNacKr8j 3ac+rvDVxkfMpvmmTm+d0p/80F56ZerV+VcHe3SdP2H38yOCwZzVf+e+X+RvPbOme/eHEmhsm XFnXWZvc70jGzjqec2jKx5MKMmcPrwjKmb5+Z7HOqSAFia2f1PfE9z0pVGIpzkg01GIuKk4EA HztH8qFAgAA X-Env-Sender: vkuznets@redhat.com X-Msg-Ref: server-9.tower-206.messagelabs.com!1479143877!70326384!1 X-Originating-IP: [209.132.183.28] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMjA5LjEzMi4xODMuMjggPT4gNTQwNjQ=\n X-StarScan-Received: X-StarScan-Version: 9.0.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 8279 invoked from network); 14 Nov 2016 17:17:59 -0000 Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by server-9.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 14 Nov 2016 17:17:59 -0000 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D93EBC04B302; Mon, 14 Nov 2016 17:17:57 +0000 (UTC) Received: from vitty.brq.redhat.com (vitty.brq.redhat.com [10.34.26.3]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id uAEHHoNW006170; Mon, 14 Nov 2016 12:17:56 -0500 From: Vitaly Kuznetsov To: xen-devel@lists.xenproject.org Date: Mon, 14 Nov 2016 18:17:48 +0100 Message-Id: <1479143869-27611-4-git-send-email-vkuznets@redhat.com> In-Reply-To: <1479143869-27611-1-git-send-email-vkuznets@redhat.com> References: <1479143869-27611-1-git-send-email-vkuznets@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Mon, 14 Nov 2016 17:17:57 +0000 (UTC) Cc: Juergen Gross , Boris Ostrovsky , x86@kernel.org, Andrew Jones , David Vrabel Subject: [Xen-devel] [RFC PATCH KERNEL 3/4] x86/xen: put setup.c, mmu.c and p2m.c under CONFIG_XEN_PV X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP These three files (mmu.c, p2m.c, setup.c) are mostly required to support PV guests, in fact p2m.c and setup.c have no code for PVHVM at all. mmu.c has some, move it to mmu_common.c and mmu_hvm.c. Some additional changes are required: - In the balloon driver we can't use xen_start_info, xen_released_pages and xen_extra_mem it is PV-only. Decorate it with #ifdef CONFIG_XEN_PV - Some PV-only functions are used by drivers and for PVHVM guests these functions have 'if (xen_feature(XENFEAT_auto_translated_physmap))' check in the beginning. Create required stubs for PVHVM-only builds. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/xen/page.h | 44 ++++++- arch/x86/xen/Makefile | 10 +- arch/x86/xen/mmu.c | 284 ---------------------------------------- arch/x86/xen/mmu_common.c | 219 +++++++++++++++++++++++++++++++ arch/x86/xen/mmu_hvm.c | 77 +++++++++++ drivers/xen/balloon.c | 30 +++-- include/xen/xen-ops.h | 13 ++ 7 files changed, 376 insertions(+), 301 deletions(-) create mode 100644 arch/x86/xen/mmu_common.c create mode 100644 arch/x86/xen/mmu_hvm.c diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h index f5fb840..3244ecf 100644 --- a/arch/x86/include/asm/xen/page.h +++ b/arch/x86/include/asm/xen/page.h @@ -43,9 +43,10 @@ extern unsigned long *xen_p2m_addr; extern unsigned long xen_p2m_size; extern unsigned long xen_max_p2m_pfn; -extern int xen_alloc_p2m_entry(unsigned long pfn); - extern unsigned long get_phys_to_machine(unsigned long pfn); + +#ifdef CONFIG_XEN_PV +extern int xen_alloc_p2m_entry(unsigned long pfn); extern bool set_phys_to_machine(unsigned long pfn, unsigned long mfn); extern bool __set_phys_to_machine(unsigned long pfn, unsigned long mfn); extern unsigned long __init set_phys_range_identity(unsigned long pfn_s, @@ -57,6 +58,38 @@ extern int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, extern int clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, struct gnttab_unmap_grant_ref *kunmap_ops, struct page **pages, unsigned int count); +#else /* CONFIG_XEN_PV */ +static inline int xen_alloc_p2m_entry(unsigned long pfn) +{ + return 0; +} + +static inline bool set_phys_to_machine(unsigned long pfn, unsigned long mfn) +{ + return true; +} + +static inline bool __set_phys_to_machine(unsigned long pfn, unsigned long mfn) +{ + return true; +} + +static inline int +set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops, + struct gnttab_map_grant_ref *kmap_ops, + struct page **pages, unsigned int count) +{ + return 0; +} + +static inline int +clear_foreign_p2m_mapping(struct gnttab_unmap_grant_ref *unmap_ops, + struct gnttab_unmap_grant_ref *kunmap_ops, + struct page **pages, unsigned int count) +{ + return 0; +} +#endif /* CONFIG_XEN_PV */ /* * Helper functions to write or read unsigned long values to/from @@ -82,6 +115,7 @@ static inline int xen_safe_read_ulong(unsigned long *addr, unsigned long *val) * - get_phys_to_machine() is to be called by __pfn_to_mfn() only in special * cases needing an extended handling. */ +#ifdef CONFIG_XEN_PV static inline unsigned long __pfn_to_mfn(unsigned long pfn) { unsigned long mfn; @@ -98,6 +132,12 @@ static inline unsigned long __pfn_to_mfn(unsigned long pfn) return mfn; } +#else +static inline unsigned long __pfn_to_mfn(unsigned long pfn) +{ + return pfn; +} +#endif static inline unsigned long pfn_to_mfn(unsigned long pfn) { diff --git a/arch/x86/xen/Makefile b/arch/x86/xen/Makefile index aa6cd5e..121a368 100644 --- a/arch/x86/xen/Makefile +++ b/arch/x86/xen/Makefile @@ -10,13 +10,13 @@ nostackp := $(call cc-option, -fno-stack-protector) CFLAGS_enlighten.o := $(nostackp) CFLAGS_mmu.o := $(nostackp) -obj-y := enlighten_common.o setup.o multicalls.o \ - mmu.o irq.o time.o xen-asm.o xen-asm_$(BITS).o \ +obj-y := enlighten_common.o multicalls.o \ + irq.o time.o xen-asm.o xen-asm_$(BITS).o \ grant-table.o suspend.o platform-pci-unplug.o \ - p2m.o apic.o pmu.o + apic.o pmu.o mmu_common.o -obj-$(CONFIG_XEN_PV) += enlighten.o -obj-$(CONFIG_XEN_PVHVM) += enlighten_hvm.o +obj-$(CONFIG_XEN_PV) += enlighten.o setup.o mmu.o p2m.o +obj-$(CONFIG_XEN_PVHVM) += enlighten_hvm.o mmu_hvm.o obj-$(CONFIG_EVENT_TRACING) += trace.o diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c index 65e184b..94b1806 100644 --- a/arch/x86/xen/mmu.c +++ b/arch/x86/xen/mmu.c @@ -80,12 +80,6 @@ #include "mmu.h" #include "debugfs.h" -/* - * Protects atomic reservation decrease/increase against concurrent increases. - * Also protects non-atomic updates of current_pages and balloon lists. - */ -DEFINE_SPINLOCK(xen_reservation_lock); - #ifdef CONFIG_X86_32 /* * Identity map, in addition to plain kernel map. This needs to be @@ -125,36 +119,6 @@ static phys_addr_t xen_pt_base, xen_pt_size __initdata; */ #define USER_LIMIT ((STACK_TOP_MAX + PGDIR_SIZE - 1) & PGDIR_MASK) -unsigned long arbitrary_virt_to_mfn(void *vaddr) -{ - xmaddr_t maddr = arbitrary_virt_to_machine(vaddr); - - return PFN_DOWN(maddr.maddr); -} - -xmaddr_t arbitrary_virt_to_machine(void *vaddr) -{ - unsigned long address = (unsigned long)vaddr; - unsigned int level; - pte_t *pte; - unsigned offset; - - /* - * if the PFN is in the linear mapped vaddr range, we can just use - * the (quick) virt_to_machine() p2m lookup - */ - if (virt_addr_valid(vaddr)) - return virt_to_machine(vaddr); - - /* otherwise we have to do a (slower) full page-table walk */ - - pte = lookup_address(address, &level); - BUG_ON(pte == NULL); - offset = address & ~PAGE_MASK; - return XMADDR(((phys_addr_t)pte_mfn(*pte) << PAGE_SHIFT) + offset); -} -EXPORT_SYMBOL_GPL(arbitrary_virt_to_machine); - void make_lowmem_page_readonly(void *vaddr) { pte_t *pte, ptev; @@ -1314,25 +1278,6 @@ unsigned long xen_read_cr2_direct(void) return this_cpu_read(xen_vcpu_info.arch.cr2); } -void xen_flush_tlb_all(void) -{ - struct mmuext_op *op; - struct multicall_space mcs; - - trace_xen_mmu_flush_tlb_all(0); - - preempt_disable(); - - mcs = xen_mc_entry(sizeof(*op)); - - op = mcs.args; - op->cmd = MMUEXT_TLB_FLUSH_ALL; - MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); - - xen_mc_issue(PARAVIRT_LAZY_MMU); - - preempt_enable(); -} static void xen_flush_tlb(void) { struct mmuext_op *op; @@ -2695,232 +2640,3 @@ void xen_destroy_contiguous_region(phys_addr_t pstart, unsigned int order) spin_unlock_irqrestore(&xen_reservation_lock, flags); } EXPORT_SYMBOL_GPL(xen_destroy_contiguous_region); - -#ifdef CONFIG_XEN_PVHVM -#ifdef CONFIG_PROC_VMCORE -/* - * This function is used in two contexts: - * - the kdump kernel has to check whether a pfn of the crashed kernel - * was a ballooned page. vmcore is using this function to decide - * whether to access a pfn of the crashed kernel. - * - the kexec kernel has to check whether a pfn was ballooned by the - * previous kernel. If the pfn is ballooned, handle it properly. - * Returns 0 if the pfn is not backed by a RAM page, the caller may - * handle the pfn special in this case. - */ -static int xen_oldmem_pfn_is_ram(unsigned long pfn) -{ - struct xen_hvm_get_mem_type a = { - .domid = DOMID_SELF, - .pfn = pfn, - }; - int ram; - - if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a)) - return -ENXIO; - - switch (a.mem_type) { - case HVMMEM_mmio_dm: - ram = 0; - break; - case HVMMEM_ram_rw: - case HVMMEM_ram_ro: - default: - ram = 1; - break; - } - - return ram; -} -#endif - -static void xen_hvm_exit_mmap(struct mm_struct *mm) -{ - struct xen_hvm_pagetable_dying a; - int rc; - - a.domid = DOMID_SELF; - a.gpa = __pa(mm->pgd); - rc = HYPERVISOR_hvm_op(HVMOP_pagetable_dying, &a); - WARN_ON_ONCE(rc < 0); -} - -static int is_pagetable_dying_supported(void) -{ - struct xen_hvm_pagetable_dying a; - int rc = 0; - - a.domid = DOMID_SELF; - a.gpa = 0x00; - rc = HYPERVISOR_hvm_op(HVMOP_pagetable_dying, &a); - if (rc < 0) { - printk(KERN_DEBUG "HVMOP_pagetable_dying not supported\n"); - return 0; - } - return 1; -} - -void __init xen_hvm_init_mmu_ops(void) -{ - if (is_pagetable_dying_supported()) - pv_mmu_ops.exit_mmap = xen_hvm_exit_mmap; -#ifdef CONFIG_PROC_VMCORE - register_oldmem_pfn_is_ram(&xen_oldmem_pfn_is_ram); -#endif -} -#endif - -#define REMAP_BATCH_SIZE 16 - -struct remap_data { - xen_pfn_t *mfn; - bool contiguous; - pgprot_t prot; - struct mmu_update *mmu_update; -}; - -static int remap_area_mfn_pte_fn(pte_t *ptep, pgtable_t token, - unsigned long addr, void *data) -{ - struct remap_data *rmd = data; - pte_t pte = pte_mkspecial(mfn_pte(*rmd->mfn, rmd->prot)); - - /* If we have a contiguous range, just update the mfn itself, - else update pointer to be "next mfn". */ - if (rmd->contiguous) - (*rmd->mfn)++; - else - rmd->mfn++; - - rmd->mmu_update->ptr = virt_to_machine(ptep).maddr; - rmd->mmu_update->val = pte_val_ma(pte); - rmd->mmu_update++; - - return 0; -} - -static int do_remap_gfn(struct vm_area_struct *vma, - unsigned long addr, - xen_pfn_t *gfn, int nr, - int *err_ptr, pgprot_t prot, - unsigned domid, - struct page **pages) -{ - int err = 0; - struct remap_data rmd; - struct mmu_update mmu_update[REMAP_BATCH_SIZE]; - unsigned long range; - int mapped = 0; - - BUG_ON(!((vma->vm_flags & (VM_PFNMAP | VM_IO)) == (VM_PFNMAP | VM_IO))); - - if (xen_feature(XENFEAT_auto_translated_physmap)) { -#ifdef CONFIG_XEN_PVH - /* We need to update the local page tables and the xen HAP */ - return xen_xlate_remap_gfn_array(vma, addr, gfn, nr, err_ptr, - prot, domid, pages); -#else - return -EINVAL; -#endif - } - - rmd.mfn = gfn; - rmd.prot = prot; - /* We use the err_ptr to indicate if there we are doing a contiguous - * mapping or a discontigious mapping. */ - rmd.contiguous = !err_ptr; - - while (nr) { - int index = 0; - int done = 0; - int batch = min(REMAP_BATCH_SIZE, nr); - int batch_left = batch; - range = (unsigned long)batch << PAGE_SHIFT; - - rmd.mmu_update = mmu_update; - err = apply_to_page_range(vma->vm_mm, addr, range, - remap_area_mfn_pte_fn, &rmd); - if (err) - goto out; - - /* We record the error for each page that gives an error, but - * continue mapping until the whole set is done */ - do { - int i; - - err = HYPERVISOR_mmu_update(&mmu_update[index], - batch_left, &done, domid); - - /* - * @err_ptr may be the same buffer as @gfn, so - * only clear it after each chunk of @gfn is - * used. - */ - if (err_ptr) { - for (i = index; i < index + done; i++) - err_ptr[i] = 0; - } - if (err < 0) { - if (!err_ptr) - goto out; - err_ptr[i] = err; - done++; /* Skip failed frame. */ - } else - mapped += done; - batch_left -= done; - index += done; - } while (batch_left); - - nr -= batch; - addr += range; - if (err_ptr) - err_ptr += batch; - cond_resched(); - } -out: - - xen_flush_tlb_all(); - - return err < 0 ? err : mapped; -} - -int xen_remap_domain_gfn_range(struct vm_area_struct *vma, - unsigned long addr, - xen_pfn_t gfn, int nr, - pgprot_t prot, unsigned domid, - struct page **pages) -{ - return do_remap_gfn(vma, addr, &gfn, nr, NULL, prot, domid, pages); -} -EXPORT_SYMBOL_GPL(xen_remap_domain_gfn_range); - -int xen_remap_domain_gfn_array(struct vm_area_struct *vma, - unsigned long addr, - xen_pfn_t *gfn, int nr, - int *err_ptr, pgprot_t prot, - unsigned domid, struct page **pages) -{ - /* We BUG_ON because it's a programmer error to pass a NULL err_ptr, - * and the consequences later is quite hard to detect what the actual - * cause of "wrong memory was mapped in". - */ - BUG_ON(err_ptr == NULL); - return do_remap_gfn(vma, addr, gfn, nr, err_ptr, prot, domid, pages); -} -EXPORT_SYMBOL_GPL(xen_remap_domain_gfn_array); - - -/* Returns: 0 success */ -int xen_unmap_domain_gfn_range(struct vm_area_struct *vma, - int numpgs, struct page **pages) -{ - if (!pages || !xen_feature(XENFEAT_auto_translated_physmap)) - return 0; - -#ifdef CONFIG_XEN_PVH - return xen_xlate_unmap_gfn_range(vma, numpgs, pages); -#else - return -EINVAL; -#endif -} -EXPORT_SYMBOL_GPL(xen_unmap_domain_gfn_range); diff --git a/arch/x86/xen/mmu_common.c b/arch/x86/xen/mmu_common.c new file mode 100644 index 0000000..d134cd9 --- /dev/null +++ b/arch/x86/xen/mmu_common.c @@ -0,0 +1,219 @@ +#include + +#include +#include + +#include + +#include "multicalls.h" +#include "mmu.h" + +/* + * Protects atomic reservation decrease/increase against concurrent increases. + * Also protects non-atomic updates of current_pages and balloon lists. + */ +DEFINE_SPINLOCK(xen_reservation_lock); + +#define REMAP_BATCH_SIZE 16 + +struct remap_data { + xen_pfn_t *mfn; + bool contiguous; + pgprot_t prot; + struct mmu_update *mmu_update; +}; + +void xen_flush_tlb_all(void) +{ + struct mmuext_op *op; + struct multicall_space mcs; + + trace_xen_mmu_flush_tlb_all(0); + + preempt_disable(); + + mcs = xen_mc_entry(sizeof(*op)); + + op = mcs.args; + op->cmd = MMUEXT_TLB_FLUSH_ALL; + MULTI_mmuext_op(mcs.mc, op, 1, NULL, DOMID_SELF); + + xen_mc_issue(PARAVIRT_LAZY_MMU); + + preempt_enable(); +} + +static int remap_area_mfn_pte_fn(pte_t *ptep, pgtable_t token, + unsigned long addr, void *data) +{ + struct remap_data *rmd = data; + pte_t pte = pte_mkspecial(mfn_pte(*rmd->mfn, rmd->prot)); + + /* If we have a contiguous range, just update the mfn itself, + else update pointer to be "next mfn". */ + if (rmd->contiguous) + (*rmd->mfn)++; + else + rmd->mfn++; + + rmd->mmu_update->ptr = virt_to_machine(ptep).maddr; + rmd->mmu_update->val = pte_val_ma(pte); + rmd->mmu_update++; + + return 0; +} + +static int do_remap_gfn(struct vm_area_struct *vma, + unsigned long addr, + xen_pfn_t *gfn, int nr, + int *err_ptr, pgprot_t prot, + unsigned domid, + struct page **pages) +{ + int err = 0; + struct remap_data rmd; + struct mmu_update mmu_update[REMAP_BATCH_SIZE]; + unsigned long range; + int mapped = 0; + + BUG_ON(!((vma->vm_flags & (VM_PFNMAP | VM_IO)) == (VM_PFNMAP | VM_IO))); + + if (xen_feature(XENFEAT_auto_translated_physmap)) { +#ifdef CONFIG_XEN_PVH + /* We need to update the local page tables and the xen HAP */ + return xen_xlate_remap_gfn_array(vma, addr, gfn, nr, err_ptr, + prot, domid, pages); +#else + return -EINVAL; +#endif + } + + rmd.mfn = gfn; + rmd.prot = prot; + /* We use the err_ptr to indicate if there we are doing a contiguous + * mapping or a discontigious mapping. */ + rmd.contiguous = !err_ptr; + + while (nr) { + int index = 0; + int done = 0; + int batch = min(REMAP_BATCH_SIZE, nr); + int batch_left = batch; + range = (unsigned long)batch << PAGE_SHIFT; + + rmd.mmu_update = mmu_update; + err = apply_to_page_range(vma->vm_mm, addr, range, + remap_area_mfn_pte_fn, &rmd); + if (err) + goto out; + + /* We record the error for each page that gives an error, but + * continue mapping until the whole set is done */ + do { + int i; + + err = HYPERVISOR_mmu_update(&mmu_update[index], + batch_left, &done, domid); + + /* + * @err_ptr may be the same buffer as @gfn, so + * only clear it after each chunk of @gfn is + * used. + */ + if (err_ptr) { + for (i = index; i < index + done; i++) + err_ptr[i] = 0; + } + if (err < 0) { + if (!err_ptr) + goto out; + err_ptr[i] = err; + done++; /* Skip failed frame. */ + } else + mapped += done; + batch_left -= done; + index += done; + } while (batch_left); + + nr -= batch; + addr += range; + if (err_ptr) + err_ptr += batch; + cond_resched(); + } +out: + + xen_flush_tlb_all(); + + return err < 0 ? err : mapped; +} + +int xen_remap_domain_gfn_range(struct vm_area_struct *vma, + unsigned long addr, + xen_pfn_t gfn, int nr, + pgprot_t prot, unsigned domid, + struct page **pages) +{ + return do_remap_gfn(vma, addr, &gfn, nr, NULL, prot, domid, pages); +} +EXPORT_SYMBOL_GPL(xen_remap_domain_gfn_range); + +int xen_remap_domain_gfn_array(struct vm_area_struct *vma, + unsigned long addr, + xen_pfn_t *gfn, int nr, + int *err_ptr, pgprot_t prot, + unsigned domid, struct page **pages) +{ + /* We BUG_ON because it's a programmer error to pass a NULL err_ptr, + * and the consequences later is quite hard to detect what the actual + * cause of "wrong memory was mapped in". + */ + BUG_ON(err_ptr == NULL); + return do_remap_gfn(vma, addr, gfn, nr, err_ptr, prot, domid, pages); +} +EXPORT_SYMBOL_GPL(xen_remap_domain_gfn_array); + +/* Returns: 0 success */ +int xen_unmap_domain_gfn_range(struct vm_area_struct *vma, + int numpgs, struct page **pages) +{ + if (!pages || !xen_feature(XENFEAT_auto_translated_physmap)) + return 0; + +#ifdef CONFIG_XEN_PVH + return xen_xlate_unmap_gfn_range(vma, numpgs, pages); +#else + return -EINVAL; +#endif +} +EXPORT_SYMBOL_GPL(xen_unmap_domain_gfn_range); + +unsigned long arbitrary_virt_to_mfn(void *vaddr) +{ + xmaddr_t maddr = arbitrary_virt_to_machine(vaddr); + + return PFN_DOWN(maddr.maddr); +} + +xmaddr_t arbitrary_virt_to_machine(void *vaddr) +{ + unsigned long address = (unsigned long)vaddr; + unsigned int level; + pte_t *pte; + unsigned offset; + + /* + * if the PFN is in the linear mapped vaddr range, we can just use + * the (quick) virt_to_machine() p2m lookup + */ + if (virt_addr_valid(vaddr)) + return virt_to_machine(vaddr); + + /* otherwise we have to do a (slower) full page-table walk */ + + pte = lookup_address(address, &level); + BUG_ON(pte == NULL); + offset = address & ~PAGE_MASK; + return XMADDR(((phys_addr_t)pte_mfn(*pte) << PAGE_SHIFT) + offset); +} +EXPORT_SYMBOL_GPL(arbitrary_virt_to_machine); diff --git a/arch/x86/xen/mmu_hvm.c b/arch/x86/xen/mmu_hvm.c new file mode 100644 index 0000000..c0ecb92 --- /dev/null +++ b/arch/x86/xen/mmu_hvm.c @@ -0,0 +1,77 @@ +#include +#include + +#include +#include + +#ifdef CONFIG_PROC_VMCORE +/* + * This function is used in two contexts: + * - the kdump kernel has to check whether a pfn of the crashed kernel + * was a ballooned page. vmcore is using this function to decide + * whether to access a pfn of the crashed kernel. + * - the kexec kernel has to check whether a pfn was ballooned by the + * previous kernel. If the pfn is ballooned, handle it properly. + * Returns 0 if the pfn is not backed by a RAM page, the caller may + * handle the pfn special in this case. + */ +static int xen_oldmem_pfn_is_ram(unsigned long pfn) +{ + struct xen_hvm_get_mem_type a = { + .domid = DOMID_SELF, + .pfn = pfn, + }; + int ram; + + if (HYPERVISOR_hvm_op(HVMOP_get_mem_type, &a)) + return -ENXIO; + + switch (a.mem_type) { + case HVMMEM_mmio_dm: + ram = 0; + break; + case HVMMEM_ram_rw: + case HVMMEM_ram_ro: + default: + ram = 1; + break; + } + + return ram; +} +#endif + +static void xen_hvm_exit_mmap(struct mm_struct *mm) +{ + struct xen_hvm_pagetable_dying a; + int rc; + + a.domid = DOMID_SELF; + a.gpa = __pa(mm->pgd); + rc = HYPERVISOR_hvm_op(HVMOP_pagetable_dying, &a); + WARN_ON_ONCE(rc < 0); +} + +static int is_pagetable_dying_supported(void) +{ + struct xen_hvm_pagetable_dying a; + int rc = 0; + + a.domid = DOMID_SELF; + a.gpa = 0x00; + rc = HYPERVISOR_hvm_op(HVMOP_pagetable_dying, &a); + if (rc < 0) { + printk(KERN_DEBUG "HVMOP_pagetable_dying not supported\n"); + return 0; + } + return 1; +} + +void __init xen_hvm_init_mmu_ops(void) +{ + if (is_pagetable_dying_supported()) + pv_mmu_ops.exit_mmap = xen_hvm_exit_mmap; +#ifdef CONFIG_PROC_VMCORE + register_oldmem_pfn_is_ram(&xen_oldmem_pfn_is_ram); +#endif +} diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index e4db19e..390168d 100644 --- a/drivers/xen/balloon.c +++ b/drivers/xen/balloon.c @@ -710,6 +710,7 @@ void free_xenballooned_pages(int nr_pages, struct page **pages) } EXPORT_SYMBOL(free_xenballooned_pages); +#ifdef CONFIG_XEN_PV static void __init balloon_add_region(unsigned long start_pfn, unsigned long pages) { @@ -733,19 +734,22 @@ static void __init balloon_add_region(unsigned long start_pfn, balloon_stats.total_pages += extra_pfn_end - start_pfn; } +#endif static int __init balloon_init(void) { - int i; - if (!xen_domain()) return -ENODEV; pr_info("Initialising balloon driver\n"); +#ifdef CONFIG_XEN_PV balloon_stats.current_pages = xen_pv_domain() ? min(xen_start_info->nr_pages - xen_released_pages, max_pfn) : get_num_physpages(); +#else + balloon_stats.current_pages = get_num_physpages(); +#endif balloon_stats.target_pages = balloon_stats.current_pages; balloon_stats.balloon_low = 0; balloon_stats.balloon_high = 0; @@ -762,14 +766,20 @@ static int __init balloon_init(void) register_sysctl_table(xen_root); #endif - /* - * Initialize the balloon with pages from the extra memory - * regions (see arch/x86/xen/setup.c). - */ - for (i = 0; i < XEN_EXTRA_MEM_MAX_REGIONS; i++) - if (xen_extra_mem[i].n_pfns) - balloon_add_region(xen_extra_mem[i].start_pfn, - xen_extra_mem[i].n_pfns); +#ifdef CONFIG_XEN_PV + { + int i; + + /* + * Initialize the balloon with pages from the extra memory + * regions (see arch/x86/xen/setup.c). + */ + for (i = 0; i < XEN_EXTRA_MEM_MAX_REGIONS; i++) + if (xen_extra_mem[i].n_pfns) + balloon_add_region(xen_extra_mem[i].start_pfn, + xen_extra_mem[i].n_pfns); + } +#endif return 0; } diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index bcf90ed..e20bbba 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -41,11 +41,24 @@ int xen_cpuhp_setup(int (*cpu_up_prepare_cb)(unsigned int), void xen_reboot(int reason); extern unsigned long *xen_contiguous_bitmap; +#ifdef CONFIG_XEN_PV int xen_create_contiguous_region(phys_addr_t pstart, unsigned int order, unsigned int address_bits, dma_addr_t *dma_handle); void xen_destroy_contiguous_region(phys_addr_t pstart, unsigned int order); +#else +static inline int xen_create_contiguous_region(phys_addr_t pstart, + unsigned int order, + unsigned int address_bits, + dma_addr_t *dma_handle) +{ + return 0; +} + +static inline void xen_destroy_contiguous_region(phys_addr_t pstart, + unsigned int order) { } +#endif struct vm_area_struct;