From patchwork Tue Jun 28 16:47:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Kuznetsov X-Patchwork-Id: 9203617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9163D6075F for ; Tue, 28 Jun 2016 16:49:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83478285FD for ; Tue, 28 Jun 2016 16:49:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7724C2860E; Tue, 28 Jun 2016 16:49:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4B5C8285FD for ; Tue, 28 Jun 2016 16:49:31 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bHwAo-0003bu-Bj; Tue, 28 Jun 2016 16:47:42 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bHwAn-0003bC-B0 for xen-devel@lists.xenproject.org; Tue, 28 Jun 2016 16:47:41 +0000 Received: from [193.109.254.147] by server-9.bemta-14.messagelabs.com id 46/A3-32749-C2AA2775; Tue, 28 Jun 2016 16:47:40 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrMLMWRWlGSWpSXmKPExsVysWW7jK7OqqJ wgzsrOSy+b5nM5MDocfjDFZYAxijWzLyk/IoE1owLb36zFzxXrjh5awJjA+NxuS5GLg4hgd1M Eo+m7GfrYuQEco4wSlz7WA9iswnoSHx/eooZxBYRUJK4t2oyE0gDs8AUZonPG7ezgiSEBWwlv qy8wNjFyMHBIqAq0XHfHyTMK+AksWv1FrASTgFnic/t29gh5jtJnFgxD2yXhIC2xPMLk9lBZk oI9DFKLJ/zjX0CI88CRoZVjBrFqUVlqUW6hgZ6SUWZ6RkluYmZObqGhiZ6uanFxYnpqTmJScV 6yfm5mxiBnmcAgh2M55Y5H2KU5GBSEuX91lsULsSXlJ9SmZFYnBFfVJqTWnyIUYaDQ0mCt30l UE6wKDU9tSItMwcYgjBpCQ4eJRHeZyuA0rzFBYm5xZnpEKlTjIpS4rxeIH0CIImM0jy4NljYX 2KUlRLmZQQ6RIinILUoN7MEVf4VozgHo5IwbzjIFJ7MvBK46a+AFjMBLWatzgdZXJKIkJJqYD wf61HcvSt1jnzMpwSl2i0enZuPlLqs7J3wQy9npdHORZdcb9zV0enbNnf6m9vprC7SoffWOss E7V6rwhI/48Pkri3lNxfsflN9wlUhpExpWlqD78J5J46vllvdaND+WqmzzUrPIIw14ouwzsqV TmcVH96uDf75mkFO/Hq0UPTzlJBXXf0TupVYijMSDbWYi4oTAbnHTt52AgAA X-Env-Sender: vkuznets@redhat.com X-Msg-Ref: server-15.tower-27.messagelabs.com!1467132458!50676418!1 X-Originating-IP: [209.132.183.28] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMjA5LjEzMi4xODMuMjggPT4gNTQwNjQ=\n X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 44330 invoked from network); 28 Jun 2016 16:47:39 -0000 Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by server-15.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 28 Jun 2016 16:47:39 -0000 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EEDDF9B0DF; Tue, 28 Jun 2016 16:47:37 +0000 (UTC) Received: from vitty.brq.redhat.com (vitty.brq.redhat.com [10.34.26.3]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u5SGlUI7004662; Tue, 28 Jun 2016 12:47:35 -0400 From: Vitaly Kuznetsov To: xen-devel@lists.xenproject.org Date: Tue, 28 Jun 2016 18:47:23 +0200 Message-Id: <1467132449-1030-3-git-send-email-vkuznets@redhat.com> In-Reply-To: <1467132449-1030-1-git-send-email-vkuznets@redhat.com> References: <1467132449-1030-1-git-send-email-vkuznets@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 28 Jun 2016 16:47:38 +0000 (UTC) Cc: Juergen Gross , Stefano Stabellini , x86@kernel.org, linux-kernel@vger.kernel.org, Julien Grall , Ingo Molnar , David Vrabel , "H. Peter Anvin" , Boris Ostrovsky , Thomas Gleixner Subject: [Xen-devel] [PATCH linux 2/8] xen: introduce xen_vcpu_id mapping X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP It may happen that Xen's and Linux's ideas of vCPU id diverge. In particular, when we crash on a secondary vCPU we may want to do kdump and unlike plain kexec where we do migrate_to_reboot_cpu() we try booting on the vCPU which crashed. This doesn't work very well for PVHVM guests as we have a number of hypercalls where we pass vCPU id as a parameter. These hypercalls either fail or do something unexpected. To solve the issue introduce percpu xen_vcpu_id mapping. ARM and PV guests get direct mapping for now. Boot CPU for PVHVM guest gets its id from CPUID. With secondary CPUs it is a bit more trickier. Currently, we initialize IPI vectors before these CPUs boot so we can't use CPUID. However, we know that physical CPU id (vLAPIC id) is Xen's vCPU id * 2, we can piggyback on that. Alternatively, we could have disabled all secondary CPUs once we detect that Xen's and Linux's ideas of vCPU id diverged. Signed-off-by: Vitaly Kuznetsov --- arch/arm/xen/enlighten.c | 10 ++++++++++ arch/x86/xen/enlighten.c | 18 +++++++++++++++++- include/xen/xen-ops.h | 1 + 3 files changed, 28 insertions(+), 1 deletion(-) diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c index 75cd734..ea99ca2 100644 --- a/arch/arm/xen/enlighten.c +++ b/arch/arm/xen/enlighten.c @@ -46,6 +46,10 @@ struct shared_info *HYPERVISOR_shared_info = (void *)&xen_dummy_shared_info; DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu); static struct vcpu_info __percpu *xen_vcpu_info; +/* Linux <-> Xen vCPU id mapping */ +DEFINE_PER_CPU(int, xen_vcpu_id) = -1; +EXPORT_SYMBOL_GPL(xen_vcpu_id); + /* These are unused until we support booting "pre-ballooned" */ unsigned long xen_released_pages; struct xen_memory_region xen_extra_mem[XEN_EXTRA_MEM_MAX_REGIONS] __initdata; @@ -179,6 +183,9 @@ static void xen_percpu_init(void) pr_info("Xen: initializing cpu%d\n", cpu); vcpup = per_cpu_ptr(xen_vcpu_info, cpu); + /* Direct vCPU id mapping for ARM guests. */ + per_cpu(xen_vcpu_id, cpu) = cpu; + info.mfn = virt_to_gfn(vcpup); info.offset = xen_offset_in_page(vcpup); @@ -328,6 +335,9 @@ static int __init xen_guest_init(void) if (xen_vcpu_info == NULL) return -ENOMEM; + /* Direct vCPU id mapping for ARM guests. */ + per_cpu(xen_vcpu_id, 0) = 0; + if (gnttab_setup_auto_xlat_frames(grant_frames)) { free_percpu(xen_vcpu_info); return -ENOMEM; diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 760789a..69f4c0c 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -59,6 +59,7 @@ #include #include #include +#include #include #include #include @@ -118,6 +119,10 @@ DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu); */ DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info); +/* Linux <-> Xen vCPU id mapping */ +DEFINE_PER_CPU(int, xen_vcpu_id) = -1; +EXPORT_SYMBOL_GPL(xen_vcpu_id); + enum xen_domain_type xen_domain_type = XEN_NATIVE; EXPORT_SYMBOL_GPL(xen_domain_type); @@ -1137,8 +1142,11 @@ void xen_setup_vcpu_info_placement(void) { int cpu; - for_each_possible_cpu(cpu) + for_each_possible_cpu(cpu) { + /* Set up direct vCPU id mapping for PV guests. */ + per_cpu(xen_vcpu_id, cpu) = cpu; xen_vcpu_setup(cpu); + } /* xen_vcpu_setup managed to place the vcpu_info within the * percpu area for all cpus, so make use of it. Note that for @@ -1797,6 +1805,12 @@ static void __init init_hvm_pv_info(void) xen_setup_features(); + cpuid(base + 4, &eax, &ebx, &ecx, &edx); + if (eax & XEN_HVM_CPUID_VCPU_ID_PRESENT) + this_cpu_write(xen_vcpu_id, ebx); + else + this_cpu_write(xen_vcpu_id, smp_processor_id()); + pv_info.name = "Xen HVM"; xen_domain_type = XEN_HVM_DOMAIN; @@ -1808,6 +1822,8 @@ static int xen_hvm_cpu_notify(struct notifier_block *self, unsigned long action, int cpu = (long)hcpu; switch (action) { case CPU_UP_PREPARE: + /* vLAPIC_ID == Xen's vCPU_ID * 2 for HVM guests */ + per_cpu(xen_vcpu_id, cpu) = cpu_physical_id(cpu) / 2; xen_vcpu_setup(cpu); if (xen_have_vector_callback) { if (xen_feature(XENFEAT_hvm_safe_pvclock)) diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 86abe07..b02a343 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -8,6 +8,7 @@ #include DECLARE_PER_CPU(struct vcpu_info *, xen_vcpu); +DECLARE_PER_CPU(int, xen_vcpu_id); void xen_arch_pre_suspend(void); void xen_arch_post_suspend(int suspend_cancelled);