From patchwork Wed May 17 15:57:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luwei Kang X-Patchwork-Id: 9732577 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7BA966020B for ; Thu, 18 May 2017 05:37:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 555CE2879E for ; Thu, 18 May 2017 05:37:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 468F0287FE; Thu, 18 May 2017 05:37:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.2 required=2.0 tests=BAYES_00, DATE_IN_PAST_12_24, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D8E7D2879E for ; Thu, 18 May 2017 05:37:06 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dBE4h-0000I4-Sr; Thu, 18 May 2017 05:34:11 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dBE4g-0000Hy-QF for xen-devel@lists.xen.org; Thu, 18 May 2017 05:34:10 +0000 Received: from [85.158.139.211] by server-6.bemta-5.messagelabs.com id A8/27-02180-2523D195; Thu, 18 May 2017 05:34:10 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrCLMWRWlGSWpSXmKPExsVywNwkVtfPSDb SYPIbYYslHxezODB6HN39mymAMYo1My8pvyKBNWPltmvsBTclKrZ1bmBuYFwk3MXIxSEkMJ1R YtmdZcxdjJwcEgK8EkeWzWCFsAMkXk56xw5iCwmUSlxY3sUEYrMJqEtsfb8RrEZEQFri2ufLj F2MHBzMAkUSi69GgISFBZwktt5bwwoSZhFQlVjcJgUS5hVwkZhw9hPUdDmJm+c6mScwci9gZF jFqFGcWlSWWqRrZKSXVJSZnlGSm5iZo2toYKqXm1pcnJiempOYVKyXnJ+7iRHo23oGBsYdjHv a/Q4xSnIwKYnyuv6TiRTiS8pPqcxILM6ILyrNSS0+xCjDwaEkwStgKBspJFiUmp5akZaZAwwy mLQEB4+SCO8vA6A0b3FBYm5xZjpE6hSjopQ4716QhABIIqM0D64NFtiXGGWlhHkZGRgYhHgKU otyM0tQ5V8xinMwKgnzMoNs58nMK4Gb/gpoMRPQ4uYH0iCLSxIRUlINjKuuG2+IiZGU/fh3be rhyri5fEcWcJrfyT9dPfHH66df1wQ5v1G7u7V+163tZlynrU7XWP6Vz9C74zjLP4P5/IpP027 zTX1deMrH19Jh0dW0dyLxl2qj5HZmul24tWD7CRZ1az5mtbAzji/jD/1xvrbEdStvQaSMq+3D Cu+F67KnV7eWm8qJXlFiKc5INNRiLipOBACiRR0rZwIAAA== X-Env-Sender: luwei.kang@intel.com X-Msg-Ref: server-16.tower-206.messagelabs.com!1495085643!82999373!1 X-Originating-IP: [192.55.52.93] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTkyLjU1LjUyLjkzID0+IDMyNDY2NQ==\n X-StarScan-Received: X-StarScan-Version: 9.4.12; banners=-,-,- X-VirusChecked: Checked Received: (qmail 46923 invoked from network); 18 May 2017 05:34:06 -0000 Received: from mga11.intel.com (HELO mga11.intel.com) (192.55.52.93) by server-16.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 18 May 2017 05:34:06 -0000 Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 May 2017 22:34:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.38,357,1491289200"; d="scan'208"; a="1170866536" Received: from vmm.bj.intel.com ([10.238.135.172]) by fmsmga002.fm.intel.com with ESMTP; 17 May 2017 22:33:58 -0700 From: Luwei Kang To: xen-devel@lists.xen.org Date: Wed, 17 May 2017 23:57:05 +0800 Message-Id: <1495036625-24071-1-git-send-email-luwei.kang@intel.com> X-Mailer: git-send-email 2.7.4 Cc: andrew.cooper3@citrix.com, boris.ostrovsky@oracle.com, Luwei Kang , jbeulich@suse.com Subject: [Xen-devel] [PATCH v2] x86/vpmu: add cpu hot unplug notifier for vpmu X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Currently, Hot unplug a physical CPU with vpmu enabled may cause system hang due to send a remote call to an offlined pCPU. This patch add a cpu hot unplug notifer to save vpmu context before cpu offline. Consider one scenario, hot unplug pCPU N with vpmu enabled. The vcpu which running on this pCPU will be switch to other online cpu. A remote call will be send to pCPU N to save the vpmu context before loading the vpmu context on this pCPU. System will hang in function on_select_cpus() because of that pCPU is offlined and can not do any respond. Signed-off-by: Luwei Kang --- v2: 1.fix some typo and coding style; 2.change "swith" to "if" in cpu_callback() because of there just have one case; 3.add VPMU_CONTEX_LOADED check before send remote call in vpmu_arch_destroy(); --- xen/arch/x86/cpu/vpmu.c | 42 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c index 03401fd..57a0e9d 100644 --- a/xen/arch/x86/cpu/vpmu.c +++ b/xen/arch/x86/cpu/vpmu.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -581,9 +582,14 @@ static void vpmu_arch_destroy(struct vcpu *v) if ( vpmu->arch_vpmu_ops && vpmu->arch_vpmu_ops->arch_vpmu_destroy ) { - /* Unload VPMU first. This will stop counters */ - on_selected_cpus(cpumask_of(vcpu_vpmu(v)->last_pcpu), - vpmu_save_force, v, 1); + /* + * Unload VPMU first if VPMU_CONTEXT_LOADED being set. + * This will stop counters. + */ + if ( vpmu_is_set(vpmu, VPMU_CONTEXT_LOADED) ) + on_selected_cpus(cpumask_of(vcpu_vpmu(v)->last_pcpu), + vpmu_save_force, v, 1); + vpmu->arch_vpmu_ops->arch_vpmu_destroy(v); } } @@ -835,6 +841,33 @@ long do_xenpmu_op(unsigned int op, XEN_GUEST_HANDLE_PARAM(xen_pmu_params_t) arg) return ret; } +static int cpu_callback( + struct notifier_block *nfb, unsigned long action, void *hcpu) +{ + unsigned int cpu = (unsigned long)hcpu; + struct vcpu *vcpu = per_cpu(last_vcpu, cpu); + struct vpmu_struct *vpmu; + + if ( !vcpu ) + return NOTIFY_DONE; + + vpmu = vcpu_vpmu(vcpu); + if ( !vpmu_is_set(vpmu, VPMU_CONTEXT_ALLOCATED) ) + return NOTIFY_DONE; + + if ( action == CPU_DYING ) + { + vpmu_save_force(vcpu); + vpmu_reset(vpmu, VPMU_CONTEXT_LOADED); + } + + return NOTIFY_DONE; +} + +static struct notifier_block vpmu_cpu_nfb = { + .notifier_call = cpu_callback +}; + static int __init vpmu_init(void) { int vendor = current_cpu_data.x86_vendor; @@ -872,8 +905,11 @@ static int __init vpmu_init(void) } if ( vpmu_mode != XENPMU_MODE_OFF ) + { + register_cpu_notifier(&vpmu_cpu_nfb); printk(XENLOG_INFO "VPMU: version " __stringify(XENPMU_VER_MAJ) "." __stringify(XENPMU_VER_MIN) "\n"); + } else opt_vpmu_enabled = 0;