From patchwork Wed Aug 3 13:00:48 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 9261213 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8DA9C6089F for ; Wed, 3 Aug 2016 13:03:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E46528407 for ; Wed, 3 Aug 2016 13:03:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 72CFA28578; Wed, 3 Aug 2016 13:03:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 39C5B28554 for ; Wed, 3 Aug 2016 13:03:21 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUvn5-0004je-SP; Wed, 03 Aug 2016 13:00:55 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bUvn4-0004jL-Pf for xen-devel@lists.xenproject.org; Wed, 03 Aug 2016 13:00:54 +0000 Received: from [85.158.139.211] by server-14.bemta-5.messagelabs.com id 56/6A-10347-60BE1A75; Wed, 03 Aug 2016 13:00:54 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNIsWRWlGSWpSXmKPExsXS6fjDS5f19cJ wg/cbJS2+b5nM5MDocfjDFZYAxijWzLyk/IoE1oyXMzazFZyyqLj/cwVrA+NtvS5GTg4hgTyJ BRM2snUxcnDwCthJzPqXCRKWEDCUePr+OhuIzSKgKnF/0lEwm01AXaLt2XZWkHIRAQOJc0eTQ ExmAX2JbetYQCqEBWIlHixtYIQYbifx4OECMJtTwF7i09w5LBCLBCX+7hAGCTMDlUzbto95Ai PPLITMLCQZCFtL4uGvWywQtrbEsoWvmWeB7ZWWWP6PAyLsIvH87H1WVCUgtr/Er8k97AsYOVY xahSnFpWlFukamuslFWWmZ5TkJmbm6BoamOrlphYXJ6an5iQmFesl5+duYgQGKgMQ7GC8eNrz EKMkB5OSKO/EgwvDhfiS8lMqMxKLM+KLSnNSiw8xynBwKEnw3nkJlBMsSk1PrUjLzAHGDExag oNHSYT31nOgNG9xQWJucWY6ROoUo6KUOO97kD4BkERGaR5cGyxOLzHKSgnzMgIdIsRTkFqUm1 mCKv+KUZyDUUmYV+IV0BSezLwSuOmvgBYzAS0+YbAAZHFJIkJKqoGRrcT45osVd94HzHsnbSl wwz4v9Wh88VynG0smXu6OS1/Q3B3JH937aHfZudNLeERf/n48e9Gbexo2LMbz1a4wXzlcJyti 9qLOaVemsX7b0v82VtGF+zvY10SUqjOc93n6O/BhS76rfV1/nzxDhHWp8g59lqe/OzRMFkqoG x6fcUl+Pu+N/7+UWIozEg21mIuKEwGIqD4VzgIAAA== X-Env-Sender: JBeulich@suse.com X-Msg-Ref: server-11.tower-206.messagelabs.com!1470229251!40550960!1 X-Originating-IP: [137.65.248.74] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.77; banners=-,-,- X-VirusChecked: Checked Received: (qmail 39513 invoked from network); 3 Aug 2016 13:00:53 -0000 Received: from prv-mh.provo.novell.com (HELO prv-mh.provo.novell.com) (137.65.248.74) by server-11.tower-206.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 3 Aug 2016 13:00:53 -0000 Received: from INET-PRV-MTA by prv-mh.provo.novell.com with Novell_GroupWise; Wed, 03 Aug 2016 07:00:50 -0600 Message-Id: <57A20720020000780010238D@prv-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 14.2.1 Date: Wed, 03 Aug 2016 07:00:48 -0600 From: "Jan Beulich" To: "xen-devel" References: <57A205C50200007800102378@prv-mh.provo.novell.com> In-Reply-To: <57A205C50200007800102378@prv-mh.provo.novell.com> Mime-Version: 1.0 Cc: Andrew Cooper Subject: [Xen-devel] [PATCH v2 2/6] x86/time: correctly honor late clearing of TSC related feature flags X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP As such clearing of flags may have an impact on the selected rendezvous function, handle such in a central place. But don't allow such feature flags to be cleared during CPU hotplug: Platform and local system times may have diverged significantly by then, potentially causing noticably (even if only temporary) strange behavior. As we're anyway expecting only sufficiently similar CPUs to appear during hotplug, this shouldn't be introducing new limitations. Reported-by: Joao Martins Signed-off-by: Jan Beulich Tested-by: Dario Faggioli Tested-by: Joao Martins x86/time: correctly honor late clearing of TSC related feature flags As such clearing of flags may have an impact on the selected rendezvous function, handle such in a central place. But don't allow such feature flags to be cleared during CPU hotplug: Platform and local system times may have diverged significantly by then, potentially causing noticably (even if only temporary) strange behavior. As we're anyway expecting only sufficiently similar CPUs to appear during hotplug, this shouldn't be introducing new limitations. Reported-by: Joao Martins Signed-off-by: Jan Beulich Tested-by: Dario Faggioli Tested-by: Joao Martins --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -1162,8 +1162,8 @@ static int mwait_idle_cpu_init(struct no } if (state > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !pm_idle_save) - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + !pm_idle_save && system_state < SYS_STATE_active) + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); cx = dev->states + dev->count; cx->type = state; --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1353,6 +1353,24 @@ static void time_calibration(void *unuse &r, 1); } +void __init clear_tsc_cap(unsigned int feature) +{ + void (*rendezvous_fn)(void *) = time_calibration_std_rendezvous; + + if ( feature ) + setup_clear_cpu_cap(feature); + + /* If we have constant-rate TSCs then scale factor can be shared. */ + if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) + { + /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ + if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) + rendezvous_fn = time_calibration_tsc_rendezvous; + } + + time_calibration_rendezvous_fn = rendezvous_fn; +} + static struct { s_time_t local_stime, master_stime; } ap_bringup_ref; @@ -1478,7 +1496,7 @@ static int __init verify_tsc_reliability { printk("%s: TSC warp detected, disabling TSC_RELIABLE\n", __func__); - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); } } @@ -1491,13 +1509,7 @@ int __init init_xen_time(void) { tsc_check_writability(); - /* If we have constant-rate TSCs then scale factor can be shared. */ - if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) - { - /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ - if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) - time_calibration_rendezvous_fn = time_calibration_tsc_rendezvous; - } + clear_tsc_cap(0); open_softirq(TIME_CALIBRATE_SOFTIRQ, local_time_calibration); --- a/xen/include/asm-x86/time.h +++ b/xen/include/asm-x86/time.h @@ -70,6 +70,7 @@ void tsc_get_info(struct domain *d, uint void force_update_vcpu_system_time(struct vcpu *v); int host_tsc_is_safe(void); +void clear_tsc_cap(unsigned int feature); void cpuid_time_leaf(uint32_t sub_idx, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx); --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -1162,8 +1162,8 @@ static int mwait_idle_cpu_init(struct no } if (state > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !pm_idle_save) - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + !pm_idle_save && system_state < SYS_STATE_active) + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); cx = dev->states + dev->count; cx->type = state; --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1353,6 +1353,24 @@ static void time_calibration(void *unuse &r, 1); } +void __init clear_tsc_cap(unsigned int feature) +{ + void (*rendezvous_fn)(void *) = time_calibration_std_rendezvous; + + if ( feature ) + setup_clear_cpu_cap(feature); + + /* If we have constant-rate TSCs then scale factor can be shared. */ + if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) + { + /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ + if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) + rendezvous_fn = time_calibration_tsc_rendezvous; + } + + time_calibration_rendezvous_fn = rendezvous_fn; +} + static struct { s_time_t local_stime, master_stime; } ap_bringup_ref; @@ -1478,7 +1496,7 @@ static int __init verify_tsc_reliability { printk("%s: TSC warp detected, disabling TSC_RELIABLE\n", __func__); - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); } } @@ -1491,13 +1509,7 @@ int __init init_xen_time(void) { tsc_check_writability(); - /* If we have constant-rate TSCs then scale factor can be shared. */ - if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) - { - /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ - if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) - time_calibration_rendezvous_fn = time_calibration_tsc_rendezvous; - } + clear_tsc_cap(0); open_softirq(TIME_CALIBRATE_SOFTIRQ, local_time_calibration); --- a/xen/include/asm-x86/time.h +++ b/xen/include/asm-x86/time.h @@ -70,6 +70,7 @@ void tsc_get_info(struct domain *d, uint void force_update_vcpu_system_time(struct vcpu *v); int host_tsc_is_safe(void); +void clear_tsc_cap(unsigned int feature); void cpuid_time_leaf(uint32_t sub_idx, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);