From patchwork Wed Jun 15 10:28:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 9178115 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7493F60776 for ; Wed, 15 Jun 2016 10:30:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 60DDB2804C for ; Wed, 15 Jun 2016 10:30:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 55C0B281FE; Wed, 15 Jun 2016 10:30:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A59902804C for ; Wed, 15 Jun 2016 10:30:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bD849-0007hp-R0; Wed, 15 Jun 2016 10:28:57 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bD849-0007hZ-3m for xen-devel@lists.xenproject.org; Wed, 15 Jun 2016 10:28:57 +0000 Received: from [85.158.137.68] by server-16.bemta-3.messagelabs.com id 48/D5-23871-8ED21675; Wed, 15 Jun 2016 10:28:56 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrLIsWRWlGSWpSXmKPExsXS6fjDS/e5bmK 4wa07ghbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8af3QeYCt6ZVXy59IypgbFft4uRk0NIIE/i +L0Z7CA2r4CdxPdvx9hAbAkBQ4l981eB2SwCqhJnPq4Eq2ETUJdoe7adtYuRg0NEwEDi3NGkL kYuDmaBbkaJHZvOMoHUCAtESby908IGMd9OYvnfLawgNqeAvcTdLw/ZQHp5BQQl/u4QBgkzA5 W8aPvINIGRZxZCZhaSDIStJfHw1y0WCFtbYtnC18wg5cwC0hLL/3FAhF0knp+9z4qqBMT2l/g 1uYd9ASPHKkb14tSistQiXQu9pKLM9IyS3MTMHF1DA2O93NTi4sT01JzEpGK95PzcTYzAYGUA gh2MF9qdDzFKcjApifJ6yCWGC/El5adUZiQWZ8QXleakFh9ilOHgUJLg3a0DlBMsSk1PrUjLz AHGDUxagoNHSYS3FSTNW1yQmFucmQ6ROsWoKCXO2wKSEABJZJTmwbXBYvUSo6yUMC8j0CFCPA WpRbmZJajyrxjFORiVhHkvgEzhycwrgZv+CmgxE9Bim+nxIItLEhFSUg2MypLb8pd8Uvb8F/b NOXG1UJpGlN7TlTVZuRNOlurnbzdwuSI91chh6lz7R6VbLk+06mHJCzI9HfHoR5rIn5vX3kzZ dcpgUtxOl1f3rP5Mdbiw71vd40/3NeZI7o0zs0lZ41u789czN6ceeQm/VIarf3yjtZTCjM66J AUkLCvyuHHTJHJB3v0lSizFGYmGWsxFxYkAGHceZtACAAA= X-Env-Sender: JBeulich@suse.com X-Msg-Ref: server-4.tower-31.messagelabs.com!1465986533!35826647!1 X-Originating-IP: [137.65.248.74] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 33475 invoked from network); 15 Jun 2016 10:28:54 -0000 Received: from prv-mh.provo.novell.com (HELO prv-mh.provo.novell.com) (137.65.248.74) by server-4.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 15 Jun 2016 10:28:54 -0000 Received: from INET-PRV-MTA by prv-mh.provo.novell.com with Novell_GroupWise; Wed, 15 Jun 2016 04:28:52 -0600 Message-Id: <57614A0402000078000F53A5@prv-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 14.2.0 Date: Wed, 15 Jun 2016 04:28:52 -0600 From: "Jan Beulich" To: "xen-devel" References: <576140F302000078000F52FE@prv-mh.provo.novell.com> In-Reply-To: <576140F302000078000F52FE@prv-mh.provo.novell.com> Mime-Version: 1.0 Cc: Andrew Cooper , Dario Faggioli , Joao Martins Subject: [Xen-devel] [PATCH 5/8] x86/time: correctly honor late clearing of TSC related feature flags X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP As such clearing of flags may have an impact on the selected rendezvous function, handle such in a central place. But don't allow such feature flags to be cleared during CPU hotplug: Platform and local system times may have diverged significantly by then, potentially causing noticably (even if only temporary) strange behavior. As we're anyway expecting only sufficiently similar CPUs to appear during hotplug, this shouldn't be introducing new limitations. Reported-by: Joao Martins Signed-off-by: Jan Beulich x86/time: correctly honor late clearing of TSC related feature flags As such clearing of flags may have an impact on the selected rendezvous function, handle such in a central place. But don't allow such feature flags to be cleared during CPU hotplug: Platform and local system times may have diverged significantly by then, potentially causing noticably (even if only temporary) strange behavior. As we're anyway expecting only sufficiently similar CPUs to appear during hotplug, this shouldn't be introducing new limitations. Reported-by: Joao Martins Signed-off-by: Jan Beulich --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -1135,8 +1135,8 @@ static int mwait_idle_cpu_init(struct no } if (state > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !pm_idle_save) - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + !pm_idle_save && system_state < SYS_STATE_active) + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); cx = dev->states + dev->count; cx->type = state; --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1358,6 +1358,24 @@ static void time_calibration(void *unuse &r, 1); } +void __init clear_tsc_cap(unsigned int feature) +{ + void (*rendezvous_fn)(void *) = time_calibration_std_rendezvous; + + if ( feature ) + setup_clear_cpu_cap(feature); + + /* If we have constant-rate TSCs then scale factor can be shared. */ + if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) + { + /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ + if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) + rendezvous_fn = time_calibration_tsc_rendezvous; + } + + time_calibration_rendezvous_fn = rendezvous_fn; +} + static struct { s_time_t local_stime, master_stime; } ap_bringup_ref; @@ -1482,7 +1500,7 @@ static int __init verify_tsc_reliability { printk("%s: TSC warp detected, disabling TSC_RELIABLE\n", __func__); - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); } } @@ -1495,13 +1513,7 @@ int __init init_xen_time(void) { tsc_check_writability(); - /* If we have constant-rate TSCs then scale factor can be shared. */ - if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) - { - /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ - if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) - time_calibration_rendezvous_fn = time_calibration_tsc_rendezvous; - } + clear_tsc_cap(0); open_softirq(TIME_CALIBRATE_SOFTIRQ, local_time_calibration); --- a/xen/include/asm-x86/time.h +++ b/xen/include/asm-x86/time.h @@ -71,6 +71,7 @@ void tsc_get_info(struct domain *d, uint void force_update_vcpu_system_time(struct vcpu *v); int host_tsc_is_safe(void); +void clear_tsc_cap(unsigned int feature); void cpuid_time_leaf(uint32_t sub_idx, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx); --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -1135,8 +1135,8 @@ static int mwait_idle_cpu_init(struct no } if (state > 2 && !boot_cpu_has(X86_FEATURE_NONSTOP_TSC) && - !pm_idle_save) - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + !pm_idle_save && system_state < SYS_STATE_active) + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); cx = dev->states + dev->count; cx->type = state; --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -1358,6 +1358,24 @@ static void time_calibration(void *unuse &r, 1); } +void __init clear_tsc_cap(unsigned int feature) +{ + void (*rendezvous_fn)(void *) = time_calibration_std_rendezvous; + + if ( feature ) + setup_clear_cpu_cap(feature); + + /* If we have constant-rate TSCs then scale factor can be shared. */ + if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) + { + /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ + if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) + rendezvous_fn = time_calibration_tsc_rendezvous; + } + + time_calibration_rendezvous_fn = rendezvous_fn; +} + static struct { s_time_t local_stime, master_stime; } ap_bringup_ref; @@ -1482,7 +1500,7 @@ static int __init verify_tsc_reliability { printk("%s: TSC warp detected, disabling TSC_RELIABLE\n", __func__); - setup_clear_cpu_cap(X86_FEATURE_TSC_RELIABLE); + clear_tsc_cap(X86_FEATURE_TSC_RELIABLE); } } @@ -1495,13 +1513,7 @@ int __init init_xen_time(void) { tsc_check_writability(); - /* If we have constant-rate TSCs then scale factor can be shared. */ - if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) ) - { - /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */ - if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ) - time_calibration_rendezvous_fn = time_calibration_tsc_rendezvous; - } + clear_tsc_cap(0); open_softirq(TIME_CALIBRATE_SOFTIRQ, local_time_calibration); --- a/xen/include/asm-x86/time.h +++ b/xen/include/asm-x86/time.h @@ -71,6 +71,7 @@ void tsc_get_info(struct domain *d, uint void force_update_vcpu_system_time(struct vcpu *v); int host_tsc_is_safe(void); +void clear_tsc_cap(unsigned int feature); void cpuid_time_leaf(uint32_t sub_idx, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);