From patchwork Fri May 20 14:19:35 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 9129679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B1ECF6048B for ; Fri, 20 May 2016 14:22:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A10A321BED for ; Fri, 20 May 2016 14:22:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93A9E27BE4; Fri, 20 May 2016 14:22:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7F73421BED for ; Fri, 20 May 2016 14:22:10 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b3lHF-0000FA-VJ; Fri, 20 May 2016 14:19:45 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b3lHD-0000F4-RD for xen-devel@lists.xen.org; Fri, 20 May 2016 14:19:44 +0000 Received: from [193.109.254.147] by server-2.bemta-14.messagelabs.com id 74/CC-03279-FFC1F375; Fri, 20 May 2016 14:19:43 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFKsWRWlGSWpSXmKPExsVyuP0Ov+5fGft wg8v/VS2WfFzM4sDocXT3b6YAxijWzLyk/IoE1oxXe06yFXxqZ6xYMusiUwPjt5IuRk4OCQEj ibcT/zF1MXJwCAksZJRodwcJCwv4SRz4e4sVxBYRcJBYdv4FI0RJtMTli7JdjFwczAL/GSX+L 1nDDlLDJqAqseH6KVaQGl4BdYnrv2NBwixA4QPNj8GmiwqES+xY6QsS5hUQlDg58wkLSJgTqP r1fHGQMLOAv8SeJWdZJjDyzkJSNQtJCsLWk9hx/RcrhC0vsf3tHGYI216i6UM3G6a4tcSlhlV wNSe/nGDHpqZlxW9GmJoNv/YyYVPzadlquJrnh9ZhcYO1xOaWN4wLGAVWMWoUpxaVpRbpGprr JRVlpmeU5CZm5ugaGpro5aYWFyemp+YkJhXrJefnbmIExmE9AwPjDsafpz0PMUpyMCmJ8qqJ2 4cL8SXlp1RmJBZnxBeV5qQWH2KU4eBQkuB9Iw2UEyxKTU+tSMvMASYEmLQEB4+SCG8zSJq3uC AxtzgzHSJ1ilFRSpx3C0hCACSRUZoH1wZLQpcYZaWEeRkZGBiEeApSi3IzS1DlXzGKczAqCfM yA1OaEE9mXgnc9FdAi5mAFt8SswFZXJKIkJJqYGSrUynf0xK6oybikSjvgpCcm+Kztk2Vyax7 fFx3kmfOChn3my8eS3/MN2FtvP77hGT3UwY76fp7nxXdJVPV92cfFru73OGOf3YXS+9qmXz79 nNcieY5y59ExAR8DmXM6Fwr+lUynM359fzZs/YHzF96uSueLY6TJ7gl3klR7Y+xyoo/b6WvKr EUZyQaajEXFScCAMu8uwY9AwAA X-Env-Sender: jgross@suse.com X-Msg-Ref: server-15.tower-27.messagelabs.com!1463753981!42875338!1 X-Originating-IP: [195.135.220.15] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.34; banners=-,-,- X-VirusChecked: Checked Received: (qmail 14678 invoked from network); 20 May 2016 14:19:41 -0000 Received: from mx2.suse.de (HELO mx2.suse.de) (195.135.220.15) by server-15.tower-27.messagelabs.com with DHE-RSA-CAMELLIA256-SHA encrypted SMTP; 20 May 2016 14:19:41 -0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 7E5FEAB9D; Fri, 20 May 2016 14:19:40 +0000 (UTC) To: Julien Grall , xen-devel@lists.xen.org References: <1463750562-20549-1-git-send-email-jgross@suse.com> <573F1AD7.8040103@arm.com> From: Juergen Gross Message-ID: <573F1CF7.7080606@suse.com> Date: Fri, 20 May 2016 16:19:35 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 In-Reply-To: <573F1AD7.8040103@arm.com> Cc: sstabellini@kernel.org, wei.liu2@citrix.com, Wei Chen , George.Dunlap@eu.citrix.com, andrew.cooper3@citrix.com, Steve Capper , ian.jackson@eu.citrix.com, tim@xen.org, jbeulich@suse.com Subject: Re: [Xen-devel] [PATCH 0/2] Support consistent reads of mapped vcpu_runstate_info X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP On 20/05/16 16:10, Julien Grall wrote: > Hi Juergen, > > On 20/05/16 14:22, Juergen Gross wrote: >> A guest mapping vcpu_runstate_info into its memory can't read this >> information from another cpu but the one the data is referring to. >> Reason is there is no reliable way for the guest to detect a concurrent >> data update by the hypervisor. >> >> This patch series adds an update flag to the mapped data which can be >> used by the guest to detect an update is occurring. As this flag is >> modifying the current interface it has to be activated by using a >> vm_assist hypercall, which in turn has to be made available for ARM. >> >> Runtime tested on x86 with a modified Linux kernel using the new >> feature. >> Compile tested only for ARM. > > I would like to give a go on ARM. Who it be possible to provide the > patch for Linux and how to test it? Sure. You'll need the four attached patches (to be applied on top of kernel 4.6). With CONFIG_PARAVIRT_TIME_ACCOUNTING set in the kernel config, full functionality will be used (without being set the runstate info of other cpus won't be read). You can verify the vm_assist hypercall has worked via "xl debug-keys q" and "xl dmesg | grep vm_assist" (value should be 00000020 on ARM). Juergen From 689b4ba8c13be73ed51e485a7f7baea593d0ce6e Mon Sep 17 00:00:00 2001 From: Juergen Gross Date: Tue, 17 May 2016 14:03:02 +0200 Subject: [PATCH v4] xen: add steal_clock support on x86 The pv_time_ops structure contains a function pointer for the "steal_clock" functionality used only by KVM and Xen on ARM. Xen on x86 uses its own mechanism to account for the "stolen" time a thread wasn't able to run due to hypervisor scheduling. Add support in Xen arch independent time handling for this feature by moving it out of the arm arch into drivers/xen and remove the x86 Xen hack. Signed-off-by: Juergen Gross Reviewed-by: Boris Ostrovsky --- V4: minor adjustments as requested by Stefano Stabellini (remove no longer needed #include, remove __init from header) V3: add #include to avoid build error on arm V2: remove the x86 do_stolen_accounting() hack --- arch/arm/xen/enlighten.c | 18 ++---------------- arch/x86/xen/time.c | 44 ++------------------------------------------ drivers/xen/time.c | 20 ++++++++++++++++++++ include/linux/kernel_stat.h | 1 - include/xen/xen-ops.h | 1 + kernel/sched/cputime.c | 10 ---------- 6 files changed, 25 insertions(+), 69 deletions(-) diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c index 75cd734..71db30c 100644 --- a/arch/arm/xen/enlighten.c +++ b/arch/arm/xen/enlighten.c @@ -12,7 +12,6 @@ #include #include #include -#include #include #include #include @@ -84,19 +83,6 @@ int xen_unmap_domain_gfn_range(struct vm_area_struct *vma, } EXPORT_SYMBOL_GPL(xen_unmap_domain_gfn_range); -static unsigned long long xen_stolen_accounting(int cpu) -{ - struct vcpu_runstate_info state; - - BUG_ON(cpu != smp_processor_id()); - - xen_get_runstate_snapshot(&state); - - WARN_ON(state.state != RUNSTATE_running); - - return state.time[RUNSTATE_runnable] + state.time[RUNSTATE_offline]; -} - static void xen_read_wallclock(struct timespec64 *ts) { u32 version; @@ -355,8 +341,8 @@ static int __init xen_guest_init(void) register_cpu_notifier(&xen_cpu_notifier); - pv_time_ops.steal_clock = xen_stolen_accounting; - static_key_slow_inc(¶virt_steal_enabled); + xen_time_setup_guest(); + if (xen_initial_domain()) pvclock_gtod_register_notifier(&xen_pvclock_gtod_notifier); diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index a0a4e55..6be31df 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -11,8 +11,6 @@ #include #include #include -#include -#include #include #include #include @@ -31,44 +29,6 @@ /* Xen may fire a timer up to this many ns early */ #define TIMER_SLOP 100000 -#define NS_PER_TICK (1000000000LL / HZ) - -/* snapshots of runstate info */ -static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate_snapshot); - -/* unused ns of stolen time */ -static DEFINE_PER_CPU(u64, xen_residual_stolen); - -static void do_stolen_accounting(void) -{ - struct vcpu_runstate_info state; - struct vcpu_runstate_info *snap; - s64 runnable, offline, stolen; - cputime_t ticks; - - xen_get_runstate_snapshot(&state); - - WARN_ON(state.state != RUNSTATE_running); - - snap = this_cpu_ptr(&xen_runstate_snapshot); - - /* work out how much time the VCPU has not been runn*ing* */ - runnable = state.time[RUNSTATE_runnable] - snap->time[RUNSTATE_runnable]; - offline = state.time[RUNSTATE_offline] - snap->time[RUNSTATE_offline]; - - *snap = state; - - /* Add the appropriate number of ticks of stolen time, - including any left-overs from last time. */ - stolen = runnable + offline + __this_cpu_read(xen_residual_stolen); - - if (stolen < 0) - stolen = 0; - - ticks = iter_div_u64_rem(stolen, NS_PER_TICK, &stolen); - __this_cpu_write(xen_residual_stolen, stolen); - account_steal_ticks(ticks); -} /* Get the TSC speed from Xen */ static unsigned long xen_tsc_khz(void) @@ -335,8 +295,6 @@ static irqreturn_t xen_timer_interrupt(int irq, void *dev_id) ret = IRQ_HANDLED; } - do_stolen_accounting(); - return ret; } @@ -431,6 +389,8 @@ static void __init xen_time_init(void) xen_setup_timer(cpu); xen_setup_cpu_clockevents(); + xen_time_setup_guest(); + if (xen_initial_domain()) pvclock_gtod_register_notifier(&xen_pvclock_gtod_notifier); } diff --git a/drivers/xen/time.c b/drivers/xen/time.c index 7107842..2257b66 100644 --- a/drivers/xen/time.c +++ b/drivers/xen/time.c @@ -6,6 +6,7 @@ #include #include +#include #include #include @@ -75,6 +76,15 @@ bool xen_vcpu_stolen(int vcpu) return per_cpu(xen_runstate, vcpu).state == RUNSTATE_runnable; } +static u64 xen_steal_clock(int cpu) +{ + struct vcpu_runstate_info state; + + BUG_ON(cpu != smp_processor_id()); + xen_get_runstate_snapshot(&state); + return state.time[RUNSTATE_runnable] + state.time[RUNSTATE_offline]; +} + void xen_setup_runstate_info(int cpu) { struct vcpu_register_runstate_memory_area area; @@ -86,3 +96,13 @@ void xen_setup_runstate_info(int cpu) BUG(); } +void __init xen_time_setup_guest(void) +{ + pv_time_ops.steal_clock = xen_steal_clock; + + static_key_slow_inc(¶virt_steal_enabled); + /* + * We can't set paravirt_steal_rq_enabled as this would require the + * capability to read another cpu's runstate info. + */ +} diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index 25a822f..44fda64 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -92,7 +92,6 @@ static inline void account_process_tick(struct task_struct *tsk, int user) extern void account_process_tick(struct task_struct *, int user); #endif -extern void account_steal_ticks(unsigned long ticks); extern void account_idle_ticks(unsigned long ticks); #endif /* _LINUX_KERNEL_STAT_H */ diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 86abe07..77bf9d1 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -21,6 +21,7 @@ void xen_resume_notifier_unregister(struct notifier_block *nb); bool xen_vcpu_stolen(int vcpu); void xen_setup_runstate_info(int cpu); +void xen_time_setup_guest(void); void xen_get_runstate_snapshot(struct vcpu_runstate_info *res); int xen_setup_shutdown_event(void); diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 75f98c5..8c4c6dc 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -490,16 +490,6 @@ void account_process_tick(struct task_struct *p, int user_tick) } /* - * Account multiple ticks of steal time. - * @p: the process from which the cpu time has been stolen - * @ticks: number of stolen ticks - */ -void account_steal_ticks(unsigned long ticks) -{ - account_steal_time(jiffies_to_cputime(ticks)); -} - -/* * Account multiple ticks of idle time. * @ticks: number of stolen ticks */