| Message ID | 1482482513-24151-1-git-send-email-yu.c.chen@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Fri, Dec 23, 2016 at 04:41:53PM +0800, Chen Yu wrote: > Currently the notifier of pvclock_gtod_notify() get invoked > frequently due to the periodic update_wall_time(). This might > slow down the system a little bit as there might be redundant > execution code path and unnecessary lock contention > in update_pvclock_gtod(), which was found when I was doing > suspend/resume speed testings. As pvclock_gtod_notify() > should be invoked only when clocksource has changed, according to > Commit 16e8d74d2da9 ("KVM: x86: notifier for clocksource changes") > , either we can add a new notifier for clocksource switch, > or we can simply bypass the following code in pvclock_gtod_notify() > earlier if there is no clocksource switch. > > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: "Radim Krcmar" <rkrcmar@redhat.com> > Cc: Marcelo Tosatti <mtosatti@redhat.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Ingo Molnar <mingo@redhat.com> > Cc: "H. Peter Anvin" <hpa@zytor.com> > Cc: Wanpeng Li <wanpeng.li@hotmail.com> > Signed-off-by: Chen Yu <yu.c.chen@intel.com> > --- > arch/x86/kvm/x86.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 445c51b..54aa32d 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -5961,13 +5961,14 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused, > struct pvclock_gtod_data *gtod = &pvclock_gtod_data; > struct timekeeper *tk = priv; > > + if (likely(gtod->clock.vclock_mode == VCLOCK_TSC)) > + return 0; I think this is only safe if any of the values in "struct pvclock_gtod_data" are unchanged. Otherwise the local (KVM) copy is kept incorrect. > update_pvclock_gtod(tk); > > /* disable master clock if host does not trust, or does not > * use, TSC clocksource > */ > - if (gtod->clock.vclock_mode != VCLOCK_TSC && > - atomic_read(&kvm_guest_has_master_clock) != 0) > + if (atomic_read(&kvm_guest_has_master_clock) != 0) > queue_work(system_long_wq, &pvclock_gtod_work); > > return 0; > -- > 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Marcelo, On Mon, Dec 26, 2016 at 05:44:25PM -0200, Marcelo Tosatti wrote: > On Fri, Dec 23, 2016 at 04:41:53PM +0800, Chen Yu wrote: > > Currently the notifier of pvclock_gtod_notify() get invoked > > frequently due to the periodic update_wall_time(). This might > > slow down the system a little bit as there might be redundant > > execution code path and unnecessary lock contention > > in update_pvclock_gtod(), which was found when I was doing > > suspend/resume speed testings. As pvclock_gtod_notify() > > should be invoked only when clocksource has changed, according to > > Commit 16e8d74d2da9 ("KVM: x86: notifier for clocksource changes") > > , either we can add a new notifier for clocksource switch, > > or we can simply bypass the following code in pvclock_gtod_notify() > > earlier if there is no clocksource switch. > > > > Cc: Paolo Bonzini <pbonzini@redhat.com> > > Cc: "Radim Krcmar" <rkrcmar@redhat.com> > > Cc: Marcelo Tosatti <mtosatti@redhat.com> > > Cc: Thomas Gleixner <tglx@linutronix.de> > > Cc: Ingo Molnar <mingo@redhat.com> > > Cc: "H. Peter Anvin" <hpa@zytor.com> > > Cc: Wanpeng Li <wanpeng.li@hotmail.com> > > Signed-off-by: Chen Yu <yu.c.chen@intel.com> > > --- > > arch/x86/kvm/x86.c | 5 +++-- > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > index 445c51b..54aa32d 100644 > > --- a/arch/x86/kvm/x86.c > > +++ b/arch/x86/kvm/x86.c > > @@ -5961,13 +5961,14 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused, > > struct pvclock_gtod_data *gtod = &pvclock_gtod_data; > > struct timekeeper *tk = priv; > > > > + if (likely(gtod->clock.vclock_mode == VCLOCK_TSC)) > > + return 0; > > I think this is only safe if any of the values in "struct > pvclock_gtod_data" are unchanged. Otherwise the local (KVM) copy is > kept incorrect. I missread the code previously and I thought only under the condition the clocksource has been switched to another one will the KVM copy be touched. Apparently it is not the case because the copy should be updated on-time during normal tick, right? thanks for your reply, Yu > > > update_pvclock_gtod(tk); > > > > /* disable master clock if host does not trust, or does not > > * use, TSC clocksource > > */ > > - if (gtod->clock.vclock_mode != VCLOCK_TSC && > > - atomic_read(&kvm_guest_has_master_clock) != 0) > > + if (atomic_read(&kvm_guest_has_master_clock) != 0) > > queue_work(system_long_wq, &pvclock_gtod_work); > > > > return 0; > > -- > > 2.7.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Dec 27, 2016 at 04:06:44PM +0800, Chen Yu wrote: > Hi Marcelo, > On Mon, Dec 26, 2016 at 05:44:25PM -0200, Marcelo Tosatti wrote: > > On Fri, Dec 23, 2016 at 04:41:53PM +0800, Chen Yu wrote: > > > Currently the notifier of pvclock_gtod_notify() get invoked > > > frequently due to the periodic update_wall_time(). This might > > > slow down the system a little bit as there might be redundant > > > execution code path and unnecessary lock contention > > > in update_pvclock_gtod(), which was found when I was doing > > > suspend/resume speed testings. As pvclock_gtod_notify() > > > should be invoked only when clocksource has changed, according to > > > Commit 16e8d74d2da9 ("KVM: x86: notifier for clocksource changes") > > > , either we can add a new notifier for clocksource switch, > > > or we can simply bypass the following code in pvclock_gtod_notify() > > > earlier if there is no clocksource switch. > > > > > > Cc: Paolo Bonzini <pbonzini@redhat.com> > > > Cc: "Radim Krcmar" <rkrcmar@redhat.com> > > > Cc: Marcelo Tosatti <mtosatti@redhat.com> > > > Cc: Thomas Gleixner <tglx@linutronix.de> > > > Cc: Ingo Molnar <mingo@redhat.com> > > > Cc: "H. Peter Anvin" <hpa@zytor.com> > > > Cc: Wanpeng Li <wanpeng.li@hotmail.com> > > > Signed-off-by: Chen Yu <yu.c.chen@intel.com> > > > --- > > > arch/x86/kvm/x86.c | 5 +++-- > > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > > index 445c51b..54aa32d 100644 > > > --- a/arch/x86/kvm/x86.c > > > +++ b/arch/x86/kvm/x86.c > > > @@ -5961,13 +5961,14 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused, > > > struct pvclock_gtod_data *gtod = &pvclock_gtod_data; > > > struct timekeeper *tk = priv; > > > > > > + if (likely(gtod->clock.vclock_mode == VCLOCK_TSC)) > > > + return 0; > > > > I think this is only safe if any of the values in "struct > > pvclock_gtod_data" are unchanged. Otherwise the local (KVM) copy is > > kept incorrect. > I missread the code previously and I thought only under the condition > the clocksource has been switched to another one will the KVM copy > be touched. Apparently it is not the case because the copy should > be updated on-time during normal tick, right? > thanks for your reply, Yes, it is updated during the normal tick, and mult/freq values change. However, if none of them change, its not necessary to call the callback. Perhaps you can check if any of the values changed and only invoke the callback in that case? -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Dec 27, 2016 at 01:32:47PM -0200, Marcelo Tosatti wrote: > On Tue, Dec 27, 2016 at 04:06:44PM +0800, Chen Yu wrote: > > Hi Marcelo, > > On Mon, Dec 26, 2016 at 05:44:25PM -0200, Marcelo Tosatti wrote: > > > On Fri, Dec 23, 2016 at 04:41:53PM +0800, Chen Yu wrote: > > > > Currently the notifier of pvclock_gtod_notify() get invoked > > > > frequently due to the periodic update_wall_time(). This might > > > > slow down the system a little bit as there might be redundant > > > > execution code path and unnecessary lock contention > > > > in update_pvclock_gtod(), which was found when I was doing > > > > suspend/resume speed testings. As pvclock_gtod_notify() > > > > should be invoked only when clocksource has changed, according to > > > > Commit 16e8d74d2da9 ("KVM: x86: notifier for clocksource changes") > > > > , either we can add a new notifier for clocksource switch, > > > > or we can simply bypass the following code in pvclock_gtod_notify() > > > > earlier if there is no clocksource switch. > > > > > > > > Cc: Paolo Bonzini <pbonzini@redhat.com> > > > > Cc: "Radim Krcmar" <rkrcmar@redhat.com> > > > > Cc: Marcelo Tosatti <mtosatti@redhat.com> > > > > Cc: Thomas Gleixner <tglx@linutronix.de> > > > > Cc: Ingo Molnar <mingo@redhat.com> > > > > Cc: "H. Peter Anvin" <hpa@zytor.com> > > > > Cc: Wanpeng Li <wanpeng.li@hotmail.com> > > > > Signed-off-by: Chen Yu <yu.c.chen@intel.com> > > > > --- > > > > arch/x86/kvm/x86.c | 5 +++-- > > > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > > > index 445c51b..54aa32d 100644 > > > > --- a/arch/x86/kvm/x86.c > > > > +++ b/arch/x86/kvm/x86.c > > > > @@ -5961,13 +5961,14 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused, > > > > struct pvclock_gtod_data *gtod = &pvclock_gtod_data; > > > > struct timekeeper *tk = priv; > > > > > > > > + if (likely(gtod->clock.vclock_mode == VCLOCK_TSC)) > > > > + return 0; > > > > > > I think this is only safe if any of the values in "struct > > > pvclock_gtod_data" are unchanged. Otherwise the local (KVM) copy is > > > kept incorrect. > > I missread the code previously and I thought only under the condition > > the clocksource has been switched to another one will the KVM copy > > be touched. Apparently it is not the case because the copy should > > be updated on-time during normal tick, right? > > thanks for your reply, > > Yes, it is updated during the normal tick, and mult/freq values change. > > However, if none of them change, its not necessary to call the callback. > Perhaps you can check if any of the values changed and only > invoke the callback in that case? > Yes, this should be an optimization, but most of the callers(workload) come from update_wall_time(), and in this code path the clock source's cycle should already be updated in most cases, so this optimization should not take much effect to reduce the burden I guess? Thanks, Yu -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Dec 29, 2016 at 04:59:02PM +0800, Chen Yu wrote: > On Tue, Dec 27, 2016 at 01:32:47PM -0200, Marcelo Tosatti wrote: > > On Tue, Dec 27, 2016 at 04:06:44PM +0800, Chen Yu wrote: > > > Hi Marcelo, > > > On Mon, Dec 26, 2016 at 05:44:25PM -0200, Marcelo Tosatti wrote: > > > > On Fri, Dec 23, 2016 at 04:41:53PM +0800, Chen Yu wrote: > > > > > Currently the notifier of pvclock_gtod_notify() get invoked > > > > > frequently due to the periodic update_wall_time(). This might > > > > > slow down the system a little bit as there might be redundant > > > > > execution code path and unnecessary lock contention > > > > > in update_pvclock_gtod(), which was found when I was doing > > > > > suspend/resume speed testings. As pvclock_gtod_notify() > > > > > should be invoked only when clocksource has changed, according to > > > > > Commit 16e8d74d2da9 ("KVM: x86: notifier for clocksource changes") > > > > > , either we can add a new notifier for clocksource switch, > > > > > or we can simply bypass the following code in pvclock_gtod_notify() > > > > > earlier if there is no clocksource switch. > > > > > > > > > > Cc: Paolo Bonzini <pbonzini@redhat.com> > > > > > Cc: "Radim Krcmar" <rkrcmar@redhat.com> > > > > > Cc: Marcelo Tosatti <mtosatti@redhat.com> > > > > > Cc: Thomas Gleixner <tglx@linutronix.de> > > > > > Cc: Ingo Molnar <mingo@redhat.com> > > > > > Cc: "H. Peter Anvin" <hpa@zytor.com> > > > > > Cc: Wanpeng Li <wanpeng.li@hotmail.com> > > > > > Signed-off-by: Chen Yu <yu.c.chen@intel.com> > > > > > --- > > > > > arch/x86/kvm/x86.c | 5 +++-- > > > > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > > > > index 445c51b..54aa32d 100644 > > > > > --- a/arch/x86/kvm/x86.c > > > > > +++ b/arch/x86/kvm/x86.c > > > > > @@ -5961,13 +5961,14 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused, > > > > > struct pvclock_gtod_data *gtod = &pvclock_gtod_data; > > > > > struct timekeeper *tk = priv; > > > > > > > > > > + if (likely(gtod->clock.vclock_mode == VCLOCK_TSC)) > > > > > + return 0; > > > > > > > > I think this is only safe if any of the values in "struct > > > > pvclock_gtod_data" are unchanged. Otherwise the local (KVM) copy is > > > > kept incorrect. > > > I missread the code previously and I thought only under the condition > > > the clocksource has been switched to another one will the KVM copy > > > be touched. Apparently it is not the case because the copy should > > > be updated on-time during normal tick, right? > > > thanks for your reply, > > > > Yes, it is updated during the normal tick, and mult/freq values change. > > > > However, if none of them change, its not necessary to call the callback. > > Perhaps you can check if any of the values changed and only > > invoke the callback in that case? > > > Yes, this should be an optimization, but most of the callers(workload) come > from update_wall_time(), and in this code path the clock source's cycle > should already be updated in most cases, so this optimization should not take > much effect to reduce the burden I guess? > > Thanks, > Yu I don't understand your reasoning. "If the clock source parameters are already updated then optimization does not make much effect". If the clock source parameters are updated (that is there has been no change in any of the values in pvclock_gtod_data), then you can skip the callback. This case reduces the burden. Right? -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 445c51b..54aa32d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5961,13 +5961,14 @@ static int pvclock_gtod_notify(struct notifier_block *nb, unsigned long unused, struct pvclock_gtod_data *gtod = &pvclock_gtod_data; struct timekeeper *tk = priv; + if (likely(gtod->clock.vclock_mode == VCLOCK_TSC)) + return 0; update_pvclock_gtod(tk); /* disable master clock if host does not trust, or does not * use, TSC clocksource */ - if (gtod->clock.vclock_mode != VCLOCK_TSC && - atomic_read(&kvm_guest_has_master_clock) != 0) + if (atomic_read(&kvm_guest_has_master_clock) != 0) queue_work(system_long_wq, &pvclock_gtod_work); return 0;
Currently the notifier of pvclock_gtod_notify() get invoked frequently due to the periodic update_wall_time(). This might slow down the system a little bit as there might be redundant execution code path and unnecessary lock contention in update_pvclock_gtod(), which was found when I was doing suspend/resume speed testings. As pvclock_gtod_notify() should be invoked only when clocksource has changed, according to Commit 16e8d74d2da9 ("KVM: x86: notifier for clocksource changes") , either we can add a new notifier for clocksource switch, or we can simply bypass the following code in pvclock_gtod_notify() earlier if there is no clocksource switch. Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krcmar" <rkrcmar@redhat.com> Cc: Marcelo Tosatti <mtosatti@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Wanpeng Li <wanpeng.li@hotmail.com> Signed-off-by: Chen Yu <yu.c.chen@intel.com> --- arch/x86/kvm/x86.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)