| Message ID | 20130714163009.22374.22100.stgit@zurg (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Sun, Jul 14, 2013 at 6:30 PM, Konstantin Khlebnikov <khlebnikov@openvz.org> wrote: > This patch fixes regression in power consumtion of sandy bridge gpu, which > exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that > it's extremely busy. After that it never reaches rc6 state. > > Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > ("drm/i915: load boot context at driver init time"). Without documentation > it's not clear what is happening here, probably this breaks internal state of > hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake > during whole initialization sequence in gen6_init_clock_gating() fixes this bug. > > References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > Signed-off-by: Konstantin Khlebnikov <khlebnikov@openvz.org> We already hold an forcewake reference while setting up the rps stuff, should we maybe hold the forcewake for the entire duration, i.e. grab it here in clock_gating and release it only in gen6/vlv_enable_rps? Can you please test that version, too? In any case the forcewake grabbing here in the clock gating function needs a big comment that otherwise setting the MCTL register might break rc6 entry. -Daniel > --- > drivers/gpu/drm/i915/intel_pm.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c > index aa01128..839a43f 100644 > --- a/drivers/gpu/drm/i915/intel_pm.c > +++ b/drivers/gpu/drm/i915/intel_pm.c > @@ -3640,6 +3640,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) > int pipe; > uint32_t dspclk_gate = ILK_VRHUNIT_CLOCK_GATE_DISABLE; > > + gen6_gt_force_wake_get(dev_priv); > + > I915_WRITE(ILK_DSPCLK_GATE_D, dspclk_gate); > > I915_WRITE(ILK_DISPLAY_CHICKEN2, > @@ -3728,6 +3730,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) > cpt_init_clock_gating(dev); > > gen6_check_mch_setup(dev); > + > + gen6_gt_force_wake_put(dev_priv); > } > > static void gen7_setup_fixed_func_scheduler(struct drm_i915_private *dev_priv) > -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
Daniel Vetter wrote: > On Sun, Jul 14, 2013 at 6:30 PM, Konstantin Khlebnikov > <khlebnikov@openvz.org> wrote: >> This patch fixes regression in power consumtion of sandy bridge gpu, which >> exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that >> it's extremely busy. After that it never reaches rc6 state. >> >> Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 >> ("drm/i915: load boot context at driver init time"). Without documentation >> it's not clear what is happening here, probably this breaks internal state of >> hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake >> during whole initialization sequence in gen6_init_clock_gating() fixes this bug. >> >> References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 >> Signed-off-by: Konstantin Khlebnikov<khlebnikov@openvz.org> > > We already hold an forcewake reference while setting up the rps stuff, > should we maybe hold the forcewake for the entire duration, i.e. grab > it here in clock_gating and release it only in gen6/vlv_enable_rps? > Can you please test that version, too? This will be racy because rps stuff is done in separate work which might be canceled if intel_disable_gt_powersave() happens before its completion. > > In any case the forcewake grabbing here in the clock gating function > needs a big comment that otherwise setting the MCTL register might > break rc6 entry. > -Daniel > >> --- >> drivers/gpu/drm/i915/intel_pm.c | 4 ++++ >> 1 file changed, 4 insertions(+) >> >> diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c >> index aa01128..839a43f 100644 >> --- a/drivers/gpu/drm/i915/intel_pm.c >> +++ b/drivers/gpu/drm/i915/intel_pm.c >> @@ -3640,6 +3640,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) >> int pipe; >> uint32_t dspclk_gate = ILK_VRHUNIT_CLOCK_GATE_DISABLE; >> >> + gen6_gt_force_wake_get(dev_priv); >> + >> I915_WRITE(ILK_DSPCLK_GATE_D, dspclk_gate); >> >> I915_WRITE(ILK_DISPLAY_CHICKEN2, >> @@ -3728,6 +3730,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) >> cpt_init_clock_gating(dev); >> >> gen6_check_mch_setup(dev); >> + >> + gen6_gt_force_wake_put(dev_priv); >> } >> >> static void gen7_setup_fixed_func_scheduler(struct drm_i915_private *dev_priv) >> > > > > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On Sun, Jul 14, 2013 at 06:52:39PM +0200, Daniel Vetter wrote: > On Sun, Jul 14, 2013 at 6:30 PM, Konstantin Khlebnikov > <khlebnikov@openvz.org> wrote: > > This patch fixes regression in power consumtion of sandy bridge gpu, which > > exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that > > it's extremely busy. After that it never reaches rc6 state. > > > > Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > > ("drm/i915: load boot context at driver init time"). Without documentation > > it's not clear what is happening here, probably this breaks internal state of > > hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake > > during whole initialization sequence in gen6_init_clock_gating() fixes this bug. > > > > References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > > Signed-off-by: Konstantin Khlebnikov <khlebnikov@openvz.org> > > We already hold an forcewake reference while setting up the rps stuff, > should we maybe hold the forcewake for the entire duration, i.e. grab > it here in clock_gating and release it only in gen6/vlv_enable_rps? > Can you please test that version, too? > > In any case the forcewake grabbing here in the clock gating function > needs a big comment that otherwise setting the MCTL register might > break rc6 entry. It is not clear why the forcewake works, but is easy to imagine one of the operations in that sequence requires the GPU to be awake at the time of programming for it to succeed. MBCTL:EnableBootFetch does seem the most suspicious from its wording in the bspec. I guess all instances of poking this bit should be protected similary (snb, ivb, vlv, hsw). Based on that reasoning and that waking the GPU up here has no negative consequences, and so long as all paths are fixed, I am happy to give this an Acked-by: Chris Wilson <chris@chris-wilson.co.uk> Also, we need to reapply the w/a after a Function Level Reset, in other words we do need to repeat the init_clock_gating after intel_gpu_reset(). -Chris
On Sun, Jul 14, 2013 at 09:56:45PM +0400, Konstantin Khlebnikov wrote: > Daniel Vetter wrote: > >On Sun, Jul 14, 2013 at 6:30 PM, Konstantin Khlebnikov > ><khlebnikov@openvz.org> wrote: > >>This patch fixes regression in power consumtion of sandy bridge gpu, which > >>exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that > >>it's extremely busy. After that it never reaches rc6 state. > >> > >>Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > >>("drm/i915: load boot context at driver init time"). Without documentation > >>it's not clear what is happening here, probably this breaks internal state of > >>hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake > >>during whole initialization sequence in gen6_init_clock_gating() fixes this bug. > >> > >>References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > >>Signed-off-by: Konstantin Khlebnikov<khlebnikov@openvz.org> > > > >We already hold an forcewake reference while setting up the rps stuff, > >should we maybe hold the forcewake for the entire duration, i.e. grab > >it here in clock_gating and release it only in gen6/vlv_enable_rps? > >Can you please test that version, too? > > This will be racy because rps stuff is done in separate work which might be canceled > if intel_disable_gt_powersave() happens before its completion. Can be fixed with a flush_delayed_work. And since that has the same requirements wrt locking to prevent deadlocks as cancel_work_sync it would be a drop-in replacement. Can I volunteer you to look into testing that out a bit? Otherwise I could volunteer someone from our team. In any case I think we should apply this trick to all platforms where we've added the MBCTL write (i.e. snb, ivb, hsw & vlv) since rps/rc6 works _very_ similar on all of those. Thanks, Daniel
On Sun, Jul 14, 2013 at 08:30:09PM +0400, Konstantin Khlebnikov wrote: > This patch fixes regression in power consumtion of sandy bridge gpu, which > exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that > it's extremely busy. After that it never reaches rc6 state. > > Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > ("drm/i915: load boot context at driver init time"). Without documentation > it's not clear what is happening here, probably this breaks internal state of > hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake > during whole initialization sequence in gen6_init_clock_gating() fixes this bug. > > References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > Signed-off-by: Konstantin Khlebnikov <khlebnikov@openvz.org> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=58971 Tested-by: Alexander Kaltsas <alexkaltsas@gmail.com> Tested-by: rocko <rockorequin@hotmail.com> > --- > drivers/gpu/drm/i915/intel_pm.c | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c > index aa01128..839a43f 100644 > --- a/drivers/gpu/drm/i915/intel_pm.c > +++ b/drivers/gpu/drm/i915/intel_pm.c > @@ -3640,6 +3640,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) > int pipe; > uint32_t dspclk_gate = ILK_VRHUNIT_CLOCK_GATE_DISABLE; > > + gen6_gt_force_wake_get(dev_priv); > + > I915_WRITE(ILK_DSPCLK_GATE_D, dspclk_gate); > > I915_WRITE(ILK_DISPLAY_CHICKEN2, > @@ -3728,6 +3730,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) > cpt_init_clock_gating(dev); > > gen6_check_mch_setup(dev); > + > + gen6_gt_force_wake_put(dev_priv); > } > > static void gen7_setup_fixed_func_scheduler(struct drm_i915_private *dev_priv) >
On Tue, Jul 16, 2013 at 10:31 AM, Daniel Vetter <daniel@ffwll.ch> wrote: > On Sun, Jul 14, 2013 at 09:56:45PM +0400, Konstantin Khlebnikov wrote: > > Daniel Vetter wrote: > > >On Sun, Jul 14, 2013 at 6:30 PM, Konstantin Khlebnikov > > ><khlebnikov@openvz.org> wrote: > > >>This patch fixes regression in power consumtion of sandy bridge gpu, > which > > >>exists since v3.6 Sometimes after resuming from s2ram gpu starts > thinking that > > >>it's extremely busy. After that it never reaches rc6 state. > > >> > > >>Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > > >>("drm/i915: load boot context at driver init time"). Without > documentation > > >>it's not clear what is happening here, probably this breaks internal > state of > > >>hardware ring buffers and confuses RPS engine. Fortunately keeping > forcewake > > >>during whole initialization sequence in gen6_init_clock_gating() fixes > this bug. > > >> > > >>References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > > >>Signed-off-by: Konstantin Khlebnikov<khlebnikov@openvz.org> > > > > > >We already hold an forcewake reference while setting up the rps stuff, > > >should we maybe hold the forcewake for the entire duration, i.e. grab > > >it here in clock_gating and release it only in gen6/vlv_enable_rps? > > >Can you please test that version, too? > > > > This will be racy because rps stuff is done in separate work which might > be canceled > > if intel_disable_gt_powersave() happens before its completion. > > Can be fixed with a flush_delayed_work. And since that has the same > requirements wrt locking to prevent deadlocks as cancel_work_sync it would > be a drop-in replacement. Can I volunteer you to look into testing that > out a bit? Otherwise I could volunteer someone from our team. > > In any case I think we should apply this trick to all platforms where > we've added the MBCTL write (i.e. snb, ivb, hsw & vlv) since rps/rc6 works > _very_ similar on all of those. > I've tested that patch and it really works for me. If you want change something for other hardware or extend range where forcewake is held prease do it in a separate patch. This will be good for bisecting new bugs in the future. > > Thanks, Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch >
On Tue, Jul 16, 2013 at 11:34:25AM +0400, Konstantin Khlebnikov wrote: > On Tue, Jul 16, 2013 at 10:31 AM, Daniel Vetter <daniel@ffwll.ch> wrote: > > > On Sun, Jul 14, 2013 at 09:56:45PM +0400, Konstantin Khlebnikov wrote: > > > Daniel Vetter wrote: > > > >On Sun, Jul 14, 2013 at 6:30 PM, Konstantin Khlebnikov > > > ><khlebnikov@openvz.org> wrote: > > > >>This patch fixes regression in power consumtion of sandy bridge gpu, > > which > > > >>exists since v3.6 Sometimes after resuming from s2ram gpu starts > > thinking that > > > >>it's extremely busy. After that it never reaches rc6 state. > > > >> > > > >>Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > > > >>("drm/i915: load boot context at driver init time"). Without > > documentation > > > >>it's not clear what is happening here, probably this breaks internal > > state of > > > >>hardware ring buffers and confuses RPS engine. Fortunately keeping > > forcewake > > > >>during whole initialization sequence in gen6_init_clock_gating() fixes > > this bug. > > > >> > > > >>References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > > > >>Signed-off-by: Konstantin Khlebnikov<khlebnikov@openvz.org> > > > > > > > >We already hold an forcewake reference while setting up the rps stuff, > > > >should we maybe hold the forcewake for the entire duration, i.e. grab > > > >it here in clock_gating and release it only in gen6/vlv_enable_rps? > > > >Can you please test that version, too? > > > > > > This will be racy because rps stuff is done in separate work which might > > be canceled > > > if intel_disable_gt_powersave() happens before its completion. > > > > Can be fixed with a flush_delayed_work. And since that has the same > > requirements wrt locking to prevent deadlocks as cancel_work_sync it would > > be a drop-in replacement. Can I volunteer you to look into testing that > > out a bit? Otherwise I could volunteer someone from our team. > > > > In any case I think we should apply this trick to all platforms where > > we've added the MBCTL write (i.e. snb, ivb, hsw & vlv) since rps/rc6 works > > _very_ similar on all of those. > > > > I've tested that patch and it really works for me. If you want change > something for other hardware or > extend range where forcewake is held prease do it in a separate patch. > This will be good for bisecting new bugs in the future. The issue I have with the current patch is that it looks a bit like duct-tape since the point where we drop the forcewake references seems to lack justification. The write to MBCTL itself will temporarily wake up the chip, so just wrapping that up in with forcewake is very likely not good enough. So I fear that we'll only hold forcewake long enough on most systems and still have a bunch of oddball broken systems out there. Holding forcewake otoh until we've fully set up rps/rc6 makes imo tons of sense, hence why I've brought up the idea. Same reasoning applies to extending the w/a to all systems supporting rc6. -Daniel
On Tue, Jul 16, 2013 at 09:44:59AM +0200, Daniel Vetter wrote: > The issue I have with the current patch is that it looks a bit like > duct-tape since the point where we drop the forcewake references seems to > lack justification. The write to MBCTL itself will temporarily wake up the > chip, so just wrapping that up in with forcewake is very likely not good > enough. So I fear that we'll only hold forcewake long enough on most > systems and still have a bunch of oddball broken systems out there. > > Holding forcewake otoh until we've fully set up rps/rc6 makes imo tons of > sense, hence why I've brought up the idea. Same reasoning applies to > extending the w/a to all systems supporting rc6. In which case disable rc6 at the start of init gating and only enable it at the end of the deferred task. That I think will better test your hypothesis and make the transistion steps clearer. -Chris
On Tue, Jul 16, 2013 at 10:31 AM, Chris Wilson <chris@chris-wilson.co.uk> wrote: > On Tue, Jul 16, 2013 at 09:44:59AM +0200, Daniel Vetter wrote: >> The issue I have with the current patch is that it looks a bit like >> duct-tape since the point where we drop the forcewake references seems to >> lack justification. The write to MBCTL itself will temporarily wake up the >> chip, so just wrapping that up in with forcewake is very likely not good >> enough. So I fear that we'll only hold forcewake long enough on most >> systems and still have a bunch of oddball broken systems out there. >> >> Holding forcewake otoh until we've fully set up rps/rc6 makes imo tons of >> sense, hence why I've brought up the idea. Same reasoning applies to >> extending the w/a to all systems supporting rc6. > > In which case disable rc6 at the start of init gating and only enable it > at the end of the deferred task. That I think will better test your > hypothesis and make the transistion steps clearer. Hm yeah, that would be much clearer instead of risky tricks with a refcount which is only dropped someplace completely else. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On Tue, Jul 16, 2013 at 08:32:40AM +0200, Daniel Vetter wrote: > On Sun, Jul 14, 2013 at 08:30:09PM +0400, Konstantin Khlebnikov wrote: > > This patch fixes regression in power consumtion of sandy bridge gpu, which > > exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that > > it's extremely busy. After that it never reaches rc6 state. > > > > Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 > > ("drm/i915: load boot context at driver init time"). Without documentation > > it's not clear what is happening here, probably this breaks internal state of > > hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake > > during whole initialization sequence in gen6_init_clock_gating() fixes this bug. > > > > References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 > > Signed-off-by: Konstantin Khlebnikov <khlebnikov@openvz.org> > > Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=58971 > Tested-by: Alexander Kaltsas <alexkaltsas@gmail.com> > Tested-by: rocko <rockorequin@hotmail.com> Tested-by: JohnMB <johnmbryant@sky.com> > > > --- > > drivers/gpu/drm/i915/intel_pm.c | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c > > index aa01128..839a43f 100644 > > --- a/drivers/gpu/drm/i915/intel_pm.c > > +++ b/drivers/gpu/drm/i915/intel_pm.c > > @@ -3640,6 +3640,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) > > int pipe; > > uint32_t dspclk_gate = ILK_VRHUNIT_CLOCK_GATE_DISABLE; > > > > + gen6_gt_force_wake_get(dev_priv); > > + > > I915_WRITE(ILK_DSPCLK_GATE_D, dspclk_gate); > > > > I915_WRITE(ILK_DISPLAY_CHICKEN2, > > @@ -3728,6 +3730,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) > > cpt_init_clock_gating(dev); > > > > gen6_check_mch_setup(dev); > > + > > + gen6_gt_force_wake_put(dev_priv); > > } > > > > static void gen7_setup_fixed_func_scheduler(struct drm_i915_private *dev_priv) > > > > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c index aa01128..839a43f 100644 --- a/drivers/gpu/drm/i915/intel_pm.c +++ b/drivers/gpu/drm/i915/intel_pm.c @@ -3640,6 +3640,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) int pipe; uint32_t dspclk_gate = ILK_VRHUNIT_CLOCK_GATE_DISABLE; + gen6_gt_force_wake_get(dev_priv); + I915_WRITE(ILK_DSPCLK_GATE_D, dspclk_gate); I915_WRITE(ILK_DISPLAY_CHICKEN2, @@ -3728,6 +3730,8 @@ static void gen6_init_clock_gating(struct drm_device *dev) cpt_init_clock_gating(dev); gen6_check_mch_setup(dev); + + gen6_gt_force_wake_put(dev_priv); } static void gen7_setup_fixed_func_scheduler(struct drm_i915_private *dev_priv)
This patch fixes regression in power consumtion of sandy bridge gpu, which exists since v3.6 Sometimes after resuming from s2ram gpu starts thinking that it's extremely busy. After that it never reaches rc6 state. Bug was introduce by commit b4ae3f22d238617ca11610b29fde16cf8c0bc6e0 ("drm/i915: load boot context at driver init time"). Without documentation it's not clear what is happening here, probably this breaks internal state of hardware ring buffers and confuses RPS engine. Fortunately keeping forcewake during whole initialization sequence in gen6_init_clock_gating() fixes this bug. References: https://bugs.freedesktop.org/show_bug.cgi?id=54089 Signed-off-by: Konstantin Khlebnikov <khlebnikov@openvz.org> --- drivers/gpu/drm/i915/intel_pm.c | 4 ++++ 1 file changed, 4 insertions(+)