diff mbox series

[RFC,3/3] opp: Power on (virtual) power domains managed by the OPP core

Message ID 20200730080146.25185-4-stephan@gerhold.net (mailing list archive)
State RFC, archived
Headers show
Series opp: required_opps: Power on genpd, scale down in reverse order | expand

Commit Message

Stephan Gerhold July 30, 2020, 8:01 a.m. UTC
dev_pm_opp_attach_genpd() allows attaching an arbitrary number of
power domains to an OPP table. In that case, the genpd core will
create a virtual device for each of the power domains.

At the moment, the OPP core only calls
dev_pm_genpd_set_performance_state() on these virtual devices.
It does not attempt to power on the power domains. Therefore
the required power domain might never get turned on.

So far, dev_pm_opp_attach_genpd() is only used in qcom-cpufreq-nvmem.c
to attach the CPR power domain to the CPU OPP table. The CPR driver
does not check if it was actually powered on so this did not cause
any problems. However, other drivers (e.g. rpmpd) might ignore the
performance state until the power domain is actually powered on.

Since these virtual devices are managed exclusively by the OPP core,
I would say that it should also be responsible to ensure they are
enabled. A similar approach is already used for regulators, see
commit 8d45719caaf5 ("opp: core: add regulators enable and disable").

This commit implements similar functionality for the virtual genpd
devices managed by the OPP core. The power domains are turned on
the first time dev_pm_opp_set_rate() is called. They are turned off
again when dev_pm_opp_set_rate(dev, 0) is called.

Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
---
Related discussion: https://lore.kernel.org/linux-arm-msm/20200426123140.GA190483@gerhold.net/

There would be also other ways to implement this, e.g. device links,
assuming that the device using the OPP table also makes use of runtime PM.
My first thought was that it would be most consistent to handle this like
regulators, bandwidth votes etc. RFC :)
---
 drivers/opp/core.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++
 drivers/opp/opp.h  |  1 +
 2 files changed, 56 insertions(+)

Comments

Viresh Kumar Aug. 24, 2020, 11:27 a.m. UTC | #1
On 30-07-20, 10:01, Stephan Gerhold wrote:
> dev_pm_opp_attach_genpd() allows attaching an arbitrary number of
> power domains to an OPP table. In that case, the genpd core will
> create a virtual device for each of the power domains.
> 
> At the moment, the OPP core only calls
> dev_pm_genpd_set_performance_state() on these virtual devices.
> It does not attempt to power on the power domains. Therefore
> the required power domain might never get turned on.
> 
> So far, dev_pm_opp_attach_genpd() is only used in qcom-cpufreq-nvmem.c
> to attach the CPR power domain to the CPU OPP table. The CPR driver
> does not check if it was actually powered on so this did not cause
> any problems. However, other drivers (e.g. rpmpd) might ignore the
> performance state until the power domain is actually powered on.
> 
> Since these virtual devices are managed exclusively by the OPP core,
> I would say that it should also be responsible to ensure they are
> enabled. A similar approach is already used for regulators, see
> commit 8d45719caaf5 ("opp: core: add regulators enable and disable").
> 
> This commit implements similar functionality for the virtual genpd
> devices managed by the OPP core. The power domains are turned on
> the first time dev_pm_opp_set_rate() is called. They are turned off
> again when dev_pm_opp_set_rate(dev, 0) is called.
> 
> Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> ---
> Related discussion: https://lore.kernel.org/linux-arm-msm/20200426123140.GA190483@gerhold.net/
> 
> There would be also other ways to implement this, e.g. device links,
> assuming that the device using the OPP table also makes use of runtime PM.
> My first thought was that it would be most consistent to handle this like
> regulators, bandwidth votes etc. RFC :)

This stuff was done ages back and I am starting to forget almost
everything now :)

Ulf, why doesn't pm_runtime_get(dev) take care of enabling multiple
power domain case ? RFP (request for patience) :)
Stephan Gerhold Aug. 24, 2020, 11:55 a.m. UTC | #2
On Mon, Aug 24, 2020 at 04:57:44PM +0530, Viresh Kumar wrote:
> On 30-07-20, 10:01, Stephan Gerhold wrote:
> > dev_pm_opp_attach_genpd() allows attaching an arbitrary number of
> > power domains to an OPP table. In that case, the genpd core will
> > create a virtual device for each of the power domains.
> > 
> > At the moment, the OPP core only calls
> > dev_pm_genpd_set_performance_state() on these virtual devices.
> > It does not attempt to power on the power domains. Therefore
> > the required power domain might never get turned on.
> > 
> > So far, dev_pm_opp_attach_genpd() is only used in qcom-cpufreq-nvmem.c
> > to attach the CPR power domain to the CPU OPP table. The CPR driver
> > does not check if it was actually powered on so this did not cause
> > any problems. However, other drivers (e.g. rpmpd) might ignore the
> > performance state until the power domain is actually powered on.
> > 
> > Since these virtual devices are managed exclusively by the OPP core,
> > I would say that it should also be responsible to ensure they are
> > enabled. A similar approach is already used for regulators, see
> > commit 8d45719caaf5 ("opp: core: add regulators enable and disable").
> > 
> > This commit implements similar functionality for the virtual genpd
> > devices managed by the OPP core. The power domains are turned on
> > the first time dev_pm_opp_set_rate() is called. They are turned off
> > again when dev_pm_opp_set_rate(dev, 0) is called.
> > 
> > Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> > ---
> > Related discussion: https://lore.kernel.org/linux-arm-msm/20200426123140.GA190483@gerhold.net/
> > 
> > There would be also other ways to implement this, e.g. device links,
> > assuming that the device using the OPP table also makes use of runtime PM.
> > My first thought was that it would be most consistent to handle this like
> > regulators, bandwidth votes etc. RFC :)
> 
> This stuff was done ages back and I am starting to forget almost
> everything now :)
> 
> Ulf, why doesn't pm_runtime_get(dev) take care of enabling multiple
> power domain case ? RFP (request for patience) :)
> 

So I'm really not an expert for power domains, but here is my
understanding:

We attach the power domains in dev_pm_opp_attach_genpd(opp_dev, names),
where opp_dev is the device the OPP table belongs to.

To do that, the genpd core creates a set of virtual devices. These
virtual devices are not related to opp_dev in any way. Therefore, the
power domains stay off until we run pm_runtime_get(virt_dev) for each of
the virtual devices. (Which is what is implemented in this patch...)

If I understand correctly, what you would like to do is to have a single
pm_runtime_get(opp_dev) call also enable all the virtual devices?

As far as I understand, this can be done by adding "device links"
between opp_dev and the virtual devices, e.g.

	device_link_add(opp_dev, virt_dev, DL_FLAG_PM_RUNTIME);

for each of the virtual devices.

But the problem with that approach is that it assumes that someone
actually calls pm_runtime_get(opp_dev), i.e. we assume that opp_dev is
managed by runtime PM. As far as I know, this isn't the case for the CPU
OPP table for example.

Maybe Ulf can correct me if I'm wrong :)

Thanks!
Stephan
Ulf Hansson Aug. 24, 2020, 2:36 p.m. UTC | #3
On Mon, 24 Aug 2020 at 13:56, Stephan Gerhold <stephan@gerhold.net> wrote:
>
> On Mon, Aug 24, 2020 at 04:57:44PM +0530, Viresh Kumar wrote:
> > On 30-07-20, 10:01, Stephan Gerhold wrote:
> > > dev_pm_opp_attach_genpd() allows attaching an arbitrary number of
> > > power domains to an OPP table. In that case, the genpd core will
> > > create a virtual device for each of the power domains.
> > >
> > > At the moment, the OPP core only calls
> > > dev_pm_genpd_set_performance_state() on these virtual devices.
> > > It does not attempt to power on the power domains. Therefore
> > > the required power domain might never get turned on.
> > >
> > > So far, dev_pm_opp_attach_genpd() is only used in qcom-cpufreq-nvmem.c
> > > to attach the CPR power domain to the CPU OPP table. The CPR driver
> > > does not check if it was actually powered on so this did not cause
> > > any problems. However, other drivers (e.g. rpmpd) might ignore the
> > > performance state until the power domain is actually powered on.
> > >
> > > Since these virtual devices are managed exclusively by the OPP core,
> > > I would say that it should also be responsible to ensure they are
> > > enabled. A similar approach is already used for regulators, see
> > > commit 8d45719caaf5 ("opp: core: add regulators enable and disable").
> > >
> > > This commit implements similar functionality for the virtual genpd
> > > devices managed by the OPP core. The power domains are turned on
> > > the first time dev_pm_opp_set_rate() is called. They are turned off
> > > again when dev_pm_opp_set_rate(dev, 0) is called.
> > >
> > > Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> > > ---
> > > Related discussion: https://lore.kernel.org/linux-arm-msm/20200426123140.GA190483@gerhold.net/
> > >
> > > There would be also other ways to implement this, e.g. device links,
> > > assuming that the device using the OPP table also makes use of runtime PM.
> > > My first thought was that it would be most consistent to handle this like
> > > regulators, bandwidth votes etc. RFC :)
> >
> > This stuff was done ages back and I am starting to forget almost
> > everything now :)
> >
> > Ulf, why doesn't pm_runtime_get(dev) take care of enabling multiple
> > power domain case ? RFP (request for patience) :)
> >
>
> So I'm really not an expert for power domains, but here is my
> understanding:
>
> We attach the power domains in dev_pm_opp_attach_genpd(opp_dev, names),
> where opp_dev is the device the OPP table belongs to.
>
> To do that, the genpd core creates a set of virtual devices. These
> virtual devices are not related to opp_dev in any way. Therefore, the
> power domains stay off until we run pm_runtime_get(virt_dev) for each of
> the virtual devices. (Which is what is implemented in this patch...)

Just to clarify. The reason why genpd creates virtual devices isn't
because of the opp table itself.

Instead this is because we can only attach one PM domain per device.
And since a device may have multiple PM domains, we need to create a
virtual device and per PM domain and attach that instead. Then it's up
to the caller to manage the virtual devices.

In some cases where the PM domains can be managed together, a device
link makes sense - while in others it doesn't.

>
> If I understand correctly, what you would like to do is to have a single
> pm_runtime_get(opp_dev) call also enable all the virtual devices?
>
> As far as I understand, this can be done by adding "device links"
> between opp_dev and the virtual devices, e.g.
>
>         device_link_add(opp_dev, virt_dev, DL_FLAG_PM_RUNTIME);
>
> for each of the virtual devices.

Yep.

>
> But the problem with that approach is that it assumes that someone
> actually calls pm_runtime_get(opp_dev), i.e. we assume that opp_dev is
> managed by runtime PM. As far as I know, this isn't the case for the CPU
> OPP table for example.
>
> Maybe Ulf can correct me if I'm wrong :)

If I understand correctly, the opp_dev is the actual consumer device.
It could represent an I/O controller for example, or a CPU in the CPU
freq case.

That said, perhaps should rely on the consumer to deploy runtime PM
support, but let the OPP core to set up the device links for the genpd
virtual devices!?

>
> Thanks!
> Stephan

Kind regards
Uffe
Stephan Gerhold Aug. 24, 2020, 3:08 p.m. UTC | #4
On Mon, Aug 24, 2020 at 04:36:57PM +0200, Ulf Hansson wrote:
> On Mon, 24 Aug 2020 at 13:56, Stephan Gerhold <stephan@gerhold.net> wrote:
> >
> > On Mon, Aug 24, 2020 at 04:57:44PM +0530, Viresh Kumar wrote:
> > > On 30-07-20, 10:01, Stephan Gerhold wrote:
> > > > dev_pm_opp_attach_genpd() allows attaching an arbitrary number of
> > > > power domains to an OPP table. In that case, the genpd core will
> > > > create a virtual device for each of the power domains.
> > > >
> > > > At the moment, the OPP core only calls
> > > > dev_pm_genpd_set_performance_state() on these virtual devices.
> > > > It does not attempt to power on the power domains. Therefore
> > > > the required power domain might never get turned on.
> > > >
> > > > So far, dev_pm_opp_attach_genpd() is only used in qcom-cpufreq-nvmem.c
> > > > to attach the CPR power domain to the CPU OPP table. The CPR driver
> > > > does not check if it was actually powered on so this did not cause
> > > > any problems. However, other drivers (e.g. rpmpd) might ignore the
> > > > performance state until the power domain is actually powered on.
> > > >
> > > > Since these virtual devices are managed exclusively by the OPP core,
> > > > I would say that it should also be responsible to ensure they are
> > > > enabled. A similar approach is already used for regulators, see
> > > > commit 8d45719caaf5 ("opp: core: add regulators enable and disable").
> > > >
> > > > This commit implements similar functionality for the virtual genpd
> > > > devices managed by the OPP core. The power domains are turned on
> > > > the first time dev_pm_opp_set_rate() is called. They are turned off
> > > > again when dev_pm_opp_set_rate(dev, 0) is called.
> > > >
> > > > Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> > > > ---
> > > > Related discussion: https://lore.kernel.org/linux-arm-msm/20200426123140.GA190483@gerhold.net/
> > > >
> > > > There would be also other ways to implement this, e.g. device links,
> > > > assuming that the device using the OPP table also makes use of runtime PM.
> > > > My first thought was that it would be most consistent to handle this like
> > > > regulators, bandwidth votes etc. RFC :)
> > >
> > > This stuff was done ages back and I am starting to forget almost
> > > everything now :)
> > >
> > > Ulf, why doesn't pm_runtime_get(dev) take care of enabling multiple
> > > power domain case ? RFP (request for patience) :)
> > >
> >
> > So I'm really not an expert for power domains, but here is my
> > understanding:
> >
> > We attach the power domains in dev_pm_opp_attach_genpd(opp_dev, names),
> > where opp_dev is the device the OPP table belongs to.
> >
> > To do that, the genpd core creates a set of virtual devices. These
> > virtual devices are not related to opp_dev in any way. Therefore, the
> > power domains stay off until we run pm_runtime_get(virt_dev) for each of
> > the virtual devices. (Which is what is implemented in this patch...)
> 
> Just to clarify. The reason why genpd creates virtual devices isn't
> because of the opp table itself.
> 
> Instead this is because we can only attach one PM domain per device.
> And since a device may have multiple PM domains, we need to create a
> virtual device and per PM domain and attach that instead. Then it's up
> to the caller to manage the virtual devices.
> 
> In some cases where the PM domains can be managed together, a device
> link makes sense - while in others it doesn't.
> 
> >
> > If I understand correctly, what you would like to do is to have a single
> > pm_runtime_get(opp_dev) call also enable all the virtual devices?
> >
> > As far as I understand, this can be done by adding "device links"
> > between opp_dev and the virtual devices, e.g.
> >
> >         device_link_add(opp_dev, virt_dev, DL_FLAG_PM_RUNTIME);
> >
> > for each of the virtual devices.
> 
> Yep.
> 
> >
> > But the problem with that approach is that it assumes that someone
> > actually calls pm_runtime_get(opp_dev), i.e. we assume that opp_dev is
> > managed by runtime PM. As far as I know, this isn't the case for the CPU
> > OPP table for example.
> >
> > Maybe Ulf can correct me if I'm wrong :)
> 
> If I understand correctly, the opp_dev is the actual consumer device.
> It could represent an I/O controller for example, or a CPU in the CPU
> freq case.
> 

Exactly.

> That said, perhaps should rely on the consumer to deploy runtime PM
> support, but let the OPP core to set up the device links for the genpd
> virtual devices!?
> 

Yes, that would be the alternative option.
I would be fine with it as long as it also works for the CPUfreq case.

I don't think anything manages runtime PM for the CPU device, just
like no-one calls dev_pm_opp_set_rate(cpu_dev, 0). So with my patch the
power domain is essentially kept always-on (except for system suspend).
At least in my case this is intended.

If device links also keep the power domains on if the consumer device
does not make use of runtime PM it should work fine for my case.

Personally, I think my original patch (without device links) fits better
into the OPP API, for the following two reasons.

With device links:

  1. Unlike regulators/interconnects, attached power domains would be
     controlled by runtime PM instead of dev_pm_opp_set_rate(opp_dev, 0).

  2. ... some driver using OPP tables might not make use of runtime PM.
     In that case, the power domains would stay on the whole time,
     even if dev_pm_opp_set_rate(opp_dev, 0) was called.

With my patch, the power domain state is directly related to the
dev_pm_opp_set_rate(opp_dev, 0) call, which is more intuitive than
relying on the runtime PM state in my opinion.

Stephan
Viresh Kumar Aug. 25, 2020, 4:43 a.m. UTC | #5
On 24-08-20, 17:08, Stephan Gerhold wrote:
> On Mon, Aug 24, 2020 at 04:36:57PM +0200, Ulf Hansson wrote:
> > That said, perhaps should rely on the consumer to deploy runtime PM
> > support, but let the OPP core to set up the device links for the genpd
> > virtual devices!?
> > 
> 
> Yes, that would be the alternative option.

That is the right option IMO.

> I would be fine with it as long as it also works for the CPUfreq case.
> 
> I don't think anything manages runtime PM for the CPU device, just
> like no-one calls dev_pm_opp_set_rate(cpu_dev, 0). So with my patch the
> power domain is essentially kept always-on (except for system suspend).
> At least in my case this is intended.
> 
> If device links also keep the power domains on if the consumer device
> does not make use of runtime PM it should work fine for my case.

With device link, you only need to do rpm enable/disable on the consumer device
and it will get propagated by itself.

> Personally, I think my original patch (without device links) fits better
> into the OPP API, for the following two reasons.
> 
> With device links:
> 
>   1. Unlike regulators/interconnects, attached power domains would be
>      controlled by runtime PM instead of dev_pm_opp_set_rate(opp_dev, 0).
> 
>   2. ... some driver using OPP tables might not make use of runtime PM.
>      In that case, the power domains would stay on the whole time,
>      even if dev_pm_opp_set_rate(opp_dev, 0) was called.
> 
> With my patch, the power domain state is directly related to the
> dev_pm_opp_set_rate(opp_dev, 0) call, which is more intuitive than
> relying on the runtime PM state in my opinion.

So opp-set-rate isn't in the best of shape TBH, some things are left for the
drivers while other are done by it. Regulator-enable/disable was moved to it
some time back as people needed something like that. While on the other hand,
clk_enable/disable doesn't happen there, nor does rpm enable/disable.

Maybe one day we may want to do that, but lets make sure someone wants to do
that first.

Anyway, even in that case both of the changes would be required. We must make
device links nevertheless first. And later on if required, we may want to do rpm
enable/disable on the consumer device itself.
Ulf Hansson Aug. 25, 2020, 6:43 a.m. UTC | #6
On Tue, 25 Aug 2020 at 06:43, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> On 24-08-20, 17:08, Stephan Gerhold wrote:
> > On Mon, Aug 24, 2020 at 04:36:57PM +0200, Ulf Hansson wrote:
> > > That said, perhaps should rely on the consumer to deploy runtime PM
> > > support, but let the OPP core to set up the device links for the genpd
> > > virtual devices!?
> > >
> >
> > Yes, that would be the alternative option.
>
> That is the right option IMO.
>
> > I would be fine with it as long as it also works for the CPUfreq case.
> >
> > I don't think anything manages runtime PM for the CPU device, just
> > like no-one calls dev_pm_opp_set_rate(cpu_dev, 0). So with my patch the
> > power domain is essentially kept always-on (except for system suspend).
> > At least in my case this is intended.
> >
> > If device links also keep the power domains on if the consumer device
> > does not make use of runtime PM it should work fine for my case.
>
> With device link, you only need to do rpm enable/disable on the consumer device
> and it will get propagated by itself.

Note that the default state for the genpd virtual device(s) is that
runtime PM has been enabled for them. This means it's left in runtime
suspended state, which allows its PM domain to be powered off (if all
other devices and child domains for it allow that too, of course).

>
> > Personally, I think my original patch (without device links) fits better
> > into the OPP API, for the following two reasons.
> >
> > With device links:
> >
> >   1. Unlike regulators/interconnects, attached power domains would be
> >      controlled by runtime PM instead of dev_pm_opp_set_rate(opp_dev, 0).
> >
> >   2. ... some driver using OPP tables might not make use of runtime PM.
> >      In that case, the power domains would stay on the whole time,
> >      even if dev_pm_opp_set_rate(opp_dev, 0) was called.
> >
> > With my patch, the power domain state is directly related to the
> > dev_pm_opp_set_rate(opp_dev, 0) call, which is more intuitive than
> > relying on the runtime PM state in my opinion.
>
> So opp-set-rate isn't in the best of shape TBH, some things are left for the
> drivers while other are done by it. Regulator-enable/disable was moved to it
> some time back as people needed something like that. While on the other hand,
> clk_enable/disable doesn't happen there, nor does rpm enable/disable.
>
> Maybe one day we may want to do that, but lets make sure someone wants to do
> that first.
>
> Anyway, even in that case both of the changes would be required. We must make
> device links nevertheless first. And later on if required, we may want to do rpm
> enable/disable on the consumer device itself.

This sounds like a reasonable step-by-step approach.

Then, to create the device links, we should use DL_FLAG_PM_RUNTIME,
DL_FLAG_STATELESS.

But whether we should use DL_FLAG_RPM_ACTIVE as well, to initially
runtime resume the supplier (the genpd virtual device), is harder to
know - as that kind of depends on expectations by the consumer device
driver.

Kind regards
Uffe
Stephan Gerhold Aug. 25, 2020, 7:33 a.m. UTC | #7
On Tue, Aug 25, 2020 at 08:43:42AM +0200, Ulf Hansson wrote:
> On Tue, 25 Aug 2020 at 06:43, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> >
> > On 24-08-20, 17:08, Stephan Gerhold wrote:
> > > On Mon, Aug 24, 2020 at 04:36:57PM +0200, Ulf Hansson wrote:
> > > > That said, perhaps should rely on the consumer to deploy runtime PM
> > > > support, but let the OPP core to set up the device links for the genpd
> > > > virtual devices!?
> > > >
> > >
> > > Yes, that would be the alternative option.
> >
> > That is the right option IMO.
> >
> > > I would be fine with it as long as it also works for the CPUfreq case.
> > >
> > > I don't think anything manages runtime PM for the CPU device, just
> > > like no-one calls dev_pm_opp_set_rate(cpu_dev, 0). So with my patch the
> > > power domain is essentially kept always-on (except for system suspend).
> > > At least in my case this is intended.
> > >
> > > If device links also keep the power domains on if the consumer device
> > > does not make use of runtime PM it should work fine for my case.
> >
> > With device link, you only need to do rpm enable/disable on the consumer device
> > and it will get propagated by itself.
> 
> Note that the default state for the genpd virtual device(s) is that
> runtime PM has been enabled for them. This means it's left in runtime
> suspended state, which allows its PM domain to be powered off (if all
> other devices and child domains for it allow that too, of course).
> 
> >
> > > Personally, I think my original patch (without device links) fits better
> > > into the OPP API, for the following two reasons.
> > >
> > > With device links:
> > >
> > >   1. Unlike regulators/interconnects, attached power domains would be
> > >      controlled by runtime PM instead of dev_pm_opp_set_rate(opp_dev, 0).
> > >
> > >   2. ... some driver using OPP tables might not make use of runtime PM.
> > >      In that case, the power domains would stay on the whole time,
> > >      even if dev_pm_opp_set_rate(opp_dev, 0) was called.
> > >
> > > With my patch, the power domain state is directly related to the
> > > dev_pm_opp_set_rate(opp_dev, 0) call, which is more intuitive than
> > > relying on the runtime PM state in my opinion.
> >
> > So opp-set-rate isn't in the best of shape TBH, some things are left for the
> > drivers while other are done by it. Regulator-enable/disable was moved to it
> > some time back as people needed something like that. While on the other hand,
> > clk_enable/disable doesn't happen there, nor does rpm enable/disable.
> >
> > Maybe one day we may want to do that, but lets make sure someone wants to do
> > that first.
> >
> > Anyway, even in that case both of the changes would be required. We must make
> > device links nevertheless first. And later on if required, we may want to do rpm
> > enable/disable on the consumer device itself.
> 
> This sounds like a reasonable step-by-step approach.
> 
> Then, to create the device links, we should use DL_FLAG_PM_RUNTIME,
> DL_FLAG_STATELESS.
> 

OK, I will give this a try later this week.

> But whether we should use DL_FLAG_RPM_ACTIVE as well, to initially
> runtime resume the supplier (the genpd virtual device), is harder to
> know - as that kind of depends on expectations by the consumer device
> driver.
> 

I'm not sure I understand the purpose of that flag. I thought we want to
link the PM state of the virtual genpd device (supplier) to the PM state
of the device of the OPP table (consumer).

Shouldn't it just determine the initial state based on the state of the
consumer device?

Thanks!
Stephan
Ulf Hansson Aug. 25, 2020, 12:42 p.m. UTC | #8
On Tue, 25 Aug 2020 at 09:34, Stephan Gerhold <stephan@gerhold.net> wrote:
>
> On Tue, Aug 25, 2020 at 08:43:42AM +0200, Ulf Hansson wrote:
> > On Tue, 25 Aug 2020 at 06:43, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > >
> > > On 24-08-20, 17:08, Stephan Gerhold wrote:
> > > > On Mon, Aug 24, 2020 at 04:36:57PM +0200, Ulf Hansson wrote:
> > > > > That said, perhaps should rely on the consumer to deploy runtime PM
> > > > > support, but let the OPP core to set up the device links for the genpd
> > > > > virtual devices!?
> > > > >
> > > >
> > > > Yes, that would be the alternative option.
> > >
> > > That is the right option IMO.
> > >
> > > > I would be fine with it as long as it also works for the CPUfreq case.
> > > >
> > > > I don't think anything manages runtime PM for the CPU device, just
> > > > like no-one calls dev_pm_opp_set_rate(cpu_dev, 0). So with my patch the
> > > > power domain is essentially kept always-on (except for system suspend).
> > > > At least in my case this is intended.
> > > >
> > > > If device links also keep the power domains on if the consumer device
> > > > does not make use of runtime PM it should work fine for my case.
> > >
> > > With device link, you only need to do rpm enable/disable on the consumer device
> > > and it will get propagated by itself.
> >
> > Note that the default state for the genpd virtual device(s) is that
> > runtime PM has been enabled for them. This means it's left in runtime
> > suspended state, which allows its PM domain to be powered off (if all
> > other devices and child domains for it allow that too, of course).
> >
> > >
> > > > Personally, I think my original patch (without device links) fits better
> > > > into the OPP API, for the following two reasons.
> > > >
> > > > With device links:
> > > >
> > > >   1. Unlike regulators/interconnects, attached power domains would be
> > > >      controlled by runtime PM instead of dev_pm_opp_set_rate(opp_dev, 0).
> > > >
> > > >   2. ... some driver using OPP tables might not make use of runtime PM.
> > > >      In that case, the power domains would stay on the whole time,
> > > >      even if dev_pm_opp_set_rate(opp_dev, 0) was called.
> > > >
> > > > With my patch, the power domain state is directly related to the
> > > > dev_pm_opp_set_rate(opp_dev, 0) call, which is more intuitive than
> > > > relying on the runtime PM state in my opinion.
> > >
> > > So opp-set-rate isn't in the best of shape TBH, some things are left for the
> > > drivers while other are done by it. Regulator-enable/disable was moved to it
> > > some time back as people needed something like that. While on the other hand,
> > > clk_enable/disable doesn't happen there, nor does rpm enable/disable.
> > >
> > > Maybe one day we may want to do that, but lets make sure someone wants to do
> > > that first.
> > >
> > > Anyway, even in that case both of the changes would be required. We must make
> > > device links nevertheless first. And later on if required, we may want to do rpm
> > > enable/disable on the consumer device itself.
> >
> > This sounds like a reasonable step-by-step approach.
> >
> > Then, to create the device links, we should use DL_FLAG_PM_RUNTIME,
> > DL_FLAG_STATELESS.
> >
>
> OK, I will give this a try later this week.
>
> > But whether we should use DL_FLAG_RPM_ACTIVE as well, to initially
> > runtime resume the supplier (the genpd virtual device), is harder to
> > know - as that kind of depends on expectations by the consumer device
> > driver.
> >
>
> I'm not sure I understand the purpose of that flag. I thought we want to
> link the PM state of the virtual genpd device (supplier) to the PM state
> of the device of the OPP table (consumer).

Correct, but this is about synchronizing the initial runtime PM state
of the consumer and supplier.

>
> Shouldn't it just determine the initial state based on the state of the
> consumer device?

We could do that. Then something along the lines of the below, should work:

pm_runtime_get_noresume(consumer) - to prevent runtime suspend, temporarily.

if(pm_runtime_active(consumer))
    create links with DL_FLAG_RPM_ACTIVE
else
    create links without DL_FLAG_RPM_ACTIVE

pm_runtime_put(consumer)

Kind regards
Uffe
Stephan Gerhold Aug. 26, 2020, 8:31 a.m. UTC | #9
On Tue, Aug 25, 2020 at 02:42:54PM +0200, Ulf Hansson wrote:
> On Tue, 25 Aug 2020 at 09:34, Stephan Gerhold <stephan@gerhold.net> wrote:
> >
> > On Tue, Aug 25, 2020 at 08:43:42AM +0200, Ulf Hansson wrote:
> > > On Tue, 25 Aug 2020 at 06:43, Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > > >
> > > > On 24-08-20, 17:08, Stephan Gerhold wrote:
> > > > > On Mon, Aug 24, 2020 at 04:36:57PM +0200, Ulf Hansson wrote:
> > > > > > That said, perhaps should rely on the consumer to deploy runtime PM
> > > > > > support, but let the OPP core to set up the device links for the genpd
> > > > > > virtual devices!?
> > > > > >
> > > > >
> > > > > Yes, that would be the alternative option.
> > > >
> > > > That is the right option IMO.
> > > >
> > > > > I would be fine with it as long as it also works for the CPUfreq case.
> > > > >
> > > > > I don't think anything manages runtime PM for the CPU device, just
> > > > > like no-one calls dev_pm_opp_set_rate(cpu_dev, 0). So with my patch the
> > > > > power domain is essentially kept always-on (except for system suspend).
> > > > > At least in my case this is intended.
> > > > >
> > > > > If device links also keep the power domains on if the consumer device
> > > > > does not make use of runtime PM it should work fine for my case.
> > > >
> > > > With device link, you only need to do rpm enable/disable on the consumer device
> > > > and it will get propagated by itself.
> > >
> > > Note that the default state for the genpd virtual device(s) is that
> > > runtime PM has been enabled for them. This means it's left in runtime
> > > suspended state, which allows its PM domain to be powered off (if all
> > > other devices and child domains for it allow that too, of course).
> > >
> > > >
> > > > > Personally, I think my original patch (without device links) fits better
> > > > > into the OPP API, for the following two reasons.
> > > > >
> > > > > With device links:
> > > > >
> > > > >   1. Unlike regulators/interconnects, attached power domains would be
> > > > >      controlled by runtime PM instead of dev_pm_opp_set_rate(opp_dev, 0).
> > > > >
> > > > >   2. ... some driver using OPP tables might not make use of runtime PM.
> > > > >      In that case, the power domains would stay on the whole time,
> > > > >      even if dev_pm_opp_set_rate(opp_dev, 0) was called.
> > > > >
> > > > > With my patch, the power domain state is directly related to the
> > > > > dev_pm_opp_set_rate(opp_dev, 0) call, which is more intuitive than
> > > > > relying on the runtime PM state in my opinion.
> > > >
> > > > So opp-set-rate isn't in the best of shape TBH, some things are left for the
> > > > drivers while other are done by it. Regulator-enable/disable was moved to it
> > > > some time back as people needed something like that. While on the other hand,
> > > > clk_enable/disable doesn't happen there, nor does rpm enable/disable.
> > > >
> > > > Maybe one day we may want to do that, but lets make sure someone wants to do
> > > > that first.
> > > >
> > > > Anyway, even in that case both of the changes would be required. We must make
> > > > device links nevertheless first. And later on if required, we may want to do rpm
> > > > enable/disable on the consumer device itself.
> > >
> > > This sounds like a reasonable step-by-step approach.
> > >
> > > Then, to create the device links, we should use DL_FLAG_PM_RUNTIME,
> > > DL_FLAG_STATELESS.
> > >
> >
> > OK, I will give this a try later this week.
> >
> > > But whether we should use DL_FLAG_RPM_ACTIVE as well, to initially
> > > runtime resume the supplier (the genpd virtual device), is harder to
> > > know - as that kind of depends on expectations by the consumer device
> > > driver.
> > >
> >
> > I'm not sure I understand the purpose of that flag. I thought we want to
> > link the PM state of the virtual genpd device (supplier) to the PM state
> > of the device of the OPP table (consumer).
> 
> Correct, but this is about synchronizing the initial runtime PM state
> of the consumer and supplier.
> 
> >
> > Shouldn't it just determine the initial state based on the state of the
> > consumer device?
> 
> We could do that. Then something along the lines of the below, should work:
> 
> pm_runtime_get_noresume(consumer) - to prevent runtime suspend, temporarily.
> 
> if(pm_runtime_active(consumer))
>     create links with DL_FLAG_RPM_ACTIVE
> else
>     create links without DL_FLAG_RPM_ACTIVE
> 
> pm_runtime_put(consumer)
> 

This seems to work as expected, thanks for the suggestion!
I will submit a v2 with that shortly.

Thanks!
Stephan
diff mbox series

Patch

diff --git a/drivers/opp/core.c b/drivers/opp/core.c
index f93f551c911e..66ecffe12f01 100644
--- a/drivers/opp/core.c
+++ b/drivers/opp/core.c
@@ -17,6 +17,7 @@ 
 #include <linux/device.h>
 #include <linux/export.h>
 #include <linux/pm_domain.h>
+#include <linux/pm_runtime.h>
 #include <linux/regulator/consumer.h>
 
 #include "opp.h"
@@ -796,6 +797,26 @@  static int _set_required_opp(struct device *dev, struct device *pd_dev,
 	return ret;
 }
 
+static int _enable_required_opp(struct device *dev, struct device *pd_dev,
+				bool on)
+{
+	int ret;
+
+	if (on) {
+		ret = pm_runtime_get_sync(pd_dev);
+		if (ret < 0) {
+			pm_runtime_put_noidle(pd_dev);
+			dev_err(dev, "Failed to enable %s: %d\n",
+				dev_name(pd_dev), ret);
+			return ret;
+		}
+	} else {
+		pm_runtime_put(pd_dev);
+	}
+
+	return 0;
+}
+
 /* This is only called for PM domain for now */
 static int _set_required_opps(struct device *dev,
 			      struct opp_table *opp_table,
@@ -803,6 +824,8 @@  static int _set_required_opps(struct device *dev,
 {
 	struct opp_table **required_opp_tables = opp_table->required_opp_tables;
 	struct device **genpd_virt_devs = opp_table->genpd_virt_devs;
+	bool power_on = opp != NULL;
+	bool already_enabled = power_on == opp_table->genpd_virt_enabled;
 	struct device *pd_dev;
 	int i, ret = 0;
 
@@ -829,6 +852,20 @@  static int _set_required_opps(struct device *dev,
 			ret = _set_required_opp(dev, pd_dev, opp, i);
 			if (ret)
 				break;
+
+			if (likely(already_enabled))
+				continue;
+
+			ret = _enable_required_opp(dev, pd_dev, power_on);
+			if (ret)
+				break;
+		}
+
+		if (ret && !already_enabled) {
+			/* Rollback (skip current since it failed) */
+			for (i--; i >= 0; i--)
+				_enable_required_opp(dev, genpd_virt_devs[i],
+						     !power_on);
 		}
 	} else {
 		/* Scaling down? Set required OPPs in reverse order */
@@ -838,8 +875,26 @@  static int _set_required_opps(struct device *dev,
 			ret = _set_required_opp(dev, pd_dev, opp, i);
 			if (ret)
 				break;
+
+			if (likely(already_enabled))
+				continue;
+
+			ret = _enable_required_opp(dev, pd_dev, power_on);
+			if (ret)
+				break;
+		}
+
+		if (ret && !already_enabled) {
+			/* Rollback (skip current since it failed) */
+			for (i++; i < opp_table->required_opp_count; i++)
+				_enable_required_opp(dev, genpd_virt_devs[i],
+						     !power_on);
 		}
 	}
+
+	if (ret == 0 && !already_enabled)
+		opp_table->genpd_virt_enabled = power_on;
+
 	mutex_unlock(&opp_table->genpd_virt_dev_lock);
 
 	return ret;
diff --git a/drivers/opp/opp.h b/drivers/opp/opp.h
index e51646ff279e..01ad9e136cc8 100644
--- a/drivers/opp/opp.h
+++ b/drivers/opp/opp.h
@@ -188,6 +188,7 @@  struct opp_table {
 	struct device **genpd_virt_devs;
 	struct opp_table **required_opp_tables;
 	unsigned int required_opp_count;
+	bool genpd_virt_enabled;
 
 	unsigned int *supported_hw;
 	unsigned int supported_hw_count;