[00/11] i.MX8MM power domain support

Message

Lucas Stach Sept. 30, 2020, 3:49 p.m. UTC

Hi all,

this adds power domain support for the i.MX8MM to the existing GPCv2
driver. It is not complete yet, as it is still missing the VPU and
display power domains, as those require support for the BLK_CTL
regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
those regions on the i.MX8MP is currently under development and we
plan to use this as a template for the i.MX8MM when the dust has
settled. The changes in this series have been made with this in
mind, so once the BLK_CTL driver exists it should be a matter of
hooking things together via DT, with no further changes required on
the GPCv2 driver side (famous last words).

Special thanks to Marek Vasut who helped with testing and debugging
of early versions of this code.

Regards,
Lucas

Lucas Stach (11):
  soc: imx: gpcv2: move to more ideomatic error handling in probe
  soc: imx: gpcv2: move domain mapping to domain driver probe
  soc: imx: gpcv2: split power up and power down sequence control
  soc: imx: gpcv2: wait for ADB400 handshake
  soc: imx: gpcv2: add runtime PM support for power-domains
  soc: imx: gpcv2: allow domains without power-sequence control
  soc: imx: gpcv2: add support for optional resets
  dt-bindings: add defines for i.MX8MM power domains
  soc: imx: gpcv2: add support for i.MX8MM power domains
  arm64: dts: imx8mm: add GPC node and power domains
  arm64: dts: imx8mm: put USB controllers into power-domains

 .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
 arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
 drivers/soc/imx/gpcv2.c                       | 501 +++++++++++++++---
 include/dt-bindings/power/imx8mm-power.h      |  22 +
 4 files changed, 516 insertions(+), 74 deletions(-)
 create mode 100644 include/dt-bindings/power/imx8mm-power.h

Comments

Frieder Schrempf Oct. 1, 2020, 7:46 a.m. UTC | #1

On 30.09.20 17:49, Lucas Stach wrote:
> Hi all,
> 
> this adds power domain support for the i.MX8MM to the existing GPCv2
> driver. It is not complete yet, as it is still missing the VPU and
> display power domains, as those require support for the BLK_CTL
> regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
> those regions on the i.MX8MP is currently under development and we
> plan to use this as a template for the i.MX8MM when the dust has
> settled. The changes in this series have been made with this in
> mind, so once the BLK_CTL driver exists it should be a matter of
> hooking things together via DT, with no further changes required on
> the GPCv2 driver side (famous last words).
> 
> Special thanks to Marek Vasut who helped with testing and debugging
> of early versions of this code.

I tested this on our i.MX8MM boards by making sure the GPUs and USBs 
come up properly. It works just fine on v5.9-rc6 and also backported to 
5.4. So for the whole series:

Tested-by: Frieder Schrempf <[e-mail]>

> 
> Regards,
> Lucas
> 
> Lucas Stach (11):
>    soc: imx: gpcv2: move to more ideomatic error handling in probe
>    soc: imx: gpcv2: move domain mapping to domain driver probe
>    soc: imx: gpcv2: split power up and power down sequence control
>    soc: imx: gpcv2: wait for ADB400 handshake
>    soc: imx: gpcv2: add runtime PM support for power-domains
>    soc: imx: gpcv2: allow domains without power-sequence control
>    soc: imx: gpcv2: add support for optional resets
>    dt-bindings: add defines for i.MX8MM power domains
>    soc: imx: gpcv2: add support for i.MX8MM power domains
>    arm64: dts: imx8mm: add GPC node and power domains
>    arm64: dts: imx8mm: put USB controllers into power-domains
> 
>   .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
>   arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
>   drivers/soc/imx/gpcv2.c                       | 501 +++++++++++++++---
>   include/dt-bindings/power/imx8mm-power.h      |  22 +
>   4 files changed, 516 insertions(+), 74 deletions(-)
>   create mode 100644 include/dt-bindings/power/imx8mm-power.h
>

Adam Ford Oct. 3, 2020, 6:03 p.m. UTC | #2

On Wed, Sep 30, 2020 at 10:50 AM Lucas Stach <[e-mail]> wrote:
>
> Hi all,
>
> this adds power domain support for the i.MX8MM to the existing GPCv2
> driver. It is not complete yet, as it is still missing the VPU and
> display power domains, as those require support for the BLK_CTL
> regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
> those regions on the i.MX8MP is currently under development and we
> plan to use this as a template for the i.MX8MM when the dust has
> settled. The changes in this series have been made with this in
> mind, so once the BLK_CTL driver exists it should be a matter of
> hooking things together via DT, with no further changes required on
> the GPCv2 driver side (famous last words).
>
> Special thanks to Marek Vasut who helped with testing and debugging
> of early versions of this code.
>
> Regards,
> Lucas
>
> Lucas Stach (11):
>   soc: imx: gpcv2: move to more ideomatic error handling in probe
>   soc: imx: gpcv2: move domain mapping to domain driver probe
>   soc: imx: gpcv2: split power up and power down sequence control
>   soc: imx: gpcv2: wait for ADB400 handshake
>   soc: imx: gpcv2: add runtime PM support for power-domains
>   soc: imx: gpcv2: allow domains without power-sequence control
>   soc: imx: gpcv2: add support for optional resets
>   dt-bindings: add defines for i.MX8MM power domains
>   soc: imx: gpcv2: add support for i.MX8MM power domains
>   arm64: dts: imx8mm: add GPC node and power domains
>   arm64: dts: imx8mm: put USB controllers into power-domains

I fully support trying to get this done.  I tried to push something
like this before, but I was turned down by someone at NXP [1].  I have
both a Mini and Nano board at my disposal, so if Nano support isn't
there yet, and this driver patch gets accepted, I'll gladly work to
help and/or test functionality.

[1] - https://lkml.org/lkml/2020/4/30/540

adam

>
>  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
>  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
>  drivers/soc/imx/gpcv2.c                       | 501 +++++++++++++++---
>  include/dt-bindings/power/imx8mm-power.h      |  22 +
>  4 files changed, 516 insertions(+), 74 deletions(-)
>  create mode 100644 include/dt-bindings/power/imx8mm-power.h
>
> --
> 2.20.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> [e-mail]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Marek Vasut Oct. 3, 2020, 6:08 p.m. UTC | #3

On 10/3/20 8:01 PM, Adam Ford wrote:
> On Wed, Sep 30, 2020 at 10:50 AM Lucas Stach <[e-mail]> wrote:
> 
>> Hi all,
>>
>> this adds power domain support for the i.MX8MM to the existing GPCv2
>> driver. It is not complete yet, as it is still missing the VPU and
>> display power domains, as those require support for the BLK_CTL
>> regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
>> those regions on the i.MX8MP is currently under development and we
>> plan to use this as a template for the i.MX8MM when the dust has
>> settled. The changes in this series have been made with this in
>> mind, so once the BLK_CTL driver exists it should be a matter of
>> hooking things together via DT, with no further changes required on
>> the GPCv2 driver side (famous last words).
>>
>> Special thanks to Marek Vasut who helped with testing and debugging
>> of early versions of this code.
>>
>>
> I fully support trying to get this done.  I tried to push something this
> before, but I was turned down by someone at NXP [1].  I have both a Mini
> and Nano board at my disposal, so if Nano support isn't there yet, and this
> driver patch gets accepted, I'll gladly work to help and/or test
> functionality.
> 
> [1] - https://lkml.org/lkml/2020/4/30/540

Note that the [1] is using poorly defined and buggy SMC interface to
TFA, I agree with Lucas that is not the way to go for MX8M. This
patchset is not using that SMC interface at all, but rather lets Linux
do the power domain handling, which I think is much better.

Adam Ford Oct. 3, 2020, 6:11 p.m. UTC | #4

On Sat, Oct 3, 2020 at 1:08 PM Marek Vasut <[e-mail]> wrote:
>
> On 10/3/20 8:01 PM, Adam Ford wrote:
> > On Wed, Sep 30, 2020 at 10:50 AM Lucas Stach <[e-mail]> wrote:
> >
> >> Hi all,
> >>
> >> this adds power domain support for the i.MX8MM to the existing GPCv2
> >> driver. It is not complete yet, as it is still missing the VPU and
> >> display power domains, as those require support for the BLK_CTL
> >> regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
> >> those regions on the i.MX8MP is currently under development and we
> >> plan to use this as a template for the i.MX8MM when the dust has
> >> settled. The changes in this series have been made with this in
> >> mind, so once the BLK_CTL driver exists it should be a matter of
> >> hooking things together via DT, with no further changes required on
> >> the GPCv2 driver side (famous last words).
> >>
> >> Special thanks to Marek Vasut who helped with testing and debugging
> >> of early versions of this code.
> >>
> >>
> > I fully support trying to get this done.  I tried to push something this
> > before, but I was turned down by someone at NXP [1].  I have both a Mini
> > and Nano board at my disposal, so if Nano support isn't there yet, and this
> > driver patch gets accepted, I'll gladly work to help and/or test
> > functionality.
> >
> > [1] - https://lkml.org/lkml/2020/4/30/540
>
> Note that the [1] is using poorly defined and buggy SMC interface to
> TFA, I agree with Lucas that is not the way to go for MX8M. This
> patchset is not using that SMC interface at all, but rather lets Linux
> do the power domain handling, which I think is much better.

I hope you're more successful in convincing NXP of that than Lucas and
I were before.  Our pleas fell on deaf ears.

adam

Adam Ford Oct. 8, 2020, 8:47 p.m. UTC | #5

On Wed, Sep 30, 2020 at 10:50 AM Lucas Stach <[e-mail]> wrote:
>
> Hi all,
>
> this adds power domain support for the i.MX8MM to the existing GPCv2
> driver. It is not complete yet, as it is still missing the VPU and
> display power domains, as those require support for the BLK_CTL
> regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
> those regions on the i.MX8MP is currently under development and we
> plan to use this as a template for the i.MX8MM when the dust has
> settled. The changes in this series have been made with this in
> mind, so once the BLK_CTL driver exists it should be a matter of
> hooking things together via DT, with no further changes required on
> the GPCv2 driver side (famous last words).

Thanks for improving this.

With this series, I was able to test the USB and it is functional.
Without it, the system would hang when attempting to use USB.

>
> Special thanks to Marek Vasut who helped with testing and debugging
> of early versions of this code.
>

For the series:

Tested-by: Adam Ford <[e-mail]>

> Regards,
> Lucas
>
> Lucas Stach (11):
>   soc: imx: gpcv2: move to more ideomatic error handling in probe
>   soc: imx: gpcv2: move domain mapping to domain driver probe
>   soc: imx: gpcv2: split power up and power down sequence control
>   soc: imx: gpcv2: wait for ADB400 handshake
>   soc: imx: gpcv2: add runtime PM support for power-domains
>   soc: imx: gpcv2: allow domains without power-sequence control
>   soc: imx: gpcv2: add support for optional resets
>   dt-bindings: add defines for i.MX8MM power domains
>   soc: imx: gpcv2: add support for i.MX8MM power domains
>   arm64: dts: imx8mm: add GPC node and power domains
>   arm64: dts: imx8mm: put USB controllers into power-domains
>
>  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
>  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
>  drivers/soc/imx/gpcv2.c                       | 501 +++++++++++++++---
>  include/dt-bindings/power/imx8mm-power.h      |  22 +
>  4 files changed, 516 insertions(+), 74 deletions(-)
>  create mode 100644 include/dt-bindings/power/imx8mm-power.h
>
> --
> 2.20.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> [e-mail]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Jacky Bai Oct. 9, 2020, 3 a.m. UTC | #6

> -----Original Message-----
> From: Lucas Stach [mailto:[e-mail]]
> Sent: Wednesday, September 30, 2020 11:50 PM
> To: Shawn Guo <[e-mail]>; Rob Herring <[e-mail]>
> Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> <[e-mail]>; Frieder Schrempf <[e-mail]>;
> Marek Vasut <[e-mail]>; [e-mail];
> [e-mail]; [e-mail];
> [e-mail]
> Subject: [PATCH 00/11] i.MX8MM power domain support
> 
> Hi all,
> 
> this adds power domain support for the i.MX8MM to the existing GPCv2 driver.
> It is not complete yet, as it is still missing the VPU and display power domains,
> as those require support for the BLK_CTL regions of the VPUMIX and
> DISPLAYMIX domains. A Linux driver for those regions on the i.MX8MP is
> currently under development and we plan to use this as a template for the
> i.MX8MM when the dust has settled. The changes in this series have been
> made with this in mind, so once the BLK_CTL driver exists it should be a
> matter of hooking things together via DT, with no further changes required on
> the GPCv2 driver side (famous last words).
> 
> Special thanks to Marek Vasut who helped with testing and debugging of early
> versions of this code.
> 

Lucas,

thanks for working on this, but I think current support for 8MM can NOT 100% work due to HW limitation.
Maybe, we need further discussion before moving forward, otherwise, we will meet awkward situation when NXP
doing LTS upgrade. Below are some info shared.

1. The GPU & VPU related power domains need to do special handling due to HW limitation, can refer to the power domain sequence
  In NXP release.
2. another reason that we do power domain control in TF-A in NXP release is that MAIN NOC power domain can only be controlled by
  TF-A, and before MAIN NOC power domain, we need to check other MIXs' power status. If other power domain is controlled by linux side,
  It is not easy to cross world status sync.
3. either 8MM, 8MN, or 8MP, the power domain design is different, I am not sure if it is the good to add hundreds line of code in GPCv2 each time
  a new SOC is added.

BR
Jacky Bai

> Regards,
> Lucas
> 
> Lucas Stach (11):
>   soc: imx: gpcv2: move to more ideomatic error handling in probe
>   soc: imx: gpcv2: move domain mapping to domain driver probe
>   soc: imx: gpcv2: split power up and power down sequence control
>   soc: imx: gpcv2: wait for ADB400 handshake
>   soc: imx: gpcv2: add runtime PM support for power-domains
>   soc: imx: gpcv2: allow domains without power-sequence control
>   soc: imx: gpcv2: add support for optional resets
>   dt-bindings: add defines for i.MX8MM power domains
>   soc: imx: gpcv2: add support for i.MX8MM power domains
>   arm64: dts: imx8mm: add GPC node and power domains
>   arm64: dts: imx8mm: put USB controllers into power-domains
> 
>  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
>  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
>  drivers/soc/imx/gpcv2.c                       | 501
> +++++++++++++++---
>  include/dt-bindings/power/imx8mm-power.h      |  22 +
>  4 files changed, 516 insertions(+), 74 deletions(-)  create mode 100644
> include/dt-bindings/power/imx8mm-power.h
> 
> --
> 2.20.1

Lucas Stach Oct. 9, 2020, 11:12 a.m. UTC | #7

Hi Jacky,

On Fr, 2020-10-09 at 03:00 +0000, Jacky Bai wrote:
> > -----Original Message-----
> > From: Lucas Stach [mailto:[e-mail]]
> > Sent: Wednesday, September 30, 2020 11:50 PM
> > To: Shawn Guo <[e-mail]>; Rob Herring <[e-mail]>
> > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > <[e-mail]>; Frieder Schrempf <[e-mail]>;
> > Marek Vasut <[e-mail]>; [e-mail];
> > [e-mail]; [e-mail];
> > [e-mail]
> > Subject: [PATCH 00/11] i.MX8MM power domain support
> > 
> > Hi all,
> > 
> > this adds power domain support for the i.MX8MM to the existing GPCv2 driver.
> > It is not complete yet, as it is still missing the VPU and display power domains,
> > as those require support for the BLK_CTL regions of the VPUMIX and
> > DISPLAYMIX domains. A Linux driver for those regions on the i.MX8MP is
> > currently under development and we plan to use this as a template for the
> > i.MX8MM when the dust has settled. The changes in this series have been
> > made with this in mind, so once the BLK_CTL driver exists it should be a
> > matter of hooking things together via DT, with no further changes required on
> > the GPCv2 driver side (famous last words).
> > 
> > Special thanks to Marek Vasut who helped with testing and debugging of early
> > versions of this code.
> > 
> 
> Lucas,
> 
> thanks for working on this, but I think current support for 8MM can NOT 100% work due to HW limitation.
> Maybe, we need further discussion before moving forward, otherwise, we will meet awkward situation when NXP
> doing LTS upgrade. Below are some info shared.
> 
> 1. The GPU & VPU related power domains need to do special handling due to HW limitation, can refer to the power domain sequence
>   In NXP release.

For the GPU this driver already does the same thing as the TF-A based
implementation by driving the GPU2D and GPU3D domains together and
triggering the SRC reset.

For the VPU I expect that we can do all the necessary syncing with a
proper VPU BLK_CTL driver.

> 2. another reason that we do power domain control in TF-A in NXP release is that MAIN NOC power domain can only be controlled by
>   TF-A, and before MAIN NOC power domain, we need to check other MIXs' power status. If other power domain is controlled by linux side,
>   It is not easy to cross world status sync.

This is a valid concern and I want to learn more about this. When do
you turn off MAIN NOC power in the TF-A? Is it just system suspend? If
so I think it's a valid requirement for the kernel driver to shut down
all the peripheral power domains before entering system suspend.

> 3. either 8MM, 8MN, or 8MP, the power domain design is different, I am not sure if it is the good to add hundreds line of code in GPCv2 each time
>   a new SOC is added.

I don't buy into this argument. We have lots of drivers in the Linux
kernel that require some changes for new SoC generations, that's what
Linux drivers are for. The complexity of the hardware doesn't disappear
just because you push some of the driver bits into TF-A, you just
handle the complexity at a different palce and IMHO that the wrong
place. The power domains have complex interactions with other drivers
in the Linux system, so debugging and deplyong fixes is much easier
when the power domain handling is fully done by a kernel driver.

Regards,
Lucas

> BR
> Jacky Bai
> 
> > Regards,
> > Lucas
> > 
> > Lucas Stach (11):
> >   soc: imx: gpcv2: move to more ideomatic error handling in probe
> >   soc: imx: gpcv2: move domain mapping to domain driver probe
> >   soc: imx: gpcv2: split power up and power down sequence control
> >   soc: imx: gpcv2: wait for ADB400 handshake
> >   soc: imx: gpcv2: add runtime PM support for power-domains
> >   soc: imx: gpcv2: allow domains without power-sequence control
> >   soc: imx: gpcv2: add support for optional resets
> >   dt-bindings: add defines for i.MX8MM power domains
> >   soc: imx: gpcv2: add support for i.MX8MM power domains
> >   arm64: dts: imx8mm: add GPC node and power domains
> >   arm64: dts: imx8mm: put USB controllers into power-domains
> > 
> >  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
> >  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
> >  drivers/soc/imx/gpcv2.c                       | 501
> > +++++++++++++++---
> >  include/dt-bindings/power/imx8mm-power.h      |  22 +
> >  4 files changed, 516 insertions(+), 74 deletions(-)  create mode 100644
> > include/dt-bindings/power/imx8mm-power.h
> > 
> > --
> > 2.20.1

Adam Ford Oct. 9, 2020, 12:57 p.m. UTC | #8

On Fri, Oct 9, 2020 at 6:16 AM Lucas Stach <[e-mail]> wrote:
>
> Hi Jacky,
>
> On Fr, 2020-10-09 at 03:00 +0000, Jacky Bai wrote:
> > > -----Original Message-----
> > > From: Lucas Stach [mailto:[e-mail]]
> > > Sent: Wednesday, September 30, 2020 11:50 PM
> > > To: Shawn Guo <[e-mail]>; Rob Herring <[e-mail]>
> > > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > > <[e-mail]>; Frieder Schrempf <[e-mail]>;
> > > Marek Vasut <[e-mail]>; [e-mail];
> > > [e-mail]; [e-mail];
> > > [e-mail]
> > > Subject: [PATCH 00/11] i.MX8MM power domain support
> > >
> > > Hi all,
> > >
> > > this adds power domain support for the i.MX8MM to the existing GPCv2 driver.
> > > It is not complete yet, as it is still missing the VPU and display power domains,
> > > as those require support for the BLK_CTL regions of the VPUMIX and
> > > DISPLAYMIX domains. A Linux driver for those regions on the i.MX8MP is
> > > currently under development and we plan to use this as a template for the
> > > i.MX8MM when the dust has settled. The changes in this series have been
> > > made with this in mind, so once the BLK_CTL driver exists it should be a
> > > matter of hooking things together via DT, with no further changes required on
> > > the GPCv2 driver side (famous last words).
> > >
> > > Special thanks to Marek Vasut who helped with testing and debugging of early
> > > versions of this code.
> > >
> >
> > Lucas,
> >
> > thanks for working on this, but I think current support for 8MM can NOT 100% work due to HW limitation.
> > Maybe, we need further discussion before moving forward, otherwise, we will meet awkward situation when NXP
> > doing LTS upgrade. Below are some info shared.
> >
> > 1. The GPU & VPU related power domains need to do special handling due to HW limitation, can refer to the power domain sequence
> >   In NXP release.
>
> For the GPU this driver already does the same thing as the TF-A based
> implementation by driving the GPU2D and GPU3D domains together and
> triggering the SRC reset.
>
> For the VPU I expect that we can do all the necessary syncing with a
> proper VPU BLK_CTL driver.
>
> > 2. another reason that we do power domain control in TF-A in NXP release is that MAIN NOC power domain can only be controlled by
> >   TF-A, and before MAIN NOC power domain, we need to check other MIXs' power status. If other power domain is controlled by linux side,
> >   It is not easy to cross world status sync.
>
> This is a valid concern and I want to learn more about this. When do
> you turn off MAIN NOC power in the TF-A? Is it just system suspend? If
> so I think it's a valid requirement for the kernel driver to shut down
> all the peripheral power domains before entering system suspend.
>
> > 3. either 8MM, 8MN, or 8MP, the power domain design is different, I am not sure if it is the good to add hundreds line of code in GPCv2 each time
> >   a new SOC is added.
>
> I don't buy into this argument. We have lots of drivers in the Linux
> kernel that require some changes for new SoC generations, that's what
> Linux drivers are for. The complexity of the hardware doesn't disappear
> just because you push some of the driver bits into TF-A, you just
> handle the complexity at a different palce and IMHO that the wrong
> place. The power domains have complex interactions with other drivers
> in the Linux system, so debugging and deplyong fixes is much easier
> when the power domain handling is fully done by a kernel driver.

In an effort to keep the code size manageable, what if we were to
propose a gpc-core configured to be a generic function common to all
SoC's, and move the tables for each unique SoC into separate files.
Making each SoC's GPC a Kconfig option could give people the ability
to disable the various options that don't apply to their specific
application, and the setup and configuration of the tables should be
easier to read.  I know of at least one touchscreen driver that does
this (tsc200x).

adam
>
> Regards,
> Lucas
>
> > BR
> > Jacky Bai
> >
> > > Regards,
> > > Lucas
> > >
> > > Lucas Stach (11):
> > >   soc: imx: gpcv2: move to more ideomatic error handling in probe
> > >   soc: imx: gpcv2: move domain mapping to domain driver probe
> > >   soc: imx: gpcv2: split power up and power down sequence control
> > >   soc: imx: gpcv2: wait for ADB400 handshake
> > >   soc: imx: gpcv2: add runtime PM support for power-domains
> > >   soc: imx: gpcv2: allow domains without power-sequence control
> > >   soc: imx: gpcv2: add support for optional resets
> > >   dt-bindings: add defines for i.MX8MM power domains
> > >   soc: imx: gpcv2: add support for i.MX8MM power domains
> > >   arm64: dts: imx8mm: add GPC node and power domains
> > >   arm64: dts: imx8mm: put USB controllers into power-domains
> > >
> > >  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
> > >  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
> > >  drivers/soc/imx/gpcv2.c                       | 501
> > > +++++++++++++++---
> > >  include/dt-bindings/power/imx8mm-power.h      |  22 +
> > >  4 files changed, 516 insertions(+), 74 deletions(-)  create mode 100644
> > > include/dt-bindings/power/imx8mm-power.h
> > >
> > > --
> > > 2.20.1
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> [e-mail]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Jacky Bai Oct. 10, 2020, 2:16 a.m. UTC | #9

> -----Original Message-----
> From: Lucas Stach [mailto:[e-mail]]
> Sent: Friday, October 9, 2020 7:12 PM
> To: Jacky Bai <[e-mail]>; Shawn Guo <[e-mail]>; Rob
> Herring <[e-mail]>
> Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> <[e-mail]>; Frieder Schrempf <[e-mail]>;
> Marek Vasut <[e-mail]>; [e-mail];
> [e-mail]; [e-mail];
> [e-mail]
> Subject: Re: [PATCH 00/11] i.MX8MM power domain support
> 
> Hi Jacky,
> 
> On Fr, 2020-10-09 at 03:00 +0000, Jacky Bai wrote:
> > > -----Original Message-----
> > > From: Lucas Stach [mailto:[e-mail]]
> > > Sent: Wednesday, September 30, 2020 11:50 PM
> > > To: Shawn Guo <[e-mail]>; Rob Herring
> > > <[e-mail]>
> > > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > > <[e-mail]>; Frieder Schrempf
> > > <[e-mail]>; Marek Vasut <[e-mail]>;
> > > [e-mail];
> > > [e-mail]; [e-mail];
> > > [e-mail]
> > > Subject: [PATCH 00/11] i.MX8MM power domain support
> > >
> > > Hi all,
> > >
> > > this adds power domain support for the i.MX8MM to the existing GPCv2
> driver.
> > > It is not complete yet, as it is still missing the VPU and display
> > > power domains, as those require support for the BLK_CTL regions of
> > > the VPUMIX and DISPLAYMIX domains. A Linux driver for those regions
> > > on the i.MX8MP is currently under development and we plan to use
> > > this as a template for the i.MX8MM when the dust has settled. The
> > > changes in this series have been made with this in mind, so once the
> > > BLK_CTL driver exists it should be a matter of hooking things
> > > together via DT, with no further changes required on the GPCv2 driver side
> (famous last words).
> > >
> > > Special thanks to Marek Vasut who helped with testing and debugging
> > > of early versions of this code.
> > >
> >
> > Lucas,
> >
> > thanks for working on this, but I think current support for 8MM can NOT 100%
> work due to HW limitation.
> > Maybe, we need further discussion before moving forward, otherwise, we
> > will meet awkward situation when NXP doing LTS upgrade. Below are some
> info shared.
> >
> > 1. The GPU & VPU related power domains need to do special handling due
> to HW limitation, can refer to the power domain sequence
> >   In NXP release.
> 
> For the GPU this driver already does the same thing as the TF-A based
> implementation by driving the GPU2D and GPU3D domains together and
> triggering the SRC reset.
> 
> For the VPU I expect that we can do all the necessary syncing with a proper
> VPU BLK_CTL driver.
> 

Ok, thanks. I saw the reset handling in this patchset.

> > 2. another reason that we do power domain control in TF-A in NXP release is
> that MAIN NOC power domain can only be controlled by
> >   TF-A, and before MAIN NOC power domain, we need to check other
> MIXs' power status. If other power domain is controlled by linux side,
> >   It is not easy to cross world status sync.
> 
> This is a valid concern and I want to learn more about this. When do you turn
> off MAIN NOC power in the TF-A? Is it just system suspend? If so I think it's a
> valid requirement for the kernel driver to shut down all the peripheral power
> domains before entering system suspend.
> 

The main NOC will be off just in system suspend case. Main NoC on/off is controlled by
GPC HW slot method. As all the MIXs with ADB400 bridge are connected on the main NoC,
we must make sure that all these ADB400 port in power down status when main NoC power off
in system suspend, otherwise system will hang when resume. Previously, all the MIX power domain
is controlled by TF-A, then TF-A knows the status of each MIX, if any MIX is on, we will skip the NOC
power down setting.

> > 3. either 8MM, 8MN, or 8MP, the power domain design is different, I am not
> sure if it is the good to add hundreds line of code in GPCv2 each time
> >   a new SOC is added.
> 
> I don't buy into this argument. We have lots of drivers in the Linux kernel that
> require some changes for new SoC generations, that's what Linux drivers are
> for. The complexity of the hardware doesn't disappear just because you push
> some of the driver bits into TF-A, you just handle the complexity at a different
> palce and IMHO that the wrong place. The power domains have complex
> interactions with other drivers in the Linux system, so debugging and
> deplyong fixes is much easier when the power domain handling is fully done
> by a kernel driver.

Actually, due to the security requirement from other system solution provider,
for example, Microsoft Azure Sphere, it has strict requirement for power domain
to be controlled by secure subsystem(either TF-A, TEE or dedicated secure domain controller).
Same requirement for reset control, and system critical clock control.

For NXP i.MX8M family, it is ok to implement in linux kernel, just a tradeoff to find out a place
to hide the complexity ^_^.

BTW, for virtualization support, it is better to put the power domain in a central place to simplify
the VM implementation.

BR
Jacky Bai

> 
> Regards,
> Lucas
> 
> > BR
> > Jacky Bai
> >
> > > Regards,
> > > Lucas
> > >
> > > Lucas Stach (11):
> > >   soc: imx: gpcv2: move to more ideomatic error handling in probe
> > >   soc: imx: gpcv2: move domain mapping to domain driver probe
> > >   soc: imx: gpcv2: split power up and power down sequence control
> > >   soc: imx: gpcv2: wait for ADB400 handshake
> > >   soc: imx: gpcv2: add runtime PM support for power-domains
> > >   soc: imx: gpcv2: allow domains without power-sequence control
> > >   soc: imx: gpcv2: add support for optional resets
> > >   dt-bindings: add defines for i.MX8MM power domains
> > >   soc: imx: gpcv2: add support for i.MX8MM power domains
> > >   arm64: dts: imx8mm: add GPC node and power domains
> > >   arm64: dts: imx8mm: put USB controllers into power-domains
> > >
> > >  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
> > >  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
> > >  drivers/soc/imx/gpcv2.c                       | 501
> > > +++++++++++++++---
> > >  include/dt-bindings/power/imx8mm-power.h      |  22 +
> > >  4 files changed, 516 insertions(+), 74 deletions(-)  create mode
> > > 100644 include/dt-bindings/power/imx8mm-power.h
> > >
> > > --
> > > 2.20.1

Lucas Stach Oct. 13, 2020, 6:26 p.m. UTC | #10

On Sa, 2020-10-10 at 02:16 +0000, Jacky Bai wrote:
> > -----Original Message-----
> > From: Lucas Stach [mailto:[e-mail]]
> > Sent: Friday, October 9, 2020 7:12 PM
> > To: Jacky Bai <[e-mail]>; Shawn Guo <[e-mail]>; Rob
> > Herring <[e-mail]>
> > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > <[e-mail]>; Frieder Schrempf <[e-mail]>;
> > Marek Vasut <[e-mail]>; [e-mail];
> > [e-mail]; [e-mail];
> > [e-mail]
> > Subject: Re: [PATCH 00/11] i.MX8MM power domain support
> > 
> > Hi Jacky,
> > 
> > On Fr, 2020-10-09 at 03:00 +0000, Jacky Bai wrote:
> > > > -----Original Message-----
> > > > From: Lucas Stach [mailto:[e-mail]]
> > > > Sent: Wednesday, September 30, 2020 11:50 PM
> > > > To: Shawn Guo <[e-mail]>; Rob Herring
> > > > <[e-mail]>
> > > > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > > > <[e-mail]>; Frieder Schrempf
> > > > <[e-mail]>; Marek Vasut <[e-mail]>;
> > > > [e-mail];
> > > > [e-mail]; [e-mail];
> > > > [e-mail]
> > > > Subject: [PATCH 00/11] i.MX8MM power domain support
> > > > 
> > > > Hi all,
> > > > 
> > > > this adds power domain support for the i.MX8MM to the existing GPCv2
> > driver.
> > > > It is not complete yet, as it is still missing the VPU and display
> > > > power domains, as those require support for the BLK_CTL regions of
> > > > the VPUMIX and DISPLAYMIX domains. A Linux driver for those regions
> > > > on the i.MX8MP is currently under development and we plan to use
> > > > this as a template for the i.MX8MM when the dust has settled. The
> > > > changes in this series have been made with this in mind, so once the
> > > > BLK_CTL driver exists it should be a matter of hooking things
> > > > together via DT, with no further changes required on the GPCv2 driver side
> > (famous last words).
> > > > Special thanks to Marek Vasut who helped with testing and debugging
> > > > of early versions of this code.
> > > > 
> > > 
> > > Lucas,
> > > 
> > > thanks for working on this, but I think current support for 8MM can NOT 100%
> > work due to HW limitation.
> > > Maybe, we need further discussion before moving forward, otherwise, we
> > > will meet awkward situation when NXP doing LTS upgrade. Below are some
> > info shared.
> > > 1. The GPU & VPU related power domains need to do special handling due
> > to HW limitation, can refer to the power domain sequence
> > >   In NXP release.
> > 
> > For the GPU this driver already does the same thing as the TF-A based
> > implementation by driving the GPU2D and GPU3D domains together and
> > triggering the SRC reset.
> > 
> > For the VPU I expect that we can do all the necessary syncing with a proper
> > VPU BLK_CTL driver.
> > 
> 
> Ok, thanks. I saw the reset handling in this patchset.
> 
> > > 2. another reason that we do power domain control in TF-A in NXP release is
> > that MAIN NOC power domain can only be controlled by
> > >   TF-A, and before MAIN NOC power domain, we need to check other
> > MIXs' power status. If other power domain is controlled by linux side,
> > >   It is not easy to cross world status sync.
> > 
> > This is a valid concern and I want to learn more about this. When do you turn
> > off MAIN NOC power in the TF-A? Is it just system suspend? If so I think it's a
> > valid requirement for the kernel driver to shut down all the peripheral power
> > domains before entering system suspend.
> > 
> 
> The main NOC will be off just in system suspend case. Main NoC on/off is controlled by
> GPC HW slot method. As all the MIXs with ADB400 bridge are connected on the main NoC,
> we must make sure that all these ADB400 port in power down status when main NoC power off
> in system suspend, otherwise system will hang when resume. Previously, all the MIX power domain
> is controlled by TF-A, then TF-A knows the status of each MIX, if any MIX is on, we will skip the NOC
> power down setting.

And that just means the SoC doesn't enter the desired low-power state
when going into suspend, right? I think it would be much easier and
less error prone to just mandate that all peripheral power domains must
be powered down before trying to enter system suspend.

Is there any use-case where you would legitimately want to enter a high
power suspend state with some of ther peripheral power domains still
active?

> > > 3. either 8MM, 8MN, or 8MP, the power domain design is different, I am not
> > sure if it is the good to add hundreds line of code in GPCv2 each time
> > >   a new SOC is added.
> > 
> > I don't buy into this argument. We have lots of drivers in the Linux kernel that
> > require some changes for new SoC generations, that's what Linux drivers are
> > for. The complexity of the hardware doesn't disappear just because you push
> > some of the driver bits into TF-A, you just handle the complexity at a different
> > palce and IMHO that the wrong place. The power domains have complex
> > interactions with other drivers in the Linux system, so debugging and
> > deplyong fixes is much easier when the power domain handling is fully done
> > by a kernel driver.
> 
> Actually, due to the security requirement from other system solution provider,
> for example, Microsoft Azure Sphere, it has strict requirement for power domain
> to be controlled by secure subsystem(either TF-A, TEE or dedicated secure domain controller).
> Same requirement for reset control, and system critical clock control.

Yes, I'm aware of those requirements, but to satisfy those you need a
full implementation of all those parts in the secure subsystem. Doing
it just for the power domains adds complexity for no gain, as you still
won't be able to meet all the requirements and frankly I don't think
this is a realistic goal to achieve with the current i.MX8M family of
SoCs. 

Meeting those requirements needs a fully system approach where the
secure subsystem parts are made sufficiently independent from the non-
secure parts on a hardware level, which I don't see on the i.MX8M SoC
and hardware design guide.

> For NXP i.MX8M family, it is ok to implement in linux kernel, just a tradeoff to find out a place
> to hide the complexity ^_^.
> 
> BTW, for virtualization support, it is better to put the power domain in a central place to simplify
> the VM implementation.

Same as above. If you can make all the relevant bits (clock, reset,
power-domain, regulator) available via a virtualization friendly API,
then I would see a point in adding complexity for this abstraction. As
long as this added abstraction only solves a very tiny bit of the
overall picture, I just don't see the point in the added complexity and
(from a Linux PoV) obfuscation.

Regards,
Lucas

> BR
> Jacky Bai
> 
> > Regards,
> > Lucas
> > 
> > > BR
> > > Jacky Bai
> > > 
> > > > Regards,
> > > > Lucas
> > > > 
> > > > Lucas Stach (11):
> > > >   soc: imx: gpcv2: move to more ideomatic error handling in probe
> > > >   soc: imx: gpcv2: move domain mapping to domain driver probe
> > > >   soc: imx: gpcv2: split power up and power down sequence control
> > > >   soc: imx: gpcv2: wait for ADB400 handshake
> > > >   soc: imx: gpcv2: add runtime PM support for power-domains
> > > >   soc: imx: gpcv2: allow domains without power-sequence control
> > > >   soc: imx: gpcv2: add support for optional resets
> > > >   dt-bindings: add defines for i.MX8MM power domains
> > > >   soc: imx: gpcv2: add support for i.MX8MM power domains
> > > >   arm64: dts: imx8mm: add GPC node and power domains
> > > >   arm64: dts: imx8mm: put USB controllers into power-domains
> > > > 
> > > >  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
> > > >  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
> > > >  drivers/soc/imx/gpcv2.c                       | 501
> > > > +++++++++++++++---
> > > >  include/dt-bindings/power/imx8mm-power.h      |  22 +
> > > >  4 files changed, 516 insertions(+), 74 deletions(-)  create mode
> > > > 100644 include/dt-bindings/power/imx8mm-power.h
> > > > 
> > > > --
> > > > 2.20.1

Peng Fan Oct. 14, 2020, 1:23 a.m. UTC | #11

> Subject: Re: [PATCH 00/11] i.MX8MM power domain support
> 
> On Sa, 2020-10-10 at 02:16 +0000, Jacky Bai wrote:
> > > -----Original Message-----
> > > From: Lucas Stach [mailto:[e-mail]]
> > > Sent: Friday, October 9, 2020 7:12 PM
> > > To: Jacky Bai <[e-mail]>; Shawn Guo <[e-mail]>;
> > > Rob Herring <[e-mail]>
> > > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > > <[e-mail]>; Frieder Schrempf
> > > <[e-mail]>; Marek Vasut <[e-mail]>;
> > > [e-mail];
> > > [e-mail]; [e-mail];
> > > [e-mail]
> > > Subject: Re: [PATCH 00/11] i.MX8MM power domain support
> > >
> > > Hi Jacky,
> > >
> > > On Fr, 2020-10-09 at 03:00 +0000, Jacky Bai wrote:
> > > > > -----Original Message-----
> > > > > From: Lucas Stach [mailto:[e-mail]]
> > > > > Sent: Wednesday, September 30, 2020 11:50 PM
> > > > > To: Shawn Guo <[e-mail]>; Rob Herring
> > > > > <[e-mail]>
> > > > > Cc: dl-linux-imx <[e-mail]>; Fabio Estevam
> > > > > <[e-mail]>; Frieder Schrempf
> > > > > <[e-mail]>; Marek Vasut <[e-mail]>;
> > > > > [e-mail];
> > > > > [e-mail]; [e-mail];
> > > > > [e-mail]
> > > > > Subject: [PATCH 00/11] i.MX8MM power domain support
> > > > >
> > > > > Hi all,
> > > > >
> > > > > this adds power domain support for the i.MX8MM to the existing
> > > > > GPCv2
> > > driver.
> > > > > It is not complete yet, as it is still missing the VPU and
> > > > > display power domains, as those require support for the BLK_CTL
> > > > > regions of the VPUMIX and DISPLAYMIX domains. A Linux driver for
> > > > > those regions on the i.MX8MP is currently under development and
> > > > > we plan to use this as a template for the i.MX8MM when the dust
> > > > > has settled. The changes in this series have been made with this
> > > > > in mind, so once the BLK_CTL driver exists it should be a matter
> > > > > of hooking things together via DT, with no further changes
> > > > > required on the GPCv2 driver side
> > > (famous last words).
> > > > > Special thanks to Marek Vasut who helped with testing and
> > > > > debugging of early versions of this code.
> > > > >
> > > >
> > > > Lucas,
> > > >
> > > > thanks for working on this, but I think current support for 8MM
> > > > can NOT 100%
> > > work due to HW limitation.
> > > > Maybe, we need further discussion before moving forward,
> > > > otherwise, we will meet awkward situation when NXP doing LTS
> > > > upgrade. Below are some
> > > info shared.
> > > > 1. The GPU & VPU related power domains need to do special handling
> > > > due
> > > to HW limitation, can refer to the power domain sequence
> > > >   In NXP release.
> > >
> > > For the GPU this driver already does the same thing as the TF-A
> > > based implementation by driving the GPU2D and GPU3D domains
> together
> > > and triggering the SRC reset.
> > >
> > > For the VPU I expect that we can do all the necessary syncing with a
> > > proper VPU BLK_CTL driver.
> > >
> >
> > Ok, thanks. I saw the reset handling in this patchset.
> >
> > > > 2. another reason that we do power domain control in TF-A in NXP
> > > > release is
> > > that MAIN NOC power domain can only be controlled by
> > > >   TF-A, and before MAIN NOC power domain, we need to check other
> > > MIXs' power status. If other power domain is controlled by linux
> > > side,
> > > >   It is not easy to cross world status sync.
> > >
> > > This is a valid concern and I want to learn more about this. When do
> > > you turn off MAIN NOC power in the TF-A? Is it just system suspend?
> > > If so I think it's a valid requirement for the kernel driver to shut
> > > down all the peripheral power domains before entering system suspend.
> > >
> >
> > The main NOC will be off just in system suspend case. Main NoC on/off
> > is controlled by GPC HW slot method. As all the MIXs with ADB400
> > bridge are connected on the main NoC, we must make sure that all these
> > ADB400 port in power down status when main NoC power off in system
> > suspend, otherwise system will hang when resume. Previously, all the
> > MIX power domain is controlled by TF-A, then TF-A knows the status of each
> MIX, if any MIX is on, we will skip the NOC power down setting.
> 
> And that just means the SoC doesn't enter the desired low-power state when
> going into suspend, right? I think it would be much easier and less error prone
> to just mandate that all peripheral power domains must be powered down
> before trying to enter system suspend.
> 
> Is there any use-case where you would legitimately want to enter a high
> power suspend state with some of ther peripheral power domains still active?
> 
> > > > 3. either 8MM, 8MN, or 8MP, the power domain design is different,
> > > > I am not
> > > sure if it is the good to add hundreds line of code in GPCv2 each
> > > time
> > > >   a new SOC is added.
> > >
> > > I don't buy into this argument. We have lots of drivers in the Linux
> > > kernel that require some changes for new SoC generations, that's
> > > what Linux drivers are for. The complexity of the hardware doesn't
> > > disappear just because you push some of the driver bits into TF-A,
> > > you just handle the complexity at a different palce and IMHO that
> > > the wrong place. The power domains have complex interactions with
> > > other drivers in the Linux system, so debugging and deplyong fixes
> > > is much easier when the power domain handling is fully done by a kernel
> driver.
> >
> > Actually, due to the security requirement from other system solution
> > provider, for example, Microsoft Azure Sphere, it has strict
> > requirement for power domain to be controlled by secure subsystem(either
> TF-A, TEE or dedicated secure domain controller).
> > Same requirement for reset control, and system critical clock control.
> 
> Yes, I'm aware of those requirements, but to satisfy those you need a full
> implementation of all those parts in the secure subsystem. Doing it just for the
> power domains adds complexity for no gain, as you still won't be able to meet
> all the requirements and frankly I don't think this is a realistic goal to achieve
> with the current i.MX8M family of SoCs.

At least we are moving to that direction.

> 
> Meeting those requirements needs a fully system approach where the secure
> subsystem parts are made sufficiently independent from the non- secure
> parts on a hardware level, which I don't see on the i.MX8M SoC and hardware
> design guide.

CSU could restrict the access permission.

> 
> > For NXP i.MX8M family, it is ok to implement in linux kernel, just a
> > tradeoff to find out a place to hide the complexity ^_^.
> >
> > BTW, for virtualization support, it is better to put the power domain
> > in a central place to simplify the VM implementation.
> 
> Same as above. If you can make all the relevant bits (clock, reset,
> power-domain, regulator) available via a virtualization friendly API, then I
> would see a point in adding complexity for this abstraction. As long as this
> added abstraction only solves a very tiny bit of the overall picture, I just don't
> see the point in the added complexity and (from a Linux PoV) obfuscation.

Could we use SCMI for power domain, system critical clocks, smc watchdog
and etc?

Or we support two approaches, one is let Linux control everything, the other
is using SCMI.

Thoughts?

Thanks,
Peng.

> 
> Regards,
> Lucas
> 
> > BR
> > Jacky Bai
> >
> > > Regards,
> > > Lucas
> > >
> > > > BR
> > > > Jacky Bai
> > > >
> > > > > Regards,
> > > > > Lucas
> > > > >
> > > > > Lucas Stach (11):
> > > > >   soc: imx: gpcv2: move to more ideomatic error handling in probe
> > > > >   soc: imx: gpcv2: move domain mapping to domain driver probe
> > > > >   soc: imx: gpcv2: split power up and power down sequence control
> > > > >   soc: imx: gpcv2: wait for ADB400 handshake
> > > > >   soc: imx: gpcv2: add runtime PM support for power-domains
> > > > >   soc: imx: gpcv2: allow domains without power-sequence control
> > > > >   soc: imx: gpcv2: add support for optional resets
> > > > >   dt-bindings: add defines for i.MX8MM power domains
> > > > >   soc: imx: gpcv2: add support for i.MX8MM power domains
> > > > >   arm64: dts: imx8mm: add GPC node and power domains
> > > > >   arm64: dts: imx8mm: put USB controllers into power-domains
> > > > >
> > > > >  .../bindings/power/fsl,imx-gpcv2.yaml         |   8 +
> > > > >  arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  59 +++
> > > > >  drivers/soc/imx/gpcv2.c                       | 501
> > > > > +++++++++++++++---
> > > > >  include/dt-bindings/power/imx8mm-power.h      |  22 +
> > > > >  4 files changed, 516 insertions(+), 74 deletions(-)  create
> > > > > mode
> > > > > 100644 include/dt-bindings/power/imx8mm-power.h
> > > > >
> > > > > --
> > > > > 2.20.1

Lucas Stach Oct. 22, 2020, 8:24 a.m. UTC | #12

Hi Peng,

On Mi, 2020-10-14 at 01:23 +0000, Peng Fan wrote:
[...]
> > > > > 3. either 8MM, 8MN, or 8MP, the power domain design is different,
> > > > > I am not
> > > > sure if it is the good to add hundreds line of code in GPCv2 each
> > > > time
> > > > >   a new SOC is added.
> > > > 
> > > > I don't buy into this argument. We have lots of drivers in the Linux
> > > > kernel that require some changes for new SoC generations, that's
> > > > what Linux drivers are for. The complexity of the hardware doesn't
> > > > disappear just because you push some of the driver bits into TF-A,
> > > > you just handle the complexity at a different palce and IMHO that
> > > > the wrong place. The power domains have complex interactions with
> > > > other drivers in the Linux system, so debugging and deplyong fixes
> > > > is much easier when the power domain handling is fully done by a kernel
> > driver.
> > > Actually, due to the security requirement from other system solution
> > > provider, for example, Microsoft Azure Sphere, it has strict
> > > requirement for power domain to be controlled by secure subsystem(either
> > TF-A, TEE or dedicated secure domain controller).
> > > Same requirement for reset control, and system critical clock control.
> > 
> > Yes, I'm aware of those requirements, but to satisfy those you need a full
> > implementation of all those parts in the secure subsystem. Doing it just for the
> > power domains adds complexity for no gain, as you still won't be able to meet
> > all the requirements and frankly I don't think this is a realistic goal to achieve
> > with the current i.MX8M family of SoCs.
> 
> At least we are moving to that direction.

To me it seems like the current way (custom TF-A interface and
implementation) is one step in the right direction, but two steps
backwards in terms of complexity.

> > Meeting those requirements needs a fully system approach where the secure
> > subsystem parts are made sufficiently independent from the non- secure
> > parts on a hardware level, which I don't see on the i.MX8M SoC and hardware
> > design guide.
> 
> CSU could restrict the access permission.

While this is true, my argument is much broader and not only focused on
on-SoC peripherals. For example some of the power domains need
different voltages for specific performance states, which means you
need to communicate with a external PMIC or other voltage regulator,
which in turn means you need to set aside the necessary i2c bus and/or
GPIO banks required for this communication at system design time, so it
isn't shared between TF-A and the rich OS. I don't see this in any of
the i.MX8M designs.

> > > For NXP i.MX8M family, it is ok to implement in linux kernel, just a
> > > tradeoff to find out a place to hide the complexity ^_^.
> > > 
> > > BTW, for virtualization support, it is better to put the power domain
> > > in a central place to simplify the VM implementation.
> > 
> > Same as above. If you can make all the relevant bits (clock, reset,
> > power-domain, regulator) available via a virtualization friendly API, then I
> > would see a point in adding complexity for this abstraction. As long as this
> > added abstraction only solves a very tiny bit of the overall picture, I just don't
> > see the point in the added complexity and (from a Linux PoV) obfuscation.
> 
> Could we use SCMI for power domain, system critical clocks, smc watchdog
> and etc?

If you could demonstrate a working solution with all those pieces
hidden behind a standard SCMI interface, this would make for a much
more compelling story supporting the secure subsystem argument.

> Or we support two approaches, one is let Linux control everything, the other
> is using SCMI.
> 
> Thoughts?

I wouldn't be opposed to such a solution. If you can put all this
behind a standard SCMI interface, I guess we wouldn't need two
different SoC specific drivers for the same purpose, so we could easily
have a Linux full-control solution (i.e. this patchset) coexist with a
SCMI based implementation, possibly with just a slightly different base
SoC DT with all the power domains, clocks and other system level
control stuff behind SCMI.

What I'm strongly opposed to is having a custom TF-A interface and all
the added complexity for little to no gain in actual system security.

Regards,
Lucas

Fabio Estevam Oct. 22, 2020, 4:36 p.m. UTC | #13

On Thu, Oct 22, 2020 at 5:24 AM Lucas Stach <[e-mail]> wrote:

> What I'm strongly opposed to is having a custom TF-A interface and all
> the added complexity for little to no gain in actual system security.

I agree with Lucas.

I would also prefer this series to be applied so that i.MX8MM upstream
progress does not get stalled any longer.

Thanks

Peng Fan Oct. 28, 2020, 1:50 p.m. UTC | #14

Hi Lucas,

> Subject: Re: [PATCH 00/11] i.MX8MM power domain support
> 
> Hi Peng,
> 
> On Mi, 2020-10-14 at 01:23 +0000, Peng Fan wrote:
> [...]
> > > > > > 3. either 8MM, 8MN, or 8MP, the power domain design is
> > > > > > different, I am not
> > > > > sure if it is the good to add hundreds line of code in GPCv2
> > > > > each time
> > > > > >   a new SOC is added.
> > > > >
> > > > > I don't buy into this argument. We have lots of drivers in the
> > > > > Linux kernel that require some changes for new SoC generations,
> > > > > that's what Linux drivers are for. The complexity of the
> > > > > hardware doesn't disappear just because you push some of the
> > > > > driver bits into TF-A, you just handle the complexity at a
> > > > > different palce and IMHO that the wrong place. The power domains
> > > > > have complex interactions with other drivers in the Linux
> > > > > system, so debugging and deplyong fixes is much easier when the
> > > > > power domain handling is fully done by a kernel
> > > driver.
> > > > Actually, due to the security requirement from other system
> > > > solution provider, for example, Microsoft Azure Sphere, it has
> > > > strict requirement for power domain to be controlled by secure
> > > > subsystem(either
> > > TF-A, TEE or dedicated secure domain controller).
> > > > Same requirement for reset control, and system critical clock control.
> > >
> > > Yes, I'm aware of those requirements, but to satisfy those you need
> > > a full implementation of all those parts in the secure subsystem.
> > > Doing it just for the power domains adds complexity for no gain, as
> > > you still won't be able to meet all the requirements and frankly I
> > > don't think this is a realistic goal to achieve with the current i.MX8M
> family of SoCs.
> >
> > At least we are moving to that direction.
> 
> To me it seems like the current way (custom TF-A interface and
> implementation) is one step in the right direction, but two steps backwards in
> terms of complexity.
> 
> > > Meeting those requirements needs a fully system approach where the
> > > secure subsystem parts are made sufficiently independent from the
> > > non- secure parts on a hardware level, which I don't see on the
> > > i.MX8M SoC and hardware design guide.
> >
> > CSU could restrict the access permission.
> 
> While this is true, my argument is much broader and not only focused on
> on-SoC peripherals. For example some of the power domains need different
> voltages for specific performance states, which means you need to
> communicate with a external PMIC or other voltage regulator, which in turn
> means you need to set aside the necessary i2c bus and/or GPIO banks
> required for this communication at system design time, so it isn't shared
> between TF-A and the rich OS. I don't see this in any of the i.MX8M designs.
> 
> > > > For NXP i.MX8M family, it is ok to implement in linux kernel, just
> > > > a tradeoff to find out a place to hide the complexity ^_^.
> > > >
> > > > BTW, for virtualization support, it is better to put the power
> > > > domain in a central place to simplify the VM implementation.
> > >
> > > Same as above. If you can make all the relevant bits (clock, reset,
> > > power-domain, regulator) available via a virtualization friendly
> > > API, then I would see a point in adding complexity for this
> > > abstraction. As long as this added abstraction only solves a very
> > > tiny bit of the overall picture, I just don't see the point in the added
> complexity and (from a Linux PoV) obfuscation.
> >
> > Could we use SCMI for power domain, system critical clocks, smc
> > watchdog and etc?
> 
> If you could demonstrate a working solution with all those pieces hidden
> behind a standard SCMI interface, this would make for a much more
> compelling story supporting the secure subsystem argument.
> 
> > Or we support two approaches, one is let Linux control everything, the
> > other is using SCMI.
> >
> > Thoughts?
> 
> I wouldn't be opposed to such a solution. If you can put all this behind a
> standard SCMI interface, I guess we wouldn't need two different SoC specific
> drivers for the same purpose, so we could easily have a Linux full-control
> solution (i.e. this patchset) coexist with a SCMI based implementation,
> possibly with just a slightly different base SoC DT with all the power domains,
> clocks and other system level control stuff behind SCMI.
> 
> What I'm strongly opposed to is having a custom TF-A interface and all the
> added complexity for little to no gain in actual system security.

Understand. There are truly the SoC design might not fit well to protect
all the stuff.

It is good that you did this patchset. Vote for you to add more support
on i.MX.

Your patchset not conflict with SCMI, as you said, this is true.

Please continue your effort.

Thanks,
Peng.

> 
> Regards,
> Lucas

Adam Ford Oct. 31, 2020, 1:56 p.m. UTC | #15

On Wed, Oct 28, 2020 at 8:51 AM Peng Fan <[e-mail]> wrote:
>
> Hi Lucas,
>
> > Subject: Re: [PATCH 00/11] i.MX8MM power domain support
> >
> > Hi Peng,
> >
> > On Mi, 2020-10-14 at 01:23 +0000, Peng Fan wrote:
> > [...]
> > > > > > > 3. either 8MM, 8MN, or 8MP, the power domain design is
> > > > > > > different, I am not
> > > > > > sure if it is the good to add hundreds line of code in GPCv2
> > > > > > each time
> > > > > > >   a new SOC is added.
> > > > > >
> > > > > > I don't buy into this argument. We have lots of drivers in the
> > > > > > Linux kernel that require some changes for new SoC generations,
> > > > > > that's what Linux drivers are for. The complexity of the
> > > > > > hardware doesn't disappear just because you push some of the
> > > > > > driver bits into TF-A, you just handle the complexity at a
> > > > > > different palce and IMHO that the wrong place. The power domains
> > > > > > have complex interactions with other drivers in the Linux
> > > > > > system, so debugging and deplyong fixes is much easier when the
> > > > > > power domain handling is fully done by a kernel
> > > > driver.
> > > > > Actually, due to the security requirement from other system
> > > > > solution provider, for example, Microsoft Azure Sphere, it has
> > > > > strict requirement for power domain to be controlled by secure
> > > > > subsystem(either
> > > > TF-A, TEE or dedicated secure domain controller).
> > > > > Same requirement for reset control, and system critical clock control.
> > > >
> > > > Yes, I'm aware of those requirements, but to satisfy those you need
> > > > a full implementation of all those parts in the secure subsystem.
> > > > Doing it just for the power domains adds complexity for no gain, as
> > > > you still won't be able to meet all the requirements and frankly I
> > > > don't think this is a realistic goal to achieve with the current i.MX8M
> > family of SoCs.
> > >
> > > At least we are moving to that direction.
> >
> > To me it seems like the current way (custom TF-A interface and
> > implementation) is one step in the right direction, but two steps backwards in
> > terms of complexity.
> >
> > > > Meeting those requirements needs a fully system approach where the
> > > > secure subsystem parts are made sufficiently independent from the
> > > > non- secure parts on a hardware level, which I don't see on the
> > > > i.MX8M SoC and hardware design guide.
> > >
> > > CSU could restrict the access permission.
> >
> > While this is true, my argument is much broader and not only focused on
> > on-SoC peripherals. For example some of the power domains need different
> > voltages for specific performance states, which means you need to
> > communicate with a external PMIC or other voltage regulator, which in turn
> > means you need to set aside the necessary i2c bus and/or GPIO banks
> > required for this communication at system design time, so it isn't shared
> > between TF-A and the rich OS. I don't see this in any of the i.MX8M designs.
> >
> > > > > For NXP i.MX8M family, it is ok to implement in linux kernel, just
> > > > > a tradeoff to find out a place to hide the complexity ^_^.
> > > > >
> > > > > BTW, for virtualization support, it is better to put the power
> > > > > domain in a central place to simplify the VM implementation.
> > > >
> > > > Same as above. If you can make all the relevant bits (clock, reset,
> > > > power-domain, regulator) available via a virtualization friendly
> > > > API, then I would see a point in adding complexity for this
> > > > abstraction. As long as this added abstraction only solves a very
> > > > tiny bit of the overall picture, I just don't see the point in the added
> > complexity and (from a Linux PoV) obfuscation.
> > >
> > > Could we use SCMI for power domain, system critical clocks, smc
> > > watchdog and etc?
> >
> > If you could demonstrate a working solution with all those pieces hidden
> > behind a standard SCMI interface, this would make for a much more
> > compelling story supporting the secure subsystem argument.
> >
> > > Or we support two approaches, one is let Linux control everything, the
> > > other is using SCMI.
> > >
> > > Thoughts?
> >
> > I wouldn't be opposed to such a solution. If you can put all this behind a
> > standard SCMI interface, I guess we wouldn't need two different SoC specific
> > drivers for the same purpose, so we could easily have a Linux full-control
> > solution (i.e. this patchset) coexist with a SCMI based implementation,
> > possibly with just a slightly different base SoC DT with all the power domains,
> > clocks and other system level control stuff behind SCMI.
> >
> > What I'm strongly opposed to is having a custom TF-A interface and all the
> > added complexity for little to no gain in actual system security.
>
> Understand. There are truly the SoC design might not fit well to protect
> all the stuff.
>
> It is good that you did this patchset. Vote for you to add more support
> on i.MX.
>
> Your patchset not conflict with SCMI, as you said, this is true.
>
> Please continue your effort.

Despite ongoing efforts to get the displaymix (and probably vpumix)
operational, it would be nice to have operational USB which needs two
of the power domains.  Could we apply the initial framework and only
the DT nodes for the functional power domains (hsiomix and pgc_otg1) .

While it's not a perfect solution, I think incremental improvements
are better than going stale while we wait for the perfect solution.
Having them pulled into a staging branch also helps coordinate
efforts.

adam
>
> Thanks,
> Peng.
>
> >
> > Regards,
> > Lucas
>
> _______________________________________________
> linux-arm-kernel mailing list
> [e-mail]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel