diff mbox series

[1/3] PCI: Add a flag to notify PCI drivers about powerdown during suspend

Message ID 20220513110027.31015-2-manivannan.sadhasivam@linaro.org (mailing list archive)
State Superseded
Headers show
Series PCI: Notify PCI drivers about powerdown during suspend | expand

Commit Message

Manivannan Sadhasivam May 13, 2022, 11 a.m. UTC
On some systems like Chromebooks based on Qcom chipsets, the OS may
powerdown all PCIe devices during system suspend for aggressive
powersaving. In that case, the PCI host controller drivers need to notify
the PCI device drivers that the power will be taken off during system
suspend so that the drivers can prepare the devices accordingly.

One prime example is the PCI NVMe driver. This flag can be used by the
driver to shutdown the NVMe device during suspend and recover it during
resume.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
 include/linux/pci.h | 1 +
 1 file changed, 1 insertion(+)

Comments

Bjorn Helgaas May 16, 2022, 8:18 p.m. UTC | #1
On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote:
> On some systems like Chromebooks based on Qcom chipsets, the OS may
> powerdown all PCIe devices during system suspend for aggressive
> powersaving. In that case, the PCI host controller drivers need to notify
> the PCI device drivers that the power will be taken off during system
> suspend so that the drivers can prepare the devices accordingly.

"The OS may powerdown all PCIe devices ..." makes it sound like this
is an OS policy decision.  Where exactly (what function) is that?

Or if it's not an OS policy decision, but rather some property of the
hardware, say that specifically.

> One prime example is the PCI NVMe driver. This flag can be used by the
> driver to shutdown the NVMe device during suspend and recover it during
> resume.
> 
> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> ---
>  include/linux/pci.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 60adf42460ab..069caf1fe88d 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -578,6 +578,7 @@ struct pci_host_bridge {
>  	unsigned int	preserve_config:1;	/* Preserve FW resource setup */
>  	unsigned int	size_windows:1;		/* Enable root bus sizing */
>  	unsigned int	msi_domain:1;		/* Bridge wants MSI domain */
> +	unsigned int	suspend_poweroff:1;	/* OS may poweroff devices during system suspend */
>  
>  	/* Resource alignment requirements */
>  	resource_size_t (*align_resource)(struct pci_dev *dev,
> -- 
> 2.25.1
>
Manivannan Sadhasivam May 17, 2022, 3:09 p.m. UTC | #2
On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote:
> On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote:
> > On some systems like Chromebooks based on Qcom chipsets, the OS may
> > powerdown all PCIe devices during system suspend for aggressive
> > powersaving. In that case, the PCI host controller drivers need to notify
> > the PCI device drivers that the power will be taken off during system
> > suspend so that the drivers can prepare the devices accordingly.
> 
> "The OS may powerdown all PCIe devices ..." makes it sound like this
> is an OS policy decision.  Where exactly (what function) is that?
> 
> Or if it's not an OS policy decision, but rather some property of the
> hardware, say that specifically.
> 

On SC7280, it is the Resource Power Manager(RPMh) that's powering the devices
down by cutting off the PCIe voltage domain. But the SC7280 RC driver itself
may put the PCIe devices into D3cold state during system suspend.
https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/

So to cover both cases (one is a hardware independent of SoC and another one is
the device driver), and to be generic, I've used the term "OS" after looking at
the previous flags.

Thanks,
Mani

> > One prime example is the PCI NVMe driver. This flag can be used by the
> > driver to shutdown the NVMe device during suspend and recover it during
> > resume.
> > 
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > ---
> >  include/linux/pci.h | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > index 60adf42460ab..069caf1fe88d 100644
> > --- a/include/linux/pci.h
> > +++ b/include/linux/pci.h
> > @@ -578,6 +578,7 @@ struct pci_host_bridge {
> >  	unsigned int	preserve_config:1;	/* Preserve FW resource setup */
> >  	unsigned int	size_windows:1;		/* Enable root bus sizing */
> >  	unsigned int	msi_domain:1;		/* Bridge wants MSI domain */
> > +	unsigned int	suspend_poweroff:1;	/* OS may poweroff devices during system suspend */
> >  
> >  	/* Resource alignment requirements */
> >  	resource_size_t (*align_resource)(struct pci_dev *dev,
> > -- 
> > 2.25.1
> >
Bjorn Helgaas May 17, 2022, 5:24 p.m. UTC | #3
On Tue, May 17, 2022 at 08:39:08PM +0530, Manivannan Sadhasivam wrote:
> On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote:
> > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote:
> > > On some systems like Chromebooks based on Qcom chipsets, the OS may
> > > powerdown all PCIe devices during system suspend for aggressive
> > > powersaving. In that case, the PCI host controller drivers need to notify
> > > the PCI device drivers that the power will be taken off during system
> > > suspend so that the drivers can prepare the devices accordingly.
> > 
> > "The OS may powerdown all PCIe devices ..." makes it sound like this
> > is an OS policy decision.  Where exactly (what function) is that?
> > 
> > Or if it's not an OS policy decision, but rather some property of the
> > hardware, say that specifically.
> 
> On SC7280, it is the Resource Power Manager(RPMh) that's powering
> the devices down by cutting off the PCIe voltage domain. But the
> SC7280 RC driver itself may put the PCIe devices into D3cold state
> during system suspend.
> https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/
> 
> So to cover both cases (one is a hardware independent of SoC and
> another one is the device driver), and to be generic, I've used the
> term "OS" after looking at the previous flags.

This sort of device-specific behavior definitely needs a pointer to an
example.  Otherwise it seems like it could be generic PCIe behavior
that should be documented in the PCIe base spec.

> > > One prime example is the PCI NVMe driver. This flag can be used by the
> > > driver to shutdown the NVMe device during suspend and recover it during
> > > resume.

Apparently nvme is broken, or at least sub-optimal, without this flag.
What other drivers will be similarly affected?

> > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > > ---
> > >  include/linux/pci.h | 1 +
> > >  1 file changed, 1 insertion(+)
> > > 
> > > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > > index 60adf42460ab..069caf1fe88d 100644
> > > --- a/include/linux/pci.h
> > > +++ b/include/linux/pci.h
> > > @@ -578,6 +578,7 @@ struct pci_host_bridge {
> > >  	unsigned int	preserve_config:1;	/* Preserve FW resource setup */
> > >  	unsigned int	size_windows:1;		/* Enable root bus sizing */
> > >  	unsigned int	msi_domain:1;		/* Bridge wants MSI domain */
> > > +	unsigned int	suspend_poweroff:1;	/* OS may poweroff devices during system suspend */
> > >  
> > >  	/* Resource alignment requirements */
> > >  	resource_size_t (*align_resource)(struct pci_dev *dev,
Manivannan Sadhasivam May 18, 2022, 3:59 a.m. UTC | #4
On Tue, May 17, 2022 at 12:24:23PM -0500, Bjorn Helgaas wrote:
> On Tue, May 17, 2022 at 08:39:08PM +0530, Manivannan Sadhasivam wrote:
> > On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote:
> > > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote:
> > > > On some systems like Chromebooks based on Qcom chipsets, the OS may
> > > > powerdown all PCIe devices during system suspend for aggressive
> > > > powersaving. In that case, the PCI host controller drivers need to notify
> > > > the PCI device drivers that the power will be taken off during system
> > > > suspend so that the drivers can prepare the devices accordingly.
> > > 
> > > "The OS may powerdown all PCIe devices ..." makes it sound like this
> > > is an OS policy decision.  Where exactly (what function) is that?
> > > 
> > > Or if it's not an OS policy decision, but rather some property of the
> > > hardware, say that specifically.
> > 
> > On SC7280, it is the Resource Power Manager(RPMh) that's powering
> > the devices down by cutting off the PCIe voltage domain. But the
> > SC7280 RC driver itself may put the PCIe devices into D3cold state
> > during system suspend.
> > https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/
> > 
> > So to cover both cases (one is a hardware independent of SoC and
> > another one is the device driver), and to be generic, I've used the
> > term "OS" after looking at the previous flags.
> 
> This sort of device-specific behavior definitely needs a pointer to an
> example.  Otherwise it seems like it could be generic PCIe behavior
> that should be documented in the PCIe base spec.
> 

Okay.

> > > > One prime example is the PCI NVMe driver. This flag can be used by the
> > > > driver to shutdown the NVMe device during suspend and recover it during
> > > > resume.
> 
> Apparently nvme is broken, or at least sub-optimal, without this flag.

Yes, broken on SC7280 or any other SoCs that turn off power.

> What other drivers will be similarly affected?
> 

I don't have a list but the drivers that don't expect the device to be turned
off or reset during suspend may experience this issue. Right now, we have only
identified the issue with NVMe because that's what used on Chromebooks.

But in the coming days, we may need to fix some of the drivers also.

Thanks,
Mani

> > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > > > ---
> > > >  include/linux/pci.h | 1 +
> > > >  1 file changed, 1 insertion(+)
> > > > 
> > > > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > > > index 60adf42460ab..069caf1fe88d 100644
> > > > --- a/include/linux/pci.h
> > > > +++ b/include/linux/pci.h
> > > > @@ -578,6 +578,7 @@ struct pci_host_bridge {
> > > >  	unsigned int	preserve_config:1;	/* Preserve FW resource setup */
> > > >  	unsigned int	size_windows:1;		/* Enable root bus sizing */
> > > >  	unsigned int	msi_domain:1;		/* Bridge wants MSI domain */
> > > > +	unsigned int	suspend_poweroff:1;	/* OS may poweroff devices during system suspend */
> > > >  
> > > >  	/* Resource alignment requirements */
> > > >  	resource_size_t (*align_resource)(struct pci_dev *dev,
Rob Herring May 26, 2022, 8:48 p.m. UTC | #5
On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote:
> On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote:
> > On some systems like Chromebooks based on Qcom chipsets, the OS may
> > powerdown all PCIe devices during system suspend for aggressive
> > powersaving. In that case, the PCI host controller drivers need to notify
> > the PCI device drivers that the power will be taken off during system
> > suspend so that the drivers can prepare the devices accordingly.
> 
> "The OS may powerdown all PCIe devices ..." makes it sound like this
> is an OS policy decision.  Where exactly (what function) is that?
> 
> Or if it's not an OS policy decision, but rather some property of the
> hardware, say that specifically.
> 
> > One prime example is the PCI NVMe driver. This flag can be used by the
> > driver to shutdown the NVMe device during suspend and recover it during
> > resume.
> > 
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > ---
> >  include/linux/pci.h | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/include/linux/pci.h b/include/linux/pci.h
> > index 60adf42460ab..069caf1fe88d 100644
> > --- a/include/linux/pci.h
> > +++ b/include/linux/pci.h
> > @@ -578,6 +578,7 @@ struct pci_host_bridge {
> >  	unsigned int	preserve_config:1;	/* Preserve FW resource setup */
> >  	unsigned int	size_windows:1;		/* Enable root bus sizing */
> >  	unsigned int	msi_domain:1;		/* Bridge wants MSI domain */
> > +	unsigned int	suspend_poweroff:1;	/* OS may poweroff devices during system suspend */

Why does this apply to the whole host bridge? What if you have multiple 
devices and some are powered off and others aren't?

Rob
diff mbox series

Patch

diff --git a/include/linux/pci.h b/include/linux/pci.h
index 60adf42460ab..069caf1fe88d 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -578,6 +578,7 @@  struct pci_host_bridge {
 	unsigned int	preserve_config:1;	/* Preserve FW resource setup */
 	unsigned int	size_windows:1;		/* Enable root bus sizing */
 	unsigned int	msi_domain:1;		/* Bridge wants MSI domain */
+	unsigned int	suspend_poweroff:1;	/* OS may poweroff devices during system suspend */
 
 	/* Resource alignment requirements */
 	resource_size_t (*align_resource)(struct pci_dev *dev,