diff mbox series

PCI: pciehp: Ignore spurious link inactive change when off

Message ID 20210409205935.41881-1-jonathan.derrick@intel.com (mailing list archive)
State Accepted
Delegated to: Bjorn Helgaas
Headers show
Series PCI: pciehp: Ignore spurious link inactive change when off | expand

Commit Message

Jon Derrick April 9, 2021, 8:59 p.m. UTC
When a specific x8 CEM card is bifurcated into x4x4 mode, and the
upstream ports both support hotplugging on each respective x4 device, a
slot management system for the CEM card requires both x4 devices to be
sysfs removed from the OS before it can safely turn-off physical power.
The implications are that Slot Control will display Powered Off status
for the device where the device is actually powered until both ports
have powered off.

When power is removed from the first half, the link remains active to
provide clocking while waiting for the second half to have power
removed. When power is then removed from the second half, the first half
starts shutdown sequence and will trigger a link status change event.
This is misinterpreted as an enabling event due to positive presence
detect and causes the first half to be re-enabled.

The spurious enable can be resolved by ignoring link status change
events when no link is active when in the off state.

Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>
---
 drivers/pci/hotplug/pciehp_ctrl.c | 5 +++++
 1 file changed, 5 insertions(+)

Comments

Raj, Ashok April 9, 2021, 9:38 p.m. UTC | #1
On Fri, Apr 09, 2021 at 02:59:35PM -0600, Derrick, Jonathan wrote:
> When a specific x8 CEM card is bifurcated into x4x4 mode, and the
> upstream ports both support hotplugging on each respective x4 device, a
> slot management system for the CEM card requires both x4 devices to be
> sysfs removed from the OS before it can safely turn-off physical power.
> The implications are that Slot Control will display Powered Off status
> for the device where the device is actually powered until both ports
> have powered off.
> 
> When power is removed from the first half, the link remains active to
> provide clocking while waiting for the second half to have power
> removed. When power is then removed from the second half, the first half
> starts shutdown sequence and will trigger a link status change event.
> This is misinterpreted as an enabling event due to positive presence
> detect and causes the first half to be re-enabled.
> 
> The spurious enable can be resolved by ignoring link status change
> events when no link is active when in the off state.
> 
> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>

Although this seems like it should never happen with normal cards, it seems
harmless otherwise. 


Reviewed by: Ashok Raj <ashok.raj@intel.com>
> ---
>  drivers/pci/hotplug/pciehp_ctrl.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp_ctrl.c
> index 529c34808440..a2c5eef03e7d 100644
> --- a/drivers/pci/hotplug/pciehp_ctrl.c
> +++ b/drivers/pci/hotplug/pciehp_ctrl.c
> @@ -265,6 +265,11 @@ void pciehp_handle_presence_or_link_change(struct controller *ctrl, u32 events)
>  		cancel_delayed_work(&ctrl->button_work);
>  		fallthrough;
>  	case OFF_STATE:
> +		if ((events & PCI_EXP_SLTSTA_DLLSC) && !link_active) {
> +			mutex_unlock(&ctrl->state_lock);
> +			break;
> +		}
> +
>  		ctrl->state = POWERON_STATE;
>  		mutex_unlock(&ctrl->state_lock);
>  		if (present)
> -- 
> 2.26.2
>
Bjorn Helgaas May 24, 2021, 10:42 p.m. UTC | #2
On Fri, Apr 09, 2021 at 02:59:35PM -0600, Jon Derrick wrote:
> When a specific x8 CEM card is bifurcated into x4x4 mode, and the
> upstream ports both support hotplugging on each respective x4 device, a
> slot management system for the CEM card requires both x4 devices to be
> sysfs removed from the OS before it can safely turn-off physical power.
> The implications are that Slot Control will display Powered Off status
> for the device where the device is actually powered until both ports
> have powered off.
> 
> When power is removed from the first half, the link remains active to
> provide clocking while waiting for the second half to have power
> removed. When power is then removed from the second half, the first half
> starts shutdown sequence and will trigger a link status change event.
> This is misinterpreted as an enabling event due to positive presence
> detect and causes the first half to be re-enabled.
> 
> The spurious enable can be resolved by ignoring link status change
> events when no link is active when in the off state.
> 
> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com>

Applied to pci/hotplug for v5.14, thanks!

> ---
>  drivers/pci/hotplug/pciehp_ctrl.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp_ctrl.c
> index 529c34808440..a2c5eef03e7d 100644
> --- a/drivers/pci/hotplug/pciehp_ctrl.c
> +++ b/drivers/pci/hotplug/pciehp_ctrl.c
> @@ -265,6 +265,11 @@ void pciehp_handle_presence_or_link_change(struct controller *ctrl, u32 events)
>  		cancel_delayed_work(&ctrl->button_work);
>  		fallthrough;
>  	case OFF_STATE:
> +		if ((events & PCI_EXP_SLTSTA_DLLSC) && !link_active) {
> +			mutex_unlock(&ctrl->state_lock);
> +			break;
> +		}
> +
>  		ctrl->state = POWERON_STATE;
>  		mutex_unlock(&ctrl->state_lock);
>  		if (present)
> -- 
> 2.26.2
>
Lukas Wunner May 25, 2021, 7:25 p.m. UTC | #3
On Mon, May 24, 2021 at 05:42:18PM -0500, Bjorn Helgaas wrote:
> On Fri, Apr 09, 2021 at 02:59:35PM -0600, Jon Derrick wrote:
> > When a specific x8 CEM card is bifurcated into x4x4 mode, and the
> > upstream ports both support hotplugging on each respective x4 device, a
> > slot management system for the CEM card requires both x4 devices to be
> > sysfs removed from the OS before it can safely turn-off physical power.
> > The implications are that Slot Control will display Powered Off status
> > for the device where the device is actually powered until both ports
> > have powered off.
> > 
> > When power is removed from the first half, the link remains active to
> > provide clocking while waiting for the second half to have power
> > removed. When power is then removed from the second half, the first half
> > starts shutdown sequence and will trigger a link status change event.
> > This is misinterpreted as an enabling event due to positive presence
> > detect and causes the first half to be re-enabled.
> > 
> > The spurious enable can be resolved by ignoring link status change
> > events when no link is active when in the off state.

Sorry for not responding earlier, I missed this patch.


> > --- a/drivers/pci/hotplug/pciehp_ctrl.c
> > +++ b/drivers/pci/hotplug/pciehp_ctrl.c
> > @@ -265,6 +265,11 @@ void pciehp_handle_presence_or_link_change(struct controller *ctrl, u32 events)
> >  		cancel_delayed_work(&ctrl->button_work);
> >  		fallthrough;
> >  	case OFF_STATE:
> > +		if ((events & PCI_EXP_SLTSTA_DLLSC) && !link_active) {
> > +			mutex_unlock(&ctrl->state_lock);
> > +			break;
> > +		}
> > +

I think this change will inadvertently ignore events that shouldn't be
ignored:  E.g., a DLLSC event may have been triggered by replacement of
the card in the slot and while Presence Detect State is 1, the link may
not yet be active.  The change above will cause not only the DLLSC but
also the PDC event to be ignored.

There are also reports of link flaps on card insertion and the above
change may result in the slot not being brought up even though it should.

The commit message sounds like powering down the CEM card takes longer
than expected.  We wait 1 second in set_slot_off() after disabling
slot power and that's apparently not sufficient.  The 1 second delay
is mandated by section 6.7.1.8 of the PCIe Base Spec.  If this card
needs a longer delay, a quirk should be added rather than changing
the algorithm for everyone.

Thanks,

Lukas
Bjorn Helgaas May 25, 2021, 8:17 p.m. UTC | #4
On Tue, May 25, 2021 at 09:25:12PM +0200, Lukas Wunner wrote:
> On Mon, May 24, 2021 at 05:42:18PM -0500, Bjorn Helgaas wrote:
> > On Fri, Apr 09, 2021 at 02:59:35PM -0600, Jon Derrick wrote:
> > > When a specific x8 CEM card is bifurcated into x4x4 mode, and the
> > > upstream ports both support hotplugging on each respective x4 device, a
> > > slot management system for the CEM card requires both x4 devices to be
> > > sysfs removed from the OS before it can safely turn-off physical power.
> > > The implications are that Slot Control will display Powered Off status
> > > for the device where the device is actually powered until both ports
> > > have powered off.
> > > 
> > > When power is removed from the first half, the link remains active to
> > > provide clocking while waiting for the second half to have power
> > > removed. When power is then removed from the second half, the first half
> > > starts shutdown sequence and will trigger a link status change event.
> > > This is misinterpreted as an enabling event due to positive presence
> > > detect and causes the first half to be re-enabled.
> > > 
> > > The spurious enable can be resolved by ignoring link status change
> > > events when no link is active when in the off state.
> 
> Sorry for not responding earlier, I missed this patch.
> 
> 
> > > --- a/drivers/pci/hotplug/pciehp_ctrl.c
> > > +++ b/drivers/pci/hotplug/pciehp_ctrl.c
> > > @@ -265,6 +265,11 @@ void pciehp_handle_presence_or_link_change(struct controller *ctrl, u32 events)
> > >  		cancel_delayed_work(&ctrl->button_work);
> > >  		fallthrough;
> > >  	case OFF_STATE:
> > > +		if ((events & PCI_EXP_SLTSTA_DLLSC) && !link_active) {
> > > +			mutex_unlock(&ctrl->state_lock);
> > > +			break;
> > > +		}
> > > +
> 
> I think this change will inadvertently ignore events that shouldn't be
> ignored:  E.g., a DLLSC event may have been triggered by replacement of
> the card in the slot and while Presence Detect State is 1, the link may
> not yet be active.  The change above will cause not only the DLLSC but
> also the PDC event to be ignored.
> 
> There are also reports of link flaps on card insertion and the above
> change may result in the slot not being brought up even though it should.
> 
> The commit message sounds like powering down the CEM card takes longer
> than expected.  We wait 1 second in set_slot_off() after disabling
> slot power and that's apparently not sufficient.  The 1 second delay
> is mandated by section 6.7.1.8 of the PCIe Base Spec.  If this card
> needs a longer delay, a quirk should be added rather than changing
> the algorithm for everyone.

I dropped this patch for now, thanks for taking a look, Lukas.

Bjorn
diff mbox series

Patch

diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp_ctrl.c
index 529c34808440..a2c5eef03e7d 100644
--- a/drivers/pci/hotplug/pciehp_ctrl.c
+++ b/drivers/pci/hotplug/pciehp_ctrl.c
@@ -265,6 +265,11 @@  void pciehp_handle_presence_or_link_change(struct controller *ctrl, u32 events)
 		cancel_delayed_work(&ctrl->button_work);
 		fallthrough;
 	case OFF_STATE:
+		if ((events & PCI_EXP_SLTSTA_DLLSC) && !link_active) {
+			mutex_unlock(&ctrl->state_lock);
+			break;
+		}
+
 		ctrl->state = POWERON_STATE;
 		mutex_unlock(&ctrl->state_lock);
 		if (present)