| Message ID | 20230607015646.558534-1-aaron.ma@canonical.com (mailing list archive) |
|---|---|
| State | Awaiting Upstream |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net,v2] igb: fix hang issue of AER error during resume | expand |
> -----Original Message----- > From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Aaron Ma > Sent: Tuesday, June 6, 2023 6:57 PM > To: Brandeburg, Jesse <jesse.brandeburg@intel.com>; Nguyen, Anthony L <anthony.l.nguyen@intel.com>; David S. Miller <davem@davemloft.net>; Eric Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>; Jeff Garzik <jgarzik@redhat.com>; Auke Kok <auke-jan.h.kok@intel.com>; intel-wired-lan@lists.osuosl.org; netdev@vger.kernel.org; linux-kernel@vger.kernel.org > Subject: [Intel-wired-lan] [PATCH net v2] igb: fix hang issue of AER error during resume > > PCIe AER error_detected caused a race issue with igb_resume. > Protect error_detected when igb is in down state. > > Error logs: > kernel: igb 0000:02:00.0: disabling already-disabled device > kernel: WARNING: CPU: 0 PID: 277 at drivers/pci/pci.c:2248 pci_disable_device+0xc4/0xf0 > kernel: RIP: 0010:pci_disable_device+0xc4/0xf0 > kernel: Call Trace: > kernel: <TASK> > kernel: igb_io_error_detected+0x3e/0x60 > kernel: report_error_detected+0xd6/0x1c0 > kernel: ? __pfx_report_normal_detected+0x10/0x10 > kernel: report_normal_detected+0x16/0x30 > kernel: pci_walk_bus+0x74/0xa0 > kernel: pcie_do_recovery+0xb9/0x340 > kernel: ? __pfx_aer_root_reset+0x10/0x10 > kernel: aer_process_err_devices+0x168/0x220 > kernel: aer_isr+0x1b5/0x1e0 > kernel: ? __pfx_irq_thread_fn+0x10/0x10 > kernel: irq_thread_fn+0x21/0x70 > kernel: irq_thread+0xf8/0x1c0 > kernel: ? __pfx_irq_thread_dtor+0x10/0x10 > kernel: ? __pfx_irq_thread+0x10/0x10 > kernel: kthread+0xef/0x120 > kernel: ? __pfx_kthread+0x10/0x10 > kernel: ret_from_fork+0x29/0x50 > kernel: </TASK> > kernel: ---[ end trace 0000000000000000 ]--- > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=217446 > Fixes: 9d5c824399de ("igb: PCI-Express 82575 Gigabit Ethernet driver") > Signed-off-by: Aaron Ma <aaron.ma@canonical.com> > Reviewed-by: Mateusz Palczewski <mateusz.palczewski@intel.com> > --- > V1->V2: Add target tree tag net and Fixes tag. > > drivers/net/ethernet/intel/igb/igb_main.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > Tested-by: Sunitha Mekala <sunithax.d.mekala@intel.com> (A Contingent worker at Intel)
Hi Aaron, On Wed, Jun 7, 2023 at 9:56 AM Aaron Ma <aaron.ma@canonical.com> wrote: > > PCIe AER error_detected caused a race issue with igb_resume. > Protect error_detected when igb is in down state. > > Error logs: > kernel: igb 0000:02:00.0: disabling already-disabled device > kernel: WARNING: CPU: 0 PID: 277 at drivers/pci/pci.c:2248 pci_disable_device+0xc4/0xf0 > kernel: RIP: 0010:pci_disable_device+0xc4/0xf0 > kernel: Call Trace: > kernel: <TASK> > kernel: igb_io_error_detected+0x3e/0x60 > kernel: report_error_detected+0xd6/0x1c0 > kernel: ? __pfx_report_normal_detected+0x10/0x10 > kernel: report_normal_detected+0x16/0x30 > kernel: pci_walk_bus+0x74/0xa0 > kernel: pcie_do_recovery+0xb9/0x340 > kernel: ? __pfx_aer_root_reset+0x10/0x10 > kernel: aer_process_err_devices+0x168/0x220 > kernel: aer_isr+0x1b5/0x1e0 > kernel: ? __pfx_irq_thread_fn+0x10/0x10 > kernel: irq_thread_fn+0x21/0x70 > kernel: irq_thread+0xf8/0x1c0 > kernel: ? __pfx_irq_thread_dtor+0x10/0x10 > kernel: ? __pfx_irq_thread+0x10/0x10 > kernel: kthread+0xef/0x120 > kernel: ? __pfx_kthread+0x10/0x10 > kernel: ret_from_fork+0x29/0x50 > kernel: </TASK> > kernel: ---[ end trace 0000000000000000 ]--- > > Link: https://bugzilla.kernel.org/show_bug.cgi?id=217446 > Fixes: 9d5c824399de ("igb: PCI-Express 82575 Gigabit Ethernet driver") > Signed-off-by: Aaron Ma <aaron.ma@canonical.com> > Reviewed-by: Mateusz Palczewski <mateusz.palczewski@intel.com> Can you please give [1] a try? I think avoid using rtnl_lock() is more ideal. [1] https://lore.kernel.org/intel-wired-lan/20230714050541.2765246-1-kai.heng.feng@canonical.com/ Kai-Heng > --- > V1->V2: Add target tree tag net and Fixes tag. > > drivers/net/ethernet/intel/igb/igb_main.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c > index 58872a4c2540..8333d4ac8169 100644 > --- a/drivers/net/ethernet/intel/igb/igb_main.c > +++ b/drivers/net/ethernet/intel/igb/igb_main.c > @@ -9581,14 +9581,21 @@ static pci_ers_result_t igb_io_error_detected(struct pci_dev *pdev, > struct net_device *netdev = pci_get_drvdata(pdev); > struct igb_adapter *adapter = netdev_priv(netdev); > > + if (test_bit(__IGB_DOWN, &adapter->state)) > + return PCI_ERS_RESULT_DISCONNECT; > + > + rtnl_lock(); > netif_device_detach(netdev); > > - if (state == pci_channel_io_perm_failure) > + if (state == pci_channel_io_perm_failure) { > + rtnl_unlock(); > return PCI_ERS_RESULT_DISCONNECT; > + } > > if (netif_running(netdev)) > igb_down(adapter); > pci_disable_device(pdev); > + rtnl_unlock(); > > /* Request a slot reset. */ > return PCI_ERS_RESULT_NEED_RESET; > -- > 2.34.1 > > _______________________________________________ > Intel-wired-lan mailing list > Intel-wired-lan@osuosl.org > https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 58872a4c2540..8333d4ac8169 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -9581,14 +9581,21 @@ static pci_ers_result_t igb_io_error_detected(struct pci_dev *pdev, struct net_device *netdev = pci_get_drvdata(pdev); struct igb_adapter *adapter = netdev_priv(netdev); + if (test_bit(__IGB_DOWN, &adapter->state)) + return PCI_ERS_RESULT_DISCONNECT; + + rtnl_lock(); netif_device_detach(netdev); - if (state == pci_channel_io_perm_failure) + if (state == pci_channel_io_perm_failure) { + rtnl_unlock(); return PCI_ERS_RESULT_DISCONNECT; + } if (netif_running(netdev)) igb_down(adapter); pci_disable_device(pdev); + rtnl_unlock(); /* Request a slot reset. */ return PCI_ERS_RESULT_NEED_RESET;
PCIe AER error_detected caused a race issue with igb_resume. Protect error_detected when igb is in down state. Error logs: kernel: igb 0000:02:00.0: disabling already-disabled device kernel: WARNING: CPU: 0 PID: 277 at drivers/pci/pci.c:2248 pci_disable_device+0xc4/0xf0 kernel: RIP: 0010:pci_disable_device+0xc4/0xf0 kernel: Call Trace: kernel: <TASK> kernel: igb_io_error_detected+0x3e/0x60 kernel: report_error_detected+0xd6/0x1c0 kernel: ? __pfx_report_normal_detected+0x10/0x10 kernel: report_normal_detected+0x16/0x30 kernel: pci_walk_bus+0x74/0xa0 kernel: pcie_do_recovery+0xb9/0x340 kernel: ? __pfx_aer_root_reset+0x10/0x10 kernel: aer_process_err_devices+0x168/0x220 kernel: aer_isr+0x1b5/0x1e0 kernel: ? __pfx_irq_thread_fn+0x10/0x10 kernel: irq_thread_fn+0x21/0x70 kernel: irq_thread+0xf8/0x1c0 kernel: ? __pfx_irq_thread_dtor+0x10/0x10 kernel: ? __pfx_irq_thread+0x10/0x10 kernel: kthread+0xef/0x120 kernel: ? __pfx_kthread+0x10/0x10 kernel: ret_from_fork+0x29/0x50 kernel: </TASK> kernel: ---[ end trace 0000000000000000 ]--- Link: https://bugzilla.kernel.org/show_bug.cgi?id=217446 Fixes: 9d5c824399de ("igb: PCI-Express 82575 Gigabit Ethernet driver") Signed-off-by: Aaron Ma <aaron.ma@canonical.com> Reviewed-by: Mateusz Palczewski <mateusz.palczewski@intel.com> --- V1->V2: Add target tree tag net and Fixes tag. drivers/net/ethernet/intel/igb/igb_main.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)