| Message ID | 20230421082606.551411-1-yajun.deng@linux.dev (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 2f0f9465ad9fa9c93f30009184c10da0f504f313 |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | net: sched: Print msecs when transmit queue time out | expand |
Hello: This patch was applied to netdev/net-next.git (main) by David S. Miller <davem@davemloft.net>: On Fri, 21 Apr 2023 16:26:06 +0800 you wrote: > The kernel will print several warnings in a short period of time > when it stalls. Like this: > > First warning: > [ 7100.097547] ------------[ cut here ]------------ > [ 7100.097550] NETDEV WATCHDOG: eno2 (xxx): transmit queue 8 timed out > [ 7100.097571] WARNING: CPU: 8 PID: 0 at net/sched/sch_generic.c:467 > dev_watchdog+0x260/0x270 > ... > > [...] Here is the summary with links: - net: sched: Print msecs when transmit queue time out https://git.kernel.org/netdev/net-next/c/2f0f9465ad9f You are awesome, thank you!
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index a9aadc4e6858..37e41f972f69 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -502,7 +502,7 @@ static void dev_watchdog(struct timer_list *t) if (netif_device_present(dev) && netif_running(dev) && netif_carrier_ok(dev)) { - int some_queue_timedout = 0; + unsigned int timedout_ms = 0; unsigned int i; unsigned long trans_start; @@ -514,16 +514,16 @@ static void dev_watchdog(struct timer_list *t) if (netif_xmit_stopped(txq) && time_after(jiffies, (trans_start + dev->watchdog_timeo))) { - some_queue_timedout = 1; + timedout_ms = jiffies_to_msecs(jiffies - trans_start); atomic_long_inc(&txq->trans_timeout); break; } } - if (unlikely(some_queue_timedout)) { + if (unlikely(timedout_ms)) { trace_net_dev_xmit_timeout(dev, i); - WARN_ONCE(1, KERN_INFO "NETDEV WATCHDOG: %s (%s): transmit queue %u timed out\n", - dev->name, netdev_drivername(dev), i); + WARN_ONCE(1, "NETDEV WATCHDOG: %s (%s): transmit queue %u timed out %u ms\n", + dev->name, netdev_drivername(dev), i, timedout_ms); netif_freeze_queues(dev); dev->netdev_ops->ndo_tx_timeout(dev, i); netif_unfreeze_queues(dev);
The kernel will print several warnings in a short period of time when it stalls. Like this: First warning: [ 7100.097547] ------------[ cut here ]------------ [ 7100.097550] NETDEV WATCHDOG: eno2 (xxx): transmit queue 8 timed out [ 7100.097571] WARNING: CPU: 8 PID: 0 at net/sched/sch_generic.c:467 dev_watchdog+0x260/0x270 ... Second warning: [ 7147.756952] rcu: INFO: rcu_preempt self-detected stall on CPU [ 7147.756958] rcu: 24-....: (59999 ticks this GP) idle=546/1/0x400000000000000 softirq=367 3137/3673146 fqs=13844 [ 7147.756960] (t=60001 jiffies g=4322709 q=133381) [ 7147.756962] NMI backtrace for cpu 24 ... We calculate that the transmit queue start stall should occur before 7095s according to watchdog_timeo, the rcu start stall at 7087s. These two times are close together, it is difficult to confirm which happened first. To let users know the exact time the stall started, print msecs when the transmit queue time out. Signed-off-by: Yajun Deng <yajun.deng@linux.dev> --- net/sched/sch_generic.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)