| Message ID | 20190502204648.5537-1-daniel.vetter@ffwll.ch (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | RFC: hung_task: taint kernel | expand |
On 2019/05/03 5:46, Daniel Vetter wrote: > There's the hung_task_panic sysctl, but that's a bit an extreme measure. > As a fallback taint at least the machine. > > Our CI uses this to decide when a reboot is necessary, plus to figure > out whether the kernel is still happy. Why your CI can't watch for "blocked for more than" message instead of setting the taint flag? How does your CI decide a reboot is necessary? There is no need to set the tainted flag when some task was just blocked for a while. It might be due to memory pressure, it might be due to setting very short timeout (e.g. a few seconds), it might be due to busy CPUs doing something else...
On Fri, May 03, 2019 at 09:47:03AM +0900, Tetsuo Handa wrote: > On 2019/05/03 5:46, Daniel Vetter wrote: > > There's the hung_task_panic sysctl, but that's a bit an extreme measure. > > As a fallback taint at least the machine. > > > > Our CI uses this to decide when a reboot is necessary, plus to figure > > out whether the kernel is still happy. > > Why your CI can't watch for "blocked for more than" message instead of > setting the taint flag? How does your CI decide a reboot is necessary? We spam an awful lot into dmesg, and at least historically had occasionally trouble capturing it all (we're better than that now I think). Plus the thing that parses dmesg isn't the thing that runs testcases, hence why we started to use taint flags (or procfs lockdep status) and similar things to check the kernel is still alive enough. > There is no need to set the tainted flag when some task was just blocked > for a while. It might be due to memory pressure, it might be due to setting > very short timeout (e.g. a few seconds), it might be due to busy CPUs doing > something else... Yeah I realize that this probably doesn't have much use outside of our CI, but maybe there's someone how likes the idea. Wrt spurious taints: You can disable the hung_tasks checker outright, which also stops the tainting. -Daniel
diff --git a/kernel/hung_task.c b/kernel/hung_task.c index f108a95882c6..d90d98f53ccb 100644 --- a/kernel/hung_task.c +++ b/kernel/hung_task.c @@ -117,6 +117,8 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout) console_verbose(); hung_task_show_lock = true; hung_task_call_panic = true; + } else { + add_taint(TAINT_WARN, LOCKDEP_STILL_OK); } /*
There's the hung_task_panic sysctl, but that's a bit an extreme measure. As a fallback taint at least the machine. Our CI uses this to decide when a reboot is necessary, plus to figure out whether the kernel is still happy. v2: Works much better when I put the else { add_taint() } at the right place. Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: "Paul E. McKenney" <paulmck@linux.ibm.com> Cc: Valdis Kletnieks <valdis.kletnieks@vt.edu> Cc: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Vitaly Kuznetsov <vkuznets@redhat.com> Cc: "Liu, Chuansheng" <chuansheng.liu@intel.com> --- kernel/hung_task.c | 2 ++ 1 file changed, 2 insertions(+)