| Message ID | 20220922195136.54575-1-tony.luck@intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Dump stack after certain machine checks | expand |
On Thu, Sep 22, 2022 at 12:51:34PM -0700, Tony Luck wrote: > I've only updated the Intel severity calculation to use this new > severity level. I'm not sure if AMD also has situations where this would > be useful. If so, then mce_severity_amd() would need to be updated too > to return different severity for IN_KERNEL and IN_KERNEL_RECOV cases. I'd look into Yazen's direction for that...
On Mon, Oct 31, 2022 at 11:30:03AM +0100, Borislav Petkov wrote: > On Thu, Sep 22, 2022 at 12:51:34PM -0700, Tony Luck wrote: > > I've only updated the Intel severity calculation to use this new > > severity level. I'm not sure if AMD also has situations where this would > > be useful. If so, then mce_severity_amd() would need to be updated too > > to return different severity for IN_KERNEL and IN_KERNEL_RECOV cases. > > I'd look into Yazen's direction for that... > Yes, I think this is something we can look into. I'm not aware of any situations at the moment. But I'd like to start focusing more on the various recovery paths and corner cases. Thanks, Yazen
In general it isn't very useful to dump the kernel stack in the panic from a fatal machine check. The problem is almost always hardware related, so knowing how the kernel got to the routine that triggered the machine check isn't useful. But Linux now has the capability to recover from most user mode and a few kernel mode memory related machine checks. Validation folks are testing that out and occasionally bring a kernel log like this to me: [69608.047771] mce: [Hardware Error]: Machine check: Data load in unrecoverable area of kernel [69608.021729] mce: [Hardware Error]: TSC 7874eb580177 ADDR 43bb84bd00 MISC 86 PPIN 9f061818e1a92082 [69608.047773] Kernel panic - not syncing: Fatal local machine check [69608.021720] mce: [Hardware Error]: RIP 10:<ffffffff8b767517> {copy_page+0x7/0x10} All I can tell them is that Linux was copying a page and hit poison in the source of the copy. But there are lots of reasons why Linux may be copying a page. A stack trace would help figure out if: 1) the test was bad and just injected an error into the wrong location 2) an injected error sat around in memory and was later consumed Case 2 will help identify places where Linux might use a "safe" copy function that returns an error to the caller which may attempt some sort of recovery. Patch 1 cleans up the Intel severity calculation by using a new severity table entry instead of some, now dubious, code to adjust the severity for errors in kernel context. Patch 2 adds a new severity level that triggers printing a stack trace. I've only updated the Intel severity calculation to use this new severity level. I'm not sure if AMD also has situations where this would be useful. If so, then mce_severity_amd() would need to be updated too to return different severity for IN_KERNEL and IN_KERNEL_RECOV cases. I've tested this out on systems that do both broadcast and local machine checks. Tony Luck (2): x86/mce: Use severity table to handle uncorrected errors in kernel x86/mce: Dump the stack for recoverable machine checks in kernel context arch/x86/kernel/cpu/mce/internal.h | 1 + arch/x86/kernel/cpu/mce/core.c | 11 +++++++++-- arch/x86/kernel/cpu/mce/severity.c | 10 ++++++---- 3 files changed, 16 insertions(+), 6 deletions(-) base-commit: 521a547ced6477c54b4b0cc206000406c221b4d6