| Message ID | 20250217071711.83735-7-adityag@linux.ibm.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Implement Firmware Assisted Dump for PSeries | expand |
On Mon Feb 17, 2025 at 5:17 PM AEST, Aditya Gupta wrote: > With all support in place, enable fadump by exporting the > "ibm,configure-kernel-dump" RTAS call in the device tree. > > Presence of "ibm,configure-kernel-dump" tells the kernel that the > platform (QEMU) supports fadump. > > Pass "fadump=on" to enable Linux to use firmware assisted dump. > > Logs of a linux boot with firmware assisted dump: > > ./build/qemu-system-ppc64 -M pseries,x-vof=on --cpu power10 --smp 4 -m 4G -kernel some-vmlinux -initrd some-initrd -append "debug fadump=on crashkernel=1G" -nographic > [ 0.000000] random: crng init done > [ 0.000000] fadump: Reserved 1024MB of memory at 0x00000040000000 (System RAM: 4096MB) > ... > [ 1.084686] rtas fadump: Registration is successful! > ... > # cat /sys/kernel/debug/powerpc/fadump_region > CPU :[0x00000040000000-0x000000400013d3] 0x13d4 bytes, Dumped: 0x0 > HPTE:[0x000000400013d4-0x000000400013d3] 0x0 bytes, Dumped: 0x0 > DUMP: Src: 0x00000000000000, Dest: 0x00000040010000, Size: 0x40000000, Dumped: 0x0 bytes > > [0x000000fffff800-0x000000ffffffff]: cmdline append: '' > # echo c > /proc/sysrq-trigger > > The fadump boot after crash: > > [ 0.000000] rtas fadump: Firmware-assisted dump is active. > [ 0.000000] fadump: Updated cmdline: debug fadump=on crashkernel=1G > [ 0.000000] fadump: Firmware-assisted dump is active. > [ 0.000000] fadump: Reserving 3072MB of memory at 0x00000040000000 for preserving crash data > .... > # file /proc/vmcore > /proc/vmcore: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, OpenPOWER ELF V2 ABI, version 1 (SYSV), SVR4-style > > Analysing the vmcore with crash-utility: > > KERNEL: vmlinux-6.14-rc2 > DUMPFILE: vmcore-a64dcfb451e2-nocma > CPUS: 4 > DATE: Thu Jan 1 05:30:00 IST 1970 > UPTIME: 00:00:30 > LOAD AVERAGE: 0.74, 0.21, 0.07 > TASKS: 94 > NODENAME: buildroot > RELEASE: 6.14.0-rc2+ > VERSION: #1 SMP Wed Feb 12 06:49:59 CST 2025 > MACHINE: ppc64le (1000 Mhz) > MEMORY: 4 GB > PANIC: "Kernel panic - not syncing: sysrq triggered crash" > PID: 270 > COMMAND: "sh" > TASK: c000000009e7cc00 [THREAD_INFO: c000000009e7cc00] > CPU: 3 > STATE: TASK_RUNNING (PANIC) > > Signed-off-by: Aditya Gupta <adityag@linux.ibm.com> This is very cool, nice work. Does it work with KVM? I think... probably it could? Are you able to add a functional test case for it? This is something that people (including me) will forget to test... Thanks, Nick > --- > hw/ppc/spapr_rtas.c | 6 +++++- > include/hw/ppc/spapr.h | 3 ++- > 2 files changed, 7 insertions(+), 2 deletions(-) > > diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c > index 0aca4270aee8..bd2ed16a46e3 100644 > --- a/hw/ppc/spapr_rtas.c > +++ b/hw/ppc/spapr_rtas.c > @@ -692,7 +692,7 @@ static void trigger_fadump_boot(target_ulong spapr_retcode) > } > > /* Papr Section 7.4.9 ibm,configure-kernel-dump RTAS call */ > -static __attribute((unused)) void rtas_configure_kernel_dump(PowerPCCPU *cpu, > +static void rtas_configure_kernel_dump(PowerPCCPU *cpu, > SpaprMachineState *spapr, > uint32_t token, uint32_t nargs, > target_ulong args, > @@ -1109,6 +1109,10 @@ static void core_rtas_register_types(void) > spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock", > rtas_ibm_nmi_interlock); > > + /* Register Fadump rtas call */ > + spapr_rtas_register(RTAS_CONFIGURE_KERNEL_DUMP, "ibm,configure-kernel-dump", > + rtas_configure_kernel_dump); > + > qtest_set_command_cb(spapr_qtest_callback); > } > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h > index fa63008e57ec..bde3bdc4b80c 100644 > --- a/include/hw/ppc/spapr.h > +++ b/include/hw/ppc/spapr.h > @@ -768,8 +768,9 @@ void push_sregs_to_kvm_pr(SpaprMachineState *spapr); > #define RTAS_IBM_SUSPEND_ME (RTAS_TOKEN_BASE + 0x2A) > #define RTAS_IBM_NMI_REGISTER (RTAS_TOKEN_BASE + 0x2B) > #define RTAS_IBM_NMI_INTERLOCK (RTAS_TOKEN_BASE + 0x2C) > +#define RTAS_CONFIGURE_KERNEL_DUMP (RTAS_TOKEN_BASE + 0x2D) > > -#define RTAS_TOKEN_MAX (RTAS_TOKEN_BASE + 0x2D) > +#define RTAS_TOKEN_MAX (RTAS_TOKEN_BASE + 0x2E) > > /* Fadump commands */ > #define FADUMP_CMD_REGISTER 1
On 27/02/25 09:03, Nicholas Piggin wrote: > On Mon Feb 17, 2025 at 5:17 PM AEST, Aditya Gupta wrote: >> With all support in place, enable fadump by exporting the >> "ibm,configure-kernel-dump" RTAS call in the device tree. >> >> Presence of "ibm,configure-kernel-dump" tells the kernel that the >> platform (QEMU) supports fadump. >> >> Pass "fadump=on" to enable Linux to use firmware assisted dump. >> >> Logs of a linux boot with firmware assisted dump: >> >> ./build/qemu-system-ppc64 -M pseries,x-vof=on --cpu power10 --smp 4 -m 4G -kernel some-vmlinux -initrd some-initrd -append "debug fadump=on crashkernel=1G" -nographic >> [ 0.000000] random: crng init done >> [ 0.000000] fadump: Reserved 1024MB of memory at 0x00000040000000 (System RAM: 4096MB) >> ... >> [ 1.084686] rtas fadump: Registration is successful! >> ... >> # cat /sys/kernel/debug/powerpc/fadump_region >> CPU :[0x00000040000000-0x000000400013d3] 0x13d4 bytes, Dumped: 0x0 >> HPTE:[0x000000400013d4-0x000000400013d3] 0x0 bytes, Dumped: 0x0 >> DUMP: Src: 0x00000000000000, Dest: 0x00000040010000, Size: 0x40000000, Dumped: 0x0 bytes >> >> [0x000000fffff800-0x000000ffffffff]: cmdline append: '' >> # echo c > /proc/sysrq-trigger >> >> The fadump boot after crash: >> >> [ 0.000000] rtas fadump: Firmware-assisted dump is active. >> [ 0.000000] fadump: Updated cmdline: debug fadump=on crashkernel=1G >> [ 0.000000] fadump: Firmware-assisted dump is active. >> [ 0.000000] fadump: Reserving 3072MB of memory at 0x00000040000000 for preserving crash data >> .... >> # file /proc/vmcore >> /proc/vmcore: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, OpenPOWER ELF V2 ABI, version 1 (SYSV), SVR4-style >> >> Analysing the vmcore with crash-utility: >> >> KERNEL: vmlinux-6.14-rc2 >> DUMPFILE: vmcore-a64dcfb451e2-nocma >> CPUS: 4 >> DATE: Thu Jan 1 05:30:00 IST 1970 >> UPTIME: 00:00:30 >> LOAD AVERAGE: 0.74, 0.21, 0.07 >> TASKS: 94 >> NODENAME: buildroot >> RELEASE: 6.14.0-rc2+ >> VERSION: #1 SMP Wed Feb 12 06:49:59 CST 2025 >> MACHINE: ppc64le (1000 Mhz) >> MEMORY: 4 GB >> PANIC: "Kernel panic - not syncing: sysrq triggered crash" >> PID: 270 >> COMMAND: "sh" >> TASK: c000000009e7cc00 [THREAD_INFO: c000000009e7cc00] >> CPU: 3 >> STATE: TASK_RUNNING (PANIC) >> >> Signed-off-by: Aditya Gupta <adityag@linux.ibm.com> > This is very cool, nice work. Does it work with KVM? I think... probably > it could? Yes it does, atleast for crashing CPU :) But there are problems with reading the CPU regs, regs don't seem correct for non-crashing CPUs. Crash is able to work perfectly for the crashing CPU as of now (as the registers are stored by the kernel in that case). > > Are you able to add a functional test case for it? This is something > that people (including me) will forget to test... Sure, I will add a test case. Thanks for your reviews Nick. It might take few weeks for me to post another version, will see into the tests in qemu and arrange the code bit more nicely. - Aditya G > > Thanks, > Nick > >> --- >> hw/ppc/spapr_rtas.c | 6 +++++- >> include/hw/ppc/spapr.h | 3 ++- >> 2 files changed, 7 insertions(+), 2 deletions(-) >> >> diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c >> index 0aca4270aee8..bd2ed16a46e3 100644 >> --- a/hw/ppc/spapr_rtas.c >> +++ b/hw/ppc/spapr_rtas.c >> @@ -692,7 +692,7 @@ static void trigger_fadump_boot(target_ulong spapr_retcode) >> } >> >> /* Papr Section 7.4.9 ibm,configure-kernel-dump RTAS call */ >> -static __attribute((unused)) void rtas_configure_kernel_dump(PowerPCCPU *cpu, >> +static void rtas_configure_kernel_dump(PowerPCCPU *cpu, >> SpaprMachineState *spapr, >> uint32_t token, uint32_t nargs, >> target_ulong args, >> @@ -1109,6 +1109,10 @@ static void core_rtas_register_types(void) >> spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock", >> rtas_ibm_nmi_interlock); >> >> + /* Register Fadump rtas call */ >> + spapr_rtas_register(RTAS_CONFIGURE_KERNEL_DUMP, "ibm,configure-kernel-dump", >> + rtas_configure_kernel_dump); >> + >> qtest_set_command_cb(spapr_qtest_callback); >> } >> >> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h >> index fa63008e57ec..bde3bdc4b80c 100644 >> --- a/include/hw/ppc/spapr.h >> +++ b/include/hw/ppc/spapr.h >> @@ -768,8 +768,9 @@ void push_sregs_to_kvm_pr(SpaprMachineState *spapr); >> #define RTAS_IBM_SUSPEND_ME (RTAS_TOKEN_BASE + 0x2A) >> #define RTAS_IBM_NMI_REGISTER (RTAS_TOKEN_BASE + 0x2B) >> #define RTAS_IBM_NMI_INTERLOCK (RTAS_TOKEN_BASE + 0x2C) >> +#define RTAS_CONFIGURE_KERNEL_DUMP (RTAS_TOKEN_BASE + 0x2D) >> >> -#define RTAS_TOKEN_MAX (RTAS_TOKEN_BASE + 0x2D) >> +#define RTAS_TOKEN_MAX (RTAS_TOKEN_BASE + 0x2E) >> >> /* Fadump commands */ >> #define FADUMP_CMD_REGISTER 1
On Thu Feb 27, 2025 at 5:07 PM AEST, Aditya Gupta wrote: > On 27/02/25 09:03, Nicholas Piggin wrote: > >> On Mon Feb 17, 2025 at 5:17 PM AEST, Aditya Gupta wrote: >>> With all support in place, enable fadump by exporting the >>> "ibm,configure-kernel-dump" RTAS call in the device tree. >>> >>> Presence of "ibm,configure-kernel-dump" tells the kernel that the >>> platform (QEMU) supports fadump. >>> >>> Pass "fadump=on" to enable Linux to use firmware assisted dump. >>> >>> Logs of a linux boot with firmware assisted dump: >>> >>> ./build/qemu-system-ppc64 -M pseries,x-vof=on --cpu power10 --smp 4 -m 4G -kernel some-vmlinux -initrd some-initrd -append "debug fadump=on crashkernel=1G" -nographic >>> [ 0.000000] random: crng init done >>> [ 0.000000] fadump: Reserved 1024MB of memory at 0x00000040000000 (System RAM: 4096MB) >>> ... >>> [ 1.084686] rtas fadump: Registration is successful! >>> ... >>> # cat /sys/kernel/debug/powerpc/fadump_region >>> CPU :[0x00000040000000-0x000000400013d3] 0x13d4 bytes, Dumped: 0x0 >>> HPTE:[0x000000400013d4-0x000000400013d3] 0x0 bytes, Dumped: 0x0 >>> DUMP: Src: 0x00000000000000, Dest: 0x00000040010000, Size: 0x40000000, Dumped: 0x0 bytes >>> >>> [0x000000fffff800-0x000000ffffffff]: cmdline append: '' >>> # echo c > /proc/sysrq-trigger >>> >>> The fadump boot after crash: >>> >>> [ 0.000000] rtas fadump: Firmware-assisted dump is active. >>> [ 0.000000] fadump: Updated cmdline: debug fadump=on crashkernel=1G >>> [ 0.000000] fadump: Firmware-assisted dump is active. >>> [ 0.000000] fadump: Reserving 3072MB of memory at 0x00000040000000 for preserving crash data >>> .... >>> # file /proc/vmcore >>> /proc/vmcore: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, OpenPOWER ELF V2 ABI, version 1 (SYSV), SVR4-style >>> >>> Analysing the vmcore with crash-utility: >>> >>> KERNEL: vmlinux-6.14-rc2 >>> DUMPFILE: vmcore-a64dcfb451e2-nocma >>> CPUS: 4 >>> DATE: Thu Jan 1 05:30:00 IST 1970 >>> UPTIME: 00:00:30 >>> LOAD AVERAGE: 0.74, 0.21, 0.07 >>> TASKS: 94 >>> NODENAME: buildroot >>> RELEASE: 6.14.0-rc2+ >>> VERSION: #1 SMP Wed Feb 12 06:49:59 CST 2025 >>> MACHINE: ppc64le (1000 Mhz) >>> MEMORY: 4 GB >>> PANIC: "Kernel panic - not syncing: sysrq triggered crash" >>> PID: 270 >>> COMMAND: "sh" >>> TASK: c000000009e7cc00 [THREAD_INFO: c000000009e7cc00] >>> CPU: 3 >>> STATE: TASK_RUNNING (PANIC) >>> >>> Signed-off-by: Aditya Gupta <adityag@linux.ibm.com> >> This is very cool, nice work. Does it work with KVM? I think... probably >> it could? > > Yes it does, atleast for crashing CPU :) > > But there are problems with reading the CPU regs, regs don't seem > correct for non-crashing CPUs. > > Crash is able to work perfectly for the crashing CPU as of now (as the > registers are stored by the kernel in that case). You may need to call cpu_synchronize_state() in the CPU_FOREACH loop before you read out the register state. Does that fix it? >> >> Are you able to add a functional test case for it? This is something >> that people (including me) will forget to test... > > Sure, I will add a test case. > > > Thanks for your reviews Nick. > > It might take few weeks for me to post another version, will see into > the tests in qemu and arrange the code bit more nicely. Yeah that's okay, I'm way behind with reviews and merging unfortunately so may have to wait for next release, but I'm keen to get it merged when we can. Sorry for the late review. Thanks, Nick
diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c index 0aca4270aee8..bd2ed16a46e3 100644 --- a/hw/ppc/spapr_rtas.c +++ b/hw/ppc/spapr_rtas.c @@ -692,7 +692,7 @@ static void trigger_fadump_boot(target_ulong spapr_retcode) } /* Papr Section 7.4.9 ibm,configure-kernel-dump RTAS call */ -static __attribute((unused)) void rtas_configure_kernel_dump(PowerPCCPU *cpu, +static void rtas_configure_kernel_dump(PowerPCCPU *cpu, SpaprMachineState *spapr, uint32_t token, uint32_t nargs, target_ulong args, @@ -1109,6 +1109,10 @@ static void core_rtas_register_types(void) spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock", rtas_ibm_nmi_interlock); + /* Register Fadump rtas call */ + spapr_rtas_register(RTAS_CONFIGURE_KERNEL_DUMP, "ibm,configure-kernel-dump", + rtas_configure_kernel_dump); + qtest_set_command_cb(spapr_qtest_callback); } diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index fa63008e57ec..bde3bdc4b80c 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -768,8 +768,9 @@ void push_sregs_to_kvm_pr(SpaprMachineState *spapr); #define RTAS_IBM_SUSPEND_ME (RTAS_TOKEN_BASE + 0x2A) #define RTAS_IBM_NMI_REGISTER (RTAS_TOKEN_BASE + 0x2B) #define RTAS_IBM_NMI_INTERLOCK (RTAS_TOKEN_BASE + 0x2C) +#define RTAS_CONFIGURE_KERNEL_DUMP (RTAS_TOKEN_BASE + 0x2D) -#define RTAS_TOKEN_MAX (RTAS_TOKEN_BASE + 0x2D) +#define RTAS_TOKEN_MAX (RTAS_TOKEN_BASE + 0x2E) /* Fadump commands */ #define FADUMP_CMD_REGISTER 1
With all support in place, enable fadump by exporting the "ibm,configure-kernel-dump" RTAS call in the device tree. Presence of "ibm,configure-kernel-dump" tells the kernel that the platform (QEMU) supports fadump. Pass "fadump=on" to enable Linux to use firmware assisted dump. Logs of a linux boot with firmware assisted dump: ./build/qemu-system-ppc64 -M pseries,x-vof=on --cpu power10 --smp 4 -m 4G -kernel some-vmlinux -initrd some-initrd -append "debug fadump=on crashkernel=1G" -nographic [ 0.000000] random: crng init done [ 0.000000] fadump: Reserved 1024MB of memory at 0x00000040000000 (System RAM: 4096MB) ... [ 1.084686] rtas fadump: Registration is successful! ... # cat /sys/kernel/debug/powerpc/fadump_region CPU :[0x00000040000000-0x000000400013d3] 0x13d4 bytes, Dumped: 0x0 HPTE:[0x000000400013d4-0x000000400013d3] 0x0 bytes, Dumped: 0x0 DUMP: Src: 0x00000000000000, Dest: 0x00000040010000, Size: 0x40000000, Dumped: 0x0 bytes [0x000000fffff800-0x000000ffffffff]: cmdline append: '' # echo c > /proc/sysrq-trigger The fadump boot after crash: [ 0.000000] rtas fadump: Firmware-assisted dump is active. [ 0.000000] fadump: Updated cmdline: debug fadump=on crashkernel=1G [ 0.000000] fadump: Firmware-assisted dump is active. [ 0.000000] fadump: Reserving 3072MB of memory at 0x00000040000000 for preserving crash data .... # file /proc/vmcore /proc/vmcore: ELF 64-bit LSB core file, 64-bit PowerPC or cisco 7500, OpenPOWER ELF V2 ABI, version 1 (SYSV), SVR4-style Analysing the vmcore with crash-utility: KERNEL: vmlinux-6.14-rc2 DUMPFILE: vmcore-a64dcfb451e2-nocma CPUS: 4 DATE: Thu Jan 1 05:30:00 IST 1970 UPTIME: 00:00:30 LOAD AVERAGE: 0.74, 0.21, 0.07 TASKS: 94 NODENAME: buildroot RELEASE: 6.14.0-rc2+ VERSION: #1 SMP Wed Feb 12 06:49:59 CST 2025 MACHINE: ppc64le (1000 Mhz) MEMORY: 4 GB PANIC: "Kernel panic - not syncing: sysrq triggered crash" PID: 270 COMMAND: "sh" TASK: c000000009e7cc00 [THREAD_INFO: c000000009e7cc00] CPU: 3 STATE: TASK_RUNNING (PANIC) Signed-off-by: Aditya Gupta <adityag@linux.ibm.com> --- hw/ppc/spapr_rtas.c | 6 +++++- include/hw/ppc/spapr.h | 3 ++- 2 files changed, 7 insertions(+), 2 deletions(-)