| Message ID | 20211213133542.2608540-1-farosas@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | target/ppc: Fix e6500 boot | expand |
On 12/13/21 14:35, Fabiano Rosas wrote: > When Altivec support was added to the e6500 kernel in 2012[1], the > QEMU code was not changed, so we don't register the VPU/VPUA > exceptions for the e6500: > > qemu: fatal: Raised an exception without defined vector 73 > > Note that the error message says 73, instead of 32, which is the IVOR > for VPU. This is because QEMU knows only knows about the VPU interrupt > for the 7400s. In theory, we should not be raising _that_ VPU > interrupt, but instead another one specific for the e6500. > > We unfortunately cannot register e6500-specific VPU/VPUA interrupts > because the SPEU/EFPDI interrupts also use IVOR32/33. These are > present only in the e500v1/2 versions. From the user manual: > > e500v1, e500v2: only SPEU/EFPDI/EFPRI > e500mc, e5500: no SPEU/EFPDI/EFPRI/VPU/VPUA > e6500: only VPU/VPUA > > So I'm leaving IVOR32/33 as SPEU/EFPDI, but altering the dispatch code > to convert the VPU #73 to a #32 when we're in the e6500. Since the > handling for SPEU and VPU is the same this is the only change that's > needed. The EFPDI is not implemented and will cause an abort. I don't > think it worth it changing the error message to take VPUA into > consideration, so I'm not changing anything there. > > This bug was discussed in the thread: > https://lists.gnu.org/archive/html/qemu-ppc/2021-06/msg00222.html > > 1- https://git.kernel.org/torvalds/c/cd66cc2ee52 > > Reported-by: <mario@locati.it> > Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com> Reviewed-by: Cédric Le Goater <clg@kaod.org> One comment, > --- > target/ppc/cpu_init.c | 6 ++++++ > target/ppc/excp_helper.c | 12 +++++++++++- > 2 files changed, 17 insertions(+), 1 deletion(-) > > diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c > index 6695985e9b..d8efcb24ed 100644 > --- a/target/ppc/cpu_init.c > +++ b/target/ppc/cpu_init.c > @@ -2273,8 +2273,14 @@ static void init_excp_e200(CPUPPCState *env, target_ulong ivpr_mask) > env->excp_vectors[POWERPC_EXCP_DTLB] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_ITLB] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_DEBUG] = 0x00000000; > + /* > + * These two are the same IVOR as POWERPC_EXCP_VPU and > + * POWERPC_EXCP_VPUA. We deal with that when dispatching at > + * powerpc_excp(). > + */ > env->excp_vectors[POWERPC_EXCP_SPEU] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_EFPDI] = 0x00000000; > + > env->excp_vectors[POWERPC_EXCP_EFPRI] = 0x00000000; > env->ivor_mask = 0x0000FFF7UL; > env->ivpr_mask = ivpr_mask; > diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c > index 17607adbe4..7bb170f440 100644 > --- a/target/ppc/excp_helper.c > +++ b/target/ppc/excp_helper.c > @@ -344,6 +344,16 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) > excp = POWERPC_EXCP_PROGRAM; > } > > +#ifdef TARGET_PPC64 > + /* > + * SPEU and VPU share the same IVOR but they exist in different > + * processors. SPEU is e500v1/2 only and VPU is e6500 only. > + */ > + if (excp_model == POWERPC_EXCP_BOOKE && excp == POWERPC_EXCP_VPU) { > + excp = POWERPC_EXCP_SPEU; > + } > +#endif I am not in favor of changing powerpc_excp() but I know you have plans for a major clean up :) Thanks, C. > switch (excp) { > case POWERPC_EXCP_NONE: > /* Should never happen */ > @@ -569,7 +579,7 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) > cpu_abort(cs, "Debug exception triggered on unsupported model\n"); > } > break; > - case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable */ > + case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable/VPU */ > env->spr[SPR_BOOKE_ESR] = ESR_SPV; > break; > case POWERPC_EXCP_EFPDI: /* Embedded floating-point data interrupt */ >
Cédric Le Goater <clg@kaod.org> writes: > On 12/13/21 14:35, Fabiano Rosas wrote: >> When Altivec support was added to the e6500 kernel in 2012[1], the >> QEMU code was not changed, so we don't register the VPU/VPUA >> exceptions for the e6500: >> >> qemu: fatal: Raised an exception without defined vector 73 >> >> Note that the error message says 73, instead of 32, which is the IVOR >> for VPU. This is because QEMU knows only knows about the VPU interrupt >> for the 7400s. In theory, we should not be raising _that_ VPU >> interrupt, but instead another one specific for the e6500. >> >> We unfortunately cannot register e6500-specific VPU/VPUA interrupts >> because the SPEU/EFPDI interrupts also use IVOR32/33. These are >> present only in the e500v1/2 versions. From the user manual: >> >> e500v1, e500v2: only SPEU/EFPDI/EFPRI >> e500mc, e5500: no SPEU/EFPDI/EFPRI/VPU/VPUA >> e6500: only VPU/VPUA >> >> So I'm leaving IVOR32/33 as SPEU/EFPDI, but altering the dispatch code >> to convert the VPU #73 to a #32 when we're in the e6500. Since the >> handling for SPEU and VPU is the same this is the only change that's >> needed. The EFPDI is not implemented and will cause an abort. I don't >> think it worth it changing the error message to take VPUA into >> consideration, so I'm not changing anything there. >> >> This bug was discussed in the thread: >> https://lists.gnu.org/archive/html/qemu-ppc/2021-06/msg00222.html >> >> 1- https://git.kernel.org/torvalds/c/cd66cc2ee52 >> >> Reported-by: <mario@locati.it> >> Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com> > > Reviewed-by: Cédric Le Goater <clg@kaod.org> > > One comment, > >> --- >> target/ppc/cpu_init.c | 6 ++++++ >> target/ppc/excp_helper.c | 12 +++++++++++- >> 2 files changed, 17 insertions(+), 1 deletion(-) >> >> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c >> index 6695985e9b..d8efcb24ed 100644 >> --- a/target/ppc/cpu_init.c >> +++ b/target/ppc/cpu_init.c >> @@ -2273,8 +2273,14 @@ static void init_excp_e200(CPUPPCState *env, target_ulong ivpr_mask) >> env->excp_vectors[POWERPC_EXCP_DTLB] = 0x00000000; >> env->excp_vectors[POWERPC_EXCP_ITLB] = 0x00000000; >> env->excp_vectors[POWERPC_EXCP_DEBUG] = 0x00000000; >> + /* >> + * These two are the same IVOR as POWERPC_EXCP_VPU and >> + * POWERPC_EXCP_VPUA. We deal with that when dispatching at >> + * powerpc_excp(). >> + */ >> env->excp_vectors[POWERPC_EXCP_SPEU] = 0x00000000; >> env->excp_vectors[POWERPC_EXCP_EFPDI] = 0x00000000; >> + >> env->excp_vectors[POWERPC_EXCP_EFPRI] = 0x00000000; >> env->ivor_mask = 0x0000FFF7UL; >> env->ivpr_mask = ivpr_mask; >> diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c >> index 17607adbe4..7bb170f440 100644 >> --- a/target/ppc/excp_helper.c >> +++ b/target/ppc/excp_helper.c >> @@ -344,6 +344,16 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) >> excp = POWERPC_EXCP_PROGRAM; >> } >> >> +#ifdef TARGET_PPC64 >> + /* >> + * SPEU and VPU share the same IVOR but they exist in different >> + * processors. SPEU is e500v1/2 only and VPU is e6500 only. >> + */ >> + if (excp_model == POWERPC_EXCP_BOOKE && excp == POWERPC_EXCP_VPU) { >> + excp = POWERPC_EXCP_SPEU; >> + } >> +#endif > > I am not in favor of changing powerpc_excp() but I know you have > plans for a major clean up :) Yep, I think is better to fix everything that is broken before the cleanup so we have more code working and being tested before the changes. I would have sent this patch months ago if I knew how to fix it then =)
On Mon, 13 Dec 2021, Fabiano Rosas wrote: > When Altivec support was added to the e6500 kernel in 2012[1], the > QEMU code was not changed, so we don't register the VPU/VPUA > exceptions for the e6500: > > qemu: fatal: Raised an exception without defined vector 73 > > Note that the error message says 73, instead of 32, which is the IVOR > for VPU. This is because QEMU knows only knows about the VPU interrupt The word "knows" is repeated in the above line. > for the 7400s. In theory, we should not be raising _that_ VPU > interrupt, but instead another one specific for the e6500. > > We unfortunately cannot register e6500-specific VPU/VPUA interrupts > because the SPEU/EFPDI interrupts also use IVOR32/33. These are > present only in the e500v1/2 versions. From the user manual: > > e500v1, e500v2: only SPEU/EFPDI/EFPRI > e500mc, e5500: no SPEU/EFPDI/EFPRI/VPU/VPUA > e6500: only VPU/VPUA > > So I'm leaving IVOR32/33 as SPEU/EFPDI, but altering the dispatch code > to convert the VPU #73 to a #32 when we're in the e6500. Since the > handling for SPEU and VPU is the same this is the only change that's > needed. The EFPDI is not implemented and will cause an abort. I don't > think it worth it changing the error message to take VPUA into > consideration, so I'm not changing anything there. > > This bug was discussed in the thread: > https://lists.gnu.org/archive/html/qemu-ppc/2021-06/msg00222.html > > 1- https://git.kernel.org/torvalds/c/cd66cc2ee52 > > Reported-by: <mario@locati.it> > Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com> > --- > target/ppc/cpu_init.c | 6 ++++++ > target/ppc/excp_helper.c | 12 +++++++++++- > 2 files changed, 17 insertions(+), 1 deletion(-) > > diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c > index 6695985e9b..d8efcb24ed 100644 > --- a/target/ppc/cpu_init.c > +++ b/target/ppc/cpu_init.c > @@ -2273,8 +2273,14 @@ static void init_excp_e200(CPUPPCState *env, target_ulong ivpr_mask) > env->excp_vectors[POWERPC_EXCP_DTLB] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_ITLB] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_DEBUG] = 0x00000000; > + /* > + * These two are the same IVOR as POWERPC_EXCP_VPU and Maybe clearar to say "The next two" or even "SPEU and EFPDI are the same..." then no need for the extra empty line but not a big deal. Regards, BALATON Zoltan > + * POWERPC_EXCP_VPUA. We deal with that when dispatching at > + * powerpc_excp(). > + */ > env->excp_vectors[POWERPC_EXCP_SPEU] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_EFPDI] = 0x00000000; > + > env->excp_vectors[POWERPC_EXCP_EFPRI] = 0x00000000; > env->ivor_mask = 0x0000FFF7UL; > env->ivpr_mask = ivpr_mask; > diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c > index 17607adbe4..7bb170f440 100644 > --- a/target/ppc/excp_helper.c > +++ b/target/ppc/excp_helper.c > @@ -344,6 +344,16 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) > excp = POWERPC_EXCP_PROGRAM; > } > > +#ifdef TARGET_PPC64 > + /* > + * SPEU and VPU share the same IVOR but they exist in different > + * processors. SPEU is e500v1/2 only and VPU is e6500 only. > + */ > + if (excp_model == POWERPC_EXCP_BOOKE && excp == POWERPC_EXCP_VPU) { > + excp = POWERPC_EXCP_SPEU; > + } > +#endif > + > switch (excp) { > case POWERPC_EXCP_NONE: > /* Should never happen */ > @@ -569,7 +579,7 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) > cpu_abort(cs, "Debug exception triggered on unsupported model\n"); > } > break; > - case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable */ > + case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable/VPU */ > env->spr[SPR_BOOKE_ESR] = ESR_SPV; > break; > case POWERPC_EXCP_EFPDI: /* Embedded floating-point data interrupt */ >
On 12/13/21 14:35, Fabiano Rosas wrote: > When Altivec support was added to the e6500 kernel in 2012[1], the > QEMU code was not changed, so we don't register the VPU/VPUA > exceptions for the e6500: > > qemu: fatal: Raised an exception without defined vector 73 > > Note that the error message says 73, instead of 32, which is the IVOR > for VPU. This is because QEMU knows only knows about the VPU interrupt > for the 7400s. In theory, we should not be raising _that_ VPU > interrupt, but instead another one specific for the e6500. > > We unfortunately cannot register e6500-specific VPU/VPUA interrupts > because the SPEU/EFPDI interrupts also use IVOR32/33. These are > present only in the e500v1/2 versions. From the user manual: > > e500v1, e500v2: only SPEU/EFPDI/EFPRI > e500mc, e5500: no SPEU/EFPDI/EFPRI/VPU/VPUA > e6500: only VPU/VPUA > > So I'm leaving IVOR32/33 as SPEU/EFPDI, but altering the dispatch code > to convert the VPU #73 to a #32 when we're in the e6500. Since the > handling for SPEU and VPU is the same this is the only change that's > needed. The EFPDI is not implemented and will cause an abort. I don't > think it worth it changing the error message to take VPUA into > consideration, so I'm not changing anything there. > > This bug was discussed in the thread: > https://lists.gnu.org/archive/html/qemu-ppc/2021-06/msg00222.html > > 1- https://git.kernel.org/torvalds/c/cd66cc2ee52 > > Reported-by: <mario@locati.it> > Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com> Applied to ppc-next. Thanks, C.
From: "BALATON Zoltan" balaton@eik.bme.hu To: "Fabiano Rosas" farosas@linux.ibm.com Cc: qemu-devel@nongnu.org, qemu-ppc@nongnu.org, clg@kaod.org, danielhb413@gmail.com, mario@locati.it Date: Mon, 13 Dec 2021 20:51:15 +0100 (CET) Subject: Re: [PATCH] target/ppc: Fix e6500 boot On Mon, 13 Dec 2021, Fabiano Rosas wrote: > When Altivec support was added to the e6500 kernel in 2012[1], the > QEMU code was not changed, so we don't register the VPU/VPUA > exceptions for the e6500: > > qemu: fatal: Raised an exception without defined vector 73 > > Note that the error message says 73, instead of 32, which is the IVOR > for VPU. This is because QEMU knows only knows about the VPU interrupt The word "knows" is repeated in the above line. > for the 7400s. In theory, we should not be raising _that_ VPU > interrupt, but instead another one specific for the e6500. > > We unfortunately cannot register e6500-specific VPU/VPUA interrupts > because the SPEU/EFPDI interrupts also use IVOR32/33. These are > present only in the e500v1/2 versions. From the user manual: > > e500v1, e500v2: only SPEU/EFPDI/EFPRI > e500mc, e5500: no SPEU/EFPDI/EFPRI/VPU/VPUA > e6500: only VPU/VPUA > > So I'm leaving IVOR32/33 as SPEU/EFPDI, but altering the dispatch code > to convert the VPU #73 to a #32 when we're in the e6500. Since the > handling for SPEU and VPU is the same this is the only change that's > needed. The EFPDI is not implemented and will cause an abort. I don't > think it worth it changing the error message to take VPUA into > consideration, so I'm not changing anything there. > > This bug was discussed in the thread: > https://lists.gnu.org/archive/html/qemu-ppc/2021-06/msg00222.html > > 1- https://git.kernel.org/torvalds/c/cd66cc2ee52 > > Reported-by: <mario@locati.it> > Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com> > --- > target/ppc/cpu_init.c | 6 ++++++ > target/ppc/excp_helper.c | 12 +++++++++++- > 2 files changed, 17 insertions(+), 1 deletion(-) > > diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c > index 6695985e9b..d8efcb24ed 100644 > --- a/target/ppc/cpu_init.c > +++ b/target/ppc/cpu_init.c > @@ -2273,8 +2273,14 @@ static void init_excp_e200(CPUPPCState *env, target_ulong ivpr_mask) > env->excp_vectors[POWERPC_EXCP_DTLB] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_ITLB] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_DEBUG] = 0x00000000; > + /* > + * These two are the same IVOR as POWERPC_EXCP_VPU and Maybe clearar to say "The next two" or even "SPEU and EFPDI are the same..." then no need for the extra empty line but not a big deal. Regards, BALATON Zoltan > + * POWERPC_EXCP_VPUA. We deal with that when dispatching at > + * powerpc_excp(). > + */ > env->excp_vectors[POWERPC_EXCP_SPEU] = 0x00000000; > env->excp_vectors[POWERPC_EXCP_EFPDI] = 0x00000000; > + > env->excp_vectors[POWERPC_EXCP_EFPRI] = 0x00000000; > env->ivor_mask = 0x0000FFF7UL; > env->ivpr_mask = ivpr_mask; > diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c > index 17607adbe4..7bb170f440 100644 > --- a/target/ppc/excp_helper.c > +++ b/target/ppc/excp_helper.c > @@ -344,6 +344,16 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) > excp = POWERPC_EXCP_PROGRAM; > } > > +#ifdef TARGET_PPC64 > + /* > + * SPEU and VPU share the same IVOR but they exist in different > + * processors. SPEU is e500v1/2 only and VPU is e6500 only. > + */ > + if (excp_model == POWERPC_EXCP_BOOKE && excp == POWERPC_EXCP_VPU) { > + excp = POWERPC_EXCP_SPEU; > + } > +#endif > + > switch (excp) { > case POWERPC_EXCP_NONE: > /* Should never happen */ > @@ -569,7 +579,7 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) > cpu_abort(cs, "Debug exception triggered on unsupported model\n"); > } > break; > - case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable */ > + case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable/VPU */ > env->spr[SPR_BOOKE_ESR] = ESR_SPV; > break; > case POWERPC_EXCP_EFPDI: /* Embedded floating-point data interrupt */ > I have tried to launch a freshly compiled qemu from git master on a NXP T2080RDB devkit that has a e6500 CPU in combination with a freshly compiled kernel 5.16-rc6 I have Debian SID ppc64 up and running using such a kernel, and when I launch qemu to run a VM with the same debian sid for ppc64 and the same kernel using --enable-kvm I end up with a kernel panic [....] Run /sbin/init as init process random: fast init done systemd[1]: illegal instruction (4) at 3fff96562ac8 nip 3fff96562ac8 lr 3fff96562aa8 code 1 in libc-2.32.so[3fff96516000+1f7000] systemd[1]: code: 60000000 38600006 9122b7e8 4801bead 60000000 60000000 8122b7e8 2c090004 systemd[1]: code: 40820014 39200005 60000000 9122b7e8 <00000000> 60000000 8122b7e8 2c090005 Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Rebooting in 180 seconds.. On the positive side, for the first time I am not flooded by milions kernel error messages from the serial console attached to the devkit, not a single error spitted out from it, great! The kernel config file I used https://repo.powerprogress.org/t2080rdb/qemu/kernel_5.16-rc6.config The resulting kernel https://repo.powerprogress.org/t2080rdb/qemu/uImage_5.16-rc6 The complete console output I get when configuring qemu before compilation https://repo.powerprogress.org/t2080rdb/qemu/2021-12-25_qemu_git_configure_on_ppc64.txt The complete console output that end up with the kernel panic when launching qemuhttps://repo.powerprogress.org/t2080rdb/qemu/2021-12-25_qemu_ppc64_e6500_kvm_debian_sid_log.txt
On Sat, 25 Dec 2021, mario@locati.it wrote: > I have tried to launch a freshly compiled qemu from git master on a NXP > T2080RDB devkit that has a e6500 CPU in combination with a freshly > compiled kernel 5.16-rc6 > I have Debian SID ppc64 up and running using such a kernel, and when I > launch qemu to run a VM with the same debian sid for ppc64 and the same > kernel using --enable-kvm I end up with a kernel panic > > > [....] > Run /sbin/init as init process > random: fast init done > systemd[1]: illegal instruction (4) at 3fff96562ac8 nip 3fff96562ac8 lr 3fff96562aa8 code 1 in libc-2.32.so[3fff96516000+1f7000] > systemd[1]: code: 60000000 38600006 9122b7e8 4801bead 60000000 60000000 8122b7e8 2c090004 > systemd[1]: code: 40820014 39200005 60000000 9122b7e8 <00000000> 60000000 8122b7e8 2c090005 Looks like it trips on a 0 opcode here in the middle of other values that look like valid code so I wonder how that 0 got there? Did something overwrite it before it tried to execute it? If it always happens on the same address maybe you could try attaching gdb and put a watch point on that address to see what writes there, otherwise I don't know how to debug this. Regards, BALATON Zoltan > Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 > Rebooting in 180 seconds.. > > > On the positive side, for the first time I am not flooded by milions > kernel error messages from the serial console attached to the devkit, > not a single error spitted out from it, great! > > > The kernel config file I used > https://repo.powerprogress.org/t2080rdb/qemu/kernel_5.16-rc6.config > > > The resulting kernel > https://repo.powerprogress.org/t2080rdb/qemu/uImage_5.16-rc6 > > > The complete console output I get when configuring qemu before compilation > https://repo.powerprogress.org/t2080rdb/qemu/2021-12-25_qemu_git_configure_on_ppc64.txt > > > The complete console output that end up with the kernel panic when launching qemu > https://repo.powerprogress.org/t2080rdb/qemu/2021-12-25_qemu_ppc64_e6500_kvm_debian_sid_log.txt
On 12/25/21 22:53, BALATON Zoltan wrote: > On Sat, 25 Dec 2021, mario@locati.it wrote: >> I have tried to launch a freshly compiled qemu from git master on a NXP T2080RDB devkit that has a e6500 CPU in combination with a freshly compiled kernel 5.16-rc6 >> I have Debian SID ppc64 up and running using such a kernel, and when I launch qemu to run a VM with the same debian sid for ppc64 and the same kernel using --enable-kvm I end up with a kernel panic Thanks for testing, >> >> [....] >> Run /sbin/init as init process >> random: fast init done >> systemd[1]: illegal instruction (4) at 3fff96562ac8 nip 3fff96562ac8 lr 3fff96562aa8 code 1 in libc-2.32.so[3fff96516000+1f7000] debian ppc64 sid has a glibc 2.33 AFAICT >> systemd[1]: code: 60000000 38600006 9122b7e8 4801bead 60000000 60000000 8122b7e8 2c090004 >> systemd[1]: code: 40820014 39200005 60000000 9122b7e8 <00000000> 60000000 8122b7e8 2c090005 > > Looks like it trips on a 0 opcode here in the middle of other values that look like valid code so I wonder how that 0 got there? Did something overwrite it before it tried to execute it? This looks like the abort() routine. > If it always happens on the same address maybe you could try attaching gdb and put a watch point on that address to see what writes there, otherwise I don't know how to debug this. Could you deduce the routine name from the nip ? Thanks, C.
From: "Cédric Le Goater" clg@kaod.org To: "BALATON Zoltan" balaton@eik.bme.hu,"mario@locati.it" mario@locati.it Cc: farosas@linux.ibm.com, qemu-devel@nongnu.org, qemu-ppc@nongnu.org, danielhb413@gmail.com Date: Sun, 26 Dec 2021 18:57:54 +0100 Subject: Re: [PATCH] target/ppc: Fix e6500 boot On 12/25/21 22:53, BALATON Zoltan wrote: > On Sat, 25 Dec 2021, mario@locati.it wrote: >> I have tried to launch a freshly compiled qemu from git master on a NXP T2080RDB devkit that has a e6500 CPU in combination with a freshly compiled kernel 5.16-rc6 >> I have Debian SID ppc64 up and running using such a kernel, and when I launch qemu to run a VM with the same debian sid for ppc64 and the same kernel using --enable-kvm I end up with a kernel panic Thanks for testing, >> >> [....] >> Run /sbin/init as init process >> random: fast init done >> systemd[1]: illegal instruction (4) at 3fff96562ac8 nip 3fff96562ac8 lr 3fff96562aa8 code 1 in libc-2.32.so[3fff96516000+1f7000] debian ppc64 sid has a glibc 2.33 AFAICT >> systemd[1]: code: 60000000 38600006 9122b7e8 4801bead 60000000 60000000 8122b7e8 2c090004 >> systemd[1]: code: 40820014 39200005 60000000 9122b7e8 <00000000> 60000000 8122b7e8 2c090005 > > Looks like it trips on a 0 opcode here in the middle of other values that look like valid code so I wonder how that 0 got there? Did something overwrite it before it tried to execute it? This looks like the abort() routine. > If it always happens on the same address maybe you could try attaching gdb and put a watch point on that address to see what writes there, otherwise I don't know how to debug this. Could you deduce the routine name from the nip ? Thanks, C. I have updated the guest VM but I get exactly the same result except that now I have libc-2.33.so installed. [...] VFS: Mounted root (ext4 filesystem) on device 254:0. devtmpfs: mounted Freeing unused kernel image (initmem) memory: 468K This architecture does not have kernel memory protection. Run /sbin/init as init process random: fast init done systemd[1]: illegal instruction (4) at 3fff8b7e615c nip 3fff8b7e615c lr 3fff8b7e613c code 1 in libc-2.33.so[3fff8b799000+1fe000] systemd[1]: code: 60000000 38600006 9122b7d0 4801bf19 60000000 60000000 8122b7d0 2c090004 systemd[1]: code: 40820014 39200005 60000000 9122b7d0 <00000000> 60000000 8122b7d0 2c090005 Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Rebooting in 180 seconds.. I don't know anything about debugging, sorry, just an average user here. Currently asking for help to more expert users in the PowerProgressCommunity in order to understand what gdb is and how to use it but so far seems quite complicated, sorry. It will taka a while before I will be able to provide what Zoltan is asking for. If anybody of you want a remote ssh access to our devkit please send me an email privately. Meanwhile, I successfully got a guest VM working with kvm simply by changing "-cpu e6500" into "-cpu e5500" and still using the same kernel I have compiled for the e6500, here the qemu commands I have used: qemu-system-ppc64 -enable-kvm -M ppce500 -cpu e5500 -smp 4 -m 2G -net nic -net user -device virtio-vga -device virtio-mouse-pci -device virtio-keyboard-pci -drive format=raw,file=hdd_debian_ppc64_new.img,index=0,if=virtio -kernel uImage -append "root=/dev/vda rw" And here a screenshot I took of the guest machine up and running quite well https://repo.powerprogress.org/t2080rdb/qemu/2021-12-27_qemu_e6500_kvm_kind_of_working.jpg What I find strange and that leave me puzzled is that running hardinfo the cpu is reported as an e6500 with altivec and not an e5500 that does not have altivec.
"mario@locati.it" <mario@locati.it> writes: > I have updated the guest VM but I get exactly the same result except that now I have libc-2.33.so installed. > > [...] > VFS: Mounted root (ext4 filesystem) on device 254:0. > devtmpfs: mounted > Freeing unused kernel image (initmem) memory: 468K > This architecture does not have kernel memory protection. > Run /sbin/init as init process > random: fast init done > systemd[1]: illegal instruction (4) at 3fff8b7e615c nip 3fff8b7e615c lr 3fff8b7e613c code 1 in libc-2.33.so[3fff8b799000+1fe000] > systemd[1]: code: 60000000 38600006 9122b7d0 4801bf19 60000000 60000000 8122b7d0 2c090004 > systemd[1]: code: 40820014 39200005 60000000 9122b7d0 <00000000> 60000000 8122b7d0 2c090005 > Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 > Rebooting in 180 seconds.. Can you make the hdd_debian_ppc64_new.img available? We won't be able to reproduce the exact same scenario because we can't run KVM, but if it boots with TCG we can at least look around the code that is failing to see if it gives us any clues.
Hello, On Mon, 27 Dec 2021, mario@locati.it wrote: > I have updated the guest VM but I get exactly the same result except that now I have libc-2.33.so installed. > > [...] > VFS: Mounted root (ext4 filesystem) on device 254:0. > devtmpfs: mounted > Freeing unused kernel image (initmem) memory: 468K > This architecture does not have kernel memory protection. > Run /sbin/init as init process > random: fast init done > systemd[1]: illegal instruction (4) at 3fff8b7e615c nip 3fff8b7e615c lr 3fff8b7e613c code 1 in libc-2.33.so[3fff8b799000+1fe000] > systemd[1]: code: 60000000 38600006 9122b7d0 4801bf19 60000000 60000000 8122b7d0 2c090004 > systemd[1]: code: 40820014 39200005 60000000 9122b7d0 <00000000> 60000000 8122b7d0 2c090005 > Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 > Rebooting in 180 seconds.. > > I don't know anything about debugging, sorry, just an average user here. > Currently asking for help to more expert users in the PowerProgressCommunity in order to understand what gdb is and how to use it but so far seems quite complicated, sorry. > It will taka a while before I will be able to provide what Zoltan is asking for. Maybe it's not needed, it was just an idea to get closer to the problem but you could also try finding this our from within the VM as Cedric suggested. As for attaching gdb to QEMU for debugging guest code here's an article but maybe you can find a better one elsewhere too: http://nickdesaulniers.github.io/blog/2018/10/24/booting-a-custom-linux-kernel-in-qemu-and-debugging-it-with-gdb/ Do you have the libc-2.33.so binary with debugging symbols? (Maybe it's available in some debug package, I don't know how Debian handles this.) If so you could try to check what is at the offset shown in the log (I think it's 1fe000 above) either with gdb or objdump or maybe there are other ways as well. > If anybody of you want a remote ssh access to our devkit please send me an email privately. > > Meanwhile, I successfully got a guest VM working with kvm simply by > changing "-cpu e6500" into "-cpu e5500" and still using the same kernel > I have compiled for the e6500, here the qemu commands I have used: > > qemu-system-ppc64 -enable-kvm -M ppce500 -cpu e5500 -smp 4 -m 2G -net > nic -net user -device virtio-vga -device virtio-mouse-pci -device > virtio-keyboard-pci -drive > format=raw,file=hdd_debian_ppc64_new.img,index=0,if=virtio -kernel > uImage -append "root=/dev/vda rw" > > And here a screenshot I took of the guest machine up and running quite well > https://repo.powerprogress.org/t2080rdb/qemu/2021-12-27_qemu_e6500_kvm_kind_of_working.jpg > > What I find strange and that leave me puzzled is that running hardinfo the cpu is reported as an e6500 with altivec and not an e5500 that does not have altivec. With KVM the code is run on the host CPU so depending on how it determines the CPU model you may still get the host CPU. I'm not sure what -cpu does with KVM but apparently it does something if it makes it boot. Probably libc takes a different path with these CPUs so you only get the problem when it has PVR set to e6500?. Regards, BALATON Zoltan
On Mon, 27 Dec 2021, Fabiano Rosas wrote: > "mario@locati.it" <mario@locati.it> writes: >> I have updated the guest VM but I get exactly the same result except that now I have libc-2.33.so installed. >> >> [...] >> VFS: Mounted root (ext4 filesystem) on device 254:0. >> devtmpfs: mounted >> Freeing unused kernel image (initmem) memory: 468K >> This architecture does not have kernel memory protection. >> Run /sbin/init as init process >> random: fast init done >> systemd[1]: illegal instruction (4) at 3fff8b7e615c nip 3fff8b7e615c lr 3fff8b7e613c code 1 in libc-2.33.so[3fff8b799000+1fe000] >> systemd[1]: code: 60000000 38600006 9122b7d0 4801bf19 60000000 60000000 8122b7d0 2c090004 >> systemd[1]: code: 40820014 39200005 60000000 9122b7d0 <00000000> 60000000 8122b7d0 2c090005 >> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 >> Rebooting in 180 seconds.. > > Can you make the hdd_debian_ppc64_new.img available? We won't be able to > reproduce the exact same scenario because we can't run KVM, but if it > boots with TCG we can at least look around the code that is failing to > see if it gives us any clues. Based on previous info it may just be a Debian install but pointing to the exact version or installer to reproduce it would help if the image cannot be made available. Regards, BALATON Zoltan
>From: "Fabiano Rosas" farosas@linux.ibm.com >To: "mario@locati.it" mario@locati.it, clg@kaod.org >Cc: danielhb413@gmail.com, qemu-ppc@nongnu.org, qemu-devel@nongnu.org, balaton@eik.bme.hu >Date: Mon, 27 Dec 2021 17:05:46 -0300 >Subject: Re: [PATCH] target/ppc: Fix e6500 boot > >"mario@locati.it" <mario@locati.it> writes: > >> I have updated the guest VM but I get exactly the same result except that now I have libc-2.33.so installed. >> >> [...] >> VFS: Mounted root (ext4 filesystem) on device 254:0. >> devtmpfs: mounted >> Freeing unused kernel image (initmem) memory: 468K >> This architecture does not have kernel memory protection. >> Run /sbin/init as init process >> random: fast init done >> systemd[1]: illegal instruction (4) at 3fff8b7e615c nip 3fff8b7e615c lr 3fff8b7e613c code 1 in libc-2.33.so[3fff8b799000+1fe000] >> systemd[1]: code: 60000000 38600006 9122b7d0 4801bf19 60000000 60000000 8122b7d0 2c090004 >> systemd[1]: code: 40820014 39200005 60000000 9122b7d0 <00000000> 60000000 8122b7d0 2c090005 >> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 >> Rebooting in 180 seconds.. > >Can you make the hdd_debian_ppc64_new.img available? We won't be able to >reproduce the exact same scenario because we can't run KVM, but if it >boots with TCG we can at least look around the code that is failing to >see if it gives us any clues. Sure, you may download the hdd of the VM I am using on the NXP T2080RDB from https://repo.powerprogress.org/t2080rdb/qemu/hdd_debian_sid_ppc64.qcow2 I have compressed the original raw disk (18.8GB) into a smaller qcow2 (3.8GB), I cannot get a smaller image sorry. Use the following kernel to launch it https://repo.powerprogress.org/t2080rdb/qemu/uImage_5.16-rc6 No matter which video output I try, but on my ubuntu 20.04 x86_64 I cannot get X11 working, so to launch qemu I use qemu-system-ppc64 -accel tcg -M ppce500 -cpu e6500 -smp 4 -m 2G -vga none -nographic -net nic -net user -drive format=qcow2,file=hdd_debian_sid_ppc64.qcow2,index=0,if=virtio -kernel uImage_5.16-rc6 -append "root=/dev/vda rw" that do now work qemu-system-ppc64 -accel tcg -M ppce500 -cpu e5500 -smp 4 -m 2G -vga none -nographic -net nic -net user -drive format=qcow2,file=hdd_debian_sid_ppc64.qcow2,index=0,if=virtio -kernel uImage_5.16-rc6 -append "root=/dev/vda rw" that works, the system sees e5500 cores, whereas running it on the devkit, the system see the host e6500 cores
Hello Mario, > > I have tried to launch a freshly compiled qemu from git master on a NXP T2080RDB devkit that has a e6500 CPU in combination with a freshly compiled kernel 5.16-rc6 > I have Debian SID ppc64 up and running using such a kernel, and when I launch qemu to run a VM with the same debian sid for ppc64 and the same kernel using --enable-kvm I end up with a kernel panic > > [....] > Run /sbin/init as init process > random: fast init done > systemd[1]: illegal instruction (4) at 3fff96562ac8 nip 3fff96562ac8 lr 3fff96562aa8 code 1 in libc-2.32.so[3fff96516000+1f7000] > systemd[1]: code: 60000000 38600006 9122b7e8 4801bead 60000000 60000000 8122b7e8 2c090004 > systemd[1]: code: 40820014 39200005 60000000 9122b7e8 <00000000> 60000000 8122b7e8 2c090005 > Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 > Rebooting in 180 seconds.. I spend sometime looking at the kernel and QEMU and I noticed that TCG e6500 machines have an issue when the kernel has KVM support. I have limited knowledge on that topic but e6500 has extra MMU features that an hypervisor could use for the guest and the e6500 emulation clearly doesn't have support for it. In the guest, could you try with to run the KVM guest with a kernel without KVM support and let us know ? Thanks, C.
>From: "Cédric Le Goater" clg@kaod.org >To: "mario@locati.it" mario@locati.it, balaton@eik.bme.hu >Cc: farosas@linux.ibm.com, qemu-devel@nongnu.org, qemu-ppc@nongnu.org, danielhb413@gmail.com >Date: Mon, 10 Jan 2022 09:04:39 +0100 >Subject: Re: [PATCH] target/ppc: Fix e6500 boot > >Hello Mario, > >> >> I have tried to launch a freshly compiled qemu from git master on a NXP T2080RDB devkit that has a e6500 CPU in combination with a freshly compiled kernel 5.16-rc6 >> I have Debian SID ppc64 up and running using such a kernel, and when I launch qemu to run a VM with the same debian sid for ppc64 and the same kernel using --enable-kvm I end up with a kernel panic >> >> [....] >> Run /sbin/init as init process >> random: fast init done >> systemd[1]: illegal instruction (4) at 3fff96562ac8 nip 3fff96562ac8 lr 3fff96562aa8 code 1 in libc-2.32.so[3fff96516000+1f7000] >> systemd[1]: code: 60000000 38600006 9122b7e8 4801bead 60000000 60000000 8122b7e8 2c090004 >> systemd[1]: code: 40820014 39200005 60000000 9122b7e8 <00000000> 60000000 8122b7e8 2c090005 >> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 >> Rebooting in 180 seconds.. > >I spend sometime looking at the kernel and QEMU and I noticed that >TCG e6500 machines have an issue when the kernel has KVM support. >I have limited knowledge on that topic but e6500 has extra MMU >features that an hypervisor could use for the guest and the e6500 >emulation clearly doesn't have support for it. > >In the guest, could you try with to run the KVM guest with a kernel >without KVM support and let us know ? > >Thanks, > >C. Unfortunately using a kernel with KVM disabled on the guest did not solve the problem on the NXP T2080RDB devkit, here's how qemu log ends [...] Loading compiled-in X.509 certificates zswap: loaded using pool lzo/zbud Key type ._fscrypt registered Key type .fscrypt registered Key type fscrypt-provisioning registered Btrfs loaded, crc32c=crc32c-generic, zoned=no, fsverity=no Key type encrypted registered ALSA device list: #0: Virtual MIDI Card 1 EXT4-fs (vda): recovery complete EXT4-fs (vda): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none. VFS: Mounted root (ext4 filesystem) on device 254:0. devtmpfs: mounted Freeing unused kernel image (initmem) memory: 460K This architecture does not have kernel memory protection. Run /sbin/init as init process random: fast init done systemd[1]: illegal instruction (4) at 3fffb412415c nip 3fffb412415c lr 3fffb412413c code 1 in libc-2.33.so[3fffb40d7000+1fe000] systemd[1]: code: 60000000 38600006 9122b7d0 4801bf19 60000000 60000000 8122b7d0 2c090004 systemd[1]: code: 40820014 39200005 60000000 9122b7d0 <00000000> 60000000 8122b7d0 2c090005 Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Rebooting in 180 seconds.. However, when using such a kernel without KVM on my x86_64 linux machine, qemu works just fine with an e6500 cpu using the following parameters qemu-system-ppc64 -M ppce500 -cpu e6500 -m 2G -kernel uImage_5.16_no_kvm -append "root=/dev/vda rw" -drive format=qcow2,file=hdd_debian_sid_ppc64.qcow2,index=0,if=virtio -net nic -net user -vga none -nographic Here the recompiled kernel 5.16 configuration used on the host that has KVM enable https://repo.powerprogress.org/t2080rdb/qemu/2022-01-11_kernel_5.16_with_kvm.config And here the compiled kernel https://repo.powerprogress.org/t2080rdb/qemu/uImage_5.16_with_kvm Here the recompiled kernel 5.16 configuration without KVM enabled for the guest https://repo.powerprogress.org/t2080rdb/qemu/2022-01-11_kernel_5.16_no_kvm.config And here the compiled kernel https://repo.powerprogress.org/t2080rdb/qemu/uImage_5.16_no_kvm Here the complete log of QEMU I have just recompiled today from git https://repo.powerprogress.org/t2080rdb/qemu/2022-01-11_qemu_git_log_kernel_5.16_no_kvm_guest.txt Cédric, I can provide you remote access (either ssh or remote desktop) to the computer with the NXP T2080 cpu for doing some debugging if you want, just drop me a private email and we will arrange the time for a session.
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c index 6695985e9b..d8efcb24ed 100644 --- a/target/ppc/cpu_init.c +++ b/target/ppc/cpu_init.c @@ -2273,8 +2273,14 @@ static void init_excp_e200(CPUPPCState *env, target_ulong ivpr_mask) env->excp_vectors[POWERPC_EXCP_DTLB] = 0x00000000; env->excp_vectors[POWERPC_EXCP_ITLB] = 0x00000000; env->excp_vectors[POWERPC_EXCP_DEBUG] = 0x00000000; + /* + * These two are the same IVOR as POWERPC_EXCP_VPU and + * POWERPC_EXCP_VPUA. We deal with that when dispatching at + * powerpc_excp(). + */ env->excp_vectors[POWERPC_EXCP_SPEU] = 0x00000000; env->excp_vectors[POWERPC_EXCP_EFPDI] = 0x00000000; + env->excp_vectors[POWERPC_EXCP_EFPRI] = 0x00000000; env->ivor_mask = 0x0000FFF7UL; env->ivpr_mask = ivpr_mask; diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c index 17607adbe4..7bb170f440 100644 --- a/target/ppc/excp_helper.c +++ b/target/ppc/excp_helper.c @@ -344,6 +344,16 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) excp = POWERPC_EXCP_PROGRAM; } +#ifdef TARGET_PPC64 + /* + * SPEU and VPU share the same IVOR but they exist in different + * processors. SPEU is e500v1/2 only and VPU is e6500 only. + */ + if (excp_model == POWERPC_EXCP_BOOKE && excp == POWERPC_EXCP_VPU) { + excp = POWERPC_EXCP_SPEU; + } +#endif + switch (excp) { case POWERPC_EXCP_NONE: /* Should never happen */ @@ -569,7 +579,7 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int excp_model, int excp) cpu_abort(cs, "Debug exception triggered on unsupported model\n"); } break; - case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable */ + case POWERPC_EXCP_SPEU: /* SPE/embedded floating-point unavailable/VPU */ env->spr[SPR_BOOKE_ESR] = ESR_SPV; break; case POWERPC_EXCP_EFPDI: /* Embedded floating-point data interrupt */
When Altivec support was added to the e6500 kernel in 2012[1], the QEMU code was not changed, so we don't register the VPU/VPUA exceptions for the e6500: qemu: fatal: Raised an exception without defined vector 73 Note that the error message says 73, instead of 32, which is the IVOR for VPU. This is because QEMU knows only knows about the VPU interrupt for the 7400s. In theory, we should not be raising _that_ VPU interrupt, but instead another one specific for the e6500. We unfortunately cannot register e6500-specific VPU/VPUA interrupts because the SPEU/EFPDI interrupts also use IVOR32/33. These are present only in the e500v1/2 versions. From the user manual: e500v1, e500v2: only SPEU/EFPDI/EFPRI e500mc, e5500: no SPEU/EFPDI/EFPRI/VPU/VPUA e6500: only VPU/VPUA So I'm leaving IVOR32/33 as SPEU/EFPDI, but altering the dispatch code to convert the VPU #73 to a #32 when we're in the e6500. Since the handling for SPEU and VPU is the same this is the only change that's needed. The EFPDI is not implemented and will cause an abort. I don't think it worth it changing the error message to take VPUA into consideration, so I'm not changing anything there. This bug was discussed in the thread: https://lists.gnu.org/archive/html/qemu-ppc/2021-06/msg00222.html 1- https://git.kernel.org/torvalds/c/cd66cc2ee52 Reported-by: <mario@locati.it> Signed-off-by: Fabiano Rosas <farosas@linux.ibm.com> --- target/ppc/cpu_init.c | 6 ++++++ target/ppc/excp_helper.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)