diff mbox series

[kvm-unit-tests,03/32] powerpc: Fix stack backtrace termination

Message ID 20240226101218.1472843-4-npiggin@gmail.com (mailing list archive)
State New, archived
Headers show
Series powerpc improvements | expand

Commit Message

Nicholas Piggin Feb. 26, 2024, 10:11 a.m. UTC
The backtrace handler terminates when it sees a NULL caller address,
but the powerpc stack setup does not keep such a NULL caller frame
at the start of the stack.

This happens to work on pseries because the memory at 0 is mapped and
it contains 0 at the location of the return address pointer if it
were a stack frame. But this is fragile, and does not work with powernv
where address 0 contains firmware instructions.

Use the existing dummy frame on stack as the NULL caller, and create a
new frame on stack for the entry code.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 powerpc/cstart64.S | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

Comments

Thomas Huth Feb. 27, 2024, 8:50 a.m. UTC | #1
On 26/02/2024 11.11, Nicholas Piggin wrote:
> The backtrace handler terminates when it sees a NULL caller address,
> but the powerpc stack setup does not keep such a NULL caller frame
> at the start of the stack.
> 
> This happens to work on pseries because the memory at 0 is mapped and
> it contains 0 at the location of the return address pointer if it
> were a stack frame. But this is fragile, and does not work with powernv
> where address 0 contains firmware instructions.
> 
> Use the existing dummy frame on stack as the NULL caller, and create a
> new frame on stack for the entry code.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   powerpc/cstart64.S | 12 ++++++++++--
>   1 file changed, 10 insertions(+), 2 deletions(-)

Thanks for tackling this! ... however, not doing powerpc work since years 
anymore, I have some ignorant questions below...

> diff --git a/powerpc/cstart64.S b/powerpc/cstart64.S
> index e18ae9a22..14ab0c6c8 100644
> --- a/powerpc/cstart64.S
> +++ b/powerpc/cstart64.S
> @@ -46,8 +46,16 @@ start:
>   	add	r1, r1, r31
>   	add	r2, r2, r31
>   
> +	/* Zero backpointers in initial stack frame so backtrace() stops */
> +	li	r0,0
> +	std	r0,0(r1)

0(r1) is the back chain pointer ...

> +	std	r0,16(r1)

... but what is 16(r1) ? I suppose that should be the "LR save word" ? But 
isn't that at 8(r1) instead?? (not sure whether I'm looking at the right ELF 
abi spec right now...)

Anyway, a comment in the source would be helpful here.

> +
> +	/* Create entry frame */
> +	stdu	r1,-INT_FRAME_SIZE(r1)

Since we already create an initial frame via stackptr from powerpc/flat.lds, 
do we really need to create this additional one here? Or does the one from 
flat.lds have to be completely empty, i.e. also no DTB pointer in it?

>   	/* save DTB pointer */
> -	std	r3, 56(r1)
> +	SAVE_GPR(3,r1)

Isn't SAVE_GPR rather meant for the interrupt frame, not for the normal C 
calling convention frames?

Sorry for asking dumb questions ... I still have a hard time understanding 
the changes here... :-/

>   	/*
>   	 * Call relocate. relocate is C code, but careful to not use
> @@ -101,7 +109,7 @@ start:
>   	stw	r4, 0(r3)
>   
>   	/* complete setup */
> -1:	ld	r3, 56(r1)
> +1:	REST_GPR(3, r1)
>   	bl	setup
>   
>   	/* run the test */

  Thomas
Thomas Huth March 1, 2024, 9:45 a.m. UTC | #2
On 27/02/2024 09.50, Thomas Huth wrote:
> On 26/02/2024 11.11, Nicholas Piggin wrote:
>> The backtrace handler terminates when it sees a NULL caller address,
>> but the powerpc stack setup does not keep such a NULL caller frame
>> at the start of the stack.
>>
>> This happens to work on pseries because the memory at 0 is mapped and
>> it contains 0 at the location of the return address pointer if it
>> were a stack frame. But this is fragile, and does not work with powernv
>> where address 0 contains firmware instructions.
>>
>> Use the existing dummy frame on stack as the NULL caller, and create a
>> new frame on stack for the entry code.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>>   powerpc/cstart64.S | 12 ++++++++++--
>>   1 file changed, 10 insertions(+), 2 deletions(-)
> 
> Thanks for tackling this! ... however, not doing powerpc work since years 
> anymore, I have some ignorant questions below...
> 
>> diff --git a/powerpc/cstart64.S b/powerpc/cstart64.S
>> index e18ae9a22..14ab0c6c8 100644
>> --- a/powerpc/cstart64.S
>> +++ b/powerpc/cstart64.S
>> @@ -46,8 +46,16 @@ start:
>>       add    r1, r1, r31
>>       add    r2, r2, r31
>> +    /* Zero backpointers in initial stack frame so backtrace() stops */
>> +    li    r0,0
>> +    std    r0,0(r1)
> 
> 0(r1) is the back chain pointer ...
> 
>> +    std    r0,16(r1)
> 
> ... but what is 16(r1) ? I suppose that should be the "LR save word" ? But 
> isn't that at 8(r1) instead?? (not sure whether I'm looking at the right ELF 
> abi spec right now...)

Ok, I was looking at the wrong ELF spec, indeed (it was an ancient 32-bit 
spec, not the 64-bit ABI). Sorry for the confusion. Having a proper #define 
or a comment for the 16 here would still be helpful, though.

  Thomas
Nicholas Piggin March 5, 2024, 2:08 a.m. UTC | #3
On Fri Mar 1, 2024 at 7:45 PM AEST, Thomas Huth wrote:
> On 27/02/2024 09.50, Thomas Huth wrote:
> > On 26/02/2024 11.11, Nicholas Piggin wrote:
> >> The backtrace handler terminates when it sees a NULL caller address,
> >> but the powerpc stack setup does not keep such a NULL caller frame
> >> at the start of the stack.
> >>
> >> This happens to work on pseries because the memory at 0 is mapped and
> >> it contains 0 at the location of the return address pointer if it
> >> were a stack frame. But this is fragile, and does not work with powernv
> >> where address 0 contains firmware instructions.
> >>
> >> Use the existing dummy frame on stack as the NULL caller, and create a
> >> new frame on stack for the entry code.
> >>
> >> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> >> ---
> >>   powerpc/cstart64.S | 12 ++++++++++--
> >>   1 file changed, 10 insertions(+), 2 deletions(-)
> > 
> > Thanks for tackling this! ... however, not doing powerpc work since years 
> > anymore, I have some ignorant questions below...
> > 
> >> diff --git a/powerpc/cstart64.S b/powerpc/cstart64.S
> >> index e18ae9a22..14ab0c6c8 100644
> >> --- a/powerpc/cstart64.S
> >> +++ b/powerpc/cstart64.S
> >> @@ -46,8 +46,16 @@ start:
> >>       add    r1, r1, r31
> >>       add    r2, r2, r31
> >> +    /* Zero backpointers in initial stack frame so backtrace() stops */
> >> +    li    r0,0
> >> +    std    r0,0(r1)
> > 
> > 0(r1) is the back chain pointer ...
> > 
> >> +    std    r0,16(r1)
> > 
> > ... but what is 16(r1) ? I suppose that should be the "LR save word" ? But 
> > isn't that at 8(r1) instead?? (not sure whether I'm looking at the right ELF 
> > abi spec right now...)
>
> Ok, I was looking at the wrong ELF spec, indeed (it was an ancient 32-bit 
> spec, not the 64-bit ABI). Sorry for the confusion. Having a proper #define 
> or a comment for the 16 here would still be helpful, though.

Thanks for the deailed reviews as always. I've been a little busy with
QEMU so may not get another series out for a bit. I'll probably wait for
Andrew's stack backtrace changes to land too before resend.

But, yes a comment makes sense. I'll add.

Thanks,
Nick
Nicholas Piggin March 5, 2024, 6:29 a.m. UTC | #4
On Tue Feb 27, 2024 at 6:50 PM AEST, Thomas Huth wrote:
> On 26/02/2024 11.11, Nicholas Piggin wrote:
> > The backtrace handler terminates when it sees a NULL caller address,
> > but the powerpc stack setup does not keep such a NULL caller frame
> > at the start of the stack.
> > 
> > This happens to work on pseries because the memory at 0 is mapped and
> > it contains 0 at the location of the return address pointer if it
> > were a stack frame. But this is fragile, and does not work with powernv
> > where address 0 contains firmware instructions.
> > 
> > Use the existing dummy frame on stack as the NULL caller, and create a
> > new frame on stack for the entry code.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   powerpc/cstart64.S | 12 ++++++++++--
> >   1 file changed, 10 insertions(+), 2 deletions(-)
>
> Thanks for tackling this! ... however, not doing powerpc work since years 
> anymore, I have some ignorant questions below...
>
> > diff --git a/powerpc/cstart64.S b/powerpc/cstart64.S
> > index e18ae9a22..14ab0c6c8 100644
> > --- a/powerpc/cstart64.S
> > +++ b/powerpc/cstart64.S
> > @@ -46,8 +46,16 @@ start:
> >   	add	r1, r1, r31
> >   	add	r2, r2, r31
> >   
> > +	/* Zero backpointers in initial stack frame so backtrace() stops */
> > +	li	r0,0
> > +	std	r0,0(r1)
>
> 0(r1) is the back chain pointer ...
>
> > +	std	r0,16(r1)
>
> ... but what is 16(r1) ? I suppose that should be the "LR save word" ? But 
> isn't that at 8(r1) instead?? (not sure whether I'm looking at the right ELF 
> abi spec right now...)
>
> Anyway, a comment in the source would be helpful here.
>
> > +
> > +	/* Create entry frame */
> > +	stdu	r1,-INT_FRAME_SIZE(r1)
>
> Since we already create an initial frame via stackptr from powerpc/flat.lds, 
> do we really need to create this additional one here? Or does the one from 
> flat.lds have to be completely empty, i.e. also no DTB pointer in it?

Oh you already figured the above questions. For this, we do have
one frame allocated already statically yes. But if we don't create
another one here then our callee will store LR into it, but the
unwinder only exits when it sees a NULL return address so it would
keep trying to walk.

We could make it terminate on NULL back chain pointer, but that's
a bit more change that also touches non-powerpc code in the generic
unwinder, and still needs some changes here. Maybe we should do
that after this series though. I'll include a comment to look at
redoing it later.

>
> >   	/* save DTB pointer */
> > -	std	r3, 56(r1)
> > +	SAVE_GPR(3,r1)
>
> Isn't SAVE_GPR rather meant for the interrupt frame, not for the normal C 
> calling convention frames?
>
> Sorry for asking dumb questions ... I still have a hard time understanding 
> the changes here... :-/

Ah, that was me being lazy and using an interrupt frame for the new
frame.

Thanks,
Nick

>
> >   	/*
> >   	 * Call relocate. relocate is C code, but careful to not use
> > @@ -101,7 +109,7 @@ start:
> >   	stw	r4, 0(r3)
> >   
> >   	/* complete setup */
> > -1:	ld	r3, 56(r1)
> > +1:	REST_GPR(3, r1)
> >   	bl	setup
> >   
> >   	/* run the test */
>
>   Thomas
Thomas Huth March 5, 2024, 6:59 a.m. UTC | #5
On 05/03/2024 07.29, Nicholas Piggin wrote:
> On Tue Feb 27, 2024 at 6:50 PM AEST, Thomas Huth wrote:
>> On 26/02/2024 11.11, Nicholas Piggin wrote:
...
>>>    	/* save DTB pointer */
>>> -	std	r3, 56(r1)
>>> +	SAVE_GPR(3,r1)
>>
>> Isn't SAVE_GPR rather meant for the interrupt frame, not for the normal C
>> calling convention frames?
>>
>> Sorry for asking dumb questions ... I still have a hard time understanding
>> the changes here... :-/
> 
> Ah, that was me being lazy and using an interrupt frame for the new
> frame.

Ah, ok. It's super-confusing (at least for me) to see an interrupt frame 
here out of no reason... could you please either add proper comments here 
explaining this, or even better switch to a normal stack frame, please?

  Thanks,
   Thomas
diff mbox series

Patch

diff --git a/powerpc/cstart64.S b/powerpc/cstart64.S
index e18ae9a22..14ab0c6c8 100644
--- a/powerpc/cstart64.S
+++ b/powerpc/cstart64.S
@@ -46,8 +46,16 @@  start:
 	add	r1, r1, r31
 	add	r2, r2, r31
 
+	/* Zero backpointers in initial stack frame so backtrace() stops */
+	li	r0,0
+	std	r0,0(r1)
+	std	r0,16(r1)
+
+	/* Create entry frame */
+	stdu	r1,-INT_FRAME_SIZE(r1)
+
 	/* save DTB pointer */
-	std	r3, 56(r1)
+	SAVE_GPR(3,r1)
 
 	/*
 	 * Call relocate. relocate is C code, but careful to not use
@@ -101,7 +109,7 @@  start:
 	stw	r4, 0(r3)
 
 	/* complete setup */
-1:	ld	r3, 56(r1)
+1:	REST_GPR(3, r1)
 	bl	setup
 
 	/* run the test */