diff mbox

Commit f5d9b7f0f9 (fix r600_enable_sclk_control()) causes kexec issues

Message ID CADnq5_OGVuYz3To8vmfgK_BjVT3hU371cank-AH6SrOws-99yQ@mail.gmail.com (mailing list archive)
State New, archived
Headers show

Commit Message

Alex Deucher July 29, 2013, 1:58 p.m. UTC
On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
<markus@trippelsdorf.de> wrote:
> On my test machine Xorg doesn't start anymore when I kexec into a
> 3.11.0-rc3 kernel.

With kexec, dpm doesn't get torn down properly which can result in a
bad hardware state when the driver loads again.  Does the attached
patch help?  It attempts to disable dpm at startup in case it wasn't
torn down properly previously.

Alex

Comments

Markus Trippelsdorf July 29, 2013, 2:09 p.m. UTC | #1
On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
> <markus@trippelsdorf.de> wrote:
> > On my test machine Xorg doesn't start anymore when I kexec into a
> > 3.11.0-rc3 kernel.
> 
> With kexec, dpm doesn't get torn down properly which can result in a
> bad hardware state when the driver loads again.  Does the attached
> patch help?  It attempts to disable dpm at startup in case it wasn't
> torn down properly previously.

dpm initialization now works, but unfortunately GPU acceleration still gets
disabled:

[drm] Initialized drm 1.1.0 20060810                                                                                                                                [135/1104]
[drm] radeon kernel modesetting enabled.
[drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D).
[drm] register mmio base: 0xFBEE0000
[drm] register mmio size: 65536
ATOM BIOS: 113
radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used)
radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF
[drm] Detected VRAM RAM=128M, BAR=128M
[drm] RAM width 32bits DDR
[TTM] Zone  kernel: Available graphics memory: 4082356 kiB
[TTM] Zone   dma32: Available graphics memory: 2097152 kiB
[TTM] Initializing pool allocator
[TTM] Initializing DMA pool allocator
[drm] radeon: 128M of VRAM memory ready
[drm] radeon: 512M of GTT memory ready.
[drm] GART: num cpu pages 131072, num gpu pages 131072
[drm] Loading RS780 Microcode
[drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).
radeon 0000:01:05.0: WB enabled
radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00
radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
[drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
[drm] Driver supports precise vblank timestamp query.
[drm] radeon: irq initialized.
radeon 0000:01:05.0: setting latency timer to 64
[drm] ring test on 0 succeeded in 1 usecs
[drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD)
radeon 0000:01:05.0: disabling GPU acceleration
radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary
[drm] Radeon Display Connectors
[drm] Connector 0:
[drm]   VGA-1
[drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
[drm]   Encoders:
[drm]     CRT1: INTERNAL_KLDSCP_DAC1
[drm] Connector 1:
[drm]   DVI-D-1
[drm]   HPD3
[drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
[drm]   Encoders:
[drm]     DFP3: INTERNAL_KLDSCP_LVTMA
== power state 0 ==
        ui class: none
        internal class: boot 
        caps: video 
        uvd    vclk: 0 dclk: 0
                power level 0    sclk: 50000 vddc_index: 2
                power level 1    sclk: 50000 vddc_index: 2
        status: c r b 
== power state 1 ==
        ui class: performance
        internal class: none
        caps: video 
        uvd    vclk: 0 dclk: 0
                power level 0    sclk: 50000 vddc_index: 1
                power level 1    sclk: 70000 vddc_index: 2
        status: 
== power state 2 ==
        ui class: none
        internal class: uvd 
        caps: video 
        uvd    vclk: 53300 dclk: 40000
                power level 0    sclk: 50000 vddc_index: 1
                power level 1    sclk: 50000 vddc_index: 1
        status: 
switching from power state:
        ui class: none
        internal class: boot 
        caps: video 
        uvd    vclk: 0 dclk: 0
                power level 0    sclk: 50000 vddc_index: 2
                power level 1    sclk: 50000 vddc_index: 2
        status: c b 
switching to power state:
        ui class: performance
        internal class: none
        caps: video 
        uvd    vclk: 0 dclk: 0
                power level 0    sclk: 50000 vddc_index: 1
                power level 1    sclk: 70000 vddc_index: 2
        status: r 
[drm] radeon: dpm initialized
[drm] fb mappable at 0xF0142000
[drm] vram apper at 0xF0000000
[drm] size 7299072
[drm] fb depth is 24
[drm]    pitch is 6912
fbcon: radeondrmfb (fb0)
Alex Deucher July 29, 2013, 2:18 p.m. UTC | #2
On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
<markus@trippelsdorf.de> wrote:
> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>> <markus@trippelsdorf.de> wrote:
>> > On my test machine Xorg doesn't start anymore when I kexec into a
>> > 3.11.0-rc3 kernel.
>>
>> With kexec, dpm doesn't get torn down properly which can result in a
>> bad hardware state when the driver loads again.  Does the attached
>> patch help?  It attempts to disable dpm at startup in case it wasn't
>> torn down properly previously.
>
> dpm initialization now works, but unfortunately GPU acceleration still gets
> disabled:

Stupid kexec complicates things.  We need to make sure dpm is torn
down before we init the rest of the GPU, but dpm needs get initialized
later in the init process since it depends on certain other state from
the driver.  I need to think about this for a bit.  I'm not sure of a
good way to handle this.

Alex

>
> [drm] Initialized drm 1.1.0 20060810                                                                                                                                [135/1104]
> [drm] radeon kernel modesetting enabled.
> [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D).
> [drm] register mmio base: 0xFBEE0000
> [drm] register mmio size: 65536
> ATOM BIOS: 113
> radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used)
> radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF
> [drm] Detected VRAM RAM=128M, BAR=128M
> [drm] RAM width 32bits DDR
> [TTM] Zone  kernel: Available graphics memory: 4082356 kiB
> [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
> [TTM] Initializing pool allocator
> [TTM] Initializing DMA pool allocator
> [drm] radeon: 128M of VRAM memory ready
> [drm] radeon: 512M of GTT memory ready.
> [drm] GART: num cpu pages 131072, num gpu pages 131072
> [drm] Loading RS780 Microcode
> [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).
> radeon 0000:01:05.0: WB enabled
> radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00
> radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
> [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
> [drm] Driver supports precise vblank timestamp query.
> [drm] radeon: irq initialized.
> radeon 0000:01:05.0: setting latency timer to 64
> [drm] ring test on 0 succeeded in 1 usecs
> [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD)
> radeon 0000:01:05.0: disabling GPU acceleration
> radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary
> [drm] Radeon Display Connectors
> [drm] Connector 0:
> [drm]   VGA-1
> [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
> [drm]   Encoders:
> [drm]     CRT1: INTERNAL_KLDSCP_DAC1
> [drm] Connector 1:
> [drm]   DVI-D-1
> [drm]   HPD3
> [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
> [drm]   Encoders:
> [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
> == power state 0 ==
>         ui class: none
>         internal class: boot
>         caps: video
>         uvd    vclk: 0 dclk: 0
>                 power level 0    sclk: 50000 vddc_index: 2
>                 power level 1    sclk: 50000 vddc_index: 2
>         status: c r b
> == power state 1 ==
>         ui class: performance
>         internal class: none
>         caps: video
>         uvd    vclk: 0 dclk: 0
>                 power level 0    sclk: 50000 vddc_index: 1
>                 power level 1    sclk: 70000 vddc_index: 2
>         status:
> == power state 2 ==
>         ui class: none
>         internal class: uvd
>         caps: video
>         uvd    vclk: 53300 dclk: 40000
>                 power level 0    sclk: 50000 vddc_index: 1
>                 power level 1    sclk: 50000 vddc_index: 1
>         status:
> switching from power state:
>         ui class: none
>         internal class: boot
>         caps: video
>         uvd    vclk: 0 dclk: 0
>                 power level 0    sclk: 50000 vddc_index: 2
>                 power level 1    sclk: 50000 vddc_index: 2
>         status: c b
> switching to power state:
>         ui class: performance
>         internal class: none
>         caps: video
>         uvd    vclk: 0 dclk: 0
>                 power level 0    sclk: 50000 vddc_index: 1
>                 power level 1    sclk: 70000 vddc_index: 2
>         status: r
> [drm] radeon: dpm initialized
> [drm] fb mappable at 0xF0142000
> [drm] vram apper at 0xF0000000
> [drm] size 7299072
> [drm] fb depth is 24
> [drm]    pitch is 6912
> fbcon: radeondrmfb (fb0)
>
> --
> Markus
Eric W. Biederman July 29, 2013, 3:50 p.m. UTC | #3
Alex Deucher <alexdeucher@gmail.com> wrote:
>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
><markus@trippelsdorf.de> wrote:
>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>>> <markus@trippelsdorf.de> wrote:
>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>>> > 3.11.0-rc3 kernel.
>>>
>>> With kexec, dpm doesn't get torn down properly which can result in a
>>> bad hardware state when the driver loads again.  Does the attached
>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>>> torn down properly previously.
>>
>> dpm initialization now works, but unfortunately GPU acceleration
>still gets
>> disabled:
>
>Stupid kexec complicates things.  We need to make sure dpm is torn
>down before we init the rest of the GPU, but dpm needs get initialized
>later in the init process since it depends on certain other state from
>the driver.  I need to think about this for a bit.  I'm not sure of a
>good way to handle this.

You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.

Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?

Eric

>Alex
>
>>
>> [drm] Initialized drm 1.1.0 20060810                                 
>                                                             [135/1104]
>> [drm] radeon kernel modesetting enabled.
>> [drm] initializing kernel modesetting (RS780 0x1002:0x9614
>0x1043:0x834D).
>> [drm] register mmio base: 0xFBEE0000
>> [drm] register mmio size: 65536
>> ATOM BIOS: 113
>> radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 -
>0x00000000C7FFFFFF (128M used)
>> radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 -
>0x00000000BFFFFFFF
>> [drm] Detected VRAM RAM=128M, BAR=128M
>> [drm] RAM width 32bits DDR
>> [TTM] Zone  kernel: Available graphics memory: 4082356 kiB
>> [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
>> [TTM] Initializing pool allocator
>> [TTM] Initializing DMA pool allocator
>> [drm] radeon: 128M of VRAM memory ready
>> [drm] radeon: 512M of GTT memory ready.
>> [drm] GART: num cpu pages 131072, num gpu pages 131072
>> [drm] Loading RS780 Microcode
>> [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).
>> radeon 0000:01:05.0: WB enabled
>> radeon 0000:01:05.0: fence driver on ring 0 use gpu addr
>0x00000000a0000c00 and cpu addr 0xffff880215c30c00
>> radeon 0000:01:05.0: fence driver on ring 3 use gpu addr
>0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
>> [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
>> [drm] Driver supports precise vblank timestamp query.
>> [drm] radeon: irq initialized.
>> radeon 0000:01:05.0: setting latency timer to 64
>> [drm] ring test on 0 succeeded in 1 usecs
>> [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed
>(0xCAFEDEAD)
>> radeon 0000:01:05.0: disabling GPU acceleration
>> radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary
>> [drm] Radeon Display Connectors
>> [drm] Connector 0:
>> [drm]   VGA-1
>> [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
>> [drm]   Encoders:
>> [drm]     CRT1: INTERNAL_KLDSCP_DAC1
>> [drm] Connector 1:
>> [drm]   DVI-D-1
>> [drm]   HPD3
>> [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
>> [drm]   Encoders:
>> [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
>> == power state 0 ==
>>         ui class: none
>>         internal class: boot
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 2
>>                 power level 1    sclk: 50000 vddc_index: 2
>>         status: c r b
>> == power state 1 ==
>>         ui class: performance
>>         internal class: none
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 1
>>                 power level 1    sclk: 70000 vddc_index: 2
>>         status:
>> == power state 2 ==
>>         ui class: none
>>         internal class: uvd
>>         caps: video
>>         uvd    vclk: 53300 dclk: 40000
>>                 power level 0    sclk: 50000 vddc_index: 1
>>                 power level 1    sclk: 50000 vddc_index: 1
>>         status:
>> switching from power state:
>>         ui class: none
>>         internal class: boot
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 2
>>                 power level 1    sclk: 50000 vddc_index: 2
>>         status: c b
>> switching to power state:
>>         ui class: performance
>>         internal class: none
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 1
>>                 power level 1    sclk: 70000 vddc_index: 2
>>         status: r
>> [drm] radeon: dpm initialized
>> [drm] fb mappable at 0xF0142000
>> [drm] vram apper at 0xF0000000
>> [drm] size 7299072
>> [drm] fb depth is 24
>> [drm]    pitch is 6912
>> fbcon: radeondrmfb (fb0)
>>
>> --
>> Markus
Joshua C. July 29, 2013, 4:14 p.m. UTC | #4
2013/7/29 Alex Deucher <alexdeucher@gmail.com>:
> On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
> <markus@trippelsdorf.de> wrote:
>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>>> <markus@trippelsdorf.de> wrote:
>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>>> > 3.11.0-rc3 kernel.
>>>
>>> With kexec, dpm doesn't get torn down properly which can result in a
>>> bad hardware state when the driver loads again.  Does the attached
>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>>> torn down properly previously.
>>
>> dpm initialization now works, but unfortunately GPU acceleration still gets
>> disabled:
>
> Stupid kexec complicates things.  We need to make sure dpm is torn
> down before we init the rest of the GPU, but dpm needs get initialized
> later in the init process since it depends on certain other state from
> the driver.  I need to think about this for a bit.  I'm not sure of a
> good way to handle this.
>
> Alex
>
>>
>> [drm] Initialized drm 1.1.0 20060810                                                                                                                                [135/1104]
>> [drm] radeon kernel modesetting enabled.
>> [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D).
>> [drm] register mmio base: 0xFBEE0000
>> [drm] register mmio size: 65536
>> ATOM BIOS: 113
>> radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used)
>> radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF
>> [drm] Detected VRAM RAM=128M, BAR=128M
>> [drm] RAM width 32bits DDR
>> [TTM] Zone  kernel: Available graphics memory: 4082356 kiB
>> [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
>> [TTM] Initializing pool allocator
>> [TTM] Initializing DMA pool allocator
>> [drm] radeon: 128M of VRAM memory ready
>> [drm] radeon: 512M of GTT memory ready.
>> [drm] GART: num cpu pages 131072, num gpu pages 131072
>> [drm] Loading RS780 Microcode
>> [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).
>> radeon 0000:01:05.0: WB enabled
>> radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00
>> radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
>> [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
>> [drm] Driver supports precise vblank timestamp query.
>> [drm] radeon: irq initialized.
>> radeon 0000:01:05.0: setting latency timer to 64
>> [drm] ring test on 0 succeeded in 1 usecs
>> [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD)
>> radeon 0000:01:05.0: disabling GPU acceleration
>> radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary
>> [drm] Radeon Display Connectors
>> [drm] Connector 0:
>> [drm]   VGA-1
>> [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
>> [drm]   Encoders:
>> [drm]     CRT1: INTERNAL_KLDSCP_DAC1
>> [drm] Connector 1:
>> [drm]   DVI-D-1
>> [drm]   HPD3
>> [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
>> [drm]   Encoders:
>> [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
>> == power state 0 ==
>>         ui class: none
>>         internal class: boot
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 2
>>                 power level 1    sclk: 50000 vddc_index: 2
>>         status: c r b
>> == power state 1 ==
>>         ui class: performance
>>         internal class: none
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 1
>>                 power level 1    sclk: 70000 vddc_index: 2
>>         status:
>> == power state 2 ==
>>         ui class: none
>>         internal class: uvd
>>         caps: video
>>         uvd    vclk: 53300 dclk: 40000
>>                 power level 0    sclk: 50000 vddc_index: 1
>>                 power level 1    sclk: 50000 vddc_index: 1
>>         status:
>> switching from power state:
>>         ui class: none
>>         internal class: boot
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 2
>>                 power level 1    sclk: 50000 vddc_index: 2
>>         status: c b
>> switching to power state:
>>         ui class: performance
>>         internal class: none
>>         caps: video
>>         uvd    vclk: 0 dclk: 0
>>                 power level 0    sclk: 50000 vddc_index: 1
>>                 power level 1    sclk: 70000 vddc_index: 2
>>         status: r
>> [drm] radeon: dpm initialized
>> [drm] fb mappable at 0xF0142000
>> [drm] vram apper at 0xF0000000
>> [drm] size 7299072
>> [drm] fb depth is 24
>> [drm]    pitch is 6912
>> fbcon: radeondrmfb (fb0)
>>
>> --
>> Markus
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

This error message seems similar to mine "[drm:r600_uvd_ring_test]
*ERROR* radeon: ring 5 test failed (0xCAFEDEAD)" Bugzilla:
https://bugs.freedesktop.org/show_bug.cgi?id=67276 In my case I blame
another commit for this. Are these bugs related?
Markus Trippelsdorf July 29, 2013, 4:35 p.m. UTC | #5
On 2013.07.29 at 18:14 +0200, Joshua C. wrote:
> 
> This error message seems similar to mine "[drm:r600_uvd_ring_test]
> *ERROR* radeon: ring 5 test failed (0xCAFEDEAD)" Bugzilla:
> https://bugs.freedesktop.org/show_bug.cgi?id=67276 In my case I blame
> another commit for this. Are these bugs related?

I guess not, because reverting commit 9cc2e0e9f13 doesn't fix the issue
for me. 
Can you check if reverting commit f5d9b7f0f9 does fixes the problem for
you?
Alex Deucher July 29, 2013, 5:58 p.m. UTC | #6
On Mon, Jul 29, 2013 at 12:14 PM, Joshua C. <joshuacov@gmail.com> wrote:
> 2013/7/29 Alex Deucher <alexdeucher@gmail.com>:
>> On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
>> <markus@trippelsdorf.de> wrote:
>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>>>> <markus@trippelsdorf.de> wrote:
>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>>>> > 3.11.0-rc3 kernel.
>>>>
>>>> With kexec, dpm doesn't get torn down properly which can result in a
>>>> bad hardware state when the driver loads again.  Does the attached
>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>>>> torn down properly previously.
>>>
>>> dpm initialization now works, but unfortunately GPU acceleration still gets
>>> disabled:
>>
>> Stupid kexec complicates things.  We need to make sure dpm is torn
>> down before we init the rest of the GPU, but dpm needs get initialized
>> later in the init process since it depends on certain other state from
>> the driver.  I need to think about this for a bit.  I'm not sure of a
>> good way to handle this.
>>
>> Alex
>>
>>>
>>> [drm] Initialized drm 1.1.0 20060810                                                                                                                                [135/1104]
>>> [drm] radeon kernel modesetting enabled.
>>> [drm] initializing kernel modesetting (RS780 0x1002:0x9614 0x1043:0x834D).
>>> [drm] register mmio base: 0xFBEE0000
>>> [drm] register mmio size: 65536
>>> ATOM BIOS: 113
>>> radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 - 0x00000000C7FFFFFF (128M used)
>>> radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 - 0x00000000BFFFFFFF
>>> [drm] Detected VRAM RAM=128M, BAR=128M
>>> [drm] RAM width 32bits DDR
>>> [TTM] Zone  kernel: Available graphics memory: 4082356 kiB
>>> [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
>>> [TTM] Initializing pool allocator
>>> [TTM] Initializing DMA pool allocator
>>> [drm] radeon: 128M of VRAM memory ready
>>> [drm] radeon: 512M of GTT memory ready.
>>> [drm] GART: num cpu pages 131072, num gpu pages 131072
>>> [drm] Loading RS780 Microcode
>>> [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).
>>> radeon 0000:01:05.0: WB enabled
>>> radeon 0000:01:05.0: fence driver on ring 0 use gpu addr 0x00000000a0000c00 and cpu addr 0xffff880215c30c00
>>> radeon 0000:01:05.0: fence driver on ring 3 use gpu addr 0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
>>> [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
>>> [drm] Driver supports precise vblank timestamp query.
>>> [drm] radeon: irq initialized.
>>> radeon 0000:01:05.0: setting latency timer to 64
>>> [drm] ring test on 0 succeeded in 1 usecs
>>> [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed (0xCAFEDEAD)
>>> radeon 0000:01:05.0: disabling GPU acceleration
>>> radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary
>>> [drm] Radeon Display Connectors
>>> [drm] Connector 0:
>>> [drm]   VGA-1
>>> [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
>>> [drm]   Encoders:
>>> [drm]     CRT1: INTERNAL_KLDSCP_DAC1
>>> [drm] Connector 1:
>>> [drm]   DVI-D-1
>>> [drm]   HPD3
>>> [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
>>> [drm]   Encoders:
>>> [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
>>> == power state 0 ==
>>>         ui class: none
>>>         internal class: boot
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 2
>>>                 power level 1    sclk: 50000 vddc_index: 2
>>>         status: c r b
>>> == power state 1 ==
>>>         ui class: performance
>>>         internal class: none
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 1
>>>                 power level 1    sclk: 70000 vddc_index: 2
>>>         status:
>>> == power state 2 ==
>>>         ui class: none
>>>         internal class: uvd
>>>         caps: video
>>>         uvd    vclk: 53300 dclk: 40000
>>>                 power level 0    sclk: 50000 vddc_index: 1
>>>                 power level 1    sclk: 50000 vddc_index: 1
>>>         status:
>>> switching from power state:
>>>         ui class: none
>>>         internal class: boot
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 2
>>>                 power level 1    sclk: 50000 vddc_index: 2
>>>         status: c b
>>> switching to power state:
>>>         ui class: performance
>>>         internal class: none
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 1
>>>                 power level 1    sclk: 70000 vddc_index: 2
>>>         status: r
>>> [drm] radeon: dpm initialized
>>> [drm] fb mappable at 0xF0142000
>>> [drm] vram apper at 0xF0000000
>>> [drm] size 7299072
>>> [drm] fb depth is 24
>>> [drm]    pitch is 6912
>>> fbcon: radeondrmfb (fb0)
>>>
>>> --
>>> Markus
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/dri-devel
>
> This error message seems similar to mine "[drm:r600_uvd_ring_test]
> *ERROR* radeon: ring 5 test failed (0xCAFEDEAD)" Bugzilla:
> https://bugs.freedesktop.org/show_bug.cgi?id=67276 In my case I blame
> another commit for this. Are these bugs related?

I doubt it's related at all.  You issue seems to be an initialization
problem with UVD.

Alex
Alex Deucher July 29, 2013, 6:02 p.m. UTC | #7
On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
<ebiederm@xmission.com> wrote:
>
>
> Alex Deucher <alexdeucher@gmail.com> wrote:
>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
>><markus@trippelsdorf.de> wrote:
>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>>>> <markus@trippelsdorf.de> wrote:
>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>>>> > 3.11.0-rc3 kernel.
>>>>
>>>> With kexec, dpm doesn't get torn down properly which can result in a
>>>> bad hardware state when the driver loads again.  Does the attached
>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>>>> torn down properly previously.
>>>
>>> dpm initialization now works, but unfortunately GPU acceleration
>>still gets
>>> disabled:
>>
>>Stupid kexec complicates things.  We need to make sure dpm is torn
>>down before we init the rest of the GPU, but dpm needs get initialized
>>later in the init process since it depends on certain other state from
>>the driver.  I need to think about this for a bit.  I'm not sure of a
>>good way to handle this.
>
> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
>
> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?

Does kexec actually call this shutdown method?  The driver implements
appropriate clean-up measures if it's shutdown properly.

Alex


>
> Eric
>
>>Alex
>>
>>>
>>> [drm] Initialized drm 1.1.0 20060810
>>                                                             [135/1104]
>>> [drm] radeon kernel modesetting enabled.
>>> [drm] initializing kernel modesetting (RS780 0x1002:0x9614
>>0x1043:0x834D).
>>> [drm] register mmio base: 0xFBEE0000
>>> [drm] register mmio size: 65536
>>> ATOM BIOS: 113
>>> radeon 0000:01:05.0: VRAM: 128M 0x00000000C0000000 -
>>0x00000000C7FFFFFF (128M used)
>>> radeon 0000:01:05.0: GTT: 512M 0x00000000A0000000 -
>>0x00000000BFFFFFFF
>>> [drm] Detected VRAM RAM=128M, BAR=128M
>>> [drm] RAM width 32bits DDR
>>> [TTM] Zone  kernel: Available graphics memory: 4082356 kiB
>>> [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
>>> [TTM] Initializing pool allocator
>>> [TTM] Initializing DMA pool allocator
>>> [drm] radeon: 128M of VRAM memory ready
>>> [drm] radeon: 512M of GTT memory ready.
>>> [drm] GART: num cpu pages 131072, num gpu pages 131072
>>> [drm] Loading RS780 Microcode
>>> [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).
>>> radeon 0000:01:05.0: WB enabled
>>> radeon 0000:01:05.0: fence driver on ring 0 use gpu addr
>>0x00000000a0000c00 and cpu addr 0xffff880215c30c00
>>> radeon 0000:01:05.0: fence driver on ring 3 use gpu addr
>>0x00000000a0000c0c and cpu addr 0xffff880215c30c0c
>>> [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
>>> [drm] Driver supports precise vblank timestamp query.
>>> [drm] radeon: irq initialized.
>>> radeon 0000:01:05.0: setting latency timer to 64
>>> [drm] ring test on 0 succeeded in 1 usecs
>>> [drm:r600_dma_ring_test] *ERROR* radeon: ring 3 test failed
>>(0xCAFEDEAD)
>>> radeon 0000:01:05.0: disabling GPU acceleration
>>> radeon 0000:01:05.0: ffff8802161cfc00 unpin not necessary
>>> [drm] Radeon Display Connectors
>>> [drm] Connector 0:
>>> [drm]   VGA-1
>>> [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
>>> [drm]   Encoders:
>>> [drm]     CRT1: INTERNAL_KLDSCP_DAC1
>>> [drm] Connector 1:
>>> [drm]   DVI-D-1
>>> [drm]   HPD3
>>> [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
>>> [drm]   Encoders:
>>> [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
>>> == power state 0 ==
>>>         ui class: none
>>>         internal class: boot
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 2
>>>                 power level 1    sclk: 50000 vddc_index: 2
>>>         status: c r b
>>> == power state 1 ==
>>>         ui class: performance
>>>         internal class: none
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 1
>>>                 power level 1    sclk: 70000 vddc_index: 2
>>>         status:
>>> == power state 2 ==
>>>         ui class: none
>>>         internal class: uvd
>>>         caps: video
>>>         uvd    vclk: 53300 dclk: 40000
>>>                 power level 0    sclk: 50000 vddc_index: 1
>>>                 power level 1    sclk: 50000 vddc_index: 1
>>>         status:
>>> switching from power state:
>>>         ui class: none
>>>         internal class: boot
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 2
>>>                 power level 1    sclk: 50000 vddc_index: 2
>>>         status: c b
>>> switching to power state:
>>>         ui class: performance
>>>         internal class: none
>>>         caps: video
>>>         uvd    vclk: 0 dclk: 0
>>>                 power level 0    sclk: 50000 vddc_index: 1
>>>                 power level 1    sclk: 70000 vddc_index: 2
>>>         status: r
>>> [drm] radeon: dpm initialized
>>> [drm] fb mappable at 0xF0142000
>>> [drm] vram apper at 0xF0000000
>>> [drm] size 7299072
>>> [drm] fb depth is 24
>>> [drm]    pitch is 6912
>>> fbcon: radeondrmfb (fb0)
>>>
>>> --
>>> Markus
>
Eric W. Biederman July 29, 2013, 6:10 p.m. UTC | #8
Alex Deucher <alexdeucher@gmail.com> writes:

> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
> <ebiederm@xmission.com> wrote:
>>
>>
>> Alex Deucher <alexdeucher@gmail.com> wrote:
>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
>>><markus@trippelsdorf.de> wrote:
>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>>>>> <markus@trippelsdorf.de> wrote:
>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>>>>> > 3.11.0-rc3 kernel.
>>>>>
>>>>> With kexec, dpm doesn't get torn down properly which can result in a
>>>>> bad hardware state when the driver loads again.  Does the attached
>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>>>>> torn down properly previously.
>>>>
>>>> dpm initialization now works, but unfortunately GPU acceleration
>>>still gets
>>>> disabled:
>>>
>>>Stupid kexec complicates things.  We need to make sure dpm is torn
>>>down before we init the rest of the GPU, but dpm needs get initialized
>>>later in the init process since it depends on certain other state from
>>>the driver.  I need to think about this for a bit.  I'm not sure of a
>>>good way to handle this.
>>
>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
>>
>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
>
> Does kexec actually call this shutdown method?  The driver implements
> appropriate clean-up measures if it's shutdown properly.

Absoltuely.  All parts of the reboot path call ->shutdown.  Including
kexec.

You don't get a device remove/hotunplug but unless this is a kexec on
panic ->shutdown is most definitely called.  Now I am talking about the
device layer/pci layer shutdown method I don't know how gpu drivers are
wired up.  GPU land was a little strange last I looked.  Hopefully it
isn't so strange that there is a method named shutdown that is not wired
up.

Eric
Alex Deucher July 29, 2013, 7:53 p.m. UTC | #9
On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
<ebiederm@xmission.com> wrote:
> Alex Deucher <alexdeucher@gmail.com> writes:
>
>> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
>> <ebiederm@xmission.com> wrote:
>>>
>>>
>>> Alex Deucher <alexdeucher@gmail.com> wrote:
>>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
>>>><markus@trippelsdorf.de> wrote:
>>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>>>>>> <markus@trippelsdorf.de> wrote:
>>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>>>>>> > 3.11.0-rc3 kernel.
>>>>>>
>>>>>> With kexec, dpm doesn't get torn down properly which can result in a
>>>>>> bad hardware state when the driver loads again.  Does the attached
>>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>>>>>> torn down properly previously.
>>>>>
>>>>> dpm initialization now works, but unfortunately GPU acceleration
>>>>still gets
>>>>> disabled:
>>>>
>>>>Stupid kexec complicates things.  We need to make sure dpm is torn
>>>>down before we init the rest of the GPU, but dpm needs get initialized
>>>>later in the init process since it depends on certain other state from
>>>>the driver.  I need to think about this for a bit.  I'm not sure of a
>>>>good way to handle this.
>>>
>>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
>>>
>>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
>>
>> Does kexec actually call this shutdown method?  The driver implements
>> appropriate clean-up measures if it's shutdown properly.
>
> Absoltuely.  All parts of the reboot path call ->shutdown.  Including
> kexec.
>
> You don't get a device remove/hotunplug but unless this is a kexec on
> panic ->shutdown is most definitely called.  Now I am talking about the
> device layer/pci layer shutdown method I don't know how gpu drivers are
> wired up.  GPU land was a little strange last I looked.  Hopefully it
> isn't so strange that there is a method named shutdown that is not wired
> up.

It doesn't look like the drm infrastructure has a shutdown callback.
The drm drivers register a drm_driver callback struct that includes an
unload callback which takes care of all the device teardown (if you
unload the module for example).  I don't know that it actually gets
called at kexec time however.  I don't know enough about how kexec
works.

Markus, does everything work ok after a reboot?  Is it just kexec that
is a problem?

Alex
Markus Trippelsdorf July 29, 2013, 7:58 p.m. UTC | #10
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
> On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
> <ebiederm@xmission.com> wrote:
> > Alex Deucher <alexdeucher@gmail.com> writes:
> >
> >> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
> >> <ebiederm@xmission.com> wrote:
> >>>
> >>>
> >>> Alex Deucher <alexdeucher@gmail.com> wrote:
> >>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
> >>>><markus@trippelsdorf.de> wrote:
> >>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
> >>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
> >>>>>> <markus@trippelsdorf.de> wrote:
> >>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
> >>>>>> > 3.11.0-rc3 kernel.
> >>>>>>
> >>>>>> With kexec, dpm doesn't get torn down properly which can result in a
> >>>>>> bad hardware state when the driver loads again.  Does the attached
> >>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
> >>>>>> torn down properly previously.
> >>>>>
> >>>>> dpm initialization now works, but unfortunately GPU acceleration
> >>>>still gets
> >>>>> disabled:
> >>>>
> >>>>Stupid kexec complicates things.  We need to make sure dpm is torn
> >>>>down before we init the rest of the GPU, but dpm needs get initialized
> >>>>later in the init process since it depends on certain other state from
> >>>>the driver.  I need to think about this for a bit.  I'm not sure of a
> >>>>good way to handle this.
> >>>
> >>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
> >>>
> >>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
> >>
> >> Does kexec actually call this shutdown method?  The driver implements
> >> appropriate clean-up measures if it's shutdown properly.
> >
> > Absoltuely.  All parts of the reboot path call ->shutdown.  Including
> > kexec.
> >
> > You don't get a device remove/hotunplug but unless this is a kexec on
> > panic ->shutdown is most definitely called.  Now I am talking about the
> > device layer/pci layer shutdown method I don't know how gpu drivers are
> > wired up.  GPU land was a little strange last I looked.  Hopefully it
> > isn't so strange that there is a method named shutdown that is not wired
> > up.
> 
> It doesn't look like the drm infrastructure has a shutdown callback.
> The drm drivers register a drm_driver callback struct that includes an
> unload callback which takes care of all the device teardown (if you
> unload the module for example).  I don't know that it actually gets
> called at kexec time however.  I don't know enough about how kexec
> works.
> 
> Markus, does everything work ok after a reboot?  Is it just kexec that
> is a problem?

Yes.
Markus Trippelsdorf July 30, 2013, 11:27 a.m. UTC | #11
On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
> On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
> <ebiederm@xmission.com> wrote:
> > Alex Deucher <alexdeucher@gmail.com> writes:
> >
> >> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
> >> <ebiederm@xmission.com> wrote:
> >>>
> >>>
> >>> Alex Deucher <alexdeucher@gmail.com> wrote:
> >>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
> >>>><markus@trippelsdorf.de> wrote:
> >>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
> >>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
> >>>>>> <markus@trippelsdorf.de> wrote:
> >>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
> >>>>>> > 3.11.0-rc3 kernel.
> >>>>>>
> >>>>>> With kexec, dpm doesn't get torn down properly which can result in a
> >>>>>> bad hardware state when the driver loads again.  Does the attached
> >>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
> >>>>>> torn down properly previously.
> >>>>>
> >>>>> dpm initialization now works, but unfortunately GPU acceleration
> >>>>still gets
> >>>>> disabled:
> >>>>
> >>>>Stupid kexec complicates things.  We need to make sure dpm is torn
> >>>>down before we init the rest of the GPU, but dpm needs get initialized
> >>>>later in the init process since it depends on certain other state from
> >>>>the driver.  I need to think about this for a bit.  I'm not sure of a
> >>>>good way to handle this.
> >>>
> >>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
> >>>
> >>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
> >>
> >> Does kexec actually call this shutdown method?  The driver implements
> >> appropriate clean-up measures if it's shutdown properly.
> >
> > Absoltuely.  All parts of the reboot path call ->shutdown.  Including
> > kexec.
> >
> > You don't get a device remove/hotunplug but unless this is a kexec on
> > panic ->shutdown is most definitely called.  Now I am talking about the
> > device layer/pci layer shutdown method I don't know how gpu drivers are
> > wired up.  GPU land was a little strange last I looked.  Hopefully it
> > isn't so strange that there is a method named shutdown that is not wired
> > up.
> 
> It doesn't look like the drm infrastructure has a shutdown callback.
> The drm drivers register a drm_driver callback struct that includes an
> unload callback which takes care of all the device teardown (if you
> unload the module for example).  I don't know that it actually gets
> called at kexec time however.  I don't know enough about how kexec
> works.

BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly
handles the kexec case during init. So maybe an r600_restore_sanity()
function is needed?

(One of the advantages of using kexec (besides the much quicker boot
time) is that the monitor is not switched off and then on during boot.
I guess that advantage would be lost if the unload callback would be
called.)
Markus Trippelsdorf July 30, 2013, 11:57 a.m. UTC | #12
On 2013.07.30 at 13:27 +0200, Markus Trippelsdorf wrote:
> On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
> > On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
> > <ebiederm@xmission.com> wrote:
> > > Alex Deucher <alexdeucher@gmail.com> writes:
> > >
> > >> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
> > >> <ebiederm@xmission.com> wrote:
> > >>>
> > >>>
> > >>> Alex Deucher <alexdeucher@gmail.com> wrote:
> > >>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
> > >>>><markus@trippelsdorf.de> wrote:
> > >>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
> > >>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
> > >>>>>> <markus@trippelsdorf.de> wrote:
> > >>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
> > >>>>>> > 3.11.0-rc3 kernel.
> > >>>>>>
> > >>>>>> With kexec, dpm doesn't get torn down properly which can result in a
> > >>>>>> bad hardware state when the driver loads again.  Does the attached
> > >>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
> > >>>>>> torn down properly previously.
> > >>>>>
> > >>>>> dpm initialization now works, but unfortunately GPU acceleration
> > >>>>still gets
> > >>>>> disabled:
> > >>>>
> > >>>>Stupid kexec complicates things.  We need to make sure dpm is torn
> > >>>>down before we init the rest of the GPU, but dpm needs get initialized
> > >>>>later in the init process since it depends on certain other state from
> > >>>>the driver.  I need to think about this for a bit.  I'm not sure of a
> > >>>>good way to handle this.
> > >>>
> > >>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
> > >>>
> > >>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
> > >>
> > >> Does kexec actually call this shutdown method?  The driver implements
> > >> appropriate clean-up measures if it's shutdown properly.
> > >
> > > Absoltuely.  All parts of the reboot path call ->shutdown.  Including
> > > kexec.
> > >
> > > You don't get a device remove/hotunplug but unless this is a kexec on
> > > panic ->shutdown is most definitely called.  Now I am talking about the
> > > device layer/pci layer shutdown method I don't know how gpu drivers are
> > > wired up.  GPU land was a little strange last I looked.  Hopefully it
> > > isn't so strange that there is a method named shutdown that is not wired
> > > up.
> > 
> > It doesn't look like the drm infrastructure has a shutdown callback.
> > The drm drivers register a drm_driver callback struct that includes an
> > unload callback which takes care of all the device teardown (if you
> > unload the module for example).  I don't know that it actually gets
> > called at kexec time however.  I don't know enough about how kexec
> > works.
> 
> BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly
> handles the kexec case during init. So maybe an r600_restore_sanity()
> function is needed?

Oh, I see r100_restore_sanity() gets also called for the other ASICs...
Alex Deucher July 30, 2013, 2:53 p.m. UTC | #13
On Tue, Jul 30, 2013 at 7:27 AM, Markus Trippelsdorf
<markus@trippelsdorf.de> wrote:
> On 2013.07.29 at 15:53 -0400, Alex Deucher wrote:
>> On Mon, Jul 29, 2013 at 2:10 PM, Eric W. Biederman
>> <ebiederm@xmission.com> wrote:
>> > Alex Deucher <alexdeucher@gmail.com> writes:
>> >
>> >> On Mon, Jul 29, 2013 at 11:50 AM, Eric W. Biederman
>> >> <ebiederm@xmission.com> wrote:
>> >>>
>> >>>
>> >>> Alex Deucher <alexdeucher@gmail.com> wrote:
>> >>>>On Mon, Jul 29, 2013 at 10:09 AM, Markus Trippelsdorf
>> >>>><markus@trippelsdorf.de> wrote:
>> >>>>> On 2013.07.29 at 09:58 -0400, Alex Deucher wrote:
>> >>>>>> On Mon, Jul 29, 2013 at 3:51 AM, Markus Trippelsdorf
>> >>>>>> <markus@trippelsdorf.de> wrote:
>> >>>>>> > On my test machine Xorg doesn't start anymore when I kexec into a
>> >>>>>> > 3.11.0-rc3 kernel.
>> >>>>>>
>> >>>>>> With kexec, dpm doesn't get torn down properly which can result in a
>> >>>>>> bad hardware state when the driver loads again.  Does the attached
>> >>>>>> patch help?  It attempts to disable dpm at startup in case it wasn't
>> >>>>>> torn down properly previously.
>> >>>>>
>> >>>>> dpm initialization now works, but unfortunately GPU acceleration
>> >>>>still gets
>> >>>>> disabled:
>> >>>>
>> >>>>Stupid kexec complicates things.  We need to make sure dpm is torn
>> >>>>down before we init the rest of the GPU, but dpm needs get initialized
>> >>>>later in the init process since it depends on certain other state from
>> >>>>the driver.  I need to think about this for a bit.  I'm not sure of a
>> >>>>good way to handle this.
>> >>>
>> >>> You might just want to implement a shutdown method that cleans things up properly.   At least as a first pass until you worry about things like kexec on panic.
>> >>>
>> >>> Or can you not shutdown the graphics stack on reboot because of the need to display the kernels shutdown progress?
>> >>
>> >> Does kexec actually call this shutdown method?  The driver implements
>> >> appropriate clean-up measures if it's shutdown properly.
>> >
>> > Absoltuely.  All parts of the reboot path call ->shutdown.  Including
>> > kexec.
>> >
>> > You don't get a device remove/hotunplug but unless this is a kexec on
>> > panic ->shutdown is most definitely called.  Now I am talking about the
>> > device layer/pci layer shutdown method I don't know how gpu drivers are
>> > wired up.  GPU land was a little strange last I looked.  Hopefully it
>> > isn't so strange that there is a method named shutdown that is not wired
>> > up.
>>
>> It doesn't look like the drm infrastructure has a shutdown callback.
>> The drm drivers register a drm_driver callback struct that includes an
>> unload callback which takes care of all the device teardown (if you
>> unload the module for example).  I don't know that it actually gets
>> called at kexec time however.  I don't know enough about how kexec
>> works.
>
> BTW there is r100_restore_sanity() in drm/radeon/r100.c that explicitly
> handles the kexec case during init. So maybe an r600_restore_sanity()
> function is needed?
>
> (One of the advantages of using kexec (besides the much quicker boot
> time) is that the monitor is not switched off and then on during boot.
> I guess that advantage would be lost if the unload callback would be
> called.)

r100_restore_sanity() is basically a set of hacks (that gets called at
driver startup) to work around the fact that with kexec the drm driver
is not torn down correctly.  So we could add a bunch more asic
specific tear down sequences to deal with dpm (and all the other
engines on the GPU that may potentially cause problems if they are not
torn down properly), but that will just turn into a mess.  All of
these hacks also add latency to the driver load.  I think the best
solution would probably be to figure how to hook up the drm unload
callback to the shutdown method that kexec uses.

Alex
diff mbox

Patch

From 8a39fdbfd4002c79ba9ab4eeb778c751fb20e173 Mon Sep 17 00:00:00 2001
From: Alex Deucher <alexander.deucher@amd.com>
Date: Mon, 29 Jul 2013 09:50:37 -0400
Subject: [PATCH] drm/radeon/dpm: disable dpm before enabling it to deal with
 kexec

Hopefully deal with kexec properly by disabling dpm prior to
enabling it in case dpm has not been torn down properly
previously.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Markus Trippelsdorf <markus@trippelsdorf.de>
---
 drivers/gpu/drm/radeon/radeon_pm.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/drivers/gpu/drm/radeon/radeon_pm.c b/drivers/gpu/drm/radeon/radeon_pm.c
index 76ffb91..c8e697e 100644
--- a/drivers/gpu/drm/radeon/radeon_pm.c
+++ b/drivers/gpu/drm/radeon/radeon_pm.c
@@ -1193,6 +1193,8 @@  static int radeon_pm_init_dpm(struct radeon_device *rdev)
 	radeon_dpm_init(rdev);
 	rdev->pm.dpm.current_ps = rdev->pm.dpm.requested_ps = rdev->pm.dpm.boot_ps;
 	radeon_dpm_print_power_states(rdev);
+	/* for cases like kexec, disable dpm just in case */
+	radeon_dpm_disable(rdev);
 	radeon_dpm_setup_asic(rdev);
 	ret = radeon_dpm_enable(rdev);
 	mutex_unlock(&rdev->pm.mutex);
-- 
1.7.7.5