diff mbox

drm/radeon: Always flush VM again on < CIK

Message ID CADnq5_P3jzB0SetPKX0F3g8CwgbqnSavKAo9+XUSE1ggVrcV=w@mail.gmail.com (mailing list archive)
State New, archived
Headers show

Commit Message

Alex Deucher Aug. 8, 2014, 1:34 p.m. UTC 
On Fri, Aug 8, 2014 at 9:31 AM, Alex Deucher <alexdeucher@gmail.com> wrote:
> On Fri, Aug 8, 2014 at 4:50 AM, Michel Dänzer <michel@daenzer.net> wrote:
>> On 08.08.2014 17:44, Christian König wrote:
>>>>>> On Thu, Aug 7, 2014 at 3:59 PM, Alex Deucher <alexdeucher@gmail.com>
>>>>>> wrote:
>>>>>>> We should be using PFP as much as possible.  Does the attached
>>>>>>> patch help?
>>>> Unfortunately not.
>>>
>>> Maybe add a readback of the VM base addr pointer to make sure that the
>>> write has really reached the SBRM?
>>
>> I'm not sure what exactly you're thinking of, but I'm happy to test any
>> patches you guys come up with. :)
>>
>
> Maybe some variant of this patch?

Ignore that one.  typo.  Try this one instead.

Alex

Comments

Michel Dänzer Aug. 11, 2014, 8:42 a.m. UTC | #1 
On 08.08.2014 22:34, Alex Deucher wrote:
> On Fri, Aug 8, 2014 at 9:31 AM, Alex Deucher <alexdeucher@gmail.com> wrote:
>> On Fri, Aug 8, 2014 at 4:50 AM, Michel Dänzer <michel@daenzer.net> wrote:
>>> On 08.08.2014 17:44, Christian König wrote:
>>>>>>> On Thu, Aug 7, 2014 at 3:59 PM, Alex Deucher <alexdeucher@gmail.com>
>>>>>>> wrote:
>>>>>>>> We should be using PFP as much as possible.  Does the attached
>>>>>>>> patch help?
>>>>> Unfortunately not.
>>>>
>>>> Maybe add a readback of the VM base addr pointer to make sure that the
>>>> write has really reached the SBRM?
>>>
>>> I'm not sure what exactly you're thinking of, but I'm happy to test any
>>> patches you guys come up with. :)
>>>
>>
>> Maybe some variant of this patch?
> 
> Ignore that one.  typo.  Try this one instead.

Thanks, but still no luck.
Alex Deucher Aug. 11, 2014, 3 p.m. UTC | #2 
On Mon, Aug 11, 2014 at 4:42 AM, Michel Dänzer <michel@daenzer.net> wrote:
> On 08.08.2014 22:34, Alex Deucher wrote:
>> On Fri, Aug 8, 2014 at 9:31 AM, Alex Deucher <alexdeucher@gmail.com> wrote:
>>> On Fri, Aug 8, 2014 at 4:50 AM, Michel Dänzer <michel@daenzer.net> wrote:
>>>> On 08.08.2014 17:44, Christian König wrote:
>>>>>>>> On Thu, Aug 7, 2014 at 3:59 PM, Alex Deucher <alexdeucher@gmail.com>
>>>>>>>> wrote:
>>>>>>>>> We should be using PFP as much as possible.  Does the attached
>>>>>>>>> patch help?
>>>>>> Unfortunately not.
>>>>>
>>>>> Maybe add a readback of the VM base addr pointer to make sure that the
>>>>> write has really reached the SBRM?
>>>>
>>>> I'm not sure what exactly you're thinking of, but I'm happy to test any
>>>> patches you guys come up with. :)
>>>>
>>>
>>> Maybe some variant of this patch?
>>
>> Ignore that one.  typo.  Try this one instead.
>
> Thanks, but still no luck.

I'm out of ideas at the moment.  I'll apply your patch unless
Christian can think of anything else.

Alex
Christian König Aug. 12, 2014, 9:05 a.m. UTC | #3 
Am 11.08.2014 um 17:00 schrieb Alex Deucher:
> On Mon, Aug 11, 2014 at 4:42 AM, Michel Dänzer <michel@daenzer.net> wrote:
>> On 08.08.2014 22:34, Alex Deucher wrote:
>>> On Fri, Aug 8, 2014 at 9:31 AM, Alex Deucher <alexdeucher@gmail.com> wrote:
>>>> On Fri, Aug 8, 2014 at 4:50 AM, Michel Dänzer <michel@daenzer.net> wrote:
>>>>> On 08.08.2014 17:44, Christian König wrote:
>>>>>>>>> On Thu, Aug 7, 2014 at 3:59 PM, Alex Deucher <alexdeucher@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> We should be using PFP as much as possible.  Does the attached
>>>>>>>>>> patch help?
>>>>>>> Unfortunately not.
>>>>>> Maybe add a readback of the VM base addr pointer to make sure that the
>>>>>> write has really reached the SBRM?
>>>>> I'm not sure what exactly you're thinking of, but I'm happy to test any
>>>>> patches you guys come up with. :)
>>>>>
>>>> Maybe some variant of this patch?
>>> Ignore that one.  typo.  Try this one instead.
>> Thanks, but still no luck.
> I'm out of ideas at the moment.  I'll apply your patch unless
> Christian can think of anything else.

Unfortunately not, so apply the patch for now.

Christian.

>
> Alex
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
Christian König Aug. 15, 2014, 2:52 p.m. UTC | #4 
Hey guys,

I think I've figured out what the cause of the remaining issues is while 
working on the implicit sync stuff.

The issue happens when the flush is done because of a CS to the DMA ring 
and a CS to the GFX ring directly after that which depends on the DMA 
submission to be finished.

In this situation we insert semaphore command so that the GFX ring wait 
for the DMA ring to finish execution and normally don't flush on the GFX 
ring a second time (the flush should be done on the DMA ring and we 
waited for that to finish).

The problem here is that semaphores can't be executed on the PFP, so the 
PFP doesn't wait for the semaphore to be completed and happily starts 
fetching commands while the flush on the DMA ring isn't completed.

@Michel: can you give this branch a try and see if it now works for you: 
http://cgit.freedesktop.org/~deathsimple/linux/log/?h=vm-flushing

We should keep that behavior in mind should we switch to put IBs into 
normal BOs, cause when those a swapped out the synchronization won't 
wait for swapping them back in using the DMA as well.

Thanks,
Christian.

Am 12.08.2014 um 11:05 schrieb Christian König:
> Am 11.08.2014 um 17:00 schrieb Alex Deucher:
>> On Mon, Aug 11, 2014 at 4:42 AM, Michel Dänzer <michel@daenzer.net> 
>> wrote:
>>> On 08.08.2014 22:34, Alex Deucher wrote:
>>>> On Fri, Aug 8, 2014 at 9:31 AM, Alex Deucher 
>>>> <alexdeucher@gmail.com> wrote:
>>>>> On Fri, Aug 8, 2014 at 4:50 AM, Michel Dänzer <michel@daenzer.net> 
>>>>> wrote:
>>>>>> On 08.08.2014 17:44, Christian König wrote:
>>>>>>>>>> On Thu, Aug 7, 2014 at 3:59 PM, Alex Deucher 
>>>>>>>>>> <alexdeucher@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>> We should be using PFP as much as possible.  Does the attached
>>>>>>>>>>> patch help?
>>>>>>>> Unfortunately not.
>>>>>>> Maybe add a readback of the VM base addr pointer to make sure 
>>>>>>> that the
>>>>>>> write has really reached the SBRM?
>>>>>> I'm not sure what exactly you're thinking of, but I'm happy to 
>>>>>> test any
>>>>>> patches you guys come up with. :)
>>>>>>
>>>>> Maybe some variant of this patch?
>>>> Ignore that one.  typo.  Try this one instead.
>>> Thanks, but still no luck.
>> I'm out of ideas at the moment.  I'll apply your patch unless
>> Christian can think of anything else.
>
> Unfortunately not, so apply the patch for now.
>
> Christian.
>
>>
>> Alex
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/dri-devel
>
diff mbox

Patch

diff --git a/drivers/gpu/drm/radeon/si.c b/drivers/gpu/drm/radeon/si.c
index dbd9d81..565201d 100644
--- a/drivers/gpu/drm/radeon/si.c
+++ b/drivers/gpu/drm/radeon/si.c
@@ -5007,6 +5007,7 @@  static void si_vm_decode_fault(struct radeon_device *rdev,
 void si_vm_flush(struct radeon_device *rdev, int ridx, struct radeon_vm *vm)
 {
 	struct radeon_ring *ring = &rdev->ring[ridx];
+	u32 reg;
 
 	if (vm == NULL)
 		return;
@@ -5017,15 +5018,23 @@  void si_vm_flush(struct radeon_device *rdev, int ridx, struct radeon_vm *vm)
 				 WRITE_DATA_DST_SEL(0)));
 
 	if (vm->id < 8) {
-		radeon_ring_write(ring,
-				  (VM_CONTEXT0_PAGE_TABLE_BASE_ADDR + (vm->id << 2)) >> 2);
+		reg = (VM_CONTEXT0_PAGE_TABLE_BASE_ADDR + (vm->id << 2)) >> 2;
 	} else {
-		radeon_ring_write(ring,
-				  (VM_CONTEXT8_PAGE_TABLE_BASE_ADDR + ((vm->id - 8) << 2)) >> 2);
+		reg = (VM_CONTEXT8_PAGE_TABLE_BASE_ADDR + ((vm->id - 8) << 2)) >> 2;
 	}
+	radeon_ring_write(ring, reg);
 	radeon_ring_write(ring, 0);
 	radeon_ring_write(ring, vm->pd_gpu_addr >> 12);
 
+	/* wait for the address change to go through */
+	radeon_ring_write(ring, PACKET3(PACKET3_WAIT_REG_MEM, 5));
+	radeon_ring_write(ring, 3); /* == */
+	radeon_ring_write(ring, reg);
+	radeon_ring_write(ring, 0);
+	radeon_ring_write(ring, vm->pd_gpu_addr >> 12);
+	radeon_ring_write(ring, 0x0fffffff);
+	radeon_ring_write(ring, 10);
+
 	/* flush hdp cache */
 	radeon_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3));
 	radeon_ring_write(ring, (WRITE_DATA_ENGINE_SEL(1) |
@@ -5034,6 +5043,14 @@  void si_vm_flush(struct radeon_device *rdev, int ridx, struct radeon_vm *vm)
 	radeon_ring_write(ring, 0);
 	radeon_ring_write(ring, 0x1);
 
+	/* clear the response reg */
+	radeon_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3));
+	radeon_ring_write(ring, (WRITE_DATA_ENGINE_SEL(1) |
+				 WRITE_DATA_DST_SEL(0)));
+	radeon_ring_write(ring, VM_INVALIDATE_RESPONSE >> 2);
+	radeon_ring_write(ring, 0);
+	radeon_ring_write(ring, 0);
+
 	/* bits 0-15 are the VM contexts0-15 */
 	radeon_ring_write(ring, PACKET3(PACKET3_WRITE_DATA, 3));
 	radeon_ring_write(ring, (WRITE_DATA_ENGINE_SEL(1) |
@@ -5042,6 +5059,15 @@  void si_vm_flush(struct radeon_device *rdev, int ridx, struct radeon_vm *vm)
 	radeon_ring_write(ring, 0);
 	radeon_ring_write(ring, 1 << vm->id);
 
+	/* wait for the invalidate */
+	radeon_ring_write(ring, PACKET3(PACKET3_WAIT_REG_MEM, 5));
+	radeon_ring_write(ring, 3); /* == */
+	radeon_ring_write(ring, VM_INVALIDATE_RESPONSE >> 2);
+	radeon_ring_write(ring, 0);
+	radeon_ring_write(ring, 1 << vm->id);
+	radeon_ring_write(ring, 1 << vm->id);
+	radeon_ring_write(ring, 10);
+
 	/* sync PFP to ME, otherwise we might get invalid PFP reads */
 	radeon_ring_write(ring, PACKET3(PACKET3_PFP_SYNC_ME, 0));
 	radeon_ring_write(ring, 0x0);