| Message ID | 20210521090959.1663703-4-daniel.vetter@ffwll.ch (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [01/11] drm/amdgpu: Comply with implicit fencing rules | expand |
Hi, On Fri, 21 May 2021 at 10:10, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: > Currently this has no practial relevance I think because there's not > many who can pull off a setup with panfrost and another gpu in the > same system. But the rules are that if you're setting an exclusive > fence, indicating a gpu write access in the implicit fencing system, > then you need to wait for all fences, not just the previous exclusive > fence. > > panfrost against itself has no problem, because it always sets the > exclusive fence (but that's probably something that will need to be > fixed for vulkan and/or multi-engine gpus, or you'll suffer badly). > Also no problem with that against display. Yeah, the 'second-generation Valhall' GPUs coming later this year / early next year are starting to get pretty weird. Firmware-mediated job scheduling out of multiple queues, userspace having direct access to the queues and can do inter-queue synchronisation (at least I think so), etc. For bonus points, synchronisation is based on $addr = $val to signal and $addr == $val to wait, with a separate fence primitive as well. Obviously Arm should be part of this conversation here, but I guess we'll have to wait for a while yet to see how everything's shaken out with this new gen, and hope that whatever's been designed upstream in the meantime is actually vaguely compatible ... Cheers, Daniel
Am 21.05.21 um 14:22 schrieb Daniel Stone: > Hi, > > On Fri, 21 May 2021 at 10:10, Daniel Vetter <daniel.vetter@ffwll.ch> wrote: >> Currently this has no practial relevance I think because there's not >> many who can pull off a setup with panfrost and another gpu in the >> same system. But the rules are that if you're setting an exclusive >> fence, indicating a gpu write access in the implicit fencing system, >> then you need to wait for all fences, not just the previous exclusive >> fence. >> >> panfrost against itself has no problem, because it always sets the >> exclusive fence (but that's probably something that will need to be >> fixed for vulkan and/or multi-engine gpus, or you'll suffer badly). >> Also no problem with that against display. > Yeah, the 'second-generation Valhall' GPUs coming later this year / > early next year are starting to get pretty weird. Firmware-mediated > job scheduling out of multiple queues, userspace having direct access > to the queues and can do inter-queue synchronisation (at least I think > so), etc. For bonus points, synchronisation is based on $addr = $val > to signal and $addr == $val to wait, with a separate fence primitive > as well. Well that sounds familiar :) > Obviously Arm should be part of this conversation here, but I guess > we'll have to wait for a while yet to see how everything's shaken out > with this new gen, and hope that whatever's been designed upstream in > the meantime is actually vaguely compatible ... Yeah, going to keep you in CC when we start to code and review user fences. Cheers, Christian. > > Cheers, > Daniel > _______________________________________________ > Linaro-mm-sig mailing list > Linaro-mm-sig@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/linaro-mm-sig
On Fri, 21 May 2021 at 13:28, Christian König <christian.koenig@amd.com> wrote: > Am 21.05.21 um 14:22 schrieb Daniel Stone: > > Yeah, the 'second-generation Valhall' GPUs coming later this year / > > early next year are starting to get pretty weird. Firmware-mediated > > job scheduling out of multiple queues, userspace having direct access > > to the queues and can do inter-queue synchronisation (at least I think > > so), etc. For bonus points, synchronisation is based on $addr = $val > > to signal and $addr == $val to wait, with a separate fence primitive > > as well. > > Well that sounds familiar :) I laughed when I first saw it, because it was better than crying I guess. If you're curious, the interface definitions are in the csf/ directory in the 'Bifrost kernel driver' r30p0 download you can get from the Arm developer site. Unfortunately the exact semantics aren't completely clear. > > Obviously Arm should be part of this conversation here, but I guess > > we'll have to wait for a while yet to see how everything's shaken out > > with this new gen, and hope that whatever's been designed upstream in > > the meantime is actually vaguely compatible ... > > Yeah, going to keep you in CC when we start to code and review user fences. Awesome, thanks Christian. Appreciate it. :) Cheers, Daniel
Am 21.05.21 um 14:54 schrieb Daniel Stone: > On Fri, 21 May 2021 at 13:28, Christian König <christian.koenig@amd.com> wrote: >> Am 21.05.21 um 14:22 schrieb Daniel Stone: >>> Yeah, the 'second-generation Valhall' GPUs coming later this year / >>> early next year are starting to get pretty weird. Firmware-mediated >>> job scheduling out of multiple queues, userspace having direct access >>> to the queues and can do inter-queue synchronisation (at least I think >>> so), etc. For bonus points, synchronisation is based on $addr = $val >>> to signal and $addr == $val to wait, with a separate fence primitive >>> as well. >> Well that sounds familiar :) > I laughed when I first saw it, because it was better than crying I guess. In Germany we say "Ich freue mich drauf wie auf Zahnschmerzen". > If you're curious, the interface definitions are in the csf/ directory > in the 'Bifrost kernel driver' r30p0 download you can get from the Arm > developer site. Unfortunately the exact semantics aren't completely > clear. Well it is actually relatively simple. Take a look at the timeline semaphores from Vulkan, everybody is basically implementing the same semantics now. When you queued up a bunch of commands on your hardware, the first one will write value 1 to a 64bit memory location, the second one will write value 2, the third value 3 and so on. After writing the value the hardware raises and interrupt signal to everybody interested. In other words pretty standard memory fence behavior. When you now have a second queue which depends on work of the first one you look at the memory location and do a compare. If you depend on the third submission you just wait for the value to be >3 and are done. Regards, Christian. > >>> Obviously Arm should be part of this conversation here, but I guess >>> we'll have to wait for a while yet to see how everything's shaken out >>> with this new gen, and hope that whatever's been designed upstream in >>> the meantime is actually vaguely compatible ... >> Yeah, going to keep you in CC when we start to code and review user fences. > Awesome, thanks Christian. Appreciate it. :) > > Cheers, > Daniel
On Fri, 21 May 2021 at 14:09, Christian König <christian.koenig@amd.com> wrote: > Am 21.05.21 um 14:54 schrieb Daniel Stone: > > If you're curious, the interface definitions are in the csf/ directory > > in the 'Bifrost kernel driver' r30p0 download you can get from the Arm > > developer site. Unfortunately the exact semantics aren't completely > > clear. > > Well it is actually relatively simple. Take a look at the timeline > semaphores from Vulkan, everybody is basically implementing the same > semantics now. > > When you queued up a bunch of commands on your hardware, the first one > will write value 1 to a 64bit memory location, the second one will write > value 2, the third value 3 and so on. After writing the value the > hardware raises and interrupt signal to everybody interested. > > In other words pretty standard memory fence behavior. > > When you now have a second queue which depends on work of the first one > you look at the memory location and do a compare. If you depend on the > third submission you just wait for the value to be >3 and are done. Right, it is clearly defined to the timeline semaphore semantics, I just meant that it's not clear how it works at a lower level wrt the synchronisation and signaling. The simplest possible interpretation is that wait_addrval blocks infinitely before kick-cmdbuf, but that seems painful with only 32 queues. And the same for fences, which are a binary signal. I guess we'll find out. My tooth hurts. Cheers, Daniel
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c index 707d912ff64a..619d6104040c 100644 --- a/drivers/gpu/drm/panfrost/panfrost_job.c +++ b/drivers/gpu/drm/panfrost/panfrost_job.c @@ -203,9 +203,8 @@ static int panfrost_acquire_object_fences(struct drm_gem_object **bos, int i, ret; for (i = 0; i < bo_count; i++) { - struct dma_fence *fence = dma_resv_get_excl_rcu(bos[i]->resv); - - ret = drm_gem_fence_array_add(deps, fence); + /* panfrost always uses write mode in its current uapi */ + ret = drm_gem_fence_array_add_implicit(deps, bos[i], true); if (ret) return ret; }
Currently this has no practial relevance I think because there's not many who can pull off a setup with panfrost and another gpu in the same system. But the rules are that if you're setting an exclusive fence, indicating a gpu write access in the implicit fencing system, then you need to wait for all fences, not just the previous exclusive fence. panfrost against itself has no problem, because it always sets the exclusive fence (but that's probably something that will need to be fixed for vulkan and/or multi-engine gpus, or you'll suffer badly). Also no problem with that against display. With the prep work done to switch over to the dependency helpers this is now a oneliner. Signed-off-by: Daniel Vetter <daniel.vetter@intel.com> Cc: Rob Herring <robh@kernel.org> Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com> Cc: Steven Price <steven.price@arm.com> Cc: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Cc: Sumit Semwal <sumit.semwal@linaro.org> Cc: "Christian König" <christian.koenig@amd.com> Cc: linux-media@vger.kernel.org Cc: linaro-mm-sig@lists.linaro.org --- drivers/gpu/drm/panfrost/panfrost_job.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)