| Message ID | 1507847254-3289-2-git-send-email-daniele.ceraolospurio@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) > diff --git a/lib/igt_dummyload.h b/lib/igt_dummyload.h > index 215425f..d20a867 100644 > --- a/lib/igt_dummyload.h > +++ b/lib/igt_dummyload.h <insert ambition for killing off using igt for i915 specific constructs> > @@ -29,6 +29,7 @@ > #include <time.h> > > #include "igt_aux.h" > +#include "igt_vgem.h" > > typedef struct igt_spin { > unsigned int handle; > @@ -51,4 +52,14 @@ void igt_spin_batch_free(int fd, igt_spin_t *spin); > > void igt_terminate_spin_batches(void); > > +typedef struct igt_cork { > + int device; > + uint32_t handle; > + uint32_t fence; > +} igt_cork_t; Nothing here depends on igt_vgem.h, that should just be for igt_dummyload.c > + > +igt_cork_t *igt_cork_new(int fd); > +void igt_cork_signal(igt_cork_t *cork); > +void igt_cork_free(int fd, igt_cork_t *cork); > + > #endif /* __IGT_DUMMYLOAD_H__ */
Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) > +typedef struct igt_cork { > + int device; > + uint32_t handle; > + uint32_t fence; > +} igt_cork_t; Oh, just struct no need for typedef when you are transparent. -Chris
Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) > +igt_cork_t *igt_cork_new(int fd); _new does not imply plugged. > +void igt_cork_signal(igt_cork_t *cork); When have you signaled a cork? > +void igt_cork_free(int fd, igt_cork_t *cork); _free does not imply unplug. -Chris
Quoting Chris Wilson (2017-10-12 23:57:38) > Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) > > +igt_cork_t *igt_cork_new(int fd); > > _new does not imply plugged. > > > +void igt_cork_signal(igt_cork_t *cork); > > When have you signaled a cork? > > > +void igt_cork_free(int fd, igt_cork_t *cork); > > _free does not imply unplug. To be clear the verbs are to plug and unplug a queue/schedule. Cork is a reference to TCP_CORK which does the same thing, but plug/unplug are more commonplace (at least in kernel code). I don't see any reason why we need a malloc here. -Chris
On 13/10/17 01:31, Chris Wilson wrote: > Quoting Chris Wilson (2017-10-12 23:57:38) >> Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) >>> +igt_cork_t *igt_cork_new(int fd); >> >> _new does not imply plugged. >> >>> +void igt_cork_signal(igt_cork_t *cork); >> >> When have you signaled a cork? >> >>> +void igt_cork_free(int fd, igt_cork_t *cork); >> >> _free does not imply unplug. > > To be clear the verbs are to plug and unplug a queue/schedule. Cork is a > reference to TCP_CORK which does the same thing, but plug/unplug are > more commonplace (at least in kernel code). > > I don't see any reason why we need a malloc here. > -Chris > I added the malloc just to use the same approach as the spin_batch, I'll get rid of it. My concern with the existing plug/unplug scheme was that the plug() function in the various tests didn't really plug anything but just created the bo and that was slightly confusing. What do you think of going with: struct igt_cork { int device; uint32_t handle; uint32_t fence; }; struct igt_cork igt_cork_create(int fd); void igt_cork_unplug(struct igt_cork *cork); void igt_cork_close(int fd, struct igt_cork *cork); void igt_cork_unplug_and_close(int fd, struct igt_cork *cork); The plug() function is still missing, as we do the actual plugging by adding the object to the execbuf and I don't think that would be clean to wrap in the library. I thought of adding something like "get_plugging_handle()" to return cork->handle and make it more explicit that that handle was responsible for the plugging but it seemed a bit overkill. Daniele
On 13/10/17 09:37, Daniele Ceraolo Spurio wrote: > > > On 13/10/17 01:31, Chris Wilson wrote: >> Quoting Chris Wilson (2017-10-12 23:57:38) >>> Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) >>>> +igt_cork_t *igt_cork_new(int fd); >>> >>> _new does not imply plugged. >>> >>>> +void igt_cork_signal(igt_cork_t *cork); >>> >>> When have you signaled a cork? >>> >>>> +void igt_cork_free(int fd, igt_cork_t *cork); >>> >>> _free does not imply unplug. >> >> To be clear the verbs are to plug and unplug a queue/schedule. Cork is a >> reference to TCP_CORK which does the same thing, but plug/unplug are >> more commonplace (at least in kernel code). >> >> I don't see any reason why we need a malloc here. >> -Chris >> > > I added the malloc just to use the same approach as the spin_batch, I'll > get rid of it. > My concern with the existing plug/unplug scheme was that the plug() > function in the various tests didn't really plug anything but just > created the bo and that was slightly confusing. > What do you think of going with: > > struct igt_cork { > int device; > uint32_t handle; > uint32_t fence; > }; > > struct igt_cork igt_cork_create(int fd); > void igt_cork_unplug(struct igt_cork *cork); > void igt_cork_close(int fd, struct igt_cork *cork); > void igt_cork_unplug_and_close(int fd, struct igt_cork *cork); > > The plug() function is still missing, as we do the actual plugging by > adding the object to the execbuf and I don't think that would be clean > to wrap in the library. I thought of adding something like > "get_plugging_handle()" to return cork->handle and make it more explicit > that that handle was responsible for the plugging but it seemed a bit > overkill. > > Daniele > Hi Chris, any feedback on this? Thanks, Daniele
Quoting Daniele Ceraolo Spurio (2017-10-18 16:49:24) > > > On 13/10/17 09:37, Daniele Ceraolo Spurio wrote: > > > > > > On 13/10/17 01:31, Chris Wilson wrote: > >> Quoting Chris Wilson (2017-10-12 23:57:38) > >>> Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) > >>>> +igt_cork_t *igt_cork_new(int fd); > >>> > >>> _new does not imply plugged. > >>> > >>>> +void igt_cork_signal(igt_cork_t *cork); > >>> > >>> When have you signaled a cork? > >>> > >>>> +void igt_cork_free(int fd, igt_cork_t *cork); > >>> > >>> _free does not imply unplug. > >> > >> To be clear the verbs are to plug and unplug a queue/schedule. Cork is a > >> reference to TCP_CORK which does the same thing, but plug/unplug are > >> more commonplace (at least in kernel code). > >> > >> I don't see any reason why we need a malloc here. > >> -Chris > >> > > > > I added the malloc just to use the same approach as the spin_batch, I'll > > get rid of it. > > My concern with the existing plug/unplug scheme was that the plug() > > function in the various tests didn't really plug anything but just > > created the bo and that was slightly confusing. It created a bo with an unsignaled fence, that's enough to plug anything attached to it. Since we can't just say plug(device) we have to say execbuf(device, plug()). > > What do you think of going with: > > > > struct igt_cork { > > int device; > > uint32_t handle; > > uint32_t fence; > > }; > > > > struct igt_cork igt_cork_create(int fd); > > void igt_cork_unplug(struct igt_cork *cork); > > void igt_cork_close(int fd, struct igt_cork *cork); > > void igt_cork_unplug_and_close(int fd, struct igt_cork *cork); close will always be unplug; there's no differentiation, in both APIs we ensure that any fence associated with the device or timeline fd is signaled upon release. We could lose the fence and still work, but for us it gives us the means by which we can do a test-and-set and report an issue where the fence was signaled too early (due to slow test setup). Similarly once unplugged, there is no use for the struct anymore, you could release the device/timeline, but we've embedded it because in terms of overhead, so far it has been insignificant. Leaving a fence dangling by separating unplug/close is a good way to leave lots of timeouts and GPU resets behind. -Chris
On 18/10/17 09:04, Chris Wilson wrote: > Quoting Daniele Ceraolo Spurio (2017-10-18 16:49:24) >> >> >> On 13/10/17 09:37, Daniele Ceraolo Spurio wrote: >>> >>> >>> On 13/10/17 01:31, Chris Wilson wrote: >>>> Quoting Chris Wilson (2017-10-12 23:57:38) >>>>> Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) >>>>>> +igt_cork_t *igt_cork_new(int fd); >>>>> >>>>> _new does not imply plugged. >>>>> >>>>>> +void igt_cork_signal(igt_cork_t *cork); >>>>> >>>>> When have you signaled a cork? >>>>> >>>>>> +void igt_cork_free(int fd, igt_cork_t *cork); >>>>> >>>>> _free does not imply unplug. >>>> >>>> To be clear the verbs are to plug and unplug a queue/schedule. Cork is a >>>> reference to TCP_CORK which does the same thing, but plug/unplug are >>>> more commonplace (at least in kernel code). >>>> >>>> I don't see any reason why we need a malloc here. >>>> -Chris >>>> >>> >>> I added the malloc just to use the same approach as the spin_batch, I'll >>> get rid of it. >>> My concern with the existing plug/unplug scheme was that the plug() >>> function in the various tests didn't really plug anything but just >>> created the bo and that was slightly confusing. > > It created a bo with an unsignaled fence, that's enough to plug anything > attached to it. Since we can't just say plug(device) we have to say > execbuf(device, plug()). > >>> What do you think of going with: >>> >>> struct igt_cork { >>> int device; >>> uint32_t handle; >>> uint32_t fence; >>> }; >>> >>> struct igt_cork igt_cork_create(int fd); >>> void igt_cork_unplug(struct igt_cork *cork); >>> void igt_cork_close(int fd, struct igt_cork *cork); >>> void igt_cork_unplug_and_close(int fd, struct igt_cork *cork); > > close will always be unplug; there's no differentiation, in both APIs we > ensure that any fence associated with the device or timeline fd is > signaled upon release. We could lose the fence and still work, but for > us it gives us the means by which we can do a test-and-set and report an > issue where the fence was signaled too early (due to slow test setup). > Similarly once unplugged, there is no use for the struct anymore, you > could release the device/timeline, but we've embedded it because in > terms of overhead, so far it has been insignificant. > > Leaving a fence dangling by separating unplug/close is a good way to > leave lots of timeouts and GPU resets behind. > -Chris > What I wanted to separate is the unplugging from the closing of the BO handle, because in some case we keep the BO around for a while after unblocking the execution. In most of those cases the BO handle is not currently closed, but it seemed dirty to me to keep that behavior and have library functions that didn't clean up after themselves. Would it be ok for you to keep both unplug and close, with the first doing just the signaling of the fence and the second doing both the signaling (if not already done) and the bo handle closure? this would basically be what is in the current version of the patches but with a different name. Alternatively, we could explicitly return the handle from the plug() call to make it clearer that it needs closing: uint32_t igt_plug(int fd, struct igt_cork *cork); void igt_unplug(struct igt_cork *cork); Could also keep the typedef in this case since the handle is returned and we don't need to peek inside the cork struct. Thanks, Daniele
Quoting Daniele Ceraolo Spurio (2017-10-18 17:50:33) > > > On 18/10/17 09:04, Chris Wilson wrote: > > Quoting Daniele Ceraolo Spurio (2017-10-18 16:49:24) > >> > >> > >> On 13/10/17 09:37, Daniele Ceraolo Spurio wrote: > >>> > >>> > >>> On 13/10/17 01:31, Chris Wilson wrote: > >>>> Quoting Chris Wilson (2017-10-12 23:57:38) > >>>>> Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) > >>>>>> +igt_cork_t *igt_cork_new(int fd); > >>>>> > >>>>> _new does not imply plugged. > >>>>> > >>>>>> +void igt_cork_signal(igt_cork_t *cork); > >>>>> > >>>>> When have you signaled a cork? > >>>>> > >>>>>> +void igt_cork_free(int fd, igt_cork_t *cork); > >>>>> > >>>>> _free does not imply unplug. > >>>> > >>>> To be clear the verbs are to plug and unplug a queue/schedule. Cork is a > >>>> reference to TCP_CORK which does the same thing, but plug/unplug are > >>>> more commonplace (at least in kernel code). > >>>> > >>>> I don't see any reason why we need a malloc here. > >>>> -Chris > >>>> > >>> > >>> I added the malloc just to use the same approach as the spin_batch, I'll > >>> get rid of it. > >>> My concern with the existing plug/unplug scheme was that the plug() > >>> function in the various tests didn't really plug anything but just > >>> created the bo and that was slightly confusing. > > > > It created a bo with an unsignaled fence, that's enough to plug anything > > attached to it. Since we can't just say plug(device) we have to say > > execbuf(device, plug()). > > > >>> What do you think of going with: > >>> > >>> struct igt_cork { > >>> int device; > >>> uint32_t handle; > >>> uint32_t fence; > >>> }; > >>> > >>> struct igt_cork igt_cork_create(int fd); > >>> void igt_cork_unplug(struct igt_cork *cork); > >>> void igt_cork_close(int fd, struct igt_cork *cork); > >>> void igt_cork_unplug_and_close(int fd, struct igt_cork *cork); > > > > close will always be unplug; there's no differentiation, in both APIs we > > ensure that any fence associated with the device or timeline fd is > > signaled upon release. We could lose the fence and still work, but for > > us it gives us the means by which we can do a test-and-set and report an > > issue where the fence was signaled too early (due to slow test setup). > > Similarly once unplugged, there is no use for the struct anymore, you > > could release the device/timeline, but we've embedded it because in > > terms of overhead, so far it has been insignificant. > > > > Leaving a fence dangling by separating unplug/close is a good way to > > leave lots of timeouts and GPU resets behind. > > -Chris > > > > What I wanted to separate is the unplugging from the closing of the BO > handle, because in some case we keep the BO around for a while after > unblocking the execution. Where? What value could it possibly have? You know the state of its fence, so presumably you want the contents. In such a situation you don't need a dummy cork to plug the queue, you have a real object with which you want interact. > In most of those cases the BO handle is not > currently closed, Every single unplug() function is closing the device; which closes the handle; gem_close() is superfluous. Early on I kept the vgem fd around and just needed to close the handle, but it looks like all of those functions have now been converted to own their device. -Chris
On 19/10/17 11:12, Chris Wilson wrote: > Quoting Daniele Ceraolo Spurio (2017-10-18 17:50:33) >> >> >> On 18/10/17 09:04, Chris Wilson wrote: >>> Quoting Daniele Ceraolo Spurio (2017-10-18 16:49:24) >>>> >>>> >>>> On 13/10/17 09:37, Daniele Ceraolo Spurio wrote: >>>>> >>>>> >>>>> On 13/10/17 01:31, Chris Wilson wrote: >>>>>> Quoting Chris Wilson (2017-10-12 23:57:38) >>>>>>> Quoting Daniele Ceraolo Spurio (2017-10-12 23:27:27) >>>>>>>> +igt_cork_t *igt_cork_new(int fd); >>>>>>> >>>>>>> _new does not imply plugged. >>>>>>> >>>>>>>> +void igt_cork_signal(igt_cork_t *cork); >>>>>>> >>>>>>> When have you signaled a cork? >>>>>>> >>>>>>>> +void igt_cork_free(int fd, igt_cork_t *cork); >>>>>>> >>>>>>> _free does not imply unplug. >>>>>> >>>>>> To be clear the verbs are to plug and unplug a queue/schedule. Cork is a >>>>>> reference to TCP_CORK which does the same thing, but plug/unplug are >>>>>> more commonplace (at least in kernel code). >>>>>> >>>>>> I don't see any reason why we need a malloc here. >>>>>> -Chris >>>>>> >>>>> >>>>> I added the malloc just to use the same approach as the spin_batch, I'll >>>>> get rid of it. >>>>> My concern with the existing plug/unplug scheme was that the plug() >>>>> function in the various tests didn't really plug anything but just >>>>> created the bo and that was slightly confusing. >>> >>> It created a bo with an unsignaled fence, that's enough to plug anything >>> attached to it. Since we can't just say plug(device) we have to say >>> execbuf(device, plug()). >>> >>>>> What do you think of going with: >>>>> >>>>> struct igt_cork { >>>>> int device; >>>>> uint32_t handle; >>>>> uint32_t fence; >>>>> }; >>>>> >>>>> struct igt_cork igt_cork_create(int fd); >>>>> void igt_cork_unplug(struct igt_cork *cork); >>>>> void igt_cork_close(int fd, struct igt_cork *cork); >>>>> void igt_cork_unplug_and_close(int fd, struct igt_cork *cork); >>> >>> close will always be unplug; there's no differentiation, in both APIs we >>> ensure that any fence associated with the device or timeline fd is >>> signaled upon release. We could lose the fence and still work, but for >>> us it gives us the means by which we can do a test-and-set and report an >>> issue where the fence was signaled too early (due to slow test setup). >>> Similarly once unplugged, there is no use for the struct anymore, you >>> could release the device/timeline, but we've embedded it because in >>> terms of overhead, so far it has been insignificant. >>> >>> Leaving a fence dangling by separating unplug/close is a good way to >>> leave lots of timeouts and GPU resets behind. >>> -Chris >>> >> >> What I wanted to separate is the unplugging from the closing of the BO >> handle, because in some case we keep the BO around for a while after >> unblocking the execution. > > Where? What value could it possibly have? You know the state of its > fence, so presumably you want the contents. In such a situation you don't > need a dummy cork to plug the queue, you have a real object with which > you want interact. > >> In most of those cases the BO handle is not >> currently closed, > > Every single unplug() function is closing the device; which closes the > handle; gem_close() is superfluous. Early on I kept the vgem fd around > and just needed to close the handle, but it looks like all of those > functions have now been converted to own their device. > -Chris > Apologies for being unclear, what I was referring to was the imported BO handle, not the one from the vgem device. In a couple of tests (gem_wait, gem_exec_latency) we still use it after the unplug. My understanding (which could be wrong, since I haven't looked at this area a lot) is that the imported handle will stay valid even if the original device is closed and that's why I wanted to explicitly clean it up. It'll still be closed when the fd is closed so we're not doing something inherently bad, but I felt it would've been cleaner to have that in the common function. I'll add a note about it in the function description instead and keep the existing plug/unplug scheme. Daniele
diff --git a/lib/igt_dummyload.c b/lib/igt_dummyload.c index bb2be55..913cc93 100644 --- a/lib/igt_dummyload.c +++ b/lib/igt_dummyload.c @@ -29,6 +29,7 @@ #include <i915_drm.h> #include "igt_core.h" +#include "drmtest.h" #include "igt_dummyload.h" #include "igt_gt.h" #include "intel_chipset.h" @@ -300,3 +301,77 @@ void igt_terminate_spin_batches(void) igt_list_for_each(iter, &spin_list, link) igt_spin_batch_end(iter); } + +/** + * igt_cork_new + * @fd: open drm file descriptor + * + * Imports a vgem bo with a fence attached to it. This bo can be used as a + * dependency during submission to stall execution until the fence is signaled. + * + * Returns: + * Structure with the handle of the imported bo and helper internal state + * for igt_cork_signal() and igt_cork_free(). + */ +igt_cork_t *igt_cork_new(int fd) +{ + igt_cork_t *cork; + struct vgem_bo bo; + int dmabuf; + + cork = calloc(1, sizeof(igt_cork_t)); + igt_assert(cork); + + cork->device = drm_open_driver(DRIVER_VGEM); + + igt_require(vgem_has_fences(cork->device)); + + bo.width = bo.height = 1; + bo.bpp = 4; + vgem_create(cork->device, &bo); + cork->fence = vgem_fence_attach(cork->device, &bo, VGEM_FENCE_WRITE); + + dmabuf = prime_handle_to_fd(cork->device, bo.handle); + cork->handle = prime_fd_to_handle(fd, dmabuf); + close(dmabuf); + + return cork; +} + +/** + * igt_cork_signal: + * @cork: cork state from igt_cork_new() + * + * This function signals the fence attached to the imported object, thus + * unblocking any stalled execution. + */ +void igt_cork_signal(igt_cork_t *cork) +{ + if (!cork) + return; + + vgem_fence_signal(cork->device, cork->fence); + close(cork->device); + cork->device = 0; +} + +/** + * igt_cork_free: + * @cork: cork state from igt_cork_new() + * @fd: open drm file descriptor + * + * This function signals the fence attached to the imported object (if it + * hasn't already been signaled by igt_cork_signal) and does the necessary + * post-processing. + */ +void igt_cork_free(int fd, igt_cork_t *cork) +{ + if (!cork) + return; + + if (cork->device) + igt_cork_signal(cork); + + gem_close(fd, cork->handle); + free(cork); +} diff --git a/lib/igt_dummyload.h b/lib/igt_dummyload.h index 215425f..d20a867 100644 --- a/lib/igt_dummyload.h +++ b/lib/igt_dummyload.h @@ -29,6 +29,7 @@ #include <time.h> #include "igt_aux.h" +#include "igt_vgem.h" typedef struct igt_spin { unsigned int handle; @@ -51,4 +52,14 @@ void igt_spin_batch_free(int fd, igt_spin_t *spin); void igt_terminate_spin_batches(void); +typedef struct igt_cork { + int device; + uint32_t handle; + uint32_t fence; +} igt_cork_t; + +igt_cork_t *igt_cork_new(int fd); +void igt_cork_signal(igt_cork_t *cork); +void igt_cork_free(int fd, igt_cork_t *cork); + #endif /* __IGT_DUMMYLOAD_H__ */
The "cork" bo (imported bo with attached fence) is used in several tests to stall execution. Moving it to a common place makes the codebase cleaner. Note that the actual test updates is done in follow up patches as it is simpler to do in one go after one more common function is added in the next patch. Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com> Cc: Chris Wilson <chris@chris-wilson.co.uk> --- lib/igt_dummyload.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/igt_dummyload.h | 11 ++++++++ 2 files changed, 86 insertions(+)