mbox series

[v5,0/6] Deliver vGPU display refresh event to userspace

Message ID 20190816023528.30210-1-tina.zhang@intel.com (mailing list archive)
Headers show
Series Deliver vGPU display refresh event to userspace | expand

Message

Zhang, Tina Aug. 16, 2019, 2:35 a.m. UTC
This series tries to send the vGPU display refresh event to user land.

Instead of delivering page flip events only or vblank events only, we 
choose to combine two of them, i.e. post display refresh event at vblanks 
and skip some of them when no page flip happens. Vblanks as upper bound 
are safe and skipping no-page-flip vblanks guarantees both trivial performance 
impacts and good user experience without screen tearing. Plus, we have the 
mask/unmask mechansim providing user space flexibility to switch between 
event-notified refresh and classic timer-based refresh.

In addition, there are some cases that guest app only uses one framebuffer 
for both drawing and display. In such case, guest OS won't do the plane page 
flip when the framebuffer is updated, thus the user land won't be notified 
about the updated framebuffer. Hence, in single framebuffer case, we apply
a heuristic to determine whether it is the case or not. If it is, notify user
land when each vblank event triggers.

v5:
- Introduce a vGPU display irq cap which can notify user space with
  both primary and cursor plane update events through one eventfd. (Gerd & Alex)
v4:
- Deliver page flip event and single framebuffer refresh event bounded 
by display vblanks. (Kechen)
v3:
- Deliver display vblank event instead of page flip event. (Zhenyu)
v2:
- Use VFIO irq chain to get eventfds from userspace instead of adding
a new ABI. (Alex)
v1:
- https://patchwork.kernel.org/cover/10962341/

Kechen Lu (2):
  drm/i915/gvt: Deliver async primary plane page flip events at vblank
  drm/i915/gvt: Add cursor plane reg update trap emulation handler

Tina Zhang (4):
  vfio: Define device specific irq type capability
  vfio: Introduce vGPU display irq type
  drm/i915/gvt: Register vGPU display event irq
  drm/i915/gvt: Deliver vGPU refresh event to userspace

 drivers/gpu/drm/i915/gvt/cmd_parser.c |   6 +-
 drivers/gpu/drm/i915/gvt/display.c    |  49 +++++-
 drivers/gpu/drm/i915/gvt/display.h    |   3 +
 drivers/gpu/drm/i915/gvt/gvt.h        |   6 +
 drivers/gpu/drm/i915/gvt/handlers.c   |  32 +++-
 drivers/gpu/drm/i915/gvt/hypercall.h  |   1 +
 drivers/gpu/drm/i915/gvt/interrupt.c  |   7 +
 drivers/gpu/drm/i915/gvt/interrupt.h  |   3 +
 drivers/gpu/drm/i915/gvt/kvmgt.c      | 230 +++++++++++++++++++++++++-
 drivers/gpu/drm/i915/gvt/mpt.h        |  17 ++
 include/uapi/linux/vfio.h             |  40 ++++-
 11 files changed, 375 insertions(+), 19 deletions(-)

Comments

Zhang, Tina Sept. 3, 2019, 1:26 a.m. UTC | #1
Hi,

Some people are asking whether the display refresh irq could be provided by qemu vfio display?

Some background: currently, we have two display timers. One is provided by QEMU UI and the other one is provided by the vgpu. The vgpu display framebuffer consumers (i.e. QEMU UIs) depend on the UI timer to consume the contents in the vgpu display framebuffer, meanwhile the display framebuffer producer (e.g. gvt-g display model) updates the framebuffer content according to the vblank timer provided by gpu vendor driver. Since these two timers are not synced, tearing could be noticed. 

So, theoretically to solve this tearing problem, we could have two options:

Option 1: Like what we have in this patch set: stop the QEMU UI timer and let both the framebuffer consumers and the framebuffer producer sync to the display refresh event provided by vendor driver. So, QEMU UI can do the refresh depending on this display refresh event to make sure to have a tear-free framebuffer.

Option 2: QEMU provides the emulated display refresh event to the vgpus provided by vendor driver. For vgpus, the display refresh event can be considered as the vblank event which is leveraged by guest window manager to do the plane update or mode-setting.

People are asking if option 2 could be a better choice.

Thanks.

BR,
Tina

> -----Original Message-----
> From: Zhang, Tina
> Sent: Friday, August 16, 2019 10:35 AM
> To: intel-gvt-dev@lists.freedesktop.org
> Cc: Zhang, Tina <tina.zhang@intel.com>; kraxel@redhat.com;
> alex.williamson@redhat.com; kvm@vger.kernel.org; linux-
> kernel@vger.kernel.org; Yuan, Hang <hang.yuan@intel.com>; Lv, Zhiyuan
> <zhiyuan.lv@intel.com>
> Subject: [PATCH v5 0/6] Deliver vGPU display refresh event to userspace
> 
> This series tries to send the vGPU display refresh event to user land.
> 
> Instead of delivering page flip events only or vblank events only, we choose
> to combine two of them, i.e. post display refresh event at vblanks and skip
> some of them when no page flip happens. Vblanks as upper bound are safe
> and skipping no-page-flip vblanks guarantees both trivial performance
> impacts and good user experience without screen tearing. Plus, we have the
> mask/unmask mechansim providing user space flexibility to switch between
> event-notified refresh and classic timer-based refresh.
> 
> In addition, there are some cases that guest app only uses one framebuffer
> for both drawing and display. In such case, guest OS won't do the plane page
> flip when the framebuffer is updated, thus the user land won't be notified
> about the updated framebuffer. Hence, in single framebuffer case, we apply
> a heuristic to determine whether it is the case or not. If it is, notify user land
> when each vblank event triggers.
> 
> v5:
> - Introduce a vGPU display irq cap which can notify user space with
>   both primary and cursor plane update events through one eventfd. (Gerd &
> Alex)
> v4:
> - Deliver page flip event and single framebuffer refresh event bounded by
> display vblanks. (Kechen)
> v3:
> - Deliver display vblank event instead of page flip event. (Zhenyu)
> v2:
> - Use VFIO irq chain to get eventfds from userspace instead of adding a new
> ABI. (Alex)
> v1:
> - https://patchwork.kernel.org/cover/10962341/
> 
> Kechen Lu (2):
>   drm/i915/gvt: Deliver async primary plane page flip events at vblank
>   drm/i915/gvt: Add cursor plane reg update trap emulation handler
> 
> Tina Zhang (4):
>   vfio: Define device specific irq type capability
>   vfio: Introduce vGPU display irq type
>   drm/i915/gvt: Register vGPU display event irq
>   drm/i915/gvt: Deliver vGPU refresh event to userspace
> 
>  drivers/gpu/drm/i915/gvt/cmd_parser.c |   6 +-
>  drivers/gpu/drm/i915/gvt/display.c    |  49 +++++-
>  drivers/gpu/drm/i915/gvt/display.h    |   3 +
>  drivers/gpu/drm/i915/gvt/gvt.h        |   6 +
>  drivers/gpu/drm/i915/gvt/handlers.c   |  32 +++-
>  drivers/gpu/drm/i915/gvt/hypercall.h  |   1 +
>  drivers/gpu/drm/i915/gvt/interrupt.c  |   7 +
>  drivers/gpu/drm/i915/gvt/interrupt.h  |   3 +
>  drivers/gpu/drm/i915/gvt/kvmgt.c      | 230 +++++++++++++++++++++++++-
>  drivers/gpu/drm/i915/gvt/mpt.h        |  17 ++
>  include/uapi/linux/vfio.h             |  40 ++++-
>  11 files changed, 375 insertions(+), 19 deletions(-)
> 
> --
> 2.17.1
Tian, Kevin Sept. 3, 2019, 7:43 a.m. UTC | #2
> From: Zhang, Tina
> Sent: Tuesday, September 3, 2019 9:27 AM
> 
> Hi,
> 
> Some people are asking whether the display refresh irq could be provided by
> qemu vfio display?
> 
> Some background: currently, we have two display timers. One is provided by
> QEMU UI and the other one is provided by the vgpu. The vgpu display
> framebuffer consumers (i.e. QEMU UIs) depend on the UI timer to consume
> the contents in the vgpu display framebuffer, meanwhile the display
> framebuffer producer (e.g. gvt-g display model) updates the framebuffer
> content according to the vblank timer provided by gpu vendor driver. Since
> these two timers are not synced, tearing could be noticed.
> 
> So, theoretically to solve this tearing problem, we could have two options:
> 
> Option 1: Like what we have in this patch set: stop the QEMU UI timer and let
> both the framebuffer consumers and the framebuffer producer sync to the
> display refresh event provided by vendor driver. So, QEMU UI can do the
> refresh depending on this display refresh event to make sure to have a tear-
> free framebuffer.
> 
> Option 2: QEMU provides the emulated display refresh event to the vgpus
> provided by vendor driver. For vgpus, the display refresh event can be
> considered as the vblank event which is leveraged by guest window manager
> to do the plane update or mode-setting.
> 
> People are asking if option 2 could be a better choice.
> 

I think the answer is specific to different usages. None is perfect in all
scenarios. The key is to find out who is the primary display to show the 
guest framebuffer, then that guy is cared about tearing thus should be 
the source of vblank event:

1) Guest framebuffer is directly programmed to, and shown in the local
display. In such case, the physical vblank event should be the source of
virtual vblank event, i.e. kernel GVT-g driver should emulate the event
in host vblank event handler.

2) Guest framebuffer is shown in the Qemu UI. Then, naturally Qemu UI
should be the source of virtual vblank event, based on its own tearing
requirement:

	a) Guest framebuffer is shown in the local application window,
which is updated by the Qemu UI timer. Then virtual vblank should be
sent at end of the UI timer handler, to make sure no change happened
when the guest framebuffer is composited as the source texture.

	b) Qemu UI may program guest framebuffer directly to the
physical plane, through whatever interface that kernel gfx driver provides.
In such case, Qemu UI should wait for vblank notification from kernel, 
and then trigger the virtual vblank event.

	c) Qemu UI may further route the guest framebuffer to remote
client, e.g. through vnc. Then virtual vblank event should be emulated
according to tearing requirement in vnc protocol.

3) combination of 1) and 2), where either local display or Qemu UI is
considered as primary display with the other for diagnostic purpose.
Then the tearing of the primary display should drive the emulation of
virtual vblank, while the other one may suffer from tearing issue.

Accordingly, we may want a kernel interface allowing user space
to specify the source of vblank emulation: in kernel or from user
space. If the former is specified, then virtual vblank is driven by 
physical vblank event. Otherwise, user space should trigger the
virtual vblank injection. Just leave all the decisions to user space. :-)

Thanks
Kevin
Gerd Hoffmann Sept. 5, 2019, 7:48 a.m. UTC | #3
Hi,

> Option 2: QEMU provides the emulated display refresh event to the
> vgpus provided by vendor driver. For vgpus, the display refresh event
> can be considered as the vblank event which is leveraged by guest
> window manager to do the plane update or mode-setting.

> People are asking if option 2 could be a better choice.

Certainly worth trying, maybe it even makes sense to implement both and
let qemu pick one, possibly even switch them at runtime.

qemu can change the refresh rate.  vnc and sdl use that to reduce the
refresh rate in case nobody is looking (no vnc client connected, sdl
window minimized).  It surely makes sense to make that visible to the
guest so it can throttle display updates too.  I'm not sure vblank is
the way to go though, guests might run into vblank irq timeouts in case
the refresh rate is very low ...

cheers,
  Gerd
Zhang, Tina Sept. 17, 2019, 9:27 a.m. UTC | #4
> -----Original Message-----
> From: intel-gvt-dev [mailto:intel-gvt-dev-bounces@lists.freedesktop.org] On
> Behalf Of kraxel@redhat.com
> Sent: Thursday, September 5, 2019 3:49 PM
> To: Zhang, Tina <tina.zhang@intel.com>
> Cc: kvm@vger.kernel.org; linux-kernel@vger.kernel.org; Yuan, Hang
> <hang.yuan@intel.com>; alex.williamson@redhat.com; Lv, Zhiyuan
> <zhiyuan.lv@intel.com>; intel-gvt-dev@lists.freedesktop.org
> Subject: Re: [PATCH v5 0/6] Deliver vGPU display refresh event to userspace
> 
>   Hi,
> 
> > Option 2: QEMU provides the emulated display refresh event to the
> > vgpus provided by vendor driver. For vgpus, the display refresh event
> > can be considered as the vblank event which is leveraged by guest
> > window manager to do the plane update or mode-setting.
> 
> > People are asking if option 2 could be a better choice.
> 
> Certainly worth trying, maybe it even makes sense to implement both and
> let qemu pick one, possibly even switch them at runtime.
> 
> qemu can change the refresh rate.  vnc and sdl use that to reduce the
> refresh rate in case nobody is looking (no vnc client connected, sdl window
> minimized).  It surely makes sense to make that visible to the guest so it can
> throttle display updates too.  I'm not sure vblank is the way to go though,
> guests might run into vblank irq timeouts in case the refresh rate is very
> low ...

Indeed, low vblank rate isn't expected by guest gfx driver. It complains about the timeout error all the time, when the vblank is low. 

Currently, gvt-g provides full virtualized display model (a.k.a. not pv). And the option 2 is more like a pv solution for performance optimization, which is of course a very good proposal. Since the two options have no dependency, this patch-set limits its scope to only include option 1. Thanks.


BR,
Tina

> 
> cheers,
>   Gerd
> 
> _______________________________________________
> intel-gvt-dev mailing list
> intel-gvt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gvt-dev