[RFC,v2,0/2] ui: Add a Wayland backend for Qemu UI (v2)

Message

Vivek Kasireddy Sept. 13, 2021, 10:20 p.m. UTC

Why does Qemu need a new Wayland UI backend?
The main reason why there needs to be a plain and simple Wayland backend
for Qemu UI is to eliminate the Blit (aka GPU copy) that happens if using
a toolkit like GTK or SDL (because they use EGL). The Blit can be eliminated
by sharing the dmabuf fd -- associated with the Guest scanout buffer --
directly with the Host compositor via the linux-dmabuf (unstable) protocol.
Once properly integrated, it would be potentially possible to have the
scanout buffer created by the Guest compositor be placed directly on a
hardware plane on the Host thereby improving performance. Only Guest 
compositors that use multiple back buffers (at-least 1 front and 1 back)
and virtio-gpu would benefit from this work.

v2:
- Add support for handling/forwarding keyboard and mouse events.
- Augment handling of events to ensure that they are not lost in a
  multi-threaded environment.
- Rebase

Cc: Gerd Hoffmann <[e-mail]>
Cc: Marc-André Lureau <[e-mail]>
Cc: Dongwon Kim <[e-mail]>
Cc: Tina Zhang <[e-mail]>
Cc: Satyeshwar Singh <[e-mail]>

Vivek Kasireddy (2):
  virtio-gpu: Add support for VIRTIO_GPU_F_RELEASE_FENCE
  ui: Add a plain Wayland backend for Qemu UI

 configure                                   |   8 +-
 hw/display/virtio-gpu-base.c                |   3 +
 hw/display/virtio-gpu-udmabuf.c             |   4 +-
 hw/display/virtio-gpu.c                     |  54 +-
 include/hw/virtio/virtio-gpu.h              |   6 +
 include/standard-headers/linux/virtio_gpu.h |   2 +
 meson.build                                 |  33 +
 meson_options.txt                           |   2 +
 qapi/ui.json                                |   3 +
 ui/meson.build                              |  52 ++
 ui/wayland.c                                | 628 ++++++++++++++++++++
 11 files changed, 791 insertions(+), 4 deletions(-)
 create mode 100644 ui/wayland.c

Comments

Daniel P. Berrangé Sept. 14, 2021, 4:29 p.m. UTC | #1

On Mon, Sep 13, 2021 at 03:20:34PM -0700, Vivek Kasireddy wrote:
> Why does Qemu need a new Wayland UI backend?
> The main reason why there needs to be a plain and simple Wayland backend
> for Qemu UI is to eliminate the Blit (aka GPU copy) that happens if using
> a toolkit like GTK or SDL (because they use EGL). The Blit can be eliminated
> by sharing the dmabuf fd -- associated with the Guest scanout buffer --
> directly with the Host compositor via the linux-dmabuf (unstable) protocol.
> Once properly integrated, it would be potentially possible to have the
> scanout buffer created by the Guest compositor be placed directly on a
> hardware plane on the Host thereby improving performance. Only Guest 
> compositors that use multiple back buffers (at-least 1 front and 1 back)
> and virtio-gpu would benefit from this work.

IME, QEMU already suffers from having too many barely maintained UI
implementations and iti s confusing to users. Using a toolkit like GTK
is generally a good thing, even if they don't enable the maximum
theoretical performance, because they reduce the long term maint burden.

I'm far from convinced that we should take on the maint of yet another
UI in QEMU, even if it does have some performance benefit, especially
if implemented using a very low level API like Wayland, that won't let
us easily add rich UI features.

Regards,
Daniel

Vivek Kasireddy Sept. 14, 2021, 11:46 p.m. UTC | #2

Hi Daniel,

> On Mon, Sep 13, 2021 at 03:20:34PM -0700, Vivek Kasireddy wrote:
> > Why does Qemu need a new Wayland UI backend?
> > The main reason why there needs to be a plain and simple Wayland backend
> > for Qemu UI is to eliminate the Blit (aka GPU copy) that happens if using
> > a toolkit like GTK or SDL (because they use EGL). The Blit can be eliminated
> > by sharing the dmabuf fd -- associated with the Guest scanout buffer --
> > directly with the Host compositor via the linux-dmabuf (unstable) protocol.
> > Once properly integrated, it would be potentially possible to have the
> > scanout buffer created by the Guest compositor be placed directly on a
> > hardware plane on the Host thereby improving performance. Only Guest
> > compositors that use multiple back buffers (at-least 1 front and 1 back)
> > and virtio-gpu would benefit from this work.
> 
> IME, QEMU already suffers from having too many barely maintained UI
> implementations and iti s confusing to users. Using a toolkit like GTK
> is generally a good thing, even if they don't enable the maximum
> theoretical performance, because they reduce the long term maint burden.
[Kasireddy, Vivek] The Wayland UI is ~600 lines of code and more than half of
that is just boilerplate; it is also fairly standalone. We don't have any major
complaints against GTK UI (which is ~3000 lines of code, just the Qemu piece
excluding libgtk) except that there is no way to dissociate EGL from it. And, to
keep the UI codebase up-to-date and leverage latest features from GTK (for
example GTK4 and beyond) it would require non-trivial amount of work.
Therefore, I think it'd not be too onerous to maintain something lightweight like
the Wayland UI module.

> 
> I'm far from convinced that we should take on the maint of yet another
> UI in QEMU, even if it does have some performance benefit, especially
[Kasireddy, Vivek] There are novel use-cases coming up particularly with 
the arrival of technologies like SRIOV where the Guest is expected to do all the
rendering and share the fully composited buffer with the Host whose job is to
just present it on the screen. And, in this scenario, if we were to use GTK UI,
the (fullscreen sized) Blits incurred would quickly add up if there are 4-6 Guests
running and presenting content at the same time locally on multiple monitors.
Wayland UI helps in this situation as it does not do any additional rendering on
the Host (no Blit) as it just merely forwards the Guest composited buffer to the
Host compositor.

> if implemented using a very low level API like Wayland, that won't let
> us easily add rich UI features.
[Kasireddy, Vivek] Yes, it is a drawback of Wayland UI that it'd not be possible to
do window decorations/rich UI features but there are users/customers that do not care
for them. I think it should be possible to have both Wayland and GTK UI co-exist
where users can choose GTK UI for fancy features and Wayland UI for performance.

Thanks,
Vivek

> 
> Regards,
> Daniel
> --
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|