Message ID | 86fuhrka4t.fsf@hiro.keithp.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
On Sat, Apr 01, 2017 at 03:58:58PM -0700, "Keith Packard" wrote: > > As a part of the DRM leasing work, we need a way to have the X server > create a lease and pass it back to an X client. Here's a proposal for > the RandR specification changes necessary for that. The basic plan is > pretty simple: > > 1. Expose the ability to create a lease for a set of CRTCs and > OUTPUTs. The X server will have to pick a suitable encoder as that's > not visible via RandR. On that, I think we could just unconditionally hand leases all encoders. Encoders turned out to be a bit an uapi mistake. Neither setcrtc nor atomic use it, the kernel always selects the right encoder for you. It's only exposed to give userspace some hints wrt routing (and it's pretty bad at describing modern restrictions, which often means you get a 1:1 encoder/connector mapping). Unconditionally exposing all encoders for all lessees would fix all these troubles. Note that there's also no properties on encoders, those only exist on crtc, connector and planes. Kinda more a comment on the kernel side than for xrandr. -Daniel > 2. Provide a way to hide some monitors from other clients using EDID > manufacturer ids and product codes. Outputs with EDID properties > matching the grab will report 'disconnected' to all clients other > than the grabbing client. This way, the desktop environment never > knows that an HMD has been plugged in, so there's no transient > flicker of desktop being presented to it. > > I wanted to make it possible for the X server to set the mode on the > leased outputs, but I'm not sure how -- I don't want to display the X > screen on them, and there's no other frame buffer I can name over the > wire. For now, that means the leasing client will need to set a mode > itself. If that fails, it can go whack the X configuration over RandR > and try again, which is all the X server would do anyways. > > Comments are welcome; I'll go try to write the code in the next few > days; it all looks pretty easy at this point. > > diff --git a/randrproto.txt b/randrproto.txt > index 74b7c36..8dded63 100644 > --- a/randrproto.txt > +++ b/randrproto.txt > @@ -1,6 +1,6 @@ > The X Resize, Rotate and Reflect Extension > - Version 1.5.0 > - 2015-03-14 > + Version 1.6.0 > + 2017-04-01 > > Jim Gettys > Jim.Gettys@hp.com > @@ -9,9 +9,7 @@ > Hewlett Packard Company > > Keith Packard > - keith.packard@intel.com > - Open Source Technology Center > - Intel Corporation > + keithp@keithp.com > > 1. Introduction > > @@ -186,6 +184,24 @@ consider as single viewable areas. > Xinerama's information now comes from the Monitors instead of directly > from the CRTCs. The Monitor marked as Primary will be listed first. > > +1.6. Introduction to version 1.6 of the extension > + > +Version 1.6 adds resource leasing. > + > + • A 'Lease' is a collection of crtcs and outputs which are made > + available to a client for direct access via kernel KMS and DRM > + APIs. This is done by passing a suitable file descriptor back to > + the client which has access to those resources. While leased, those > + resources aren't used by the X server. > + > +Version 1.6 adds EDID-based output 'grabbing'. > + > + • An 'Output Grab' matches a set of EDID Manufacturer ID and product > + codes. For outputs with matching EDID values, the connected status > + of the output is only visible to the grabbing client. Other clients > + will see the output as disconnected. Attempts to configure the > + grabbed output by other clients will fail. > + > 1.99 Acknowledgments > > Our thanks to the contributors to the design found on the xpert mailing > @@ -273,6 +289,10 @@ Mode > Provider > A value for a PROVIDER argument does not name a defined PROVIDER. > > +OutputGrab > + A value for an OUTPUTGRAB argument does not name a defined > + OUTPUTGRAB > + > ❧❧❧❧❧❧❧❧❧❧❧ > > 5. Protocol Types > @@ -419,6 +439,23 @@ MONITORINFO { name: ATOM > > ❧❧❧❧❧❧❧❧❧❧❧ > > +5.7. Protocol Types added in version 1.6 of the extension > + > +OUTPUTGRAB { XID } > + > +EDIDMATCH { id: CARD16 > + code-min: CARD16 > + code-max: CARD16 } > + > + These values come from the EDID specification. 'id' is the > + Manufacturer ID value which is bytes 8 and 9 in the EDID > + packet, stored in big endian order (MSB first). 'code-min' and > + 'code-max' define a closed-interval of Manufacturer product > + codes, which is byte 10 and 11 of the EDID packet, stored in > + little endian order (LSB first). > + > + ❧❧❧❧❧❧❧❧❧❧❧ > + > 6. Extension Initialization > > The name of this extension is "RANDR". > @@ -1666,6 +1703,67 @@ dynamic changes in the display environment. > window of the screen. > > ❧❧❧❧❧❧❧❧❧❧❧ > + > +7.6. Extension Requests added in version 1.6 of the extension. > + > +┌─── > + RRCreateLease > + window : WINDOW > + crtcs: LISTofCRTC > + outputs: LISTofOUTPUT > + ▶ > + nfd: CARD8 > + lease: FD > +└─── > + Errors: Window, Access, Value, CRTC, Output > + > + Creates a new Lease for the specified crtcs and outputs from > + the screen defined by 'window'. Returns a KMS/DRM file > + descriptor which can control the leased objects directly > + through the kernel. While leased, all resources will appear to > + be 'useless' to clients other than the leasing client as > + follows: > + > + • Crtcs are reported as having no 'possible-outputs' and all > + other values reported as if the crtc were disabled. > + > + • Outputs are reported as having no crtcs they can be > + connected to, no clones they can share a crtc with, will > + report a connection status of Disconnected, and will show > + the current crtc as Disabled. > + > + The lease remains in effect until the file descriptor is > + closed, even if the client holding the lease disconnects from > + the X server. > + > + Returns an Access error if any of the named resources are in > + use or already leased to another client. > + > + Returns a Match error if any of the named resources are in use > + by the X server. > + > +┌─── > + RRCreateOutputGrab > + window : WINDOW > + outputgrab: OUTPUTGRAB > + matches: LISTofEDIDMATCH > +└─── > + Errors: Window, Access, Value, CRTC, Output > + > + Creates an Output Grab with the specified ID on the screen > + associated with 'window'. Any output containing an EDID > + property matching the grab will appear to be disconnected to > + all clients other than the grabbing client. > + > +┌─── > + RRDestroyOutputGrab > + outputgrab: OUTPUTGRAB > +└─── > + Errors: OutputGrab > + > + Destroys the named OUTPUTGRAB. > + > + ❧❧❧❧❧❧❧❧❧❧❧ > 8. Extension Events > > Clients MAY select for ConfigureNotify on the root window to be > > > -- > -keith > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel
Daniel Vetter <daniel@ffwll.ch> writes: > On that, I think we could just unconditionally hand leases all encoders. > Encoders turned out to be a bit an uapi mistake. Well, given the complexity of hardware these days, even crtcs would probably best have been hidden... > Neither setcrtc nor atomic use it, the kernel always selects the right > encoder for you. It's only exposed to give userspace some hints wrt > routing (and it's pretty bad at describing modern restrictions, which > often means you get a 1:1 encoder/connector mapping). Unconditionally > exposing all encoders for all lessees would fix all these troubles. Yeah, I can't find encoders passed into any kernel api, other than the getencoder call. It seems like we can leave them as shared objects not subject to leasing as they are query-only. I'll go ahead and do that. The encoder crtc set still needs to be filtered in the query operation so that the client knows which crtc to use for each output. Now as to how we report the kernel objects that have been leased, we have two options: 1) Report them back via the X protocol 2) Have the client query the resulting lease for its contents I already suggested that the drm query API should be changed to report both type and id for each leased object, that would make it sufficient here. Instead of duplicating that over the X protocol, I suggest we just use the adjusted kernel API. > Note that there's also no properties on encoders, those only exist on > crtc, connector and planes. Any thoughts on access control for properties? Right now, the set property ioctl checks for access to the containing object, but there's no check when querying properties. This means that you don't need to add leases on properties. > Kinda more a comment on the kernel side than for xrandr. It's all tied together :-)
On Sun, Apr 02, 2017 at 12:58:33PM -0700, Keith Packard wrote: > Daniel Vetter <daniel@ffwll.ch> writes: > > > On that, I think we could just unconditionally hand leases all encoders. > > Encoders turned out to be a bit an uapi mistake. > > Well, given the complexity of hardware these days, even crtcs would > probably best have been hidden... > > > Neither setcrtc nor atomic use it, the kernel always selects the right > > encoder for you. It's only exposed to give userspace some hints wrt > > routing (and it's pretty bad at describing modern restrictions, which > > often means you get a 1:1 encoder/connector mapping). Unconditionally > > exposing all encoders for all lessees would fix all these troubles. > > Yeah, I can't find encoders passed into any kernel api, other than the > getencoder call. It seems like we can leave them as shared objects not > subject to leasing as they are query-only. I'll go ahead and do > that. The encoder crtc set still needs to be filtered in the query > operation so that the client knows which crtc to use for each output. > > Now as to how we report the kernel objects that have been leased, we > have two options: > > 1) Report them back via the X protocol > > 2) Have the client query the resulting lease for its contents > > I already suggested that the drm query API should be changed to report > both type and id for each leased object, that would make it sufficient > here. Instead of duplicating that over the X protocol, I suggest we just > use the adjusted kernel API. Hm, if you restrict getresources and getplanes, you'll get your leased objects query api. Iirc that part was missing in your kernel patch. And it gives you exaclty what you want: per-type list of object ids. > > > Note that there's also no properties on encoders, those only exist on > > crtc, connector and planes. > > Any thoughts on access control for properties? Right now, the set > property ioctl checks for access to the containing object, but there's > no check when querying properties. This means that you don't need to add > leases on properties. This changes with atomic - without properties you can't change anything using atomic. Otoh maybe we just want to add checks on the containing object for now, some of the finer semantic points like "don't modeset" or "don't steal shared resources at all" can't be expressed (in a generic way at least) by restricting to specific properties anyway. E.g. on many chips changing the fb is only ok if you keep the same stride, pixel format and modifier (~ tiling).
Daniel Vetter <daniel@ffwll.ch> writes: > Hm, if you restrict getresources and getplanes, you'll get your leased > objects query api. Iirc that part was missing in your kernel patch. And it > gives you exaclty what you want: per-type list of object ids. Hrm. I think that's one Dave didn't want to restrict so that applications don't see resources disappear and get confused. But, perhaps a simple variant of those instead of a completely different API. > This changes with atomic - without properties you can't change anything > using atomic. Otoh maybe we just want to add checks on the containing > object for now, That's what is already in place -- property setting is always relative to the containing resource, so we don't need any specific controls on the properties themselves unless we wanted to restrict queries, which doesn't seem necessary.
On Mon, Apr 03, 2017 at 09:41:34AM -0700, Keith Packard wrote: > Daniel Vetter <daniel@ffwll.ch> writes: > > > Hm, if you restrict getresources and getplanes, you'll get your leased > > objects query api. Iirc that part was missing in your kernel patch. And it > > gives you exaclty what you want: per-type list of object ids. > > Hrm. I think that's one Dave didn't want to restrict so that > applications don't see resources disappear and get confused. But, > perhaps a simple variant of those instead of a completely different API. On the X side we don't want to restrict. But the VR client better be able to deal with not seeing or being able to use everything. Also if this confuses VR, then another reason why we want to make leases invariant and only allow pure revoke, not changing the list. -Daniel
Daniel Vetter <daniel@ffwll.ch> writes: > Also if this confuses VR, then another reason why we want to make leases > invariant and only allow pure revoke, not changing the list. I'm not sure why you want this to be asymmetrical, nor why you would expect lessees to be any more competent at dealing with hotplug than the lessor. One use case already proposed for this API was to allow for multi-seat, where the lessee would be an existing window system, which we already accept as being incompetent at dealing with resource hotplug. I imagine that in this case, a newly plugged monitor would be detected by the multi-seat manager (logind?) and added to one of the existing leases, along with a suitable CRTC resource. For this to work, the lessee will need to already know about those resources and only have their connectivity status hidden.
On Mon, Apr 03, 2017 at 03:50:33PM -0700, Keith Packard wrote: > Daniel Vetter <daniel@ffwll.ch> writes: > > > Also if this confuses VR, then another reason why we want to make leases > > invariant and only allow pure revoke, not changing the list. > > I'm not sure why you want this to be asymmetrical, nor why you would > expect lessees to be any more competent at dealing with hotplug than the > lessor. > > One use case already proposed for this API was to allow for multi-seat, > where the lessee would be an existing window system, which we already > accept as being incompetent at dealing with resource hotplug. I imagine > that in this case, a newly plugged monitor would be detected by the > multi-seat manager (logind?) and added to one of the existing leases, > along with a suitable CRTC resource. For this to work, the lessee will > need to already know about those resources and only have their > connectivity status hidden. The multi-seat thing sounds like vapourware, I think we should care about the vr use-case for now, and only that one. And for that restricting stuff is perfectly fine. Of course we can design the entire thing in a way that doesn't draw us into a corner in the future right away, but that's mostly on the implementation side. For VR itself I'd go as far as saying that probably our "create lease" ioctl should have only the semantics we need to pass one crtc+primary plane for pageflipping in a VR compositor, expressed in a flag. All the details about additional corner cases are just so unclear to me (and there's not even a clear use case that will materialize) that I don't think having the uapi is worth it. Too close to the "I'll regret this immediately" bucket :-) Cheers, Daniel
Daniel Vetter <daniel@ffwll.ch> writes: > The multi-seat thing sounds like vapourware, I think we should care about > the vr use-case for now, and only that one. Ok, I can live with that, even if I like the idea of a slightly more general solution. > For VR itself I'd go as far as saying that probably our "create lease" > ioctl should have only the semantics we need to pass one crtc+primary > plane for pageflipping in a VR compositor, expressed in a flag. Yeah, we can't express planes through X anyways. I'll leave the kernel API with multiple planes as that's actually simpler than having it validate that only a single plane is in the lease. > All the details about additional corner cases are just so unclear to > me (and there's not even a clear use case that will materialize) that > I don't think having the uapi is worth it. Too close to the "I'll > regret this immediately" bucket :-) Removing the 'ChangeLease' ioctl eliminates a bunch of complexity in the code, and means I don't even have to think about sending events. I'll also go ahead and remove the ability to hide resources from the lessor. Thanks, as always, for your thoughtful review. ps -- Any thoughts on whether the X request should include the mode to use? Doing that would let us restrict the lessee from setting modes, and avoid potential resource issues with the window system. However, it would also require providing a scanout buffer in the request.
On Tue, Apr 04, 2017 at 08:53:45AM -0700, Keith Packard wrote: > Daniel Vetter <daniel@ffwll.ch> writes: > > > The multi-seat thing sounds like vapourware, I think we should care about > > the vr use-case for now, and only that one. > > Ok, I can live with that, even if I like the idea of a slightly more > general solution. > > > For VR itself I'd go as far as saying that probably our "create lease" > > ioctl should have only the semantics we need to pass one crtc+primary > > plane for pageflipping in a VR compositor, expressed in a flag. > > Yeah, we can't express planes through X anyways. I'll leave the kernel > API with multiple planes as that's actually simpler than having it > validate that only a single plane is in the lease. > > > All the details about additional corner cases are just so unclear to > > me (and there's not even a clear use case that will materialize) that > > I don't think having the uapi is worth it. Too close to the "I'll > > regret this immediately" bucket :-) > > Removing the 'ChangeLease' ioctl eliminates a bunch of complexity in the > code, and means I don't even have to think about sending events. I'll > also go ahead and remove the ability to hide resources from the lessor. > > Thanks, as always, for your thoughtful review. > > ps -- Any thoughts on whether the X request should include the mode to > use? Doing that would let us restrict the lessee from setting modes, > and avoid potential resource issues with the window system. However, it > would also require providing a scanout buffer in the request. Yeah I think that's a pretty neat idea to reduce the lease complexity even more. If the VR compositor is unhappy and wants a different mode, it can simply nuke the lease and as for a new one. Forgot to say that :-) -Daniel
Daniel Vetter <daniel@ffwll.ch> writes: > Yeah I think that's a pretty neat idea to reduce the lease complexity even > more. If the VR compositor is unhappy and wants a different mode, it can > simply nuke the lease and as for a new one. Forgot to say that :-) Not sure it changes the lease complexity, but it reduces the potential interference with the X server after the lease is created. Hrm. Thinking about the impact on X a bit more, this seems hard - you can't just display the root window in the HMD, so you need a frame buffer to use. The VR compositor can construct this knowing the planned X mode, but, we then have to wire it through the whole X mode set infrastructure, which is not exactly set up to do that. I'll go look at the code in more detail, but I suspect the easiest plan will be to have the VR compositor set its own mode. That may fail if X is consuming too many display resources, but that doesn't seem significantly different from having the lease fail. This doesn't change the kernel API at all, so we can figure out the X bits separately from the kernel bits.
On 02/04/17 07:58 AM, Keith Packard wrote: > > 2. Provide a way to hide some monitors from other clients using EDID > manufacturer ids and product codes. Outputs with EDID properties > matching the grab will report 'disconnected' to all clients other > than the grabbing client. This way, the desktop environment never > knows that an HMD has been plugged in, so there's no transient > flicker of desktop being presented to it. When no such special client (Steam?) is running, the desktop environment will try to use the HMD anyway, right? So the expected use case would be for the user to start a special client first and only plug in the HMD afterwards? That seems a bit inconvenient.
Michel Dänzer <michel@daenzer.net> writes: > When no such special client (Steam?) is running, the desktop environment > will try to use the HMD anyway, right? So the expected use case would be > for the user to start a special client first and only plug in the HMD > afterwards? That seems a bit inconvenient. I'd love a better alternative, but this is the best I've come up with. Of course, it needn't be the actual VR client, it could be a stub application that popped open a 'which app would you like to run on the HMD' chooser of some kind, or even hooks in the desktop system that managed that. Suggestions very much encouraged.
On Thu, 06 Apr 2017 20:02:23 -0700 Keith Packard <keithp@keithp.com> wrote: > Michel Dänzer <michel@daenzer.net> writes: > > > When no such special client (Steam?) is running, the desktop environment > > will try to use the HMD anyway, right? So the expected use case would be > > for the user to start a special client first and only plug in the HMD > > afterwards? That seems a bit inconvenient. > > I'd love a better alternative, but this is the best I've come up > with. > > Of course, it needn't be the actual VR client, it could be a stub > application that popped open a 'which app would you like to run on the > HMD' chooser of some kind, or even hooks in the desktop system that > managed that. > > Suggestions very much encouraged. Hi, we need some kind of a database to recognize HMDs in any case, right? It would be best if the database was shared, so that everyone could recognize all HMDs, at least as far as one can do that based on EDID. If we had such a database, perhaps interfaced with a library, when how about Xorg using that library to automatically recognize and hide HMDs? That library would of course be also used by all HMD-supporting Wayland servers. If there was such a library, maybe it could also handle EDID parsing once and for all... a libinput for outputs? Btw. I was also told at #openhmd that some HMDs do not appear as connected outputs until you specifically turn it on via USB. So there is going to be a hotplug, and you'd want to avoid sending that to normal X11 clients so that they won't race with the special VR client grabbing it. Thanks, pq
Pekka Paalanen <ppaalanen@gmail.com> writes: > we need some kind of a database to recognize HMDs in any case, right? > It would be best if the database was shared, so that everyone could > recognize all HMDs, at least as far as one can do that based on EDID. Yeah, I think you've got some good ideas here. Here are some questions which this raises: * What should we do with an HMD which is in the database but for which no application is installed on the host? * Entries in the database should probably have names an versions, so that clients could have some way to identify whether they support what has been connected. Would providing the manufacturer ID and product codes be sufficient here? * Should the database be visible and managed over X? Or should it be hidden from clients and only visible to the X server? * We want to generate events when the HMD is plugged in; how should clients select for these? > If we had such a database, perhaps interfaced with a library, when how > about Xorg using that library to automatically recognize and hide HMDs? > That library would of course be also used by all HMD-supporting Wayland > servers. I'm not sure a library is necessary, if the database was a directory full of files. > If there was such a library, maybe it could also handle EDID parsing > once and for all... a libinput for outputs? Yeah, not a terrible idea, but possibly more yak shaving than we really need to get into here. > Btw. I was also told at #openhmd that some HMDs do not appear as > connected outputs until you specifically turn it on via USB. So there > is going to be a hotplug, and you'd want to avoid sending that to > normal X11 clients so that they won't race with the special VR client > grabbing it. It'd still get 'hot plugged' in the X server when enabled over USB, so we'll want to hide that from regular clients in some way.
On Sun, 09 Apr 2017 10:27:31 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > > we need some kind of a database to recognize HMDs in any case, right? > > It would be best if the database was shared, so that everyone could > > recognize all HMDs, at least as far as one can do that based on EDID. > > Yeah, I think you've got some good ideas here. Here are some questions > which this raises: > > * What should we do with an HMD which is in the database but for which > no application is installed on the host? IMHO, if nothing is providing a picture intended for the HMD, the HMD should be off. There is no use in showing an arbitrary image that does not look right, is there? For the rest of the questions I can't really say anything, except that even if the database was just a bunch of files, you still need code to access it, and probably something to ensure the entries are well-formed, so that everyone will agree on how to parse them. One should probably decide whether the database will only answer the question "is it a HMD?" or can it provide other information as well. Thanks, pq
On 04/04/2017 06:48 PM, Keith Packard wrote: > Daniel Vetter <daniel@ffwll.ch> writes: > >> Yeah I think that's a pretty neat idea to reduce the lease complexity even >> more. If the VR compositor is unhappy and wants a different mode, it can >> simply nuke the lease and as for a new one. Forgot to say that :-) > > Not sure it changes the lease complexity, but it reduces the potential > interference with the X server after the lease is created. > > Hrm. Thinking about the impact on X a bit more, this seems hard - you > can't just display the root window in the HMD, so you need a frame > buffer to use. The VR compositor can construct this knowing the planned > X mode, but, we then have to wire it through the whole X mode set > infrastructure, which is not exactly set up to do that. > > I'll go look at the code in more detail, but I suspect the easiest > plan will be to have the VR compositor set its own mode. That may fail > if X is consuming too many display resources, but that doesn't seem > significantly different from having the lease fail. > > This doesn't change the kernel API at all, so we can figure out the X > bits separately from the kernel bits. > Hi, as input from a highly interested future user of such api's: An additional use case beyond VR compositors, at least highly valuable for my kind of apps (= neuroscience / vision science / medical research toolkits) would be to get fullscreen exclusive control over regular outputs / displays. My use cases run about 98% of the time in fullscreen exclusive mode and want as little interference from the windowing system / desktop environment as possible, with as much low level control as possible. It still needs windowed mode for same cases and needs a windowing system up and running. Atm. under X i have to hope that fullscreen unredirection works to get me page flipping for single display monoscopic stimulation and dual-display stereoscopic stuff. And then there's the popular "Regular desktop GUI for interaction" + separate displays for "fullscreen + page flipping for controlled presentation" case. Atm. i have to use custom xorg.confs + dual/multi-x-screen + ZaphodHeads, a static configuration with all the logout/login dance / no display hotplug for configuration change. Workable under X (minus the occasional ZaphodHeads corner case bugs), but somewhat clunky. So if this would give me a plug & play dynamic replacement for ZaphodHeads and xorg.conf fiddling that would be _very_ valuable. The RRCreateLease requests looks as if i could get that for regular non-HMD video outputs as well? And the RRCreateOutputGrab would be mostly to avoid flicker when plugging HMD's or other special purpose devices, but wouldn't be strictly needed for a regular X-client to get a lease on a set of outputs? As far as controllable properties on a lease go, i'd find it very useful if i could have control over framebuffer formats, e.g., being able to select 10 bit scanout formats would be very useful, but is afaik something that X currently doesn't expose with most drivers, especially not for regular desktop mode. Set/Get Gamma tables, dithering properties, other output properties, modesetting would be also important. On X i can do that via RandR, but in the Wayland world, much of this stuff is afaik often restricted to privileged clients or proprietary per-compositor protocols. That's a big upcoming problem for me in the Wayland world, and lease support could be a very good solution for me. If the underlying DRM leases allow me to control this stuff, and Wayland would gain similar extensions to lease outputs for fullscreen exclusive control, i could have one drm/kms backend that can be mostly agnostic of X vs. Wayland / different Wayland compositor flavors. Basically my vote to expose as much flexility in modesetting / properties for the exposed leases as possible on the kernel and X side. thanks, -mario > > > _______________________________________________ > xorg-devel@lists.x.org: X.Org development > Archives: http://lists.x.org/archives/xorg-devel > Info: https://lists.x.org/mailman/listinfo/xorg-devel >
Mario Kleiner <mario.kleiner.de@gmail.com> writes: > as input from a highly interested future user of such api's: Thanks much for taking a look at this. > My use cases run about 98% of the time in fullscreen > exclusive mode and want as little interference from the windowing system > / desktop environment as possible, with as much low level control as > possible. It still needs windowed mode for same cases and needs a > windowing system up and running. The new APIs can give you exclusive access to the display resources, bypassing the window system entirely. > Atm. under X i have to hope that fullscreen unredirection works to get > me page flipping for single display monoscopic stimulation and > dual-display stereoscopic stuff. And then there's the popular "Regular > desktop GUI for interaction" + separate displays for "fullscreen + page > flipping for controlled presentation" case. You're still depending on the window system server for timely page flips though; the main reason we're doing the leasing work is to get that variable out of the equation, eliminating any window system scheduling jitter from the regular screen updates on the HMD. > The RRCreateLease requests looks as if i could get that for regular > non-HMD video outputs as well? Yes, there's almost no way we could restrict it even if we wanted to. I'm doing testing with two standard monitors, but any hardware at all will work. > And the RRCreateOutputGrab would be mostly to avoid flicker when > plugging HMD's or other special purpose devices, but wouldn't be > strictly needed for a regular X-client to get a lease on a set of > outputs? Yes, just to avoid having the desktop extend itself onto the HMD, even briefly. The two components of the RandR changes are logically separate, but should be useful in combination when using HMD displays. > As far as controllable properties on a lease go, i'd find it very useful > if i could have control over framebuffer formats, e.g., being able to > select 10 bit scanout formats would be very useful, but is afaik > something that X currently doesn't expose with most drivers, especially > not for regular desktop mode. You have the full KMS api at your disposal; do whatever you like :-) > If the underlying DRM leases allow me to control this stuff, and Wayland > would gain similar extensions to lease outputs for fullscreen exclusive > control, i could have one drm/kms backend that can be mostly agnostic of > X vs. Wayland / different Wayland compositor flavors. Right, that's another benefit here -- allowing HMD applications to be largely window system independent, except for acquiring the initial lease. > Basically my vote to expose as much flexility in modesetting / > properties for the exposed leases as possible on the kernel and X > side. I'll have a second cut of the kernel API changes ready in another day or so; that will eliminate the ability to change an existing lease, so you'll have to acquire all of the resources you need all at once. Otherwise, it will look quite similar at the user API level. The RandR protocol changes will also need another spin; it sounds like we want to configure the set of 'special' monitors and have those never reported as connected via the current API. I don't think that will affect your use case at all, and the other part of the protocol for creating a lease won't change.
On Tue, Apr 4, 2017 at 3:02 AM, Daniel Vetter <daniel@ffwll.ch> wrote: > On Mon, Apr 03, 2017 at 03:50:33PM -0700, Keith Packard wrote: >> Daniel Vetter <daniel@ffwll.ch> writes: >> >> > Also if this confuses VR, then another reason why we want to make leases >> > invariant and only allow pure revoke, not changing the list. >> >> I'm not sure why you want this to be asymmetrical, nor why you would >> expect lessees to be any more competent at dealing with hotplug than the >> lessor. >> >> One use case already proposed for this API was to allow for multi-seat, >> where the lessee would be an existing window system, which we already >> accept as being incompetent at dealing with resource hotplug. I imagine >> that in this case, a newly plugged monitor would be detected by the >> multi-seat manager (logind?) and added to one of the existing leases, >> along with a suitable CRTC resource. For this to work, the lessee will >> need to already know about those resources and only have their >> connectivity status hidden. > > The multi-seat thing sounds like vapourware, I think we should care about > the vr use-case for now, and only that one. And for that restricting stuff > is perfectly fine. Of course we can design the entire thing in a way that > doesn't draw us into a corner in the future right away, but that's mostly > on the implementation side. For VR itself I'd go as far as saying that > probably our "create lease" ioctl should have only the semantics we need > to pass one crtc+primary plane for pageflipping in a VR compositor, > expressed in a flag. All the details about additional corner cases are > just so unclear to me (and there's not even a clear use case that will > materialize) that I don't think having the uapi is worth it. Too close to > the "I'll regret this immediately" bucket :-) > I don't know about vaprware. There were a number of attempts to provide static allocation if display resources over the years to solve things like custom zaphod configs and users wanting to use multiple heads for separate users (which currently ends up in various zephyr hacks IRC). Lots of patches were proposed, none landed. I think there is a definite use case there. Why do we need to make X aware of the lease stuff? What about having some pre-X configuration that decides how to split up the display resources. It could be user defined static or dynamic based on what is attached. You can just start X on the device nodes with whatever resources they are assigned. In the multi-user case, you can statically assign resources to each node; in the VR case, you can detect the HMD and automatically assign it's resources to a separate node or override if necessary. Alex
Alex Deucher <alexdeucher@gmail.com> writes: > I think there is a definite use case there. I agree. What we're missing right now is someone interested in driving an implementation of that use case to help define the right interfaces. Lacking that, we're not likely to come up with a good solution on our own. I think that's what Daniel is concerned about -- specifying something in the absence of an implementation using it. I took a stab at this and added the ability to change leases on the fly, and to create 'sub-leases' from an existing lease. He's pushing back on those features, and I think his reasons are sound. > Why do we need to make X aware of the lease stuff? What about having > some pre-X configuration that decides how to split up the display > resources. For multi-user, this makes a lot of sense; you'd want the system to allocate resources between users to allow them to operate fairly independently. For single-user with a hot-plugged HMD, I'd suggest that having X involved makes a lot of sense -- you may have to interact with the user to reduce resource consumption so that the HMD can be driven successfully, and that will involve poking at the X configuration. > It could be user defined static or dynamic based on what is attached. > You can just start X on the device nodes with whatever resources they > are assigned. In the multi-user case, you can statically assign > resources to each node; in the VR case, you can detect the HMD and > automatically assign it's resources to a separate node or override if > necessary. I don't think we've precluded this for a multi-user environment, and I think it's a good plan in the abstract. I will, however, suggest that asking for VR applications to wait for the desktop environment to be re-architected so that display resources can be centrally allocated by a new 'display resource manager' seems like a rather steep requirement. What I do want to ensure is that these two use cases can both be supported by the kernel interfaces we define, and that we can work in user space on either design going forward. If you'd like to start exploring the design of such a central service, that'd be awesome.
On 04/10/2017 08:11 PM, Keith Packard wrote: > Mario Kleiner <mario.kleiner.de@gmail.com> writes: > >> as input from a highly interested future user of such api's: > > Thanks much for taking a look at this. > >> My use cases run about 98% of the time in fullscreen >> exclusive mode and want as little interference from the windowing system >> / desktop environment as possible, with as much low level control as >> possible. It still needs windowed mode for same cases and needs a >> windowing system up and running. > > The new APIs can give you exclusive access to the display resources, > bypassing the window system entirely. > >> Atm. under X i have to hope that fullscreen unredirection works to get >> me page flipping for single display monoscopic stimulation and >> dual-display stereoscopic stuff. And then there's the popular "Regular >> desktop GUI for interaction" + separate displays for "fullscreen + page >> flipping for controlled presentation" case. > > You're still depending on the window system server for timely page flips > though; the main reason we're doing the leasing work is to get that > variable out of the equation, eliminating any window system scheduling > jitter from the regular screen updates on the HMD. > Yes, and that will be an advantage for me as well, especially for some more exotic situations. That said, in my experience good old X is holding up rather well for page-flipped windows if one gets unredirected fullscreen with no DE interference. I have users which runs a 1920x1080@240 Hz display at 240 fps stable update rates without skipping frames and proper frame accurate timing on AMD gfx with the open-source graphics stack on a standard Ubuntu 16.04 + Server 1.18 :) In fact, it still beats current standard Wayland compositors by a large margin, mostly due to how display update scheduling is done in the current incarnations and because Wayland doesn't have a full DRI/Present or OML_sync_control equivalent yet. >> The RRCreateLease requests looks as if i could get that for regular >> non-HMD video outputs as well? > > Yes, there's almost no way we could restrict it even if we wanted > to. I'm doing testing with two standard monitors, but any hardware at > all will work. > >> And the RRCreateOutputGrab would be mostly to avoid flicker when >> plugging HMD's or other special purpose devices, but wouldn't be >> strictly needed for a regular X-client to get a lease on a set of >> outputs? > > Yes, just to avoid having the desktop extend itself onto the HMD, even > briefly. The two components of the RandR changes are logically separate, > but should be useful in combination when using HMD displays. > >> As far as controllable properties on a lease go, i'd find it very useful >> if i could have control over framebuffer formats, e.g., being able to >> select 10 bit scanout formats would be very useful, but is afaik >> something that X currently doesn't expose with most drivers, especially >> not for regular desktop mode. > > You have the full KMS api at your disposal; do whatever you like :-) > >> If the underlying DRM leases allow me to control this stuff, and Wayland >> would gain similar extensions to lease outputs for fullscreen exclusive >> control, i could have one drm/kms backend that can be mostly agnostic of >> X vs. Wayland / different Wayland compositor flavors. > > Right, that's another benefit here -- allowing HMD applications to be largely > window system independent, except for acquiring the initial lease. > Great! Haven't looked at your patches yet, only at this thread and your blog, but this sounds all very promising. >> Basically my vote to expose as much flexility in modesetting / >> properties for the exposed leases as possible on the kernel and X >> side. > > I'll have a second cut of the kernel API changes ready in another day or > so; that will eliminate the ability to change an existing lease, so > you'll have to acquire all of the resources you need all at > once. Otherwise, it will look quite similar at the user API level. > > The RandR protocol changes will also need another spin; it sounds like > we want to configure the set of 'special' monitors and have those never > reported as connected via the current API. I don't think that will > affect your use case at all, and the other part of the protocol for > creating a lease won't change. > Yes, sounds like i could deal with that. -mario
Mario Kleiner <mario.kleiner.de@gmail.com> writes: > Great! Haven't looked at your patches yet, only at this thread and your > blog, but this sounds all very promising. I'll write up another blog post when I finish with the first round of review. That should describe the kernel API at least. I think the X API will be pretty simple though -- tell me which output and crtc and I'll hand you a lease for those.
Pekka Paalanen <ppaalanen@gmail.com> writes: > IMHO, if nothing is providing a picture intended for the HMD, the HMD > should be off. There is no use in showing an arbitrary image that does > not look right, is there? Well, if the HMD is being worn and the application crashes, then what you want is something that keeps responding to your motion so you can get the HMD off without falling or running into stuff... But, yeah, in general, if you don't have an HMD application running, the HMD can't usefully show anything from a bare window system. The trick is to make sure existing desktop applications don't try to use it by mistake. > even if the database was just a bunch of files, you still need code to > access it, and probably something to ensure the entries are > well-formed, so that everyone will agree on how to parse them. One > should probably decide whether the database will only answer the > question "is it a HMD?" or can it provide other information as well. Yup, we need a spec for the files that is reasonably sane, and also extensible so that if we want to add new data, we can. I discussed this with Eric Anholt at breakfast this morning and we came up with a couple of ideas: 1) A directory full of files, each file can contain one or more monitor entries 2) Monitors should be identified (at a minimum) using the EDID Manufacturer ID and Manufacturer product code. I can imagine also allowing the use of a serial number and/or date code if we end up using this for more stuff later. 3) Window system independent. We obviously need this for X, but other window systems should share the same data. 4) Use an existing format. Both of us would rather use JSON, but there's already XML in the DRM world, so that might make sense. These are functionally equivalent, but XML syntax is rough on the eyes. 5) Allow multiple definitions in each file, but allow for multiple files too. Here's a JSON-formatted example: { "monitors": [ { "Manufacturer" : "LGD", "Product" : 437, "desktop" : true } ] "copyright" : "Copyright © 2017, Keith Packard", "license" : "GPLv3" } One can imagine the same done in XML, but I'm too lazy to type that here. In any case, as you can see above, I've added a "desktop" field; if false, the monitor would be hidden to 'normal' desktop applications and only made visible to others. Questions: Q) Where should the directory live. A) /usr/share/monitors for distribution-provided files, /etc/monitors for application-provided files. Q) How about RandR protocol. A) I'm thinking of just creating a new randr request like RRGetOutputInfo but which will return even "hidden" outputs with non-spoofed 'connection' value. Q) What file names should the entries use. A) How about just the manufacturer and product of the first entry? Q) Ranges of product ids? A) Yeah, we might want this to avoid a lot of duplicate entries.
On Fri, 28 Apr 2017 15:03:17 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > > IMHO, if nothing is providing a picture intended for the HMD, the HMD > > should be off. There is no use in showing an arbitrary image that does > > not look right, is there? > > Well, if the HMD is being worn and the application crashes, then > what you want is something that keeps responding to your motion so you > can get the HMD off without falling or running into stuff... Hi, I was under the impression that if you have a VR application running on the HMD, it necessarily also implies that you have a VR compositor running, which means that there is always some process providing a valid image to the HMD. (At least in end-user setups.) Here the assumption is that the application may crash or stall, but the VR compositor never will. Obviously that's not exactly true, but if we design for also the VR compositor crashing or stalling, then we have a requirement for a tertiary process that is essentially just like the VR compositor. And so the turtles pile up... which is also why I don't think a backup plan for the VR compositor is necessary. Or did you have an idea of something extremely simple that could still provide a reasonable image for the HMD even when the compositors have crashed and the GPU is frozen? However, your original question was: > * What should we do with an HMD which is in the database but for which > no application is installed on the host? I assumed that means there are no VR apps nor a VR compositor that could handle the HMD. In that case I think the HMD should be always off. > But, yeah, in general, if you don't have an HMD application running, the > HMD can't usefully show anything from a bare window system. The trick is > to make sure existing desktop applications don't try to use it by mistake. Sure. > > even if the database was just a bunch of files, you still need code to > > access it, and probably something to ensure the entries are > > well-formed, so that everyone will agree on how to parse them. One > > should probably decide whether the database will only answer the > > question "is it a HMD?" or can it provide other information as well. > > Yup, we need a spec for the files that is reasonably sane, and also > extensible so that if we want to add new data, we can. I discussed this > with Eric Anholt at breakfast this morning and we came up with a couple > of ideas: > > 1) A directory full of files, each file can contain one or more monitor > entries > > 2) Monitors should be identified (at a minimum) using the EDID > Manufacturer ID and Manufacturer product code. I can imagine > also allowing the use of a serial number and/or date code if we end > up using this for more stuff later. > > 3) Window system independent. We obviously need this for X, but > other window systems should share the same data. > > 4) Use an existing format. Both of us would rather use JSON, but > there's already XML in the DRM world, so that might make > sense. These are functionally equivalent, but XML syntax is rough on > the eyes. > > 5) Allow multiple definitions in each file, but allow for multiple > files too. > > Here's a JSON-formatted example: > > { > "monitors": [ > { > "Manufacturer" : "LGD", > "Product" : 437, > "desktop" : true > } > ] > "copyright" : "Copyright © 2017, Keith Packard", > "license" : "GPLv3" > } > > One can imagine the same done in XML, but I'm too lazy to type that > here. In any case, as you can see above, I've added a "desktop" field; > if false, the monitor would be hidden to 'normal' desktop applications > and only made visible to others. I presume that if "desktop" is set to "true", it implies that the HMD is capable of showing a simple 2D canvas in stereo without any special rendering and with the default video mode. That is, creating a sort of a virtual 2D monitor. That would be nice. > Questions: > > Q) Where should the directory live. > > A) /usr/share/monitors for distribution-provided files, /etc/monitors > for application-provided files. > > Q) How about RandR protocol. > > A) I'm thinking of just creating a new randr request like > RRGetOutputInfo but which will return even "hidden" outputs with > non-spoofed 'connection' value. > > Q) What file names should the entries use. > > A) How about just the manufacturer and product of the first entry? > > Q) Ranges of product ids? > > A) Yeah, we might want this to avoid a lot of duplicate entries. > FWIW, it all sounds good to me! I don't really have an opinion on XML vs. JSON, I'm just happy it's not another ad hoc format. Thanks, pq
Pekka Paalanen <ppaalanen@gmail.com> writes: > I was under the impression that if you have a VR application running on > the HMD, it necessarily also implies that you have a VR compositor > running, which means that there is always some process providing a valid > image to the HMD. (At least in end-user setups.) Yes, I think the point I was trying to make was that X should never attempt to talk to the HMD and provide an image. > I assumed that means there are no VR apps nor a VR compositor that > could handle the HMD. In that case I think the HMD should be always > off. Agreed. > I presume that if "desktop" is set to "true", it implies that the HMD > is capable of showing a simple 2D canvas in stereo without any special > rendering and with the default video mode. That is, creating a sort of > a virtual 2D monitor. That would be nice. I was thinking that 'desktop' would be true for non-HMD displays that didn't need the VR compositor. If you've got a VR compositor and want to paint the desktop inside the VR environment, then I think you'd want to create a synthetic monitor and hand images from that to the VR compositor each frame. > FWIW, it all sounds good to me! > > I don't really have an opinion on XML vs. JSON, I'm just happy it's not > another ad hoc format. Yeah, I think we're done with ad-hoc formats. I've done both XML and JSON, and JSON is way easier to hand write, but we already use XML in a bunch of places, so the necessary libaries are already linked into the server...
On Tue, 02 May 2017 07:45:25 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > I presume that if "desktop" is set to "true", it implies that the HMD > > is capable of showing a simple 2D canvas in stereo without any special > > rendering and with the default video mode. That is, creating a sort of > > a virtual 2D monitor. That would be nice. > > I was thinking that 'desktop' would be true for non-HMD displays that > didn't need the VR compositor. If you've got a VR compositor and want to > paint the desktop inside the VR environment, then I think you'd want to > create a synthetic monitor and hand images from that to the VR > compositor each frame. Hi Keith, do you mean to list all kinds of display devices in the database? I was assuming it would list only HMDs, so not in database would imply it's a normal display and good for extending the desktop to. Or did you mean it for exceptions? As in, define a range of HMDs, but the vendor put a few normal displays in the middle of the range, so one needs to be able to exclude those? The reason I mentioned "virtual 2D display" was that I recall hearing that actually exists in some HMD hardware. If you don't do anything to enable a 3D mode, the HMD will process the signal to produce a virtual 2D display in front the user. In such case, there is no need for a VR compositor, the plain old 2D image signal will be shown correctly on a plane in the virtual space by the HMD hardware itself. I asked on #openhmd and was pointed to "Cinematic mode" on PSVR, and found e.g. this: http://www.psu.com/feature/31372/How-to-use-PlayStation-VR-cinematic-mode-effectively Mind the note towards the bottom: you don't actually need a PS4 to use it - so it must be something built into the HMD. However, reading more details from https://blog.us.playstation.com/2016/10/03/playstation-vr-the-ultimate-faq/ reveals that there is actually a separate processor box providing the cinematic mode. Sounds like it's your VR compositor as a middle-man hardware device rather than just a program. :-) Thanks, pq
Pekka Paalanen <ppaalanen@gmail.com> writes: > do you mean to list all kinds of display devices in the database? I was > assuming it would list only HMDs, so not in database would imply it's a > normal display and good for extending the desktop to. I intended for it to be a general database to which we could add almost anything; you can imagine using this to replace broken EDID values, configure alternate preferred modes or whatever. The 'desktop' boolean says whether the desktop should be expected to use the device or not; that's all I need for the HMD case. > Or did you mean it for exceptions? As in, define a range of HMDs, but > the vendor put a few normal displays in the middle of the range, so one > needs to be able to exclude those? Oh, that's a great thought; I hadn't considered what we would do with conflicting entries that mapped the same device. I'd like to make that invalid, and potentially spit out a warning message somewhere... > The reason I mentioned "virtual 2D display" was that I recall hearing > that actually exists in some HMD hardware. If you don't do anything to > enable a 3D mode, the HMD will process the signal to produce a virtual > 2D display in front the user. In such case, there is no need for a VR > compositor, the plain old 2D image signal will be shown correctly on a > plane in the virtual space by the HMD hardware itself. Oh, cool! That doesn't help as it means the user will want to pick if this happens or not. Maybe just don't include it in the database and let the VR application turn off the X output before creating a lease? > Mind the note towards the bottom: you don't actually need a PS4 to use > it - so it must be something built into the HMD. However, reading more > details from > https://blog.us.playstation.com/2016/10/03/playstation-vr-the-ultimate-faq/ > reveals that there is actually a separate processor box providing the > cinematic mode. Sounds like it's your VR compositor as a middle-man > hardware device rather than just a program. :-) Interesting. I guess it's a way to make it work without hacking the desktop environment at all? Thanks for your suggestions; I hope we're getting closer to some kind of prototype at least...
On Wed, 03 May 2017 19:04:38 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > > do you mean to list all kinds of display devices in the database? I was > > assuming it would list only HMDs, so not in database would imply it's a > > normal display and good for extending the desktop to. > > I intended for it to be a general database to which we could add almost > anything; you can imagine using this to replace broken EDID values, > configure alternate preferred modes or whatever. Ooh, a much much larger scope than I assumed. Nice. That means you need an explicit key to denote HMDs. More below. We could probably use it to recognize projectors and such, too.... which makes me think that there should probably be an easy way to add more entries: you plug in a projector, it's not recognized as such, you tell your DE it is a projector, the DE goes and saves the match in the database. Should be pretty easy with the directory full of files approach you had in mind, if there just is write access. > The 'desktop' boolean says whether the desktop should be expected to use > the device or not; that's all I need for the HMD case. > > > Or did you mean it for exceptions? As in, define a range of HMDs, but > > the vendor put a few normal displays in the middle of the range, so one > > needs to be able to exclude those? > > Oh, that's a great thought; I hadn't considered what we would do with > conflicting entries that mapped the same device. I'd like to make that > invalid, and potentially spit out a warning message somewhere... I'm not sure... disfavouring overlapping entries at least would save us from inventing heuristics on which one of the overlapping entries should be picked for a device. And if those heuristics would need to be implemented in each display server, more room for inconsistent behaviour. I think your idea is the better one. > > The reason I mentioned "virtual 2D display" was that I recall hearing > > that actually exists in some HMD hardware. If you don't do anything to > > enable a 3D mode, the HMD will process the signal to produce a virtual > > 2D display in front the user. In such case, there is no need for a VR > > compositor, the plain old 2D image signal will be shown correctly on a > > plane in the virtual space by the HMD hardware itself. > > Oh, cool! That doesn't help as it means the user will want to pick if > this happens or not. Maybe just don't include it in the database and let > the VR application turn off the X output before creating a lease? Or, list it in the database as both "desktop" and "HMD" for the same effect? That way the desktop would extend there automatically, but it would also be advertised as a HMD to clients. If it's not listed at all, would it be possible to advertise it as a HMD and DRM-leased out etc.? At least for Wayland, I'd like to not advertise "everything" as possible HMDs but only those that really are. > > Mind the note towards the bottom: you don't actually need a PS4 to use > > it - so it must be something built into the HMD. However, reading more > > details from > > https://blog.us.playstation.com/2016/10/03/playstation-vr-the-ultimate-faq/ > > reveals that there is actually a separate processor box providing the > > cinematic mode. Sounds like it's your VR compositor as a middle-man > > hardware device rather than just a program. :-) > > Interesting. I guess it's a way to make it work without hacking the > desktop environment at all? It seems it should be capable of displaying the desktop out of the box, but I don't think there is a separate HDMI-in for actual VR content, so you'd still need to cooperate with the display server to turn it to VR mode and feed the VR video stream. I also read some discussions about 3D TV modes, but couldn't really get a conclusion if they are supposed to render like if you were in a 3D cinema theatre or not. Apparently that was enabled in a firmware upgrade and maybe had some extra requirements. I would guess the real purpose behind the cinematic mode is to get more content workable on the HMD earlier, as it allows you to play almost all existing games and movies that were not made for VR/HMD to begin with. > Thanks for your suggestions; I hope we're getting closer to some kind of > prototype at least... > Thanks, pq
Pekka Paalanen <ppaalanen@gmail.com> writes: > Ooh, a much much larger scope than I assumed. Nice. Well, it's more out of a sense of fear than future planning. If all we ever use it for is as a list of monitors that the desktop should ignore, that'd be fine. > That means you need an explicit key to denote HMDs. More below. I don't think so. Presumably the VR system will have a known list of HMDs it works with, and so it will probably have a list of EDID information; I expect that's where the data for monitors-to-ignore will come from in many cases. After all, the goal is to prevent the desktop From shifting onto a display only to get kicked off as the VR app takes over the HMD. > We could probably use it to recognize projectors and such, too.... which > makes me think that there should probably be an easy way to add more > entries: you plug in a projector, it's not recognized as such, you tell > your DE it is a projector, the DE goes and saves the match in the > database. Should be pretty easy with the directory full of files > approach you had in mind, if there just is write access. Yeah, one can imagine an application designed to manage that. One can also imagine having an additional database which the desktop itself would read on behalf of the user; I'd hate to have the user directly providing data for the window system server... > I think your idea is the better one. Thanks, it seems simpler and more predictable to me as well. > Or, list it in the database as both "desktop" and "HMD" for the same > effect? All that the database needs to do today is to hide entries from the desktop; with that, the VR app can go find the monitors and hook them up without needing additional data. > That way the desktop would extend there automatically, but it would > also be advertised as a HMD to clients. If it's not listed at all, > would it be possible to advertise it as a HMD and DRM-leased out etc.? I think all we'd do is offer a new RandR request which listed "all" monitors, including those hidden from the desktop, and expect that the monitor-specific applications would use that to detect the presence of their magic outputs and use them. For an HMD, you'll also have other devices connected, so we just need some way of positively associating the various input devices with the specific monitor (in case there is more than one). > At least for Wayland, I'd like to not advertise "everything" as > possible HMDs but only those that really are. I don't think the window system needs to know that the displays are HMD, only that it shouldn't present them as part of the desktop. Everything else about them can be left outside of this spec. > It seems it should be capable of displaying the desktop out of the box, > but I don't think there is a separate HDMI-in for actual VR content, so > you'd still need to cooperate with the display server to turn it to > VR mode and feed the VR video stream. Sounds like it might be worth exploring how those work, and whether we might want to provide some additional RandR protocol to make a specific monitor 'visible' so that the desktop would extend onto it with a command from a client. I think we can safely ignore this for now and plan on coming up with a solution that extends on this basic work. > I also read some discussions about 3D TV modes, but couldn't really get > a conclusion if they are supposed to render like if you were in a 3D > cinema theatre or not. Apparently that was enabled in a firmware > upgrade and maybe had some extra requirements. Hard to imagine this being relevant for the Linux desktop at least... > I would guess the real purpose behind the cinematic mode is to get more > content workable on the HMD earlier, as it allows you to play almost > all existing games and movies that were not made for VR/HMD to begin > with. Seems like a safe assumption. Once you install a VR app and the associated 'ignore this monitor' files for the window system, I'd expect that to be the primary use case. Thanks again; I'm sensing that a simple 'ignore this monitor for the desktop' might suffice for now, but that we'll end up wanting something more complicated in the future. I think the key is to try and avoid making that harder by what we do now, but I don't think we should be trying to implement a larger solution too soon.
On Thu, 04 May 2017 11:02:44 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > > That means you need an explicit key to denote HMDs. More below. > > I don't think so. Presumably the VR system will have a known list of > HMDs it works with, and so it will probably have a list of EDID > information; I expect that's where the data for monitors-to-ignore will > come from in many cases. After all, the goal is to prevent the desktop > From shifting onto a display only to get kicked off as the VR app takes > over the HMD. Hi, I disagree on the details, more below. > > That way the desktop would extend there automatically, but it would > > also be advertised as a HMD to clients. If it's not listed at all, > > would it be possible to advertise it as a HMD and DRM-leased out etc.? > > I think all we'd do is offer a new RandR request which listed "all" > monitors, including those hidden from the desktop, and expect that the > monitor-specific applications would use that to detect the presence of > their magic outputs and use them. For an HMD, you'll also have other > devices connected, so we just need some way of positively associating > the various input devices with the specific monitor (in case there is > more than one). Such a RandR request is something I would not like to have to replicate on Wayland. The display server contains the policy, it should not just expose everything up for grabs. This is a fundamental difference between X11 and Wayland architectures, and I think the output information database should support both ways. > > At least for Wayland, I'd like to not advertise "everything" as > > possible HMDs but only those that really are. > > I don't think the window system needs to know that the displays are HMD, > only that it shouldn't present them as part of the desktop. Everything > else about them can be left outside of this spec. I disagree. Wayland will likely have a special protocol extension exclusively for advertising HMDs, so the display server will need to know which ones are HMDs and which one are not, regardless whether the desktop is extended there or not. This extension could also offer the DRM leasing capabilities. I gave the PSVR as an example of hardware where one both can meaningfully extend the desktop there and offer it as a HMD. When that output is taken into HMD use, the display will automatically stop putting the desktop there - but only temporarily. All windows that were on that extended output should still be there when the VR app/compositor finishes. This is window management policy, so the window manager must know what is going on. (That is similar to the fullscreening with video mode changing mechanism, where the video mode requested by an application is temporary, and normality will be restored automatically when e.g. the window disappears by client crash or the user switches to another window.) Having the window manager know what is a HMD and which client is active on it will let it make better management decisions and even allow switching between VR apps, or temporarily switching from VR app to the desktop when supported, and back. > > I also read some discussions about 3D TV modes, but couldn't really get > > a conclusion if they are supposed to render like if you were in a 3D > > cinema theatre or not. Apparently that was enabled in a firmware > > upgrade and maybe had some extra requirements. > > Hard to imagine this being relevant for the Linux desktop at least... Depends on how a video player will deliver the 3D movie content. Can it be done as a normal GUI app providing stereo buffers, or does one need to turn the video player into a VR app and reimplement the cinematic mode in software. The latter option works for any HMD, but the former option needs less power from the computer as head tracking etc. is outsourced to the "HMD" itself. > Thanks again; I'm sensing that a simple 'ignore this monitor for the > desktop' might suffice for now, but that we'll end up wanting something > more complicated in the future. I think the key is to try and avoid > making that harder by what we do now, but I don't think we should be > trying to implement a larger solution too soon. Yes, that I agree with. Ultimately I would envision an output device database describing what kind of a device it is (a normal monitor, a video projector, a HMD, a TV, ...) and then the software that implements the desktop or UI policy will decide how that will be used. Some policy examples: - A projector: do not extend desktop UI, but have the output normally available, so one can put regular windows there (presentation software). Turned on by default. - A HMD: Do not extend desktop, do not expose to normal apps, keep it off until special request. - A HMD with cinematic mode support: Extend desktop, turn on by default, allow special HMD requests. - A TV: Try to disable overscan or try to compensate for it by default. Also, a normal monitor and maybe a TV might have physical image size relevant, while for a projector and a HMD the physical size is mostly irrelevant. So, how can we start simple while avoiding the need to break everything if we later want to extend support for all the fancy stuff... I would suggest to have a "output device type" attribute in the database, and support only one value for now: "HMD". Then all display servers can encode the policy to hide all HMDs by default. Implementationwise it is no different from your idea, but separating device description from usage policy would be a good thing in my opinion. You can still document suggested policies in the spec if you wish, only the wording will be more of a recommendation than a requirement. Thanks, pq
Pekka Paalanen <ppaalanen@gmail.com> writes: > I disagree on the details, more below. > Such a RandR request is something I would not like to have to replicate > on Wayland. The display server contains the policy, it should not just > expose everything up for grabs. This is a fundamental difference > between X11 and Wayland architectures, and I think the output > information database should support both ways. Sounds like a good idea; if you want to work on how this should appear in Wayland, that would be great as it would provide two implementations, rather than just one. > I disagree. Wayland will likely have a special protocol extension > exclusively for advertising HMDs, so the display server will need to > know which ones are HMDs and which one are not, regardless whether the > desktop is extended there or not. This extension could also offer the > DRM leasing capabilities. What I was thinking that a display which the window system cannot drive natively should probably just be ignored entirely. An HMD which can natively handle a desktop (such as the PSVR system) might well be advertised as 'desktop capable', even though it is an HMD. However, I also think you're right -- while the window system can't deal with it *today*, that doesn't mean the window system won't be able to do that in the future. Hrm. I think the important distinction here is that the user installed an application which is designed to talk directly to this display. From that, we should probably infer that the user would like to use that application with the display. > Having the window manager know what is a HMD and which client is active > on it will let it make better management decisions and even allow > switching between VR apps, or temporarily switching from VR app to the > desktop when supported, and back. The window system will know when outputs are leased to another client, so it can tell when to bring them back to the desktop. In the PSVR instance, you'd list the device as 'desktop', but still allow leasing. When no lease was active, the desktop could extend into the PSVR environment. When the custom application started, it would request a lease at which time the desktop would move off of the PSVR. >> Thanks again; I'm sensing that a simple 'ignore this monitor for the >> desktop' might suffice for now, but that we'll end up wanting something >> more complicated in the future. I think the key is to try and avoid >> making that harder by what we do now, but I don't think we should be >> trying to implement a larger solution too soon. > > Yes, that I agree with. I guess that's the question -- is a simple command to ignore a monitor for purposes of the desktop sufficient for now? Or do we need to worry about a possible future in which the window system implements native HMD support? > Ultimately I would envision an output device database describing > what kind of a device it is (a normal monitor, a video projector, a > HMD, a TV, ...) and then the software that implements the desktop or UI > policy will decide how that will be used. Some policy examples: > > - A projector: do not extend desktop UI, but have the output normally > available, so one can put regular windows there (presentation > software). Turned on by default. > > - A HMD: Do not extend desktop, do not expose to normal apps, keep it > off until special request. > > - A HMD with cinematic mode support: Extend desktop, turn on by > default, allow special HMD requests. > > - A TV: Try to disable overscan or try to compensate for it by > default. Those all sound useful. I wonder how much we might be able to guess from EDID info and whether we want to programmatically do some of this (especially the TV thing; that's really annoying :-). In any case, a problem for the future. > I would suggest to have a "output device type" attribute in the > database, and support only one value for now: "HMD". Then all display > servers can encode the policy to hide all HMDs by default. > Implementationwise it is no different from your idea, but separating > device description from usage policy would be a good thing in my > opinion. You can still document suggested policies in the spec if you > wish, only the wording will be more of a recommendation than a > requirement. The only issue here is that now we're encoding policy in code, which is hard to change for the average user, rather than in a configuration file, which is easy to change. However, as we've defined it, these files are installed by the system and it would be nice if they weren't expected to be overridden by the user, so encoding policy there is almost worse than in the code. Hrm. How about we adopt your design and encode the device type in the file, provide a fixed policy in the window system for now and consider the possibility of changing the window system in the future to support more advanced policy mechanisms. I don't think that shuts any doors permanently.
On 05/05/2017 04:25 PM, Keith Packard wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > >> I disagree on the details, more below. > >> Such a RandR request is something I would not like to have to replicate >> on Wayland. The display server contains the policy, it should not just >> expose everything up for grabs. This is a fundamental difference >> between X11 and Wayland architectures, and I think the output >> information database should support both ways. > > Sounds like a good idea; if you want to work on how this should appear > in Wayland, that would be great as it would provide two implementations, > rather than just one. > >> I disagree. Wayland will likely have a special protocol extension >> exclusively for advertising HMDs, so the display server will need to >> know which ones are HMDs and which one are not, regardless whether the >> desktop is extended there or not. This extension could also offer the >> DRM leasing capabilities. > > What I was thinking that a display which the window system cannot drive > natively should probably just be ignored entirely. An HMD which can > natively handle a desktop (such as the PSVR system) might well be > advertised as 'desktop capable', even though it is an HMD. > > However, I also think you're right -- while the window system can't deal > with it *today*, that doesn't mean the window system won't be able to do > that in the future. > > Hrm. I think the important distinction here is that the user installed > an application which is designed to talk directly to this display. From > that, we should probably infer that the user would like to use that > application with the display. > >> Having the window manager know what is a HMD and which client is active >> on it will let it make better management decisions and even allow >> switching between VR apps, or temporarily switching from VR app to the >> desktop when supported, and back. > > The window system will know when outputs are leased to another client, > so it can tell when to bring them back to the desktop. In the PSVR > instance, you'd list the device as 'desktop', but still allow > leasing. When no lease was active, the desktop could extend into the > PSVR environment. When the custom application started, it would request > a lease at which time the desktop would move off of the PSVR. > >>> Thanks again; I'm sensing that a simple 'ignore this monitor for the >>> desktop' might suffice for now, but that we'll end up wanting something >>> more complicated in the future. I think the key is to try and avoid >>> making that harder by what we do now, but I don't think we should be >>> trying to implement a larger solution too soon. >> >> Yes, that I agree with. > > I guess that's the question -- is a simple command to ignore a monitor > for purposes of the desktop sufficient for now? Or do we need to worry > about a possible future in which the window system implements native HMD > support? > >> Ultimately I would envision an output device database describing >> what kind of a device it is (a normal monitor, a video projector, a >> HMD, a TV, ...) and then the software that implements the desktop or UI >> policy will decide how that will be used. Some policy examples: >> >> - A projector: do not extend desktop UI, but have the output normally >> available, so one can put regular windows there (presentation >> software). Turned on by default. >> >> - A HMD: Do not extend desktop, do not expose to normal apps, keep it >> off until special request. >> >> - A HMD with cinematic mode support: Extend desktop, turn on by >> default, allow special HMD requests. >> >> - A TV: Try to disable overscan or try to compensate for it by >> default. > > Those all sound useful. I wonder how much we might be able to guess from > EDID info and whether we want to programmatically do some of this > (especially the TV thing; that's really annoying :-). In any case, a > problem for the future. > >> I would suggest to have a "output device type" attribute in the >> database, and support only one value for now: "HMD". Then all display >> servers can encode the policy to hide all HMDs by default. >> Implementationwise it is no different from your idea, but separating >> device description from usage policy would be a good thing in my >> opinion. You can still document suggested policies in the spec if you >> wish, only the wording will be more of a recommendation than a >> requirement. > > The only issue here is that now we're encoding policy in code, which is > hard to change for the average user, rather than in a configuration > file, which is easy to change. However, as we've defined it, these files > are installed by the system and it would be nice if they weren't > expected to be overridden by the user, so encoding policy there is > almost worse than in the code. > > Hrm. How about we adopt your design and encode the device type in the > file, provide a fixed policy in the window system for now and consider > the possibility of changing the window system in the future to support > more advanced policy mechanisms. I don't think that shuts any doors > permanently. > Just please make sure that one (user configurable/opt-in if necessary) policy from the beginning is to allow leasing out any output to applications, not just HMDs. My type of scientific/medical applications would benefit as soon as it has the option to get a drm lease for a given output on both X and Wayland based systems. That's not a theoretical future use case, but one i'd try to offer to my users as soon as a stable protocol/implementation is available in a regular Linux distribution. It wouldn't be fun for inexperienced users if they had to hack the database for every model of display they want to use, or if every untrusted user would have to have a root password to do so. thanks, -mario > > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel >
Mario Kleiner <mario.kleiner.de@gmail.com> writes: > Just please make sure that one (user configurable/opt-in if necessary) > policy from the beginning is to allow leasing out any output to > applications, not just HMDs. That's entirely a given -- the leasing API is under the control of the application which can lease any set of resources it likes. The only question here is how to avoid flashing the desktop contents onto the HMD when you plug it in, and that means not listing some displays as 'connected' in the default RandR requests. There will be a separate request which lists them all.
On Sat, 6 May 2017 13:34:44 +0200 Mario Kleiner <mario.kleiner.de@gmail.com> wrote: > Just please make sure that one (user configurable/opt-in if necessary) > policy from the beginning is to allow leasing out any output to > applications, not just HMDs. My type of scientific/medical applications > would benefit as soon as it has the option to get a drm lease for a > given output on both X and Wayland based systems. That's not a > theoretical future use case, but one i'd try to offer to my users as > soon as a stable protocol/implementation is available in a regular Linux > distribution. It wouldn't be fun for inexperienced users if they had to > hack the database for every model of display they want to use, or if > every untrusted user would have to have a root password to do so. Hi Mario, as Keith said, the DRM leasing API is ok for that. Such policy must not be part of the database, IMO, so it's not an issue for the database. This is another reason why I think the database should only describe the hardware, not set usage patterns nor be an extension of display server configuration. For Wayland, we would need to experiment. I would not start with the assumption that the Wayland extension used for grabbing HMD outputs could also be used to grab an arbitrary output that the compositor is already using for the desktop. Since an output is a part of the desktop, it is possible it needs more strict or slightly different grabbing semantics. I believe one should design both cases separately and then see how much they have overlap. Thanks, pq PS. Keith, I started writing a reply to you, but didn't finish yet.
On Fri, 05 May 2017 07:25:14 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > > I disagree on the details, more below. > > > Such a RandR request is something I would not like to have to replicate > > on Wayland. The display server contains the policy, it should not just > > expose everything up for grabs. This is a fundamental difference > > between X11 and Wayland architectures, and I think the output > > information database should support both ways. > > Sounds like a good idea; if you want to work on how this should appear > in Wayland, that would be great as it would provide two implementations, > rather than just one. Hi Keith, I would want to, but unfortunately I have barely enough time for this fly-by commenting currently. However, I do have a fairly clear idea of how all this talked about so far would be implemented in a Wayland server, except for potential window management issues in case a VR app connects both directly for a normal window and via a VR compositor. I forget if I mentioned this to you personally yet: https://gist.github.com/ppaalanen/e0d2744ff71188e9a294726a37ca83c2 I wrote that before I knew about your DRM leasing work, which means it is well outdated. The biggest difference is that instead of flipping through the Wayland compositor, one would use DRM leases. > > I disagree. Wayland will likely have a special protocol extension > > exclusively for advertising HMDs, so the display server will need to > > know which ones are HMDs and which one are not, regardless whether the > > desktop is extended there or not. This extension could also offer the > > DRM leasing capabilities. > > What I was thinking that a display which the window system cannot drive > natively should probably just be ignored entirely. An HMD which can > natively handle a desktop (such as the PSVR system) might well be > advertised as 'desktop capable', even though it is an HMD. > > However, I also think you're right -- while the window system can't deal > with it *today*, that doesn't mean the window system won't be able to do > that in the future. > > Hrm. I think the important distinction here is that the user installed > an application which is designed to talk directly to this display. From > that, we should probably infer that the user would like to use that > application with the display. Right. > >> Thanks again; I'm sensing that a simple 'ignore this monitor for the > >> desktop' might suffice for now, but that we'll end up wanting something > >> more complicated in the future. I think the key is to try and avoid > >> making that harder by what we do now, but I don't think we should be > >> trying to implement a larger solution too soon. > > > > Yes, that I agree with. > > I guess that's the question -- is a simple command to ignore a monitor > for purposes of the desktop sufficient for now? Or do we need to worry > about a possible future in which the window system implements native HMD > support? I believe my proposal at the end of the email solves this question both ways. > > Ultimately I would envision an output device database describing > > what kind of a device it is (a normal monitor, a video projector, a > > HMD, a TV, ...) and then the software that implements the desktop or UI > > policy will decide how that will be used. Some policy examples: > > > > - A projector: do not extend desktop UI, but have the output normally > > available, so one can put regular windows there (presentation > > software). Turned on by default. > > > > - A HMD: Do not extend desktop, do not expose to normal apps, keep it > > off until special request. > > > > - A HMD with cinematic mode support: Extend desktop, turn on by > > default, allow special HMD requests. > > > > - A TV: Try to disable overscan or try to compensate for it by > > default. > > Those all sound useful. I wonder how much we might be able to guess from > EDID info and whether we want to programmatically do some of this > (especially the TV thing; that's really annoying :-). In any case, a > problem for the future. Indeed for the future. In the worst case, users will need to match their devices by serial number or some other individual way one by one. Thinking again, I believe we have to have a way to override database entries somehow. If we ship catch-all entries for, say, all Sony TVs, we are bound to get some assignment wrong for someone who then wants to make a correction without breaking everyone else. We are moving away from configuring outputs in a display server by the connector name and towards configuring outputs based the actual output device identity and type. > > I would suggest to have a "output device type" attribute in the > > database, and support only one value for now: "HMD". Then all display > > servers can encode the policy to hide all HMDs by default. > > Implementationwise it is no different from your idea, but separating > > device description from usage policy would be a good thing in my > > opinion. You can still document suggested policies in the spec if you > > wish, only the wording will be more of a recommendation than a > > requirement. > > The only issue here is that now we're encoding policy in code, which is > hard to change for the average user, rather than in a configuration > file, which is easy to change. However, as we've defined it, these files > are installed by the system and it would be nice if they weren't > expected to be overridden by the user, so encoding policy there is > almost worse than in the code. > > Hrm. How about we adopt your design and encode the device type in the > file, provide a fixed policy in the window system for now and consider > the possibility of changing the window system in the future to support > more advanced policy mechanisms. I don't think that shuts any doors > permanently. That sounds perfect and future-proof to me. I would like the output database to strictly describe the hardware, not set up any policy or usage patterns. It would be something like udev rules adding tags that you can then use in the display server configuration for matching, except for outputs which cannot be handled by udev. I believe we should not have the user poke around in the output device database unless he has a piece of equipment that is either not recognized at all or is recognized or tagged wrong. Seems we are in agreement here. The policy to deal with output categories and individual outputs would be in the display server configuration. Something like: Section "OutputClass" ... MatchType "HMD" Option "HideFromRandR" "true" Option "DPMS" "off" EndSection Heh, I wasn't even aware there actually was OutputClass already, with only MatchDriver. Wouldn't that be a good fit for static policy configuration in Xorg? Thanks, pq
Pekka Paalanen <ppaalanen@gmail.com> writes: > I forget if I mentioned this to you personally yet: > https://gist.github.com/ppaalanen/e0d2744ff71188e9a294726a37ca83c2 Thanks, that's a helpful bit of thinking. It looks like most of that is still relevant, the only piece we'd swap out is how the bits actually get to the screen, which is a technical detail. > Thinking again, I believe we have to have a way to override > database entries somehow. If we ship catch-all entries for, say, > all Sony TVs, we are bound to get some assignment wrong for > someone who then wants to make a correction without breaking > everyone else. I agree this would be useful, but it raises a pile of complications: 1) Obviously these overrides will conflict with the existing entries, which goes against the 'don't allow conflicting entries' clause we discussed earlier. 2) Where are these new entries stored? If in files, then the window system is going to need to read them. They are user-provided, and so the window system is going need to be pointed at them somehow. Handing data files to the window system (which may run as a different user) seems likely to introduce security concerns. 3) Would we provide a protocol mechanism to update them? Or would the window system be expected to scan for new entries automatically? Can we safely put this off to the future? > We are moving away from configuring outputs in a display server by the > connector name and towards configuring outputs based the actual output > device identity and type. That works where EDID is available; it's more complicated when the EDID data is missing or corrupted (as often happens in presentation environments). Fortunately, I think this is a problem entirely outside the scope of the current discussion :-) > The policy to deal with output categories and individual outputs would > be in the display server configuration. Something like: Oh, that's a cool idea. > Heh, I wasn't even aware there actually was OutputClass already, with > only MatchDriver. Wouldn't that be a good fit for static policy > configuration in Xorg? I don't think so; we want it to be independent of the driver. I think, for now, we can have a static policy implemented in the X server code and plan on making that configurable when we figure out where we want to configure it. I'll go implement what we've agreed on so far and make the rest just hard-coded for now. That way we'll have a starting point at least. Anyone care if it's JSON or XML? I'm afraid I'm leaning to XML right now because the X server already loads an XML parser (for DRM config files), and I feel like introducing yet another configuration language is probably worse than extending the use of XML? I could be convinced either way as XML is pretty awful.
On Mon, 08 May 2017 08:29:30 -0700 Keith Packard <keithp@keithp.com> wrote: > Pekka Paalanen <ppaalanen@gmail.com> writes: > > > Thinking again, I believe we have to have a way to override > > database entries somehow. If we ship catch-all entries for, say, > > all Sony TVs, we are bound to get some assignment wrong for > > someone who then wants to make a correction without breaking > > everyone else. > > I agree this would be useful, but it raises a pile of complications: > > 1) Obviously these overrides will conflict with the existing entries, > which goes against the 'don't allow conflicting entries' clause we > discussed earlier. > > 2) Where are these new entries stored? If in files, then the window > system is going to need to read them. They are user-provided, and so > the window system is going need to be pointed at them > somehow. Handing data files to the window system (which may run as a > different user) seems likely to introduce security concerns. Hi Keith, I suppose a sysadmin could add files to the system location... in /etc, to supplement the distributed files somewhere in /usr/share or something? For user-friendly settings, maybe those would need to be specific to the display server or the DE, and we should just not spec any per-user things. Oh, but you need Xorg use them before the DE is up... umm... erk. :-/ > 3) Would we provide a protocol mechanism to update them? Or would the > window system be expected to scan for new entries automatically? *shrug* > Can we safely put this off to the future? Unsure about "safely", but I think it would be the best. We have acknowledged the issue and that's fine for now. > > The policy to deal with output categories and individual outputs would > > be in the display server configuration. Something like: > > Oh, that's a cool idea. > > > Heh, I wasn't even aware there actually was OutputClass already, with > > only MatchDriver. Wouldn't that be a good fit for static policy > > configuration in Xorg? > > I don't think so; we want it to be independent of the driver. I think, > for now, we can have a static policy implemented in the X server code > and plan on making that configurable when we figure out where we want to > configure it. I later thought about another mismatch with the current usage of OutputClass. Since the only thing it can be used for (according to the manual) is matching against the kernel driver and assigning the Xorg driver, it is matching to card devices, not individual (RandR) outputs. Right? So it's matching on the wrong level. That means we probably cannot use the otherwise perfect name OutputClass for, well, output classes. > I'll go implement what we've agreed on so far and make the rest just > hard-coded for now. That way we'll have a starting point at least. Awesome. Thanks, pq
> > Just please make sure that one (user configurable/opt-in if necessary) > policy from the beginning is to allow leasing out any output to > applications, not just HMDs. My type of scientific/medical applications > would benefit as soon as it has the option to get a drm lease for a given > output on both X and Wayland based systems. That's not a theoretical future > use case, but one i'd try to offer to my users as soon as a stable > protocol/implementation is available in a regular Linux distribution. It > wouldn't be fun for inexperienced users if they had to hack the database for > every model of display they want to use, or if every untrusted user would > have to have a root password to do so. > I think we need to restrict what outputs can be leased, but I might just be paranoid, maybe the is no security problem with having an app accessing an exclusive mode, just makes me think of screensaver bypasses or something. Do your apps need input? because if we lease out you won't be getting any input from X and won't be able to open the input devices X has open. VR is special since the input devices are all their own thing. Dave.
diff --git a/randrproto.txt b/randrproto.txt index 74b7c36..8dded63 100644 --- a/randrproto.txt +++ b/randrproto.txt @@ -1,6 +1,6 @@ The X Resize, Rotate and Reflect Extension - Version 1.5.0 - 2015-03-14 + Version 1.6.0 + 2017-04-01 Jim Gettys Jim.Gettys@hp.com @@ -9,9 +9,7 @@ Hewlett Packard Company Keith Packard - keith.packard@intel.com - Open Source Technology Center - Intel Corporation + keithp@keithp.com 1. Introduction @@ -186,6 +184,24 @@ consider as single viewable areas. Xinerama's information now comes from the Monitors instead of directly from the CRTCs. The Monitor marked as Primary will be listed first. +1.6. Introduction to version 1.6 of the extension + +Version 1.6 adds resource leasing. + + • A 'Lease' is a collection of crtcs and outputs which are made + available to a client for direct access via kernel KMS and DRM + APIs. This is done by passing a suitable file descriptor back to + the client which has access to those resources. While leased, those + resources aren't used by the X server. + +Version 1.6 adds EDID-based output 'grabbing'. + + • An 'Output Grab' matches a set of EDID Manufacturer ID and product + codes. For outputs with matching EDID values, the connected status + of the output is only visible to the grabbing client. Other clients + will see the output as disconnected. Attempts to configure the + grabbed output by other clients will fail. + 1.99 Acknowledgments Our thanks to the contributors to the design found on the xpert mailing @@ -273,6 +289,10 @@ Mode Provider A value for a PROVIDER argument does not name a defined PROVIDER. +OutputGrab + A value for an OUTPUTGRAB argument does not name a defined + OUTPUTGRAB + ❧❧❧❧❧❧❧❧❧❧❧ 5. Protocol Types @@ -419,6 +439,23 @@ MONITORINFO { name: ATOM ❧❧❧❧❧❧❧❧❧❧❧ +5.7. Protocol Types added in version 1.6 of the extension + +OUTPUTGRAB { XID } + +EDIDMATCH { id: CARD16 + code-min: CARD16 + code-max: CARD16 } + + These values come from the EDID specification. 'id' is the + Manufacturer ID value which is bytes 8 and 9 in the EDID + packet, stored in big endian order (MSB first). 'code-min' and + 'code-max' define a closed-interval of Manufacturer product + codes, which is byte 10 and 11 of the EDID packet, stored in + little endian order (LSB first). + + ❧❧❧❧❧❧❧❧❧❧❧ + 6. Extension Initialization The name of this extension is "RANDR". @@ -1666,6 +1703,67 @@ dynamic changes in the display environment. window of the screen. ❧❧❧❧❧❧❧❧❧❧❧ + +7.6. Extension Requests added in version 1.6 of the extension. + +┌─── + RRCreateLease + window : WINDOW + crtcs: LISTofCRTC + outputs: LISTofOUTPUT + ▶ + nfd: CARD8 + lease: FD +└─── + Errors: Window, Access, Value, CRTC, Output + + Creates a new Lease for the specified crtcs and outputs from + the screen defined by 'window'. Returns a KMS/DRM file + descriptor which can control the leased objects directly + through the kernel. While leased, all resources will appear to + be 'useless' to clients other than the leasing client as + follows: + + • Crtcs are reported as having no 'possible-outputs' and all + other values reported as if the crtc were disabled. + + • Outputs are reported as having no crtcs they can be + connected to, no clones they can share a crtc with, will + report a connection status of Disconnected, and will show + the current crtc as Disabled. + + The lease remains in effect until the file descriptor is + closed, even if the client holding the lease disconnects from + the X server. + + Returns an Access error if any of the named resources are in + use or already leased to another client. + + Returns a Match error if any of the named resources are in use + by the X server. + +┌─── + RRCreateOutputGrab + window : WINDOW + outputgrab: OUTPUTGRAB + matches: LISTofEDIDMATCH +└─── + Errors: Window, Access, Value, CRTC, Output + + Creates an Output Grab with the specified ID on the screen + associated with 'window'. Any output containing an EDID + property matching the grab will appear to be disconnected to + all clients other than the grabbing client. + +┌─── + RRDestroyOutputGrab + outputgrab: OUTPUTGRAB +└─── + Errors: OutputGrab + + Destroys the named OUTPUTGRAB. + + ❧❧❧❧❧❧❧❧❧❧❧ 8. Extension Events Clients MAY select for ConfigureNotify on the root window to be