[v9,0/4] Input: support overlay objects on touchscreens

Message

Javier Carrasco April 22, 2024, 10:07 a.m. UTC

Some touchscreens are shipped with a physical layer on top of them where
a number of buttons and a resized touchscreen surface might be available.

In order to generate proper key events by overlay buttons and adjust the
touch events to a clipped surface, this series offers a documented,
device-tree-based solution by means of helper functions.
An implementation for a specific touchscreen driver is also included.

The functions in touch-overlay provide a simple workflow to acquire
physical objects from the device tree, map them into a list and generate
events according to the object descriptions.

This feature has been tested with a JT240MHQS-E3 display, which consists
of an st1624 as the base touchscreen and an overlay with two buttons and
a frame that clips its effective surface mounted on it.

To: Dmitry Torokhov <[e-mail]>
To: Rob Herring <[e-mail]>
To: Krzysztof Kozlowski <[e-mail]>
To: Conor Dooley <[e-mail]>
To: Bastian Hecht <[e-mail]>
To: Michael Riesch <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Signed-off-by: Javier Carrasco <[e-mail]>

Changes in v9:
- touch-overlay.c: trigger a button release if the finger slides out of
  the button segment to be consistent with the button press when sliding
  into a button segment (see touch_overlay_button_event()).
- touch-overlay.c: (nit) remove braces in if with a single statement in
  touch_overaly_process_event().
- Link to v8: https://lore.kernel.org/r/[e-mail]

Changes in v8:
- touchscreen bindings: fix description formatting.
- Link to v7: https://lore.kernel.org/r/[e-mail]

Changes in v7:
- General: return to a single input device implementation.
- touchscreen bindings: segment instead of button in the label
  description.
- touch-overlay.c: define button-specific data inside segment struct.
- touch-overlay.c: remove fwnode_property_present() and check return
  value of fwnode_property_read_u32() in touch_overlay_get_segment().
- touch-overlay.c: simplify return path in touch_overlay_map().
- Link to v6: https://lore.kernel.org/r/[e-mail]

Changes in v6:
- General: use a single list to manage a single type of object.
- General: swap patches to have bindings preceding the code.
- touch-overlay.c: minor code-sytle fixes.
- Link to v5: https://lore.kernel.org/r/[e-mail]

Changes in v5:
- touchscreen bindings: move overlay common properties to a $def entry (Rob Herring)
- st1232 bindings: move overlays to the existing example instead of
  making a new one (Rob Herring)
- Link to v4: https://lore.kernel.org/r/[e-mail]

Changes in v4:
- General: rename "touchscreen" to "touch" to include other consumers.
- PATCH 1/4: move touch-overlay feature to input core.
- PATCH 1/4, 3/4: set key caps and report key events without consumer's
  intervention.
- PATCH 2/4: add missing 'required' field with the required properties.
- Link to v3: https://lore.kernel.org/r/[e-mail]

Changes in v3:
- General: rename "virtobj" and "virtual" to "overlay"
- PATCH 1/4: Make feature bool instead of tristate (selected by
  supported touchscreens)
- Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v2:
- PATCH 1/4: remove preprocessor directives (the module is selected by
  the drivers that support the feature). Typo in the commit message.
- PATCH 2/4: more detailed documentation. Images and examples were added.
- PATCH 3/4: select ts-virtobj automatically.
- Link to v1: https://lore.kernel.org/r/[e-mail]

---
Javier Carrasco (4):
      dt-bindings: touchscreen: add touch-overlay property
      Input: touch-overlay - Add touchscreen overlay handling
      dt-bindings: input: touchscreen: st1232: add touch-overlay example
      Input: st1232 - add touch overlays handling

 .../input/touchscreen/sitronix,st1232.yaml         |  29 +++
 .../bindings/input/touchscreen/touchscreen.yaml    | 119 ++++++++++
 MAINTAINERS                                        |   7 +
 drivers/input/Makefile                             |   2 +-
 drivers/input/touch-overlay.c                      | 263 +++++++++++++++++++++
 drivers/input/touchscreen/st1232.c                 |  48 ++--
 include/linux/input/touch-overlay.h                |  22 ++
 7 files changed, 475 insertions(+), 15 deletions(-)
---
base-commit: 0bbac3facb5d6cc0171c45c9873a2dc96bea9680
change-id: 20230510-feature-ts_virtobj_patch-e267540aae74

Best regards,

Comments

Gregor Riepl April 22, 2024, 11:22 a.m. UTC | #1

> In order to generate proper key events by overlay buttons and adjust the
> touch events to a clipped surface, this series offers a documented,
> device-tree-based solution by means of helper functions.
> An implementation for a specific touchscreen driver is also included.

I believe there's at least some x86 tablets that have such a layout, so 
maybe ACPI bindings would also make sense? Or can this be supported by 
your DT-based solution?

I'm not sure if it would really be needed for existing devices, though. 
It's possible they were all handled by touchscreen controller firmware 
so far.

Hans, do you remember if we've encountered any Silead or Goodix devices 
where the soft button overlay didn't work due to missing firmware support?

Hans de Goede April 22, 2024, 11:28 a.m. UTC | #2

Hi Gregor,

On 4/22/24 1:22 PM, Gregor Riepl wrote:
>> In order to generate proper key events by overlay buttons and adjust the
>> touch events to a clipped surface, this series offers a documented,
>> device-tree-based solution by means of helper functions.
>> An implementation for a specific touchscreen driver is also included.
> 
> I believe there's at least some x86 tablets that have such a layout, so maybe ACPI bindings would also make sense? Or can this be supported by your DT-based solution?
> 
> I'm not sure if it would really be needed for existing devices, though. It's possible they were all handled by touchscreen controller firmware so far.
> 
> Hans, do you remember if we've encountered any Silead or Goodix devices where the soft button overlay didn't work due to missing firmware support?

The x86 devices with Silead or Goodix touchscreens with soft buttons which are
part of the touchscreen digitizer that I know off all handle the soft buttons
in firmware and do not report regular touches for the soft buttons area.

So I don't think that something like this is necessary for x86 devices atm.

Regards,

Hans