[0/4] media: camss: sm8250: Virtual channels support for SM8250

Message

Milen Mitkov (Consultant) Sept. 21, 2022, 2:10 p.m. UTC

From: Milen Mitkov <[e-mail]>

(Resent with hopefully the correct recipients this time)

These patches add support for multiple CSID virtual channels for Qualcomm's SM8250
platform.

NOTE: These changes depend on the multistream series that as of yet is
still not merged officially upstream. The multistream changes
allow for multiple clients on the same pipeline, which is otherwise not
possible. For development and testing multistream series v14 was used:

https://patchwork.kernel.org/project/linux-media/patch/[e-mail]/

The CSID hardware on SM8250 can demux the input data stream into
maximum of 4 multiple streams depending on virtual channel (vc)
or data type (dt) configuration.

Situations in which this is useful:
- HDR sensors that produce a 2-frame HDR output, e.g. a light and a dark frame
  (the setup we used  for testing, with the imx412 sensor),
  or a 3-frame HDR output - light, medium-lit, dark frame.
- sensors with additional metadata that is streamed over a different
  virtual channel/datatype.
- sensors that stream frames with multiple resolutions in the same pixel
  data stream

With these changes, the CSID entity has, as it did previously, a single
sink port (0), and always exposes 4 source ports (1, 2,3, 4). The
virtual channel configuration is determined by which of the source ports
are linked to an output VFE line. For example, the link below will
configure the CSID driver to enable vc 0 and vc 1:

media-ctl -l '"msm_csid0":1->"msm_vfe0_rdi0":0[1]'
media-ctl -l '"msm_csid0":2->"msm_vfe0_rdi1":0[1]'

which will be demuxed and propagated into /dev/video0
and /dev/video1 respectively. With this, the userspace can use
any normal V4L2 client app to start/stop/queue/dequeue from these
video nodes. Tested with the yavta app.

Note also that implicit format propagation has been removed, so format
for the linked source ports has to be set manually, e.g.:

media-ctl -V '"msm_csid0":1[fmt:SRGGB10/3840x2160]'
media-ctl -V '"msm_csid0":2[fmt:SRGGB10/3840x2160]'

Previously setting the format on the sink port of the CSID entity would
set the same format on the source port. However, with CSID demuxing there's
no guarantee that the format on the source ports should be the same,
in fact it can be different for each source port.

Milen Mitkov (4):
  media: camss: sm8250: Virtual channels for CSID
  media: camss: vfe: Reserve VFE lines on stream start and link to CSID
  media: camss: vfe-480: Multiple outputs support for SM8250
  media: camss: sm8250: Pipeline starting and stopping for multiple
    virtual channels

 .../platform/qcom/camss/camss-csid-gen2.c     | 55 ++++++++++-------
 .../media/platform/qcom/camss/camss-csid.c    | 41 +++++++------
 .../media/platform/qcom/camss/camss-csid.h    | 11 +++-
 .../media/platform/qcom/camss/camss-vfe-480.c | 60 ++++++++++++-------
 drivers/media/platform/qcom/camss/camss-vfe.c |  7 +++
 .../media/platform/qcom/camss/camss-video.c   | 21 ++++++-
 drivers/media/platform/qcom/camss/camss.c     |  2 +-
 7 files changed, 133 insertions(+), 64 deletions(-)