| Message ID | 20211213232849.40071-1-djrscally@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Introduce ancillary links | expand |
Em Mon, 13 Dec 2021 23:28:44 +0000 Daniel Scally <djrscally@gmail.com> escreveu: > Hello all > > At present there's no means in the kernel of describing the supporting > relationship between subdevices that work together to form an effective single > unit - the type example in this case being a camera sensor and its > corresponding vcm. To attempt to solve that, this series adds a new type of > media link called MEDIA_LNK_FL_ANCILLARY_LINK, which connects two instances of > struct media_entity. > > The mechanism of connection I have modelled as a notifier and async subdev, > which seemed the best route since sensor drivers already typically will call > v4l2_async_register_subdev_sensor() on probe, and that function already looks > for a reference to a firmware node with the reference named "lens-focus". To > avoid boilerplate in the sensor drivers, I added some new functions in > v4l2-async that are called in v4l2_async_match_notify() to create the ancillary > links - checking the entity.function of both notifier and subdev to make sure > that's appropriate. I haven't gone further than that yet, but I suspect we could > cut down on code elsewhere by, for example, also creating pad-to-pad links in > the same place. > > Thoughts and comments very welcome The idea of ancillary link sounds interesting. I did a quick look at the series. Laurent already did some good points during his review. Besides that, one thing it is missing, though, is an implementation on a driver. At least vimc should gain an implementation at this series, in order to allow media developers to test and see how the graph will be after the patch series. Regards, Mauro > > Dan > > Daniel Scally (5): > media: media.h: Add new media link type > media: entity: Add link_type() helper > media: entity: Skip non-data links in graph iteration > media: entity: Add support for ancillary links > media: v4l2-async: Create links during v4l2_async_match_notify() > > drivers/media/mc/mc-entity.c | 56 ++++++++++++++++++++++++++-- > drivers/media/v4l2-core/v4l2-async.c | 51 +++++++++++++++++++++++++ > include/media/media-entity.h | 29 ++++++++++++++ > include/uapi/linux/media.h | 1 + > 4 files changed, 134 insertions(+), 3 deletions(-) > Thanks, Mauro
Hi Mauro On 15/12/2021 09:25, Mauro Carvalho Chehab wrote: > Em Mon, 13 Dec 2021 23:28:44 +0000 > Daniel Scally <djrscally@gmail.com> escreveu: > >> Hello all >> >> At present there's no means in the kernel of describing the supporting >> relationship between subdevices that work together to form an effective single >> unit - the type example in this case being a camera sensor and its >> corresponding vcm. To attempt to solve that, this series adds a new type of >> media link called MEDIA_LNK_FL_ANCILLARY_LINK, which connects two instances of >> struct media_entity. >> >> The mechanism of connection I have modelled as a notifier and async subdev, >> which seemed the best route since sensor drivers already typically will call >> v4l2_async_register_subdev_sensor() on probe, and that function already looks >> for a reference to a firmware node with the reference named "lens-focus". To >> avoid boilerplate in the sensor drivers, I added some new functions in >> v4l2-async that are called in v4l2_async_match_notify() to create the ancillary >> links - checking the entity.function of both notifier and subdev to make sure >> that's appropriate. I haven't gone further than that yet, but I suspect we could >> cut down on code elsewhere by, for example, also creating pad-to-pad links in >> the same place. >> >> Thoughts and comments very welcome > The idea of ancillary link sounds interesting. I did a quick look at > the series. > > Laurent already did some good points during his review. > Besides that, one thing it is missing, though, is an implementation on > a driver. At least vimc should gain an implementation at this series, > in order to allow media developers to test and see how the graph will > be after the patch series. We have this running through libcamera at the moment; this series piggy backs onto the notifier that's set up by v4l2_async_create_sensor_subdev() so that the connection is made and the links created when a lens controller is linked to the sensor via the lens-focus firmware property. I've been testing this using the dw9719 driver I posted [1], plus a series that adds support to libcamera [2]. I believe that some folks from Cros have also tested it with the dw9714 driver too. [1] https://lore.kernel.org/linux-media/20211128232115.38833-1-djrscally@gmail.com/ [2] https://lists.libcamera.org/pipermail/libcamera-devel/2021-December/028082.html > > Regards, > Mauro > >> Dan >> >> Daniel Scally (5): >> media: media.h: Add new media link type >> media: entity: Add link_type() helper >> media: entity: Skip non-data links in graph iteration >> media: entity: Add support for ancillary links >> media: v4l2-async: Create links during v4l2_async_match_notify() >> >> drivers/media/mc/mc-entity.c | 56 ++++++++++++++++++++++++++-- >> drivers/media/v4l2-core/v4l2-async.c | 51 +++++++++++++++++++++++++ >> include/media/media-entity.h | 29 ++++++++++++++ >> include/uapi/linux/media.h | 1 + >> 4 files changed, 134 insertions(+), 3 deletions(-) >> > > > Thanks, > Mauro
Hi Mauro, On Wed, Dec 15, 2021 at 10:25:09AM +0100, Mauro Carvalho Chehab wrote: > Em Mon, 13 Dec 2021 23:28:44 +0000 Daniel Scally escreveu: > > > Hello all > > > > At present there's no means in the kernel of describing the supporting > > relationship between subdevices that work together to form an effective single > > unit - the type example in this case being a camera sensor and its > > corresponding vcm. To attempt to solve that, this series adds a new type of > > media link called MEDIA_LNK_FL_ANCILLARY_LINK, which connects two instances of > > struct media_entity. > > > > The mechanism of connection I have modelled as a notifier and async subdev, > > which seemed the best route since sensor drivers already typically will call > > v4l2_async_register_subdev_sensor() on probe, and that function already looks > > for a reference to a firmware node with the reference named "lens-focus". To > > avoid boilerplate in the sensor drivers, I added some new functions in > > v4l2-async that are called in v4l2_async_match_notify() to create the ancillary > > links - checking the entity.function of both notifier and subdev to make sure > > that's appropriate. I haven't gone further than that yet, but I suspect we could > > cut down on code elsewhere by, for example, also creating pad-to-pad links in > > the same place. > > > > Thoughts and comments very welcome > > The idea of ancillary link sounds interesting. I did a quick look at > the series. > > Laurent already did some good points during his review. > Besides that, one thing it is missing, though, is an implementation on > a driver. At least vimc should gain an implementation at this series, > in order to allow media developers to test and see how the graph will > be after the patch series. The whole point of patch 4/4 is to create those links in the core without requiring manual involvement of the drivers :-) > > Daniel Scally (5): > > media: media.h: Add new media link type > > media: entity: Add link_type() helper > > media: entity: Skip non-data links in graph iteration > > media: entity: Add support for ancillary links > > media: v4l2-async: Create links during v4l2_async_match_notify() > > > > drivers/media/mc/mc-entity.c | 56 ++++++++++++++++++++++++++-- > > drivers/media/v4l2-core/v4l2-async.c | 51 +++++++++++++++++++++++++ > > include/media/media-entity.h | 29 ++++++++++++++ > > include/uapi/linux/media.h | 1 + > > 4 files changed, 134 insertions(+), 3 deletions(-)