| Message ID | 20240510100347.102392-1-niklas.soderlund+renesas@ragnatech.se (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | media: staging: max96712: Store format in subdev active state | expand |
Hi Niklas, Thank you for the patch. On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote: > Create and store the subdevice format in the subdevices active state. > This change do not have a huge effect on the driver as it do not yet > support changing the format. > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se> > --- > Hello, > > This change is mostly to align all driver used in the R-Car VIN pipeline > to use the same set of API and to make it easier to extend them in > tandem going forward. > > This should be seen as a compliment to Jacopo's larger work in "[PATCH > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which > updates other drivers in the VIN pipeline to use the active state. > --- > drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------ > 1 file changed, 27 insertions(+), 9 deletions(-) > > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c > index c44145284aa1..2da65ccd2c57 100644 > --- a/drivers/staging/media/max96712/max96712.c > +++ b/drivers/staging/media/max96712/max96712.c > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = { > .s_stream = max96712_s_stream, > }; > > -static int max96712_get_pad_format(struct v4l2_subdev *sd, > - struct v4l2_subdev_state *sd_state, > - struct v4l2_subdev_format *format) > +static int max96712_init_state(struct v4l2_subdev *sd, > + struct v4l2_subdev_state *state) > { > - format->format.width = 1920; > - format->format.height = 1080; > - format->format.code = MEDIA_BUS_FMT_RGB888_1X24; > - format->format.field = V4L2_FIELD_NONE; > + static const struct v4l2_mbus_framefmt default_fmt = { > + .width = 1920, > + .height = 1080, > + .code = MEDIA_BUS_FMT_RGB888_1X24, > + .colorspace = V4L2_COLORSPACE_SRGB, > + .field = V4L2_FIELD_NONE, > + .ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT, > + .quantization = V4L2_QUANTIZATION_DEFAULT, > + .xfer_func = V4L2_XFER_FUNC_DEFAULT, > + }; > + struct v4l2_mbus_framefmt *fmt; > + > + fmt = v4l2_subdev_state_get_format(state, 0); > + *fmt = default_fmt; > > return 0; > } > > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = { > + .init_state = max96712_init_state, > +}; > + > static const struct v4l2_subdev_pad_ops max96712_pad_ops = { > - .get_fmt = max96712_get_pad_format, > - .set_fmt = max96712_get_pad_format, > + .get_fmt = v4l2_subdev_get_fmt, > + .set_fmt = v4l2_subdev_get_fmt, > }; > > static const struct v4l2_subdev_ops max96712_subdev_ops = { > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > long pixel_rate; > int ret; > > + priv->sd.internal_ops = &max96712_internal_ops; > v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops); > priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; > priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE; > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > v4l2_set_subdevdata(&priv->sd, priv); > I think it would be nice to already use the control handler lock as the active state lock. Apart from that, the patch looks good to me. > + ret = v4l2_subdev_init_finalize(&priv->sd); > + if (ret) > + goto error; > + > ret = v4l2_async_register_subdev(&priv->sd); > if (ret < 0) { > dev_err(&priv->client->dev, "Unable to register subdevice\n");
Hello Laurent, Thanks for your feedback. On 2024-05-10 13:13:40 +0300, Laurent Pinchart wrote: > Hi Niklas, > > Thank you for the patch. > > On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote: > > Create and store the subdevice format in the subdevices active state. > > This change do not have a huge effect on the driver as it do not yet > > support changing the format. > > > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se> > > --- > > Hello, > > > > This change is mostly to align all driver used in the R-Car VIN pipeline > > to use the same set of API and to make it easier to extend them in > > tandem going forward. > > > > This should be seen as a compliment to Jacopo's larger work in "[PATCH > > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which > > updates other drivers in the VIN pipeline to use the active state. > > --- > > drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------ > > 1 file changed, 27 insertions(+), 9 deletions(-) > > > > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c > > index c44145284aa1..2da65ccd2c57 100644 > > --- a/drivers/staging/media/max96712/max96712.c > > +++ b/drivers/staging/media/max96712/max96712.c > > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = { > > .s_stream = max96712_s_stream, > > }; > > > > -static int max96712_get_pad_format(struct v4l2_subdev *sd, > > - struct v4l2_subdev_state *sd_state, > > - struct v4l2_subdev_format *format) > > +static int max96712_init_state(struct v4l2_subdev *sd, > > + struct v4l2_subdev_state *state) > > { > > - format->format.width = 1920; > > - format->format.height = 1080; > > - format->format.code = MEDIA_BUS_FMT_RGB888_1X24; > > - format->format.field = V4L2_FIELD_NONE; > > + static const struct v4l2_mbus_framefmt default_fmt = { > > + .width = 1920, > > + .height = 1080, > > + .code = MEDIA_BUS_FMT_RGB888_1X24, > > + .colorspace = V4L2_COLORSPACE_SRGB, > > + .field = V4L2_FIELD_NONE, > > + .ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT, > > + .quantization = V4L2_QUANTIZATION_DEFAULT, > > + .xfer_func = V4L2_XFER_FUNC_DEFAULT, > > + }; > > + struct v4l2_mbus_framefmt *fmt; > > + > > + fmt = v4l2_subdev_state_get_format(state, 0); > > + *fmt = default_fmt; > > > > return 0; > > } > > > > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = { > > + .init_state = max96712_init_state, > > +}; > > + > > static const struct v4l2_subdev_pad_ops max96712_pad_ops = { > > - .get_fmt = max96712_get_pad_format, > > - .set_fmt = max96712_get_pad_format, > > + .get_fmt = v4l2_subdev_get_fmt, > > + .set_fmt = v4l2_subdev_get_fmt, > > }; > > > > static const struct v4l2_subdev_ops max96712_subdev_ops = { > > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > long pixel_rate; > > int ret; > > > > + priv->sd.internal_ops = &max96712_internal_ops; > > v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops); > > priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; > > priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE; > > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > > > v4l2_set_subdevdata(&priv->sd, priv); > > > > I think it would be nice to already use the control handler lock as the > active state lock. Apart from that, the patch looks good to me. I also worked on this to learn a bit more about the active state API. I assume what you are asking for here is ? priv->sd.state_lock = priv->ctrl_handler.lock; What is the rational for using the control handler lock over the default state lock? I checked Jacopo's series when working on this and there he uses no lock for the R-Car CSI-2 driver and a private driver global state lock for ADV748x. > > > + ret = v4l2_subdev_init_finalize(&priv->sd); > > + if (ret) > > + goto error; > > + > > ret = v4l2_async_register_subdev(&priv->sd); > > if (ret < 0) { > > dev_err(&priv->client->dev, "Unable to register subdevice\n"); > > -- > Regards, > > Laurent Pinchart
Hi Niklas, On Fri, May 10, 2024 at 12:36:56PM +0200, Niklas Söderlund wrote: > On 2024-05-10 13:13:40 +0300, Laurent Pinchart wrote: > > On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote: > > > Create and store the subdevice format in the subdevices active state. > > > This change do not have a huge effect on the driver as it do not yet > > > support changing the format. > > > > > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se> > > > --- > > > Hello, > > > > > > This change is mostly to align all driver used in the R-Car VIN pipeline > > > to use the same set of API and to make it easier to extend them in > > > tandem going forward. > > > > > > This should be seen as a compliment to Jacopo's larger work in "[PATCH > > > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which > > > updates other drivers in the VIN pipeline to use the active state. > > > --- > > > drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------ > > > 1 file changed, 27 insertions(+), 9 deletions(-) > > > > > > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c > > > index c44145284aa1..2da65ccd2c57 100644 > > > --- a/drivers/staging/media/max96712/max96712.c > > > +++ b/drivers/staging/media/max96712/max96712.c > > > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = { > > > .s_stream = max96712_s_stream, > > > }; > > > > > > -static int max96712_get_pad_format(struct v4l2_subdev *sd, > > > - struct v4l2_subdev_state *sd_state, > > > - struct v4l2_subdev_format *format) > > > +static int max96712_init_state(struct v4l2_subdev *sd, > > > + struct v4l2_subdev_state *state) > > > { > > > - format->format.width = 1920; > > > - format->format.height = 1080; > > > - format->format.code = MEDIA_BUS_FMT_RGB888_1X24; > > > - format->format.field = V4L2_FIELD_NONE; > > > + static const struct v4l2_mbus_framefmt default_fmt = { > > > + .width = 1920, > > > + .height = 1080, > > > + .code = MEDIA_BUS_FMT_RGB888_1X24, > > > + .colorspace = V4L2_COLORSPACE_SRGB, > > > + .field = V4L2_FIELD_NONE, > > > + .ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT, > > > + .quantization = V4L2_QUANTIZATION_DEFAULT, > > > + .xfer_func = V4L2_XFER_FUNC_DEFAULT, > > > + }; > > > + struct v4l2_mbus_framefmt *fmt; > > > + > > > + fmt = v4l2_subdev_state_get_format(state, 0); > > > + *fmt = default_fmt; > > > > > > return 0; > > > } > > > > > > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = { > > > + .init_state = max96712_init_state, > > > +}; > > > + > > > static const struct v4l2_subdev_pad_ops max96712_pad_ops = { > > > - .get_fmt = max96712_get_pad_format, > > > - .set_fmt = max96712_get_pad_format, > > > + .get_fmt = v4l2_subdev_get_fmt, > > > + .set_fmt = v4l2_subdev_get_fmt, > > > }; > > > > > > static const struct v4l2_subdev_ops max96712_subdev_ops = { > > > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > > long pixel_rate; > > > int ret; > > > > > > + priv->sd.internal_ops = &max96712_internal_ops; > > > v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops); > > > priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; > > > priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE; > > > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > > > > > v4l2_set_subdevdata(&priv->sd, priv); > > > > > > > I think it would be nice to already use the control handler lock as the > > active state lock. Apart from that, the patch looks good to me. > > I also worked on this to learn a bit more about the active state API. I > assume what you are asking for here is ? > > priv->sd.state_lock = priv->ctrl_handler.lock; That's right. > What is the rational for using the control handler lock over the default > state lock? When there are interactions between controls and formats, using separate locks can lead to AB/BA deadlocks if you need to access controls from a context where the state is locked (e.g. calling v4l2_ctrl_handler_setup() at stream enable time), and access the active state from a context where the control handler is locked (e.g. accessing the state in .s_ctrl()). Using the same lock helps there, you can then call __v4l2_ctrl_handler_setup() at stream enable time, and get the state without locking it in .s_ctrl(). At some point in the not too distant future I would like to move storage of controls to the state. I don't know how painful that will be though, the control framework has grown to be a very complex beast, with lots of features that are not useful for MC-based drivers (such as control handler inheritance). I'm worried that we'll need a full rewrite... We'll see :-) > I checked Jacopo's series when working on this and there he > uses no lock for the R-Car CSI-2 driver and a private driver global > state lock for ADV748x. For the CSI-2 RX driver, that's because it doesn't have a control handler. For the ADV748x driver, that's because the control handler is configured to use the device-wide mutex. > > > + ret = v4l2_subdev_init_finalize(&priv->sd); > > > + if (ret) > > > + goto error; > > > + > > > ret = v4l2_async_register_subdev(&priv->sd); > > > if (ret < 0) { > > > dev_err(&priv->client->dev, "Unable to register subdevice\n");
Hi Laurent, On 2024-05-10 18:27:24 +0300, Laurent Pinchart wrote: > Hi Niklas, > > On Fri, May 10, 2024 at 12:36:56PM +0200, Niklas Söderlund wrote: > > On 2024-05-10 13:13:40 +0300, Laurent Pinchart wrote: > > > On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote: > > > > Create and store the subdevice format in the subdevices active state. > > > > This change do not have a huge effect on the driver as it do not yet > > > > support changing the format. > > > > > > > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se> > > > > --- > > > > Hello, > > > > > > > > This change is mostly to align all driver used in the R-Car VIN pipeline > > > > to use the same set of API and to make it easier to extend them in > > > > tandem going forward. > > > > > > > > This should be seen as a compliment to Jacopo's larger work in "[PATCH > > > > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which > > > > updates other drivers in the VIN pipeline to use the active state. > > > > --- > > > > drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------ > > > > 1 file changed, 27 insertions(+), 9 deletions(-) > > > > > > > > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c > > > > index c44145284aa1..2da65ccd2c57 100644 > > > > --- a/drivers/staging/media/max96712/max96712.c > > > > +++ b/drivers/staging/media/max96712/max96712.c > > > > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = { > > > > .s_stream = max96712_s_stream, > > > > }; > > > > > > > > -static int max96712_get_pad_format(struct v4l2_subdev *sd, > > > > - struct v4l2_subdev_state *sd_state, > > > > - struct v4l2_subdev_format *format) > > > > +static int max96712_init_state(struct v4l2_subdev *sd, > > > > + struct v4l2_subdev_state *state) > > > > { > > > > - format->format.width = 1920; > > > > - format->format.height = 1080; > > > > - format->format.code = MEDIA_BUS_FMT_RGB888_1X24; > > > > - format->format.field = V4L2_FIELD_NONE; > > > > + static const struct v4l2_mbus_framefmt default_fmt = { > > > > + .width = 1920, > > > > + .height = 1080, > > > > + .code = MEDIA_BUS_FMT_RGB888_1X24, > > > > + .colorspace = V4L2_COLORSPACE_SRGB, > > > > + .field = V4L2_FIELD_NONE, > > > > + .ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT, > > > > + .quantization = V4L2_QUANTIZATION_DEFAULT, > > > > + .xfer_func = V4L2_XFER_FUNC_DEFAULT, > > > > + }; > > > > + struct v4l2_mbus_framefmt *fmt; > > > > + > > > > + fmt = v4l2_subdev_state_get_format(state, 0); > > > > + *fmt = default_fmt; > > > > > > > > return 0; > > > > } > > > > > > > > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = { > > > > + .init_state = max96712_init_state, > > > > +}; > > > > + > > > > static const struct v4l2_subdev_pad_ops max96712_pad_ops = { > > > > - .get_fmt = max96712_get_pad_format, > > > > - .set_fmt = max96712_get_pad_format, > > > > + .get_fmt = v4l2_subdev_get_fmt, > > > > + .set_fmt = v4l2_subdev_get_fmt, > > > > }; > > > > > > > > static const struct v4l2_subdev_ops max96712_subdev_ops = { > > > > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > > > long pixel_rate; > > > > int ret; > > > > > > > > + priv->sd.internal_ops = &max96712_internal_ops; > > > > v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops); > > > > priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; > > > > priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE; > > > > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv) > > > > > > > > v4l2_set_subdevdata(&priv->sd, priv); > > > > > > > > > > I think it would be nice to already use the control handler lock as the > > > active state lock. Apart from that, the patch looks good to me. > > > > I also worked on this to learn a bit more about the active state API. I > > assume what you are asking for here is ? > > > > priv->sd.state_lock = priv->ctrl_handler.lock; > > That's right. I will send a v2 of this patch using the controller lock. > > > What is the rational for using the control handler lock over the default > > state lock? > > When there are interactions between controls and formats, using separate > locks can lead to AB/BA deadlocks if you need to access controls from a > context where the state is locked (e.g. calling > v4l2_ctrl_handler_setup() at stream enable time), and access the active > state from a context where the control handler is locked (e.g. accessing > the state in .s_ctrl()). Using the same lock helps there, you can then > call __v4l2_ctrl_handler_setup() at stream enable time, and get the > state without locking it in .s_ctrl(). > > At some point in the not too distant future I would like to move storage > of controls to the state. I don't know how painful that will be though, > the control framework has grown to be a very complex beast, with lots of > features that are not useful for MC-based drivers (such as control > handler inheritance). I'm worried that we'll need a full rewrite... > We'll see :-) > > > I checked Jacopo's series when working on this and there he > > uses no lock for the R-Car CSI-2 driver and a private driver global > > state lock for ADV748x. > > For the CSI-2 RX driver, that's because it doesn't have a control > handler. For the ADV748x driver, that's because the control handler is > configured to use the device-wide mutex. Thanks for this through description of why using the control handler lock for the stat is a good idea. I really like your idea of trying to make the control framework less complex for MC-based drivers in the future. > > > > > + ret = v4l2_subdev_init_finalize(&priv->sd); > > > > + if (ret) > > > > + goto error; > > > > + > > > > ret = v4l2_async_register_subdev(&priv->sd); > > > > if (ret < 0) { > > > > dev_err(&priv->client->dev, "Unable to register subdevice\n"); > > -- > Regards, > > Laurent Pinchart
diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c index c44145284aa1..2da65ccd2c57 100644 --- a/drivers/staging/media/max96712/max96712.c +++ b/drivers/staging/media/max96712/max96712.c @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = { .s_stream = max96712_s_stream, }; -static int max96712_get_pad_format(struct v4l2_subdev *sd, - struct v4l2_subdev_state *sd_state, - struct v4l2_subdev_format *format) +static int max96712_init_state(struct v4l2_subdev *sd, + struct v4l2_subdev_state *state) { - format->format.width = 1920; - format->format.height = 1080; - format->format.code = MEDIA_BUS_FMT_RGB888_1X24; - format->format.field = V4L2_FIELD_NONE; + static const struct v4l2_mbus_framefmt default_fmt = { + .width = 1920, + .height = 1080, + .code = MEDIA_BUS_FMT_RGB888_1X24, + .colorspace = V4L2_COLORSPACE_SRGB, + .field = V4L2_FIELD_NONE, + .ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT, + .quantization = V4L2_QUANTIZATION_DEFAULT, + .xfer_func = V4L2_XFER_FUNC_DEFAULT, + }; + struct v4l2_mbus_framefmt *fmt; + + fmt = v4l2_subdev_state_get_format(state, 0); + *fmt = default_fmt; return 0; } +static const struct v4l2_subdev_internal_ops max96712_internal_ops = { + .init_state = max96712_init_state, +}; + static const struct v4l2_subdev_pad_ops max96712_pad_ops = { - .get_fmt = max96712_get_pad_format, - .set_fmt = max96712_get_pad_format, + .get_fmt = v4l2_subdev_get_fmt, + .set_fmt = v4l2_subdev_get_fmt, }; static const struct v4l2_subdev_ops max96712_subdev_ops = { @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv) long pixel_rate; int ret; + priv->sd.internal_ops = &max96712_internal_ops; v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops); priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE; priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE; @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv) v4l2_set_subdevdata(&priv->sd, priv); + ret = v4l2_subdev_init_finalize(&priv->sd); + if (ret) + goto error; + ret = v4l2_async_register_subdev(&priv->sd); if (ret < 0) { dev_err(&priv->client->dev, "Unable to register subdevice\n");
Create and store the subdevice format in the subdevices active state. This change do not have a huge effect on the driver as it do not yet support changing the format. Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se> --- Hello, This change is mostly to align all driver used in the R-Car VIN pipeline to use the same set of API and to make it easier to extend them in tandem going forward. This should be seen as a compliment to Jacopo's larger work in "[PATCH v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which updates other drivers in the VIN pipeline to use the active state. --- drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------ 1 file changed, 27 insertions(+), 9 deletions(-)