From patchwork Sat May 18 01:07:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10948833 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC217933 for ; Sat, 18 May 2019 01:07:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D5F0528521 for ; Sat, 18 May 2019 01:07:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C97B82852A; Sat, 18 May 2019 01:07:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59F6828521 for ; Sat, 18 May 2019 01:07:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728012AbfERBH4 (ORCPT ); Fri, 17 May 2019 21:07:56 -0400 Received: from vsp-unauthed02.binero.net ([195.74.38.227]:16489 "EHLO vsp-unauthed02.binero.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726200AbfERBHz (ORCPT ); Fri, 17 May 2019 21:07:55 -0400 X-Halon-ID: 5be6fd5a-7909-11e9-8d05-005056917f90 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [89.233.230.99]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 5be6fd5a-7909-11e9-8d05-005056917f90; Sat, 18 May 2019 03:07:51 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: Helen Koike , linux-media@vger.kernel.org Cc: libcamera-devel@lists.libcamera.org, =?utf-8?q?Niklas_S=C3=B6derlund?= , =?utf-8?q?Niklas_S=C3=B6derlund?= Subject: [PATCH 1/3] vimc: Add usage count to subdevices Date: Sat, 18 May 2019 03:07:42 +0200 Message-Id: <20190518010744.15195-2-niklas.soderlund+renesas@ragnatech.se> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190518010744.15195-1-niklas.soderlund+renesas@ragnatech.se> References: <20190518010744.15195-1-niklas.soderlund+renesas@ragnatech.se> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Prepare for multiple video streams from the same sensor by adding a use counter to each subdevice. The counter is increase for every s_stream(1) and decremented for every s_stream(0) call. The subdevice stream is not started or stopped unless the usage count go from 0 to 1 (started) or from 1 to 0 (stopped). This allow for multiple s_stream() calls to try to either start or stop the device while only the first/last call will actually effect the state of the device. Signed-off-by: Niklas Söderlund --- drivers/media/platform/vimc/vimc-debayer.c | 8 ++++++++ drivers/media/platform/vimc/vimc-scaler.c | 8 ++++++++ drivers/media/platform/vimc/vimc-sensor.c | 7 +++++++ 3 files changed, 23 insertions(+) diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c index 281f9c1a7249ad1d..624fc23ce3077d40 100644 --- a/drivers/media/platform/vimc/vimc-debayer.c +++ b/drivers/media/platform/vimc/vimc-debayer.c @@ -56,6 +56,7 @@ struct vimc_deb_device { struct vimc_ent_device ved; struct v4l2_subdev sd; struct device *dev; + atomic_t use_count; /* The active format */ struct v4l2_mbus_framefmt sink_fmt; u32 src_code; @@ -337,6 +338,9 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable) const struct v4l2_format_info *pix_info; unsigned int frame_size; + if (atomic_inc_return(&vdeb->use_count) != 1) + return 0; + if (vdeb->src_frame) return 0; @@ -374,6 +378,9 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable) return -ENOMEM; } else { + if (atomic_dec_return(&vdeb->use_count) != 0) + return 0; + if (!vdeb->src_frame) return 0; @@ -562,6 +569,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master, vdeb->ved.process_frame = vimc_deb_process_frame; dev_set_drvdata(comp, &vdeb->ved); vdeb->dev = comp; + atomic_set(&vdeb->use_count, 0); /* Initialize the frame format */ vdeb->sink_fmt = sink_fmt_default; diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c index 8aecf8e920310608..37d2020d987a7d80 100644 --- a/drivers/media/platform/vimc/vimc-scaler.c +++ b/drivers/media/platform/vimc/vimc-scaler.c @@ -45,6 +45,7 @@ struct vimc_sca_device { struct vimc_ent_device ved; struct v4l2_subdev sd; struct device *dev; + atomic_t use_count; /* NOTE: the source fmt is the same as the sink * with the width and hight multiplied by mult */ @@ -213,6 +214,9 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable) const struct v4l2_format_info *pix_info; unsigned int frame_size; + if (atomic_inc_return(&vsca->use_count) != 1) + return 0; + if (vsca->src_frame) return 0; @@ -242,6 +246,9 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable) return -ENOMEM; } else { + if (atomic_dec_return(&vsca->use_count) != 0) + return 0; + if (!vsca->src_frame) return 0; @@ -396,6 +403,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master, vsca->ved.process_frame = vimc_sca_process_frame; dev_set_drvdata(comp, &vsca->ved); vsca->dev = comp; + atomic_set(&vsca->use_count, 0); /* Initialize the frame format */ vsca->sink_fmt = sink_fmt_default; diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c index baca9ca67ce0af0b..36c3cea85a185f4b 100644 --- a/drivers/media/platform/vimc/vimc-sensor.c +++ b/drivers/media/platform/vimc/vimc-sensor.c @@ -34,6 +34,7 @@ struct vimc_sen_device { struct vimc_ent_device ved; struct v4l2_subdev sd; struct device *dev; + atomic_t use_count; struct tpg_data tpg; struct task_struct *kthread_sen; u8 *frame; @@ -197,6 +198,9 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable) const struct v4l2_format_info *pix_info; unsigned int frame_size; + if (atomic_inc_return(&vsen->use_count) != 1) + return 0; + if (vsen->kthread_sen) /* tpg is already executing */ return 0; @@ -218,6 +222,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable) vimc_sen_tpg_s_format(vsen); } else { + if (atomic_dec_return(&vsen->use_count) != 0) + return 0; vfree(vsen->frame); vsen->frame = NULL; @@ -367,6 +373,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master, vsen->ved.process_frame = vimc_sen_process_frame; dev_set_drvdata(comp, &vsen->ved); vsen->dev = comp; + atomic_set(&vsen->use_count, 0); /* Initialize the frame format */ vsen->mbus_format = fmt_default; From patchwork Sat May 18 01:07:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10948835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E35513AD for ; Sat, 18 May 2019 01:07:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17E6F28521 for ; Sat, 18 May 2019 01:07:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0C0192852B; Sat, 18 May 2019 01:07:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD76628521 for ; Sat, 18 May 2019 01:07:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728037AbfERBH5 (ORCPT ); Fri, 17 May 2019 21:07:57 -0400 Received: from bin-mail-out-06.binero.net ([195.74.38.229]:16505 "EHLO bin-mail-out-06.binero.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726200AbfERBH5 (ORCPT ); Fri, 17 May 2019 21:07:57 -0400 X-Halon-ID: 5cb8cb74-7909-11e9-8d05-005056917f90 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [89.233.230.99]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 5cb8cb74-7909-11e9-8d05-005056917f90; Sat, 18 May 2019 03:07:53 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: Helen Koike , linux-media@vger.kernel.org Cc: libcamera-devel@lists.libcamera.org, =?utf-8?q?Niklas_S=C3=B6derlund?= , =?utf-8?q?Niklas_S=C3=B6derlund?= Subject: [PATCH 2/3] vimc: Serialize vimc_streamer_s_stream() Date: Sat, 18 May 2019 03:07:43 +0200 Message-Id: <20190518010744.15195-3-niklas.soderlund+renesas@ragnatech.se> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190518010744.15195-1-niklas.soderlund+renesas@ragnatech.se> References: <20190518010744.15195-1-niklas.soderlund+renesas@ragnatech.se> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Prepare for multiple video streams from the same sensor by serializing vimc_streamer_s_stream(). Multiple streams will allow for multiple concurrent calls to this function that could involve the same subdevices. If that happens the internal state of the involved subdevices could go out of sync as they are being started and stopped at the same time, prevent this by serializing starting and stopping of the vimc streamer. Signed-off-by: Niklas Söderlund --- drivers/media/platform/vimc/vimc-streamer.c | 23 ++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c index 26b6742594890b16..514690dca3b1187b 100644 --- a/drivers/media/platform/vimc/vimc-streamer.c +++ b/drivers/media/platform/vimc/vimc-streamer.c @@ -153,39 +153,48 @@ int vimc_streamer_s_stream(struct vimc_stream *stream, struct vimc_ent_device *ved, int enable) { + static DEFINE_MUTEX(vimc_streamer_lock); int ret; if (!stream || !ved) return -EINVAL; + ret = mutex_lock_interruptible(&vimc_streamer_lock); + if (ret) + return ret; + if (enable) { if (stream->kthread) - return 0; + goto out; ret = vimc_streamer_pipeline_init(stream, ved); if (ret) - return ret; + goto out; stream->kthread = kthread_run(vimc_streamer_thread, stream, "vimc-streamer thread"); - if (IS_ERR(stream->kthread)) - return PTR_ERR(stream->kthread); + if (IS_ERR(stream->kthread)) { + ret = PTR_ERR(stream->kthread); + goto out; + } } else { if (!stream->kthread) - return 0; + goto out; ret = kthread_stop(stream->kthread); if (ret) - return ret; + goto out; stream->kthread = NULL; vimc_streamer_pipeline_terminate(stream); } +out: + mutex_unlock(&vimc_streamer_lock); - return 0; + return ret; } EXPORT_SYMBOL_GPL(vimc_streamer_s_stream); From patchwork Sat May 18 01:07:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Niklas_S=C3=B6derlund?= X-Patchwork-Id: 10948837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 503AC1515 for ; Sat, 18 May 2019 01:07:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B9B128521 for ; Sat, 18 May 2019 01:07:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 302312852B; Sat, 18 May 2019 01:07:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDA2028528 for ; Sat, 18 May 2019 01:07:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728046AbfERBH6 (ORCPT ); Fri, 17 May 2019 21:07:58 -0400 Received: from bin-mail-out-06.binero.net ([195.74.38.229]:16512 "EHLO bin-mail-out-06.binero.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728024AbfERBH5 (ORCPT ); Fri, 17 May 2019 21:07:57 -0400 X-Halon-ID: 5d5e763e-7909-11e9-8d05-005056917f90 Authorized-sender: niklas@soderlund.pp.se Received: from bismarck.berto.se (unknown [89.233.230.99]) by bin-vsp-out-02.atm.binero.net (Halon) with ESMTPA id 5d5e763e-7909-11e9-8d05-005056917f90; Sat, 18 May 2019 03:07:54 +0200 (CEST) From: =?utf-8?q?Niklas_S=C3=B6derlund?= To: Helen Koike , linux-media@vger.kernel.org Cc: libcamera-devel@lists.libcamera.org, =?utf-8?q?Niklas_S=C3=B6derlund?= , =?utf-8?q?Niklas_S=C3=B6derlund?= Subject: [PATCH 3/3] vimc: Join pipeline if one already exists Date: Sat, 18 May 2019 03:07:44 +0200 Message-Id: <20190518010744.15195-4-niklas.soderlund+renesas@ragnatech.se> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190518010744.15195-1-niklas.soderlund+renesas@ragnatech.se> References: <20190518010744.15195-1-niklas.soderlund+renesas@ragnatech.se> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A sensor which is running is already part of a pipeline and trying to start a new pipeline is not possible. This prevents two capture devices connected to the same sensor from running at the same time. Instead of failing to start the second capture device allow it to join the already running pipeline by looking it up at the sensor. This allows two (or more) capture devices to independently be started and stopped while still being connected to the same sensor. Signed-off-by: Niklas Söderlund --- drivers/media/platform/vimc/vimc-capture.c | 35 +++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c index e7d0fc2228a6f0c1..f9eb1e327e311b4a 100644 --- a/drivers/media/platform/vimc/vimc-capture.c +++ b/drivers/media/platform/vimc/vimc-capture.c @@ -264,16 +264,49 @@ static void vimc_cap_return_all_buffers(struct vimc_cap_device *vcap, spin_unlock(&vcap->qlock); } +static struct media_entity *vimc_cap_get_sensor(struct vimc_cap_device *vcap) +{ + struct media_entity *entity = &vcap->vdev.entity; + struct media_device *mdev = entity->graph_obj.mdev; + struct media_graph graph; + + mutex_lock(&mdev->graph_mutex); + if (media_graph_walk_init(&graph, mdev)) { + mutex_unlock(&mdev->graph_mutex); + return NULL; + } + + media_graph_walk_start(&graph, entity); + + while ((entity = media_graph_walk_next(&graph))) + if (entity->function == MEDIA_ENT_F_CAM_SENSOR) + break; + + mutex_unlock(&mdev->graph_mutex); + + media_graph_walk_cleanup(&graph); + + return entity; +} + static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count) { struct vimc_cap_device *vcap = vb2_get_drv_priv(vq); struct media_entity *entity = &vcap->vdev.entity; + struct media_pipeline *pipe = NULL; + struct media_entity *sensorent; int ret; vcap->sequence = 0; /* Start the media pipeline */ - ret = media_pipeline_start(entity, &vcap->stream.pipe); + sensorent = vimc_cap_get_sensor(vcap); + if (sensorent && sensorent->pipe) + pipe = sensorent->pipe; + else + pipe = &vcap->stream.pipe; + + ret = media_pipeline_start(entity, pipe); if (ret) { vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED); return ret;